Gzip сжатие для ускорения загрузки сайта
Содержание:
DESCRIPTION
gunzip takes a list of files on its command line and replaces each file whose name ends with .gz, -gz, .z, -z, _z or .Z and which begins with the correct magic number with an uncompressed file without the original extension. gunzip also recognizes the special extensions .tgz and .taz as shorthands for .tar.gz and .tar.Z respectively. When compressing, gzip uses the .tgz extension if necessary instead of truncating a file with a .tar extension.
gunzip can currently decompress files created by gzip, zip, compress, compress -H or pack. The detection of the input format is automatic. When using the first two formats, gunzip checks a 32 bit CRC. For pack, gunzip checks the uncompressed length. The standard compress format was not designed to allow consistency checks. However gunzip is sometimes able to detect a bad .Z file. If you get an error when uncompressing a .Z file, do not assume that the .Z file is correct simply because the standard uncompress does not complain. This generally means that the standard uncompress does not check its input, and happily generates garbage output. The SCO compress -H format (lzh compression method) does not include a CRC but also allows some consistency checks.
Files created by zip can be uncompressed by gzip only if they have a single member compressed with the ‘deflation’ method. This feature is only intended to help conversion of tar.zip files to the tar.gz format. To extract a zip file with a single member, use a command like gunzip <foo.zip or gunzip -S .zip foo.zip. To extract zip files with several members, use unzip instead of gunzip.
Advanced Usage
Multiple compressed files can be concatenated. In this case, gunzip will extract all members at once. For example:
gzip -c file1 > foo.gz
gzip -c file2 >> foo.gz
Then:
gunzip -c foo
is equivalent to:
cat file1 file2
In case of damage to one member of a .gz file, other members can still be recovered (if the damaged member is removed). However, you can get better compression by compressing all members at once:
cat file1 file2 | gzip > foo.gz
compresses better than:
gzip -c file1 file2 > foo.gz
If you want to recompress concatenated files to get better compression, do:
gzip -cd old.gz | gzip > new.gz
If a compressed file consists of several members, the uncompressed size and CRC reported by the —list option applies to the last member only. If you need the uncompressed size for all members, you can use:
gzip -cd file.gz | wc -c
If you want to create a single archive file with multiple members so that members can later be extracted independently, use an archiver such as tar or zip. GNU tar supports the -z option to invoke gzip transparently. gzip is designed as a complement to tar, not as a replacement.
Examples
zcat myfiles.tar.gz | more
Prints the contents of all files within the gzipped tar archive myfiles.tar.gz, piping the output to the more command, which pages the output (pauses at the end of each page).
gzip backup.tar
Compresses the tar archive backup.tar, and renames it backup.tar.gz.
gunzip backup.tar.gz
Uncompresses the gzipped file backup.tar.gz, renaming it backup.tar.
gzip -r backupfolder
Recursively compresses all files in the folder backupfolder, and all files in any subdirectories of that folder, and gives them the extension .gz.
gunzip -r backupfolder
Recursively uncompresses all gzipped files in the folder backupfolder, and removes the extension .gz from the filenames.
gzip -c myfile.txt > myfile.txt.gz
Compresses the file myfile.txt, writing the compressed file data to standard output, which in this case is redirected to a file, myfile.txt.gz. The result is that the original file and the gzipped file will both exist after the command is run.
zcat myfile.txt.gz
Prints the uncompressed contents of the compressed file myfile.txt.gz. The output is identical to printing the contents of the uncompressed file with the command cat myfile.txt.
gunzip -c myfile.txt.gz > myfile.txt
Uncompresses the gzipped file myfile.txt.gz to standard output, which in this case is redirected to the file myfile.txt. If myfile.txt already exists, the shell will prompt you to overwrite it (or not).
Tip
See the above for additional examples.
ADVANCED USAGE
gunzip
gzip -c file1 > foo.gz
gzip -c file2 >> foo.gz
Then
gunzip -c foo
is equivalent to
cat file1 file2
In case of damage to one member of a .gz file, other members can still be recovered (if the damaged member is removed). However, you can get better compression by compressing all members at once:
cat file1 file2 | gzip > foo.gz
compresses better than
gzip -c file1 file2 > foo.gz
If you want to recompress concatenated files to get better compression, do:
gzip -cd old.gz | gzip > new.gz
If a compressed file consists of several members, the uncompressed size and CRC reported by the —list option applies to the last member only. If you need the uncompressed size for all members, you can use:
gzip -cd file.gz | wc -c
If you wish to create a single archive file with multiple members so that members can later be extracted independently, use an archiver such as tar or zip. GNU tar supports the -z option to invoke gzip transparently. gzip is designed as a complement
to tar, not as a replacement.
Options
| -a, —ascii | ASCII text mode: convert end-of-lines using local conventions. This option is supported only on some non-Unix systems. For MSDOS, CR LF is converted to LF when compressing, and LF is converted to CR LF when decompressing. | ||||||||||||||
| -c, —stdout, —to-stdout | Write output on standard output; keep original files unchanged. If there are several input files, the output consists of a sequence of independently compressed members. To obtain better compression, concatenate all input files before compressing them. | ||||||||||||||
| -d, —decompress, —uncompress | Decompress. | ||||||||||||||
| -f, —force | Force compression or decompression even if the file has multiple links or the corresponding file already exists, or if the compressed data is read from or written to a terminal. If the input data is not in a format recognized by gzip, and if the option —stdout is also given, copy the input data without change to the standard output: let zcat behave as cat. If -f is not given, and when not running in the background, gzip prompts to verify whether an existing file should be overwritten. | ||||||||||||||
| -h, —help | Display a help screen and quit. | ||||||||||||||
| -l, —list | For each compressed file, list the following fields:
The uncompressed size is given as -1 for files not in gzip format, such as compressed .Z files. To get the uncompressed size for such a file, you can use: zcat file.Z | wc -c In combination with the —verbose option, the following fields are also displayed:
The compression methods currently supported are deflate, compress, lzh (SCO compress -H) and pack. The crc is given as ffffffff for a file not in gzip format. |
||||||||||||||
| -L, —license | Display the gzip license and exit. | ||||||||||||||
| -n, —no-name | When compressing, do not save the original file name and timestamp by default. (The original name is always saved if the name had to be truncated.) When decompressing, do not restore the original file name if present (remove only the gzip suffix from the compressed file name) and do not restore the original time stamp if present (copy it from the compressed file). This option is the default when decompressing. | ||||||||||||||
| -N, —name | When compressing, always save the original file name and time stamp; this is the default. When decompressing, restore the original file name and time stamp if present. This option is useful on systems which have a limit on file name length or when the time stamp has been lost after a file transfer. | ||||||||||||||
| -q, —quiet | Suppress all warnings. | ||||||||||||||
| -r, —recursive | Travel the directory structure recursively. If any of the file names specified on the command line are directories, gzip will descend into the directory and compress all the files it finds there (or decompress them in the case of gunzip). | ||||||||||||||
| -S .suf, —suffix .suf | When compressing, use suffix .suf instead of .gz. Any non-empty suffix can be given, but suffixes other than .z and .gz should be avoided to avoid confusion when files are transferred to other systems. When decompressing, add .suf to the beginning of the list of suffixes to try, when deriving an output file name from an input file name. | ||||||||||||||
| -t, —test | Test. Check the compressed file integrity. | ||||||||||||||
| -v, —verbose | Verbose. Display the name and percentage reduction for each file compressed or decompressed. | ||||||||||||||
| -V, —version | Version. Display the version number and compilation options then quit. | ||||||||||||||
| —#, —fast, —best | Regulate the speed of compression using the specified digit #, where -1 or —fast indicates the fastest compression method (less compression) and -9 or —best indicates the slowest compression method (best compression). The default compression level is -6 (that is, biased towards high compression at expense of speed). |
DIAGNOSTICS
- Usage: gzip
- Invalid options were specified on the command line.
- file: not in gzip format
- The file specified to gunzip has not been compressed.
- file: Corrupt input. Use zcat to recover some data.
- The compressed file has been damaged. The data up to the point of failure can be recovered using
zcat file > recover
- file: compressed with xx bits, can only handle yy bits
- File was compressed (using LZW) by a program that could deal with more bits than the decompress code on this machine. Recompress the file with gzip, which compresses better and uses less memory.
- file: already has .gz suffix — no change
- The file is assumed to be already compressed. Rename the file and try again.
- file already exists; do you wish to overwrite (y or n)?
- Respond «y» if you want the output file to be replaced; «n» if not.
- gunzip: corrupt input
- A SIGSEGV violation was detected which usually means that the input file has been corrupted.
- xx.x% Percentage of the input saved by compression.
- (Relevant only for -v and -l.)
- — not a regular file or directory: ignored
- When the input file is not a regular file or directory, (e.g. a symbolic link, socket, FIFO, device file), it is left unaltered.
- — has xx other links: unchanged
- The input file has links; it is left unchanged. See ln(1) for more information. Use the -f flag to force compression of multiply-linked files.
Устранение неполадок при открытии файлов GZIP
Общие проблемы с открытием файлов GZIP
Gzip Compressed Archive не установлен
Дважды щелкнув по файлу GZIP вы можете увидеть системное диалоговое окно, в котором сообщается «Не удается открыть этот тип файла». В этом случае обычно это связано с тем, что на вашем компьютере не установлено Gzip Compressed Archive для %%os%%. Так как ваша операционная система не знает, что делать с этим файлом, вы не сможете открыть его дважды щелкнув на него.
Совет: Если вам извстна другая программа, которая может открыть файл GZIP, вы можете попробовать открыть данный файл, выбрав это приложение из списка возможных программ.
Установлена неправильная версия Gzip Compressed Archive
В некоторых случаях у вас может быть более новая (или более старая) версия файла GNU Zip Compressed Archive, не поддерживаемая установленной версией приложения. При отсутствии правильной версии ПО Gzip Compressed Archive (или любой из других программ, перечисленных выше), может потребоваться загрузить другую версию ПО или одного из других прикладных программных средств, перечисленных выше. Такая проблема чаще всего возникает при работе в более старой версии прикладного программного средства с файлом, созданным в более новой версии, который старая версия не может распознать.
Совет: Иногда вы можете получить общее представление о версии файла GZIP, щелкнув правой кнопкой мыши на файл, а затем выбрав «Свойства» (Windows) или «Получить информацию» (Mac OSX).
Резюме: В любом случае, большинство проблем, возникающих во время открытия файлов GZIP, связаны с отсутствием на вашем компьютере установленного правильного прикладного программного средства.
Даже если на вашем компьютере уже установлено Gzip Compressed Archive или другое программное обеспечение, связанное с GZIP, вы все равно можете столкнуться с проблемами во время открытия файлов GNU Zip Compressed Archive. Если проблемы открытия файлов GZIP до сих пор не устранены, возможно, причина кроется в других проблемах, не позволяющих открыть эти файлы. Такие проблемы включают (представлены в порядке от наиболее до наименее распространенных):
DESCRIPTION
Gzip.gz,-gzzGzip
If the compressed file name is too long for its file system, gzip truncates it. Gzip attempts to truncate only the parts of the file name longer than 3 characters. (A part is delimited by dots.) If the name consists of small parts only,
the longest parts are truncated. For example, if file names are limited to 14 characters, gzip.msdos.exe is compressed to gzi.msd.exe.gz. Names are not truncated on systems which do not have a limit on file name length.
By default, gzip keeps the original file name and timestamp in the compressed file. These are used when decompressing the file with the -N option. This is useful when the compressed file name was truncated or when the time
stamp was not preserved after a file transfer.
Compressed files can be restored to their original form using gzip -d or gunzip or zcat. If the original name saved in the compressed file is not suitable for its file system, a new name is constructed from the original one to
make it legal.
gunzip takes a list of files on its command line and replaces each file whose name ends with .gz, -gz, .z, -z, or _z (ignoring case) and which begins with the correct magic number with an uncompressed file without the original extension. gunzip also recognizes the special extensions .tgz and .taz as shorthands for .tar.gz and .tar.Z respectively. When compressing, gzip uses the .tgz extension if necessary instead of truncating a file with a .tar extension.
gunzip can currently decompress files created by gzip, zip, compress, compress -H or pack. The detection of the input format is automatic. When using the first two formats, gunzip checks a 32 bit CRC. For pack,
gunzip checks the uncompressed length. The standard compress format was not designed to allow consistency checks. However gunzip is sometimes able to detect a bad .Z file. If you get an error when uncompressing a .Z file, do not assume
that the .Z file is correct simply because the standard uncompress does not complain. This generally means that the standard uncompress does not check its input, and happily generates garbage output. The SCO compress -H format (lzh
compression method) does not include a CRC but also allows some consistency checks.
Files created by zip can be uncompressed by gzip only if they have a single member compressed with the ‘deflation’ method. This feature is only intended to help conversion of tar.zip files to the tar.gz format. To extract a zip file with a single member, use a command like gunzip <foo.zip or gunzip -S .zip foo.zip. To extract zip files with several members, use unzip instead of gunzip.
zcat is identical to gunzip -c. (On some systems, zcat may be installed as gzcat to preserve the original link to compress.) zcat uncompresses either a list of files on the command line or its
standard input and writes the uncompressed data on standard output. zcat will uncompress files that have the correct magic number whether they have a .gz suffix or not.
Gzip uses the Lempel-Ziv algorithm used in zip and PKZIP. The amount of compression obtained depends on the size of the input and the distribution of common substrings. Typically, text such as source code or English is reduced by 60-70%.
Compression is generally much better than that achieved by LZW (as used in compress), Huffman coding (as used in pack), or adaptive Huffman coding (compact).
Compression is always performed, even if the compressed file is slightly larger than the original. The worst case expansion is a few bytes for the gzip file header, plus 5 bytes every 32K block, or an expansion ratio of 0.015% for large files. Note
that the actual number of used disk blocks almost never increases. gzip preserves the mode, ownership and timestamps of files when compressing or decompressing.
Faster Compression and Decompression With pigz
gzip and gunzip, like most Linux tools, only use a single CPU core. So, compressing large files can take a while.
That’s why pigz exists, a “parallel implementation of gzip”. pigz takes advantage of both multiple CPUs and multiple CPU cores for higher compression and decompression speed. pigz is an anagram of gzip and is pronounced “pig-zee”. We can install it with either yum or apt.
pigz is compatible with gzip, and unpigz is compatible with gunzip. As such, pigz produces files that gunzip can decompress and uses the same options as gzip. Likewise, unpigz decompresses files that gzip created and also uses the same options as gunzip.
How much faster is pigz?
To find out, we ran a quick test on a modern computer with six CPU cores and hyperthreading. The test data was an 818 MB CSV file. We used the maximum compression level 9 with both gzip and pigz.
First, we compressed a file with pigz:
And then, we decompressed this file using unpigz:
- The fastest compression with gzip took 112 seconds and reduced the file size by 88.3%, down to 95 MB
- pigz took just 15 seconds to compress the same file and therefore was 7.5 times faster, and the compressed file was 0.3% smaller than the one gzip created
- Decompressing with gunzip took 3 seconds, while pigz took about a second to decompress the same file, merely about three times faster; note that measuring a duration of just a few seconds isn’t very precise in this setup
So, pigz/unpigz does indeed speed up compressing and decompressing files significantly with multiple CPUs or multiple CPU cores!
Включаем gzip в файле .htaccess
Иногда бывает и так. У вас на хостинге gzip-сжатие успешно работает. Но после проверки PageSpeed Insights выдает рекомендацию включить сжатие. Что делать? Все достаточно просто. Проверьте, не включено ли оно в файле .htaccess.
Не забывайте: неважно, какие изменения вы будете проводить в этом файле, обязательно сохраняйте до этого резервную копию. Как сделать все практически? Открываем для редактирования в корневом каталоге сайта на сервере файл .htaccess
В нем ищем определенные строки:
Как сделать все практически? Открываем для редактирования в корневом каталоге сайта на сервере файл .htaccess. В нем ищем определенные строки:
Если таковых нет, то сжатие, скорее всего, у вас просто не включено. Чтобы включить добавьте следующий код и сохраните файл.
Но если вам все же удалось отыскать строки, обязательно проверьте, совпадает ли в них код с тем, который мы навели выше. Если совпадений нет, то отсутствующие параметры следует добавить.
И только после этого повторно проверьте скорость сайта, используя PageSpeed Insights.
Сигналом того, что все сделано правильно, будет отсутствие предупреждения, что нужно включить сжатие – все очень просто. При этом оценка сайта обязательно повыситься – это еще один приятный бонус.
Преимущества и недостатки сжатия
Gzip – сжатие, которое осуществляется на стороне веб-сервера HTML-страниц, тех, которые получает браузер юзера. Такое сжатие никоим образом не отражается на юзабилити сайта, но благодаря снижению передаваемого трафика обладает очень хорошим влиянием.
Gzip помогает архивировать исключительно текстовую информацию. В то время как видео, изображения, аудио и прочие медиаресурсы не меняются. Во время сжатия архивируется текстовая информация.
В таком виде она отправляется на компьютер пользователя. Затем такие данные автоматически распаковываются браузером. Учитывая этот факт, можно сразу же обозначить ключевой недостаток технологии – повышенная нагрузка на сервер во время сжатия.
Если вы используете слабый хостинг, а посещаемость ресурса является очень высокой, такая нагрузка может быть критичной.
Если вы планируете использовать gzip, то обязательно понаблюдайте некоторое время за сайтом. Если не будет никаких сбоев, то можно продолжать дальше.
File format
| Filename extension |
.gz |
|---|---|
| Internet media type |
application/gzip |
| Uniform Type Identifier (UTI) | org.gnu.gnu-zip-archive |
| Developed by | Jean-loup Gailly and Mark Adler |
| Type of format | Data compression |
| Open format? | Yes |
| Website | (obsolete) |
gzip is based on the DEFLATE algorithm, which is a combination of LZ77 and Huffman coding. DEFLATE was intended as a replacement for LZW and other patent-encumbered data compression algorithms which, at the time, limited the usability of compress and other popular archivers.
«gzip» is often also used to refer to the gzip file format, which is:
- a 10-byte header, containing a magic number (), the compression method ( for DEFLATE), 1-byte of header flags, a 4-byte timestamp, compression flags and the operating system ID.
- optional extra headers as allowed by the header flags, including the original filename, a comment field, an «extra» field, and the lower half of a CRC-32 checksum for the header section.
- a body, containing a DEFLATE-compressed payload
- an 8-byte footer, containing a CRC-32 checksum and the length of the original uncompressed data, modulo 232.
Although its file format also allows for multiple such streams to be concatenated (gzipped files are simply decompressed concatenated as if they were originally one file), gzip is normally used to compress just single files. Compressed archives are typically created by assembling collections of files into a single tar archive (also called tarball), and then compressing that archive with gzip. The final compressed file usually has the extension .tar.gz or .tgz.
gzip is not to be confused with the ZIP archive format, which also uses DEFLATE. The ZIP format can hold collections of files without an external archiver, but is less compact than compressed tarballs holding the same data, because it compresses files individually and cannot take advantage of redundancy between files (solid compression).
