#Tar compress multithread archive#
Using pipe |, we could first archive multiple files or directories first to. We would need to rely on tar, the archive tool. However, vanilla pigz is not very friendly to compressing multiple files into one single file and custom output filepath. # Always use -k to keep the original file All arguments after "-" are treated as filesĪ typical command for compressing and decompressing a file is like the following: 1 z, -zlib Compress to zlib (.zz) instead of gzip format Y -synchronous Force output file write to permanent storage t, -test Test the integrity of the compressed input R, -rsyncable Input-determined block locations for rsync r, -recursive Process the contents of all subdirectories q, -quiet Print no messages, even on error Number of online processors, or 8 if unknown) p, -processes n Allow up to n compression threads (default is the O -oneblock Do not split into smaller blocks for -11
#Tar compress multithread mod#
N, -name Store or restore file name and mod time n, -no-name Do not store or restore file name or mod time m, -no-time Do not store or restore mod time L, -license Display the pigz license and quit l, -list List the contents of the compressed input K, -zip Compress to PKWare zip (.zip) single entry format k, -keep Do not delete original file after processing J, -maxsplits n Maximum number of split blocks for -11 I, -iterations n Number of iterations for -11 optimization i, -independent Compress blocks independently for damage recovery F -first Do iterations first, before block split for -11 d, -decompress Decompress the compressed input c, -stdout Write all processed output to stdout (won 't delete) b, -blocksize mmm Set compression block size to mmmK (default 128K) fast, -best Compression levels 1 and 9 respectively 0 to -9, -11 Compression level (level 11, zopfli, is much slower) pigz does what gzip does,īut spreads the work over multiple processors and cores when compressing. Specified, stdin will be compressed to stdout. Will compress files in place, adding the suffix '.gz'. However, its usages on other Linux operating systems should be almost the same. The pigz usages in the blog post are mainly targeted for Ubuntu systems.
#Tar compress multithread how to#
In this blog post, I would like to briefly discuss how to use pigz. Using pigz could greatly save us the time spent on compression and decompression. Pigz is one of the parallel implementation for gzip and zip. When the number of files are large or the file sizes are large, compression and decompression would take a lot of time using single thread. However, none of these tools on Linux uses multicore and multithread during compression and decompression. It is very common to use tools such as gzip, zip, or 7zip to create or decompress. Sometimes, we would like to compress one or several files into one zipped file or decompress a zipped file.
0 kommentar(er)
