diff options
| -rw-r--r-- | ACKNOWLEDGEMENTS | 7 | ||||
| -rw-r--r-- | CHANGES | 103 | ||||
| -rw-r--r-- | INSTALL | 6 | ||||
| -rw-r--r-- | RELEASE-README | 191 | ||||
| -rw-r--r-- | RELEASE-READMEs/DONATIONS | 4 | ||||
| -rw-r--r-- | RELEASE-READMEs/README-4.3 | 182 |
6 files changed, 427 insertions, 66 deletions
diff --git a/ACKNOWLEDGEMENTS b/ACKNOWLEDGEMENTS index e346f7a..2a80700 100644 --- a/ACKNOWLEDGEMENTS +++ b/ACKNOWLEDGEMENTS @@ -1,6 +1,6 @@ ACKNOWLEDGEMENTS -Thanks to everyone who have download squashfs. I appreciate people +Thanks to everyone who have downloaded Squashfs. I appreciate people using it, and any feedback you have. The following have provided useful feedback, which has guided @@ -8,6 +8,11 @@ some of the extra features in squashfs. This is a randomly ordered (roughly in chronological order) list, which is updated when I remember... +Acknowledgements for Squashfs 4.3 +--------------------------------- + +Thanks to Bruno Wolff III and Andy Lutomirski for useful feedback +during the long development process of Squashfs 4.3. Acknowledgements for Squashfs 4.2 --------------------------------- @@ -1,5 +1,108 @@ SQUASHFS CHANGE LOG +4.3 12 MAY 2014 New compressor options, new Mksquashfs/Unsquashfs + functionality, duplicate checking optimisations, + stability improvements (option/file parsing, + buffer/memory overflow checks, filesystem hardening + on corrupted filesystems), CVE fixes. + + Too many changes to do the traditional custom changelog. But, this + is now unnecessary, so instead list most significant 15% of commits + from git changelog in chronological order. + + - unsquashfs: add checks for corrupted data in opendir functions + - unsquashfs: completely empty filesystems incorrectly generate an error + - unsquashfs: fix open file limit + - mksquashfs: Use linked list to store directory entries rather + - mksquashfs: Remove qsort and add a bottom up linked list merge sort + - mksquashfs: optimise lookup_inode2() for dirs + - pseudo: fix handling of modify pseudo files + - pseudo: fix handling of directory pseudo files + - xattr: Fix ERROR() so that it is synchronised with the progress bar + - mksquashfs/sort: Fix INFO() so that it is synced with the progress bar + - mksquashfs: Add -progress to force progress bar when using -info + - error.h: consolidate the various error macros into one header file + - mksquashfs: fix stack overflow in write_fragment_table() + - mksquashfs: move list allocation from off the stack + - unsquashfs: fix oversight in directory permission setting + - mksquashfs: dynamically allocate recovery_file + - mksquashfs: dynamically allocate buffer in subpathname() + - mksquashfs: dynamically allocate buffer in pathname() + - unsquashfs: fix CVE-2012-4024 + - unsquashfs: fix CVE-2012-4025 + - mksquashfs: fix potential stack overflow in get_component() + - mksquashfs: add parse_number() helper for numeric command line options + - mksquasfs: check return value of fstat() in reader_read_file() + - mksquashfs: dynamically allocate filename in old_add_exclude() + - unsquashfs: dynamically allocate pathname in dir_scan() + - unsquashfs: dynamically allocate pathname in pre_scan() + - sort: dynamically allocate filename in add_sort_list() + - mksquashfs: fix dir_scan() exit if lstat of source directory fails + - pseudo: fix memory leak in read_pseudo_def() if exec_file() fails + - pseudo: dynamically allocate path in dump_pseudo() + - mksquashfs: dynamically allocate path in display_path2() + - mksquashfs: dynamically allocate b_buffer in getbase() + - pseudo: fix potential stack overflow in get_component() + - pseudo: avoid buffer overflow in read_pseudo_def() using sscanf() + - pseudo: dynamically allocate filename in exec_file() + - pseudo: avoid buffer overflow in read_sort_file() using fscanf() + - sort: tighten up sort file parsing + - unsquashfs: fix name under-allocation in process_extract_files() + - unsquashfs: avoid buffer overflow in print_filename() using sprintf() + - Fix some limits in the file parsing routines + - pseudo: Rewrite pseudo file processing + - read_fs: fix small memory leaks in read_filesystem() + - mksquashfs: fix fclose leak in reader_read_file() on I/O error + - mksquashfs: fix frag struct leak in write_file_{process|blocks|frag} + - unsquashfs_xattr: fix memory leak in write_xattr() + - read_xattrs: fix xattr free in get_xattr() in error path + - unsquashfs: add -user-xattrs option to only extract user.xxx xattrs + - unsquashfs: add code to only print "not superuser" error message once + - unsquashfs: check for integer overflow in user input + - mksquashfs: check for integer overflow in user input + - mksquashfs: fix "new" variable leak in dir_scan1() + - read_fs: prevent buffer {over|under}flow in read_block() with + corrupted filesystems + - read_fs: check metadata blocks are expected size in scan_inode_table() + - read_fs: check the root inode block is found in scan_inode_table() + - read_fs: Further harden scan_inode_table() against corrupted + filesystems + - unsquashfs: prevent buffer {over|under}flow in read_block() with + corrupted filesystems + - read_xattrs: harden xattr data reading against corrupted filesystems + - unsquash-[23]: harden frag table reading against corrupted filesystems + - unsquash-4.c: harden uid/gid & frag table reading against corruption + - unsquashfs: harden inode/directory table reading against corruption + - mksquashfs: improve out of space in output filesystem handling + - mksquashfs: flag lseek error in writer as probable out of space + - mksquashfs: flag lseek error in write_destination as probable out of + space + - mksquashfs: print file being squashed when ^\ (SIGQUIT) typed + - mksquashfs: make EXIT_MKSQUASHFS() etc restore via new restore thread + - mksquashfs: fix recursive restore failure check + - info: dump queue and cache status if ^\ hit twice within one second + - mksquashfs: fix rare race condition in "locked fragment" queueing + - lz4: add experimental support for lz4 compression + - lz4: add support for lz4 "high compression" + - lzo_wrapper: new implementation with compression options + - gzip_wrapper: add compression options + - mksquashfs: redo -comp <compressor> parsing + - mksquashfs: display compressor options when -X option isn't recognised + - mksquashfs: add -Xhelp option + - mksquashfs/unsquashfs: fix mtime signedness + - Mksquashfs: optimise duplicate checking when appending + - Mksquashfs: introduce additional per CPU fragment process threads + - Mksquashfs: significantly optimise fragment duplicate checking + - read_fs: scan_inode_table(), fix memory leak on filesystem corruption + - pseudo: add_pseudo(), fix use of freed variable + - mksquashfs/unsquashfs: exclude/extract/pseudo files, fix handling of + leaf name + - mksquashfs: rewrite default queue size so it's based on physical mem + - mksquashfs: add a new -mem <mbytes> option + - mksquashfs: fix limit on the number of dynamic pseudo files + - mksquashfs: make -mem take a normal byte value, optionally with a + K, M or G + 4.2 28 FEB 2011 XZ compression, and compression options support 1. Filesystem improvements: @@ -1,6 +1,6 @@ INSTALLING SQUASHFS -The squashfs4.2.tar.gz file contains the squashfs-tools directory containing +The squashfs4.3.tar.gz file contains the squashfs-tools directory containing mksquashfs and unsquashfs. 1. Kernel support @@ -16,6 +16,8 @@ LZO compression support requires 2.6.36 or newer kernels. XZ compression support requires 2.6.38 or newer kernels. +LZ4 support is not yet in any mainline kernel. + 2. Building squashfs tools -------------------------- @@ -24,5 +26,5 @@ These can be made by typing make (or make install to install in /usr/local/bin). By default the tools are built with GZIP compression and extended attribute support. Read the Makefile in squashfs-tools/ for instructions on building -LZO and XZ compression support, and for instructions on disabling GZIP +LZO, LZ4 and XZ compression support, and for instructions on disabling GZIP and extended attribute support if desired. diff --git a/RELEASE-README b/RELEASE-README index 351f61d..a033a4b 100644 --- a/RELEASE-README +++ b/RELEASE-README @@ -1,17 +1,17 @@ - SQUASHFS 4.2 - A squashed read-only filesystem for Linux + SQUASHFS 4.3 - A squashed read-only filesystem for Linux - Copyright 2002-2011 Phillip Lougher <phillip@lougher.demon.co.uk> + Copyright 2002-2014 Phillip Lougher <phillip@lougher.demon.co.uk> Released under the GPL licence (version 2 or later). -Welcome to Squashfs version 4.2. Please read the README-4.2 and CHANGES files +Welcome to Squashfs version 4.3. Please read the README-4.3 and CHANGES files for details of changes. Squashfs is a highly compressed read-only filesystem for Linux. -It uses zlib compression to compress both files, inodes and directories. -Inodes in the system are very small and all blocks are packed to minimise -data overhead. Block sizes greater than 4K are supported up to a maximum -of 1Mbytes (default block size 128K). +It uses either gzip/xz/lzo/lz4 compression to compress both files, inodes +and directories. Inodes in the system are very small and all blocks are +packed to minimise data overhead. Block sizes greater than 4K are supported +up to a maximum of 1Mbytes (default block size 128K). Squashfs is intended for general read-only filesystem use, for archival use (i.e. in cases where a .tar.gz file may be used), and in constrained @@ -39,6 +39,9 @@ needed. 6. File duplicates are detected and removed. +7. Filesystems can be compressed with gzip, xz (lzma2), lzo or lz4 + compression algorithms. + 1.1 Extended attributes (xattrs) -------------------------------- @@ -84,12 +87,11 @@ can be used to mount it (loopback support must be in the kernel), e.g. Will mount the squashfs filesystem in the file "image" under the directory "/mnt". - 3. MKSQUASHFS ------------- -3.1 Mksquashfs options and overview. ------------------------------------- +3.1 Mksquashfs options and overview +----------------------------------- As squashfs is a read-only filesystem, the mksquashfs program must be used to create populated squashfs filesystems. @@ -102,8 +104,11 @@ Filesystem build options: Compressors available: gzip (default) lzo + lz4 xz --b <block_size> set data block to <block_size>. Default 131072 bytes +-b <block_size> set data block to <block_size>. Default 128 Kbytes + Optionally a suffix of K or M can be given to specify + Kbytes or Mbytes respectively -no-exports don't make the filesystem exportable via NFS -no-sparse don't detect sparse files -no-xattrs don't store extended attributes @@ -144,15 +149,17 @@ Filesystem append options: Mksquashfs runtime options: -version print version, licence and copyright message +-exit-on-error treat normally ignored errors as fatal -recover <name> recover filesystem data using recovery file <name> -no-recovery don't generate a recovery file -info print files written to filesystem -no-progress don't display the progress bar +-progress display progress bar when using the -info option -processors <number> Use <number> processors. By default will use number of processors available --read-queue <size> Set input queue to <size> Mbytes. Default 64 Mbytes --write-queue <size> Set output queue to <size> Mbytes. Default 512 Mbytes --fragment-queue <size> Set fragment queue to <size> Mbytes. Default 64 Mbytes +-mem <size> Use <size> physical memory. Currently set to 1922M + Optionally a suffix of K, M or G can be given to specify + Kbytes, Mbytes or Gbytes respectively Miscellaneous options: -root-owned alternative name for -all-root @@ -161,9 +168,33 @@ Miscellaneous options: -noFragmentCompression alternative name for -noF -noXattrCompression alternative name for -noX +-Xhelp print compressor options for selected compressor + Compressors available and compressor specific options: - gzip (no options) (default) - lzo (no options) + gzip (default) + -Xcompression-level <compression-level> + <compression-level> should be 1 .. 9 (default 9) + -Xwindow-size <window-size> + <window-size> should be 8 .. 15 (default 15) + -Xstrategy strategy1,strategy2,...,strategyN + Compress using strategy1,strategy2,...,strategyN in turn + and choose the best compression. + Available strategies: default, filtered, huffman_only, + run_length_encoded and fixed + lzo + -Xalgorithm <algorithm> + Where <algorithm> is one of: + lzo1x_1 + lzo1x_1_11 + lzo1x_1_12 + lzo1x_1_15 + lzo1x_999 (default) + -Xcompression-level <compression-level> + <compression-level> should be 1 .. 9 (default 8) + Only applies to lzo1x_999 algorithm + lz4 + -Xhc + Compress using LZ4 High Compression xz -Xbcj filter1,filter2,...,filterN Compress using filter1,filter2,...,filterN in turn @@ -178,7 +209,6 @@ Compressors available and compressor specific options: Example dict-sizes are 75%, 50%, 37.5%, 25%, or 32K, 16K, 8K etc. - Source1 source2 ... are the source directories/files containing the files/directories that will form the squashfs filesystem. If a single directory is specified (i.e. mksquashfs source output_fs) the squashfs @@ -249,37 +279,70 @@ By default Mksquashfs will compress using the gzip compression algorithm. This algorithm offers a good trade-off between compression ratio, and memory and time taken to decompress. -Squashfs also supports LZO and XZ (LZMA2) compression. LZO offers worse +Squashfs also supports LZ4, LZO and XZ (LZMA2) compression. LZO offers worse compression ratio than gzip, but is faster to decompress. XZ offers better compression ratio than gzip, but at the expense of greater memory and time -to decompress (and significantly more time to compress). +to decompress (and significantly more time to compress). LZ4 is similar +to LZO, but, support for it is not yet in the mainline kernel, and so +its usefulness is currently limited to using Squashfs with Mksquashfs/Unsquashfs +as an archival system like tar. If you're not building the squashfs-tools and kernel from source, then -the tools and kernel may or may not have been built with support for LZO or +the tools and kernel may or may not have been built with support for LZ4, LZO or XZ compression. The compression algorithms supported by the build of Mksquashfs can be found by typing mksquashfs without any arguments. The compressors available are displayed at the end of the help message, e.g. Compressors available and compressor specific options: - gzip (no options) (default) - lzo (no options) - xz - -Xbcj filter1,filter2,...,filterN - Compress using filter1,filter2,...,filterN in turn - (in addition to no filter), and choose the best compression. - Available filters: x86, arm, armthumb, powerpc, sparc, ia64 - -Xdict-size <dict-size> - Use <dict-size> as the XZ dictionary size. The dictionary size - can be specified as a percentage of the block size, or as an - absolute value. The dictionary size must be less than or equal - to the block size and 8192 bytes or larger. It must also be - storable in the xz header as either 2^n or as 2^n+2^(n+1). - Example dict-sizes are 75%, 50%, 37.5%, 25%, or 32K, 16K, 8K - etc. - -If the compressor offers compression specific options then these -options are also displayed (.i.e. in the above XZ is shown with two -compression specific options). + gzip (default) + -Xcompression-level <compression-level> + <compression-level> should be 1 .. 9 (default 9) + -Xwindow-size <window-size> + <window-size> should be 8 .. 15 (default 15) + -Xstrategy strategy1,strategy2,...,strategyN + Compress using strategy1,strategy2,...,strategyN in turn + and choose the best compression. + Available strategies: default, filtered, huffman_only, + run_length_encoded and fixed + lzo + -Xalgorithm <algorithm> + Where <algorithm> is one of: + lzo1x_1 + lzo1x_1_11 + lzo1x_1_12 + lzo1x_1_15 + lzo1x_999 (default) + -Xcompression-level <compression-level> + <compression-level> should be 1 .. 9 (default 8) + Only applies to lzo1x_999 algorithm + lz4 + -Xhc + Compress using LZ4 High Compression + xz + -Xbcj filter1,filter2,...,filterN + Compress using filter1,filter2,...,filterN in turn + (in addition to no filter), and choose the best compression. + Available filters: x86, arm, armthumb, powerpc, sparc, ia64 + -Xdict-size <dict-size> + Use <dict-size> as the XZ dictionary size. The dictionary size + can be specified as a percentage of the block size, or as an + absolute value. The dictionary size must be less than or equal + to the block size and 8192 bytes or larger. It must also be + storable in the xz header as either 2^n or as 2^n+2^(n+1). + Example dict-sizes are 75%, 50%, 37.5%, 25%, or 32K, 16K, 8K + etc. + +If the compressor offers compression specific options (all the compressors now +have compression specific options except the deprecated lzma1 compressor) +then these options are also displayed (.i.e. in the above XZ is shown with two +compression specific options). The compression specific options are, obviously, +specific to the compressor in question, and the compressor documentation and +web sites should be consulted to understand their behaviour. In general +the Mksquashfs compression defaults for each compressor are optimised to +give the best performance for each compressor, where what constitutes +best depends on the compressor. For gzip/xz best means highest compression, +for LZO/LZ4 best means a tradeoff between compression and (de)-compression +overhead (LZO/LZ4 by definition are intended for weaker processors). 3.3 Changing global compression defaults used in mksquashfs ----------------------------------------------------------- @@ -287,6 +350,14 @@ compression specific options). There are a large number of options that can be used to control the compression in mksquashfs. By and large the defaults are the most optimum settings and should only be changed in exceptional circumstances! +Note, this does not apply to the block size, increasing the block size +from the default of 128Kbytes will increase compression (especially +for the xz compressor) and should increase I/O performance too. However, +a block size of greater than 128Kbytes may increase latency in certain +cases (where the filesystem contains lots of fragments, and no locality +of reference is observed). For this reason the block size default is +configured to the less optimal 128Kbytes. Users should experiment +with 256Kbyte sizes or above. The -noI, -noD and -noF options (also -noInodeCompression, -noDataCompression and -noFragmentCompression) can be used to force mksquashfs to not compress @@ -295,7 +366,7 @@ generates an uncompressed filesystem. The -no-fragments tells mksquashfs to not generate fragment blocks, and rather generate a filesystem similar to a Squashfs 1.x filesystem. It will of course -still be a Squashfs 3.1 filesystem but without fragments, and so it won't be +still be a Squashfs 4.0 filesystem but without fragments, and so it won't be mountable on a Squashfs 1.x system. The -always-use-fragments option tells mksquashfs to always generate @@ -353,6 +424,7 @@ retained for backwards compatibility with earlier versions of Mksquashfs. Extended wildcard matching should be used in preference. 3.5.1 Basic exclude matching +---------------------------- Each exclude file is treated as an exact match of a file/directory in the source directories. If an exclude file/directory is absolute (i.e. @@ -366,6 +438,7 @@ Will generate exclude files /tmp/source1/ex2, /tmp/source1/ex1, source2/ex1, /tmp/source1/out/ex3 and source2/out/ex3. 3.5.2 Extended exclude file handling +------------------------------------ Extended exclude file matching treats each exclude file as a wildcard or regex expression. To enable wildcard matching specify the -wildcards @@ -429,6 +502,7 @@ Non-anchored excludes except those with "data" in the name. 3.5.3 Exclude files summary +--------------------------- The -e and -ef exclude options are usefully used in archiving the entire filesystem, where it is wished to avoid archiving /proc, and the filesystem @@ -513,7 +587,7 @@ built, rather than requiring them to be present in the source directories. This, for example, allows device nodes to be added to the filesystem without requiring root access. -Mksquashfs 4.1 adds support for "dynamic pseudo files" and a modify operation. +Mksquashfs 4.1 added support for "dynamic pseudo files" and a modify operation. Dynamic pseudo files allow files to be dynamically created when Mksquashfs is run, their contents being the result of running a command or piece of shell script. The modifiy operation allows the mode/uid/gid of an existing @@ -523,11 +597,8 @@ Two Mksquashfs options are supported, -p allows one pseudo file to be specified on the command line, and -pf allows a pseudo file to be specified containing a list of pseduo definitions, one per line. -Pseudo operations ------------------ - 3.8.1. Creating a dynamic file --------------------------- +------------------------------ Pseudo definition @@ -584,7 +655,7 @@ Squashfs filesystem, the above allows input from these special files to be captured and placed in the Squashfs filesystem. 3.8.2. Creating a block or character device ---------------------------------------- +------------------------------------------- Pseudo definition @@ -608,7 +679,7 @@ a block device "/dev/blk_dev" with major:minor 200:200, both with root uid/gid and a mode of rw-rw-rw. 3.8.3. Creating a directory ------------------------ +--------------------------- Pseudo definition @@ -624,8 +695,8 @@ For example: creates a directory "/pseudo_dir" with root uid/gid and mode of rw-rw-rw. -4.8.4. Modifying attributes of an existing file -------------------------------------------- +3.8.4. Modifying attributes of an existing file +----------------------------------------------- Pseudo definition @@ -666,12 +737,14 @@ file or directory. The Unsquashfs usage info is: -SYNTAX: unsquashfs [options] filesystem [directories or files to extract] +SYNTAX: ./unsquashfs [options] filesystem [directories or files to extract] -v[ersion] print version, licence and copyright information -d[est] <pathname> unsquash to <pathname>, default "squashfs-root" -n[o-progress] don't display the progress bar -no[-xattrs] don't extract xattrs in file system -x[attrs] extract xattrs in file system (default) + -u[ser-xattrs] only extract user xattrs in file system. + Enables extracting xattrs -p[rocessors] <number> use <number> processors. By default will use number of processors available -i[nfo] print files as they are unsquashed @@ -686,17 +759,17 @@ SYNTAX: unsquashfs [options] filesystem [directories or files to extract] One per line -da[ta-queue] <size> Set data queue to <size> Mbytes. Default 256 Mbytes - -fr[ag-queue] <size> Set fagment queue to <size> Mbytes. Default 256 - Mbytes + -fr[ag-queue] <size> Set fragment queue to <size> Mbytes. Default + 256 Mbytes -r[egex] treat extract names as POSIX regular expressions rather than use the default shell wildcard expansion (globbing) Decompressors available: gzip - lzma - xz - + lzo + lz4 + xz To extract a subset of the filesystem, the filenames or directory trees that are to be extracted can be specified on the command line. The @@ -755,8 +828,8 @@ The "-stat" option displays filesystem superblock information. This is useful to discover the filesystem version, byte ordering, whether it has a NFS export table, and what options were used to compress the filesystem, etc. -Unsquashfs can decompress all Squashfs filesystem versions, 1.x, 2.x and 3.x -filesystems. +Unsquashfs can decompress all Squashfs filesystem versions, 1.x, 2.x, 3.x and +4.0 filesystems. 5. FILESYSTEM LAYOUT -------------------- @@ -793,8 +866,8 @@ byte alignment: Compressed data blocks are written to the filesystem as files are read from the source directory, and checked for duplicates. Once all file data has been -written the completed inode, directory, fragment, export and uid/gid lookup -tables are written. +written the completed super-block, compression options, inode, directory, +fragment, export, uid/gid lookup and xattr tables are written. 5.1 Compression options ----------------------- diff --git a/RELEASE-READMEs/DONATIONS b/RELEASE-READMEs/DONATIONS index 9df2f09..b4653bb 100644 --- a/RELEASE-READMEs/DONATIONS +++ b/RELEASE-READMEs/DONATIONS @@ -6,10 +6,6 @@ development is funded soley by the author, partially supported by donations from companies and individuals that want to improve Squashfs for themselves and others. -Mainlining of Squashfs only became possible when CELF (Consumer Electronics -Linux Forum) offered to contribute to the costs, which allowed the author -to work full-time on the project. - There's lots of exciting new improvements to Squashfs in the pipeline, and if your company is a serious user of Squashfs, please consider accelerating development of Squashfs by donating. diff --git a/RELEASE-READMEs/README-4.3 b/RELEASE-READMEs/README-4.3 new file mode 100644 index 0000000..d2370a0 --- /dev/null +++ b/RELEASE-READMEs/README-4.3 @@ -0,0 +1,182 @@ + SQUASHFS 4.3 - A squashed read-only filesystem for Linux + + Copyright 2002-2014 Phillip Lougher <phillip@lougher.demon.co.uk> + + Released under the GPL licence (version 2 or later). + +Welcome to Squashfs 4.3. This is the first release in over 3 years, and +there are substantial improvements to stability, new compression options +and compressors, speed optimisations, and new options for Mksquashfs/Unsquashfs. + +This is a tools only release, support for Squashfs filesystems is +in mainline (2.6.29 and later). + +Changes in Squashfs-tools 4.3 +----------------------------- + +1. Stability improvements. Better checking of user input for out of + range/invalid values. Better handling of corrupted Squashfs filesystems + (Mksquashfs append mode, and Unsquashfs). Better handling of buffer + overflow/underflow. + +2. GZIP compressor now supports compression options, allowing different + compression levels to be used. + +3. Rewritten LZO compressor with compression options, allowing different + LZO algorithms and different compression levels to be used. + +4. New LZ4 compressor (note not yet in mainline kernel) + +5. Better default memory usage for Mksquashfs. Mksquashfs by default now + uses 25% of physical memory. + +6. Duplicate checking in Mksquashfs further optimised. With certain + "problem filesystems" greater than 2x performance improvement. + Filesystems with a lot of duplicates should see at least 10-20% speed + improvement. + +7. The -stat option in Unsquashfs now displays the compression options + used to generate the original filesystem. Previously -stat only displayed + the compression algorithm used. + +8. The file being compressed/uncompressed in Mksquashfs/Unsquashfs is now + displayed if CTRL-\ (SIGQUIT from keyboard) typed. + +9. The status of the internal queues/caches in Mksquashfs/Unsquashfs is + now displayed if CTRL-\ (SIGQUIT from keyboard) is typed twice within + one second. Normally only useful for "power users", but it can be + used to discover if there's any bottlenecks affecting performance + (the bottleneck will normally be the compressors/fragment compressors). + +10. Miscellaneous new options for Mksquashfs/Unsquashfs to fine tune behaviour. + +11. Fixes for CVE-2012-4024 and CVE-2012-4025. + +Compatiblity +------------ + +Mksquashfs 4.3 generates 4.0 filesystems. These filesystems are fully +compatible/interchangable with filesystems generated by Mksquashfs 4.0 and are +mountable on 2.6.29 and later kernels. + +Compressors +----------- + +New compression options and compressors are now supported. + +The new options and compressors are: + +1. gzip + -Xcompression-level <compression-level> + <compression-level> should be 1 .. 9 (default 9) + -Xwindow-size <window-size> + <window-size> should be 8 .. 15 (default 15) + -Xstrategy strategy1,strategy2,...,strategyN + Compress using strategy1,strategy2,...,strategyN in turn + and choose the best compression. + Available strategies: default, filtered, huffman_only, + run_length_encoded and fixed + +2. lzo + -Xalgorithm <algorithm> + Where <algorithm> is one of: + lzo1x_1 + lzo1x_1_11 + lzo1x_1_12 + lzo1x_1_15 + lzo1x_999 (default) + -Xcompression-level <compression-level> + <compression-level> should be 1 .. 9 (default 8) + Only applies to lzo1x_999 algorithm + +3. lz4 + -Xhc + Compress using LZ4 High Compression + +The compression specific options are, obviously, specific to the compressor +in question, and you should read the compressor documentation and check +their web sites to understand their behaviour. + +In general the defaults used by Mksquashfs for each compressor are optimised +to give the best performance for each compressor, where what constitutes +best depends on the compressor. For gzip/xz best means highest compression +(trying multiple filters/strategies can improve compression, but this is +extremely expensive computationally, and hence, not suitable for the defaults), +for LZO/LZ4 best means a tradeoff between compression and (de)-compression +overhead (LZO/LZ4 by definition are intended for weaker processors). + +New Mksquashfs options +---------------------- + +1. -mem <size> + + Set the amount of memory used by Mksquashfs to <size> bytes. G/M and K + post-fixes are supported. + + By default Mksquashfs uses 25% of the physical memory. Increasing + this with the -mem option can increase performance (note it does not have + any effect on compression). Reducing it can prevent thrashing if the + system is busy and there is not 25% of physical memory free (again, note + it does not have any effect on compression). + +2. -exit-on-error + + By default Mksquashfs treats certain errors as benign, if these + errors occur Mksquashfs prints the error on the console but continues. + These errors are typically failure to read a file from the source filesystem. + This is deliberate, in many cases users prefer Mksquashfs to flag + the error but continue rather than abort what may be hours of compression. + + But there are times where failure to read any file is considered critical, + and users (especially in the case of automated scripts where the + errors output to the console may be missed) prefer Mksquashfs to exit. + + The new -exit-on-error option can be used in this scenario. This option + makes Mksquashfs treat all benign errors as fatal. + +3. -progress + + By default if -info is specified, the progress bar is disabled as it gets + in the way. Occasionally you might want the progress bar enabled whilst + -info is enabled. This option forces Mksquashfs to output the progress + bar when -info is specified. + +4. -Xhelp + + Display the usage text for the currently selected compressor. + +New Unsquashfs options +---------------------- + +1. -u[ser-xattrs] + + Only write user xattrs. This forces Unsquashfs to ignore system xattrs. + This is useful when Unsquashing a filesystem as a non-root user, and the + filesystem contains system xattrs which are only writable by root. + +Major bugs fixed +---------------- + +1. If Mksquashfs ran out of space in the destination filesystem, this + would not cause Mksquashfs to immediately abort, and Mksquashfs would + continue to process the source filesystem. Mksquashfs now immediately + aborts on out of space in the destination filesystem. + +2. Unsquashfs ignored the maximum number of open files limit, and if that + was lower than the default limit for Linux, it would run out of file + descriptors. Unsquashfs now limits the number of open files to the + limit currently in force (e.g. specified by setrlimit). + +3. If huge numbers of dynamic pseudo files were specified, Mksquashfs + could exceed the maximum number of open files limit. This was because + Mksquashfs created all the dynamic file processes up front before + commencing source filesystem reading and compression. Mksquashfs + now creates the dynamic file processes on demand whilst reading + and compressing the source filesystem, thus limiting the number of + dynamic pseudo file processes in existence at any one time. + +4. When outputting Unsquashfs used to set the permissions of directories + as it recursively descended. This in hindsight had an obvious oversight, + if a directory had only read permission (or was otherwise restricted), then + Unsquashfs would fail to write its contents when descending into it. Fixed + by setting directory permissions as Unsquashfs recursively unwinds. |