From 33d134030248633ffa7d60c0a35a783c46da034b Mon Sep 17 00:00:00 2001 From: Julian Seward Date: Thu, 7 Aug 1997 22:13:13 +0200 Subject: bzip2-0.1 --- ALGORITHMS | 47 + LICENSE | 339 +++++ Makefile | 30 + README | 243 +++ README.DOS | 20 + bzip2.1 | 441 ++++++ bzip2.1.preformatted | 462 ++++++ bzip2.c | 4036 ++++++++++++++++++++++++++++++++++++++++++++++++++ bzip2.exe | Bin 0 -> 45716 bytes bzip2.txt | 462 ++++++ bzip2recover.c | 399 +++++ sample1.bz2 | Bin 0 -> 32348 bytes sample1.ref | Bin 0 -> 98696 bytes sample2.bz2 | Bin 0 -> 73732 bytes sample2.ref | Bin 0 -> 212340 bytes test.bat | 9 + test.cmd | 9 + words0 | 7 + words1 | 5 + words2 | 6 + words3 | 23 + words3sh | 12 + 22 files changed, 6550 insertions(+) create mode 100644 ALGORITHMS create mode 100644 LICENSE create mode 100644 Makefile create mode 100644 README create mode 100644 README.DOS create mode 100644 bzip2.1 create mode 100644 bzip2.1.preformatted create mode 100644 bzip2.c create mode 100644 bzip2.exe create mode 100644 bzip2.txt create mode 100644 bzip2recover.c create mode 100644 sample1.bz2 create mode 100644 sample1.ref create mode 100644 sample2.bz2 create mode 100644 sample2.ref create mode 100644 test.bat create mode 100644 test.cmd create mode 100644 words0 create mode 100644 words1 create mode 100644 words2 create mode 100644 words3 create mode 100644 words3sh diff --git a/ALGORITHMS b/ALGORITHMS new file mode 100644 index 0000000..7c7d2ca --- /dev/null +++ b/ALGORITHMS @@ -0,0 +1,47 @@ + +Bzip2 is not research work, in the sense that it doesn't present any +new ideas. Rather, it's an engineering exercise based on existing +ideas. + +Four documents describe essentially all the ideas behind bzip2: + + Michael Burrows and D. J. Wheeler: + "A block-sorting lossless data compression algorithm" + 10th May 1994. + Digital SRC Research Report 124. + ftp://ftp.digital.com/pub/DEC/SRC/research-reports/SRC-124.ps.gz + + Daniel S. Hirschberg and Debra A. LeLewer + "Efficient Decoding of Prefix Codes" + Communications of the ACM, April 1990, Vol 33, Number 4. + You might be able to get an electronic copy of this + from the ACM Digital Library. + + David J. Wheeler + Program bred3.c and accompanying document bred3.ps. + This contains the idea behind the multi-table Huffman + coding scheme. + ftp://ftp.cl.cam.ac.uk/pub/user/djw3/ + + Jon L. Bentley and Robert Sedgewick + "Fast Algorithms for Sorting and Searching Strings" + Available from Sedgewick's web page, + www.cs.princeton.edu/~rs + +The following paper gives valuable additional insights into the +algorithm, but is not immediately the basis of any code +used in bzip2. + + Peter Fenwick: + Block Sorting Text Compression + Proceedings of the 19th Australasian Computer Science Conference, + Melbourne, Australia. Jan 31 - Feb 2, 1996. + ftp://ftp.cs.auckland.ac.nz/pub/peter-f/ACSC96paper.ps + +All three are well written, and make fascinating reading. If you want +to modify bzip2 in any non-trivial way, I strongly suggest you obtain, +read and understand these papers. + +I am much indebted to the various authors for their help, support and +advice. + diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..a43ea21 --- /dev/null +++ b/LICENSE @@ -0,0 +1,339 @@ + GNU GENERAL PUBLIC LICENSE + Version 2, June 1991 + + Copyright (C) 1989, 1991 Free Software Foundation, Inc. + 675 Mass Ave, Cambridge, MA 02139, USA + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The licenses for most software are designed to take away your +freedom to share and change it. By contrast, the GNU General Public +License is intended to guarantee your freedom to share and change free +software--to make sure the software is free for all its users. This +General Public License applies to most of the Free Software +Foundation's software and to any other program whose authors commit to +using it. (Some other Free Software Foundation software is covered by +the GNU Library General Public License instead.) You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +this service if you wish), that you receive source code or can get it +if you want it, that you can change the software or use pieces of it +in new free programs; and that you know you can do these things. + + To protect your rights, we need to make restrictions that forbid +anyone to deny you these rights or to ask you to surrender the rights. +These restrictions translate to certain responsibilities for you if you +distribute copies of the software, or if you modify it. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must give the recipients all the rights that +you have. You must make sure that they, too, receive or can get the +source code. And you must show them these terms so they know their +rights. + + We protect your rights with two steps: (1) copyright the software, and +(2) offer you this license which gives you legal permission to copy, +distribute and/or modify the software. + + Also, for each author's protection and ours, we want to make certain +that everyone understands that there is no warranty for this free +software. If the software is modified by someone else and passed on, we +want its recipients to know that what they have is not the original, so +that any problems introduced by others will not reflect on the original +authors' reputations. + + Finally, any free program is threatened constantly by software +patents. We wish to avoid the danger that redistributors of a free +program will individually obtain patent licenses, in effect making the +program proprietary. To prevent this, we have made it clear that any +patent must be licensed for everyone's free use or not licensed at all. + + The precise terms and conditions for copying, distribution and +modification follow. + + GNU GENERAL PUBLIC LICENSE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + + 0. This License applies to any program or other work which contains +a notice placed by the copyright holder saying it may be distributed +under the terms of this General Public License. The "Program", below, +refers to any such program or work, and a "work based on the Program" +means either the Program or any derivative work under copyright law: +that is to say, a work containing the Program or a portion of it, +either verbatim or with modifications and/or translated into another +language. (Hereinafter, translation is included without limitation in +the term "modification".) Each licensee is addressed as "you". + +Activities other than copying, distribution and modification are not +covered by this License; they are outside its scope. The act of +running the Program is not restricted, and the output from the Program +is covered only if its contents constitute a work based on the +Program (independent of having been made by running the Program). +Whether that is true depends on what the Program does. + + 1. You may copy and distribute verbatim copies of the Program's +source code as you receive it, in any medium, provided that you +conspicuously and appropriately publish on each copy an appropriate +copyright notice and disclaimer of warranty; keep intact all the +notices that refer to this License and to the absence of any warranty; +and give any other recipients of the Program a copy of this License +along with the Program. + +You may charge a fee for the physical act of transferring a copy, and +you may at your option offer warranty protection in exchange for a fee. + + 2. You may modify your copy or copies of the Program or any portion +of it, thus forming a work based on the Program, and copy and +distribute such modifications or work under the terms of Section 1 +above, provided that you also meet all of these conditions: + + a) You must cause the modified files to carry prominent notices + stating that you changed the files and the date of any change. + + b) You must cause any work that you distribute or publish, that in + whole or in part contains or is derived from the Program or any + part thereof, to be licensed as a whole at no charge to all third + parties under the terms of this License. + + c) If the modified program normally reads commands interactively + when run, you must cause it, when started running for such + interactive use in the most ordinary way, to print or display an + announcement including an appropriate copyright notice and a + notice that there is no warranty (or else, saying that you provide + a warranty) and that users may redistribute the program under + these conditions, and telling the user how to view a copy of this + License. (Exception: if the Program itself is interactive but + does not normally print such an announcement, your work based on + the Program is not required to print an announcement.) + +These requirements apply to the modified work as a whole. If +identifiable sections of that work are not derived from the Program, +and can be reasonably considered independent and separate works in +themselves, then this License, and its terms, do not apply to those +sections when you distribute them as separate works. But when you +distribute the same sections as part of a whole which is a work based +on the Program, the distribution of the whole must be on the terms of +this License, whose permissions for other licensees extend to the +entire whole, and thus to each and every part regardless of who wrote it. + +Thus, it is not the intent of this section to claim rights or contest +your rights to work written entirely by you; rather, the intent is to +exercise the right to control the distribution of derivative or +collective works based on the Program. + +In addition, mere aggregation of another work not based on the Program +with the Program (or with a work based on the Program) on a volume of +a storage or distribution medium does not bring the other work under +the scope of this License. + + 3. You may copy and distribute the Program (or a work based on it, +under Section 2) in object code or executable form under the terms of +Sections 1 and 2 above provided that you also do one of the following: + + a) Accompany it with the complete corresponding machine-readable + source code, which must be distributed under the terms of Sections + 1 and 2 above on a medium customarily used for software interchange; or, + + b) Accompany it with a written offer, valid for at least three + years, to give any third party, for a charge no more than your + cost of physically performing source distribution, a complete + machine-readable copy of the corresponding source code, to be + distributed under the terms of Sections 1 and 2 above on a medium + customarily used for software interchange; or, + + c) Accompany it with the information you received as to the offer + to distribute corresponding source code. (This alternative is + allowed only for noncommercial distribution and only if you + received the program in object code or executable form with such + an offer, in accord with Subsection b above.) + +The source code for a work means the preferred form of the work for +making modifications to it. For an executable work, complete source +code means all the source code for all modules it contains, plus any +associated interface definition files, plus the scripts used to +control compilation and installation of the executable. However, as a +special exception, the source code distributed need not include +anything that is normally distributed (in either source or binary +form) with the major components (compiler, kernel, and so on) of the +operating system on which the executable runs, unless that component +itself accompanies the executable. + +If distribution of executable or object code is made by offering +access to copy from a designated place, then offering equivalent +access to copy the source code from the same place counts as +distribution of the source code, even though third parties are not +compelled to copy the source along with the object code. + + 4. You may not copy, modify, sublicense, or distribute the Program +except as expressly provided under this License. Any attempt +otherwise to copy, modify, sublicense or distribute the Program is +void, and will automatically terminate your rights under this License. +However, parties who have received copies, or rights, from you under +this License will not have their licenses terminated so long as such +parties remain in full compliance. + + 5. You are not required to accept this License, since you have not +signed it. However, nothing else grants you permission to modify or +distribute the Program or its derivative works. These actions are +prohibited by law if you do not accept this License. Therefore, by +modifying or distributing the Program (or any work based on the +Program), you indicate your acceptance of this License to do so, and +all its terms and conditions for copying, distributing or modifying +the Program or works based on it. + + 6. Each time you redistribute the Program (or any work based on the +Program), the recipient automatically receives a license from the +original licensor to copy, distribute or modify the Program subject to +these terms and conditions. You may not impose any further +restrictions on the recipients' exercise of the rights granted herein. +You are not responsible for enforcing compliance by third parties to +this License. + + 7. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot +distribute so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you +may not distribute the Program at all. For example, if a patent +license would not permit royalty-free redistribution of the Program by +all those who receive copies directly or indirectly through you, then +the only way you could satisfy both it and this License would be to +refrain entirely from distribution of the Program. + +If any portion of this section is held invalid or unenforceable under +any particular circumstance, the balance of the section is intended to +apply and the section as a whole is intended to apply in other +circumstances. + +It is not the purpose of this section to induce you to infringe any +patents or other property right claims or to contest validity of any +such claims; this section has the sole purpose of protecting the +integrity of the free software distribution system, which is +implemented by public license practices. Many people have made +generous contributions to the wide range of software distributed +through that system in reliance on consistent application of that +system; it is up to the author/donor to decide if he or she is willing +to distribute software through any other system and a licensee cannot +impose that choice. + +This section is intended to make thoroughly clear what is believed to +be a consequence of the rest of this License. + + 8. If the distribution and/or use of the Program is restricted in +certain countries either by patents or by copyrighted interfaces, the +original copyright holder who places the Program under this License +may add an explicit geographical distribution limitation excluding +those countries, so that distribution is permitted only in or among +countries not thus excluded. In such case, this License incorporates +the limitation as if written in the body of this License. + + 9. The Free Software Foundation may publish revised and/or new versions +of the General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + +Each version is given a distinguishing version number. If the Program +specifies a version number of this License which applies to it and "any +later version", you have the option of following the terms and conditions +either of that version or of any later version published by the Free +Software Foundation. If the Program does not specify a version number of +this License, you may choose any version ever published by the Free Software +Foundation. + + 10. If you wish to incorporate parts of the Program into other free +programs whose distribution conditions are different, write to the author +to ask for permission. For software which is copyrighted by the Free +Software Foundation, write to the Free Software Foundation; we sometimes +make exceptions for this. Our decision will be guided by the two goals +of preserving the free status of all derivatives of our free software and +of promoting the sharing and reuse of software generally. + + NO WARRANTY + + 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY +FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN +OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES +PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED +OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS +TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE +PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, +REPAIR OR CORRECTION. + + 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR +REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, +INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING +OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED +TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY +YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER +PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE +POSSIBILITY OF SUCH DAMAGES. + + END OF TERMS AND CONDITIONS + + Appendix: How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +convey the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) 19yy + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 2 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + +Also add information on how to contact you by electronic and paper mail. + +If the program is interactive, make it output a short notice like this +when it starts in an interactive mode: + + Gnomovision version 69, Copyright (C) 19yy name of author + Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, the commands you use may +be called something other than `show w' and `show c'; they could even be +mouse-clicks or menu items--whatever suits your program. + +You should also get your employer (if you work as a programmer) or your +school, if any, to sign a "copyright disclaimer" for the program, if +necessary. Here is a sample; alter the names: + + Yoyodyne, Inc., hereby disclaims all copyright interest in the program + `Gnomovision' (which makes passes at compilers) written by James Hacker. + + , 1 April 1989 + Ty Coon, President of Vice + +This General Public License does not permit incorporating your program into +proprietary programs. If your program is a subroutine library, you may +consider it more useful to permit linking proprietary applications with the +library. If this is what you want to do, use the GNU Library General +Public License instead of this License. diff --git a/Makefile b/Makefile new file mode 100644 index 0000000..d124743 --- /dev/null +++ b/Makefile @@ -0,0 +1,30 @@ + +CC = gcc +SH = /bin/sh + +CFLAGS = -O3 -fomit-frame-pointer -funroll-loops -Wall -Winline -W + + + +all: + cat words0 + $(CC) $(CFLAGS) -o bzip2 bzip2.c + $(CC) $(CFLAGS) -o bzip2recover bzip2recover.c + rm -f bunzip2 + ln -s ./bzip2 ./bunzip2 + cat words1 + ./bzip2 -1 < sample1.ref > sample1.rb2 + ./bzip2 -2 < sample2.ref > sample2.rb2 + ./bunzip2 < sample1.bz2 > sample1.tst + ./bunzip2 < sample2.bz2 > sample2.tst + cat words2 + cmp sample1.bz2 sample1.rb2 + cmp sample2.bz2 sample2.rb2 + cmp sample1.tst sample1.ref + cmp sample2.tst sample2.ref + cat words3 + + +clean: + rm -f bzip2 bunzip2 bzip2recover sample*.tst sample*.rb2 + diff --git a/README b/README new file mode 100644 index 0000000..d77830f --- /dev/null +++ b/README @@ -0,0 +1,243 @@ + +GREETINGS! + + This is the README for bzip2, my block-sorting file compressor, + version 0.1. + + bzip2 is distributed under the GNU General Public License version 2; + for details, see the file LICENSE. Pointers to the algorithms used + are in ALGORITHMS. Instructions for use are in bzip2.1.preformatted. + + Please read this file carefully. + + + +HOW TO BUILD + + -- for UNIX: + + Type `make'. (tough, huh? :-) + + This creates binaries "bzip2", and "bunzip2", + which is a symbolic link to "bzip2". + + It also runs four compress-decompress tests to make sure + things are working properly. If all goes well, you should be up & + running. Please be sure to read the output from `make' + just to be sure that the tests went ok. + + To install bzip2 properly: + + -- Copy the binary "bzip2" to a publically visible place, + possibly /usr/bin, /usr/common/bin or /usr/local/bin. + + -- In that directory, make "bunzip2" be a symbolic link + to "bzip2". + + -- Copy the manual page, bzip2.1, to the relevant place. + Probably the right place is /usr/man/man1/. + + -- for Windows 95 and NT: + + For a start, do you *really* want to recompile bzip2? + The standard distribution includes a pre-compiled version + for Windows 95 and NT, `bzip2.exe'. + + This executable was created with Jacob Navia's excellent + port to Win32 of Chris Fraser & David Hanson's excellent + ANSI C compiler, "lcc". You can get to it at the pages + of the CS department of Princeton University, + www.cs.princeton.edu. + I have not tried to compile this version of bzip2 with + a commercial C compiler such as MS Visual C, as I don't + have one available. + + Note that lcc is designed primarily to be portable and + fast. Code quality is a secondary aim, so bzip2.exe + runs perhaps 40% slower than it could if compiled with + a good optimising compiler. + + I compiled a previous version of bzip (0.21) with Borland + C 5.0, which worked fine, and with MS VC++ 2.0, which + didn't. Here is an comment from the README for bzip-0.21. + + MS VC++ 2.0's optimising compiler has a bug which, at + maximum optimisation, gives an executable which produces + garbage compressed files. Proceed with caution. + I do not know whether or not this happens with later + versions of VC++. + + Edit the defines starting at line 86 of bzip.c to + select your platform/compiler combination, and then compile. + Then check that the resulting executable (assumed to be + called bzip.exe) works correctly, using the SELFTEST.BAT file. + Bearing in mind the previous paragraph, the self-test is + important. + + Note that the defines which bzip-0.21 had, to support + compilation with VC 2.0 and BC 5.0, are gone. Windows + is not my preferred operating system, and I am, for the + moment, content with the modestly fast executable created + by lcc-win32. + + A manual page is supplied, unformatted (bzip2.1), + preformatted (bzip2.1.preformatted), and preformatted + and sanitised for MS-DOS (bzip2.txt). + + + +COMPILATION NOTES + + bzip2 should work on any 32 or 64-bit machine. It is known to work + [meaning: it has compiled and passed self-tests] on the + following platform-os combinations: + + Intel i386/i486 running Linux 2.0.21 + Sun Sparcs (various) running SunOS 4.1.4 and Solaris 2.5 + Intel i386/i486 running Windows 95 and NT + DEC Alpha running Digital Unix 4.0 + + Following the release of bzip-0.21, many people mailed me + from around the world to say they had made it work on all sorts + of weird and wonderful machines. Chances are, if you have + a reasonable ANSI C compiler and a 32-bit machine, you can + get it to work. + + The #defines starting at around line 82 of bzip2.c supply some + degree of platform-independance. If you configure bzip2 for some + new far-out platform which is not covered by the existing definitions, + please send me the relevant definitions. + + I recommend GNU C for compilation. The code is standard ANSI C, + except for the Unix-specific file handling, so any ANSI C compiler + should work. Note however that the many routines marked INLINE + should be inlined by your compiler, else performance will be very + poor. Asking your compiler to unroll loops gives some + small improvement too; for gcc, the relevant flag is + -funroll-loops. + + On a 386/486 machines, I'd recommend giving gcc the + -fomit-frame-pointer flag; this liberates another register for + allocation, which measurably improves performance. + + I used the abovementioned lcc compiler to develop bzip2. + I would highly recommend this compiler for day-to-day development; + it is fast, reliable, lightweight, has an excellent profiler, + and is generally excellent. And it's fun to retarget, if you're + into that kind of thing. + + If you compile bzip2 on a new platform or with a new compiler, + please be sure to run the four compress-decompress tests, either + using the Makefile, or with the test.bat (MSDOS) or test.cmd (OS/2) + files. Some compilers have been seen to introduce subtle bugs + when optimising, so this check is important. Ideally you should + then go on to test bzip2 on a file several megabytes or even + tens of megabytes long, just to be 110% sure. ``Professional + programmers are paranoid programmers.'' (anon). + + + +VALIDATION + + Correct operation, in the sense that a compressed file can always be + decompressed to reproduce the original, is obviously of paramount + importance. To validate bzip2, I used a modified version of + Mark Nelson's churn program. Churn is an automated test driver + which recursively traverses a directory structure, using bzip2 to + compress and then decompress each file it encounters, and checking + that the decompressed data is the same as the original. As test + material, I used several runs over several filesystems of differing + sizes. + + One set of tests was done on my base Linux filesystem, + 410 megabytes in 23,000 files. There were several runs over + this filesystem, in various configurations designed to break bzip2. + That filesystem also contained some specially constructed test + files designed to exercise boundary cases in the code. + This included files of zero length, various long, highly repetitive + files, and some files which generate blocks with all values the same. + + The other set of tests was done just with the "normal" configuration, + but on a much larger quantity of data. + + Tests are: + + Linux FS, 410M, 23000 files + + As above, with --repetitive-fast + + As above, with -1 + + Low level disk image of a disk containing + Windows NT4.0; 420M in a single huge file + + Linux distribution, incl Slackware, + all GNU sources. 1900M in 2300 files. + + Approx ~100M compiler sources and related + programming tools, running under Purify. + + About 500M of data in 120 files of around + 4 M each. This is raw data from a + biomagnetometer (SQUID-based thing). + + Overall, total volume of test data is about + 3300 megabytes in 25000 files. + + The distribution does four tests after building bzip. These tests + include test decompressions of pre-supplied compressed files, so + they not only test that bzip works correctly on the machine it was + built on, but can also decompress files compressed on a different + machine. This guards against unforseen interoperability problems. + + +Please read and be aware of the following: + +WARNING: + + This program (attempts to) compress data by performing several + non-trivial transformations on it. Unless you are 100% familiar + with *all* the algorithms contained herein, and with the + consequences of modifying them, you should NOT meddle with the + compression or decompression machinery. Incorrect changes can and + very likely *will* lead to disastrous loss of data. + + +DISCLAIMER: + + I TAKE NO RESPONSIBILITY FOR ANY LOSS OF DATA ARISING FROM THE + USE OF THIS PROGRAM, HOWSOEVER CAUSED. + + Every compression of a file implies an assumption that the + compressed file can be decompressed to reproduce the original. + Great efforts in design, coding and testing have been made to + ensure that this program works correctly. However, the complexity + of the algorithms, and, in particular, the presence of various + special cases in the code which occur with very low but non-zero + probability make it impossible to rule out the possibility of bugs + remaining in the program. DO NOT COMPRESS ANY DATA WITH THIS + PROGRAM UNLESS YOU ARE PREPARED TO ACCEPT THE POSSIBILITY, HOWEVER + SMALL, THAT THE DATA WILL NOT BE RECOVERABLE. + + That is not to say this program is inherently unreliable. Indeed, + I very much hope the opposite is true. bzip2 has been carefully + constructed and extensively tested. + +End of nasty legalities. + + +I hope you find bzip2 useful. Feel free to contact me at + jseward@acm.org +if you have any suggestions or queries. Many people mailed me with +comments, suggestions and patches after the releases of 0.15 and 0.21, +and the changes in bzip2 are largely a result of this feedback. +I thank you for your comments. + +Julian Seward + +Manchester, UK +18 July 1996 (version 0.15) +25 August 1996 (version 0.21) + +Guildford, Surrey, UK +7 August 1997 (bzip2, version 0.0) \ No newline at end of file diff --git a/README.DOS b/README.DOS new file mode 100644 index 0000000..d522b81 --- /dev/null +++ b/README.DOS @@ -0,0 +1,20 @@ + +Windows 95 & Windows NT users: + +1. There's a pre-built executable, bzip2.exe, which + should work. You don't need to compile anything. + You can run the `test.bat' batch file to check + the executable is working ok, if you want. + +2. The control-C signal catcher seems pretty dodgy + under Windows, at least for the executable supplied. + When it catches a control-C, bzip2 tries to delete + its output file, so you don't get left with a half- + baked file. But this sometimes seems to fail + under Windows. Caveat Emptor! I think I am doing + something not-quite-right in the signal catching. + Windows-&-C gurus got any suggestions? + + Control-C handling all seems to work fine under Unix. + +7 Aug 97 diff --git a/bzip2.1 b/bzip2.1 new file mode 100644 index 0000000..9094c7c --- /dev/null +++ b/bzip2.1 @@ -0,0 +1,441 @@ +.PU +.TH bzip2 1 +.SH NAME +bzip2, bunzip2 \- a block-sorting file compressor, v0.1 +.br +bzip2recover \- recovers data from damaged bzip2 files + +.SH SYNOPSIS +.ll +8 +.B bzip2 +.RB [ " \-cdfkstvVL123456789 " ] +[ +.I "filenames \&..." +] +.ll -8 +.br +.B bunzip2 +.RB [ " \-kvsVL " ] +[ +.I "filenames \&..." +] +.br +.B bzip2recover +.I "filename" + +.SH DESCRIPTION +.I Bzip2 +compresses files using the Burrows-Wheeler block-sorting +text compression algorithm, and Huffman coding. +Compression is generally considerably +better than that +achieved by more conventional LZ77/LZ78-based compressors, +and approaches the performance of the PPM family of statistical +compressors. + +The command-line options are deliberately very similar to +those of +.I GNU Gzip, +but they are not identical. + +.I Bzip2 +expects a list of file names to accompany the command-line flags. +Each file is replaced by a compressed version of itself, +with the name "original_name.bz2". +Each compressed file has the same modification date and permissions +as the corresponding original, so that these properties can be +correctly restored at decompression time. File name handling is +naive in the sense that there is no mechanism for preserving +original file names, permissions and dates in filesystems +which lack these concepts, or have serious file name length +restrictions, such as MS-DOS. + +.I Bzip2 +and +.I bunzip2 +will not overwrite existing files; if you want this to happen, +you should delete them first. + +If no file names are specified, +.I bzip2 +compresses from standard input to standard output. +In this case, +.I bzip2 +will decline to write compressed output to a terminal, as +this would be entirely incomprehensible and therefore pointless. + +.I Bunzip2 +(or +.I bzip2 \-d +) decompresses and restores all specified files whose names +end in ".bz2". +Files without this suffix are ignored. +Again, supplying no filenames +causes decompression from standard input to standard output. + +You can also compress or decompress files to +the standard output by giving the \-c flag. +You can decompress multiple files like this, but you may +only compress a single file this way, since it would otherwise +be difficult to separate out the compressed representations of +the original files. + +Compression is always performed, even if the compressed file is +slightly larger than the original. Files of less than about +one hundred bytes tend to get larger, since the compression +mechanism has a constant overhead in the region of 50 bytes. +Random data (including the output of most file compressors) +is coded at about 8.05 bits per byte, giving an expansion of +around 0.5%. + +As a self-check for your protection, +.I bzip2 +uses 32-bit CRCs to make sure that the decompressed +version of a file is identical to the original. +This guards against corruption of the compressed data, +and against undetected bugs in +.I bzip2 +(hopefully very unlikely). +The chances of data corruption going undetected is +microscopic, about one chance in four billion +for each file processed. Be aware, though, that the check +occurs upon decompression, so it can only tell you that +that something is wrong. It can't help you recover the +original uncompressed data. +You can use +.I bzip2recover +to try to recover data from damaged files. + +Return values: +0 for a normal exit, +1 for environmental +problems (file not found, invalid flags, I/O errors, &c), +2 to indicate a corrupt compressed file, +3 for an internal consistency error (eg, bug) which caused +.I bzip2 +to panic. + +.SH MEMORY MANAGEMENT +.I Bzip2 +compresses large files in blocks. The block size affects both the +compression ratio achieved, and the amount of memory needed both for +compression and decompression. The flags \-1 through \-9 +specify the block size to be 100,000 bytes through 900,000 bytes +(the default) respectively. At decompression-time, the block size used for +compression is read from the header of the compressed file, and +.I bunzip2 +then allocates itself just enough memory to decompress the file. +Since block sizes are stored in compressed files, it follows that the flags +\-1 to \-9 +are irrelevant to and so ignored during decompression. +Compression and decompression requirements, in bytes, can be estimated as: + + Compression: 400k + ( 7 x block size ) + + Decompression: 100k + ( 5 x block size ), or +.br + 100k + ( 2.5 x block size ) + +Larger block sizes give rapidly diminishing marginal returns; most +of the +compression comes from the first two or three hundred k of block size, +a fact worth bearing in mind when using +.I bzip2 +on small machines. It is also important to appreciate that the +decompression memory requirement is set at compression-time by the +choice of block size. + +For files compressed with the default 900k block size, +.I bunzip2 +will require about 4600 kbytes to decompress. +To support decompression of any file on a 4 megabyte machine, +.I bunzip2 +has an option to decompress using approximately half this +amount of memory, about 2300 kbytes. Decompression speed is +also halved, so you should use this option only where necessary. +The relevant flag is \-s. + +In general, try and use the largest block size +memory constraints allow, since that maximises the compression +achieved. Compression and decompression +speed are virtually unaffected by block size. + +Another significant point applies to files which fit in a single +block -- that means most files you'd encounter using a large +block size. The amount of real memory touched is proportional +to the size of the file, since the file is smaller than a block. +For example, compressing a file 20,000 bytes long with the flag +\-9 +will cause the compressor to allocate around +6700k of memory, but only touch 400k + 20000 * 7 = 540 +kbytes of it. Similarly, the decompressor will allocate 4600k but +only touch 100k + 20000 * 5 = 200 kbytes. + +Here is a table which summarises the maximum memory usage for +different block sizes. Also recorded is the total compressed +size for 14 files of the Calgary Text Compression Corpus +totalling 3,141,622 bytes. This column gives some feel for how +compression varies with block size. These figures tend to understate +the advantage of larger block sizes for larger files, since the +Corpus is dominated by smaller files. + + Compress Decompress Decompress Corpus + Flag usage usage -s usage Size + + -1 1100k 600k 350k 914704 + -2 1800k 1100k 600k 877703 + -3 2500k 1600k 850k 860338 + -4 3200k 2100k 1100k 846899 + -5 3900k 2600k 1350k 845160 + -6 4600k 3100k 1600k 838626 + -7 5400k 3600k 1850k 834096 + -8 6000k 4100k 2100k 828642 + -9 6700k 4600k 2350k 828642 + +.SH OPTIONS +.TP +.B \-c --stdout +Compress or decompress to standard output. \-c will decompress +multiple files to stdout, but will only compress a single file to +stdout. +.TP +.B \-d --decompress +Force decompression. +.I Bzip2 +and +.I bunzip2 +are really the same program, and the decision about whether to +compress or decompress is done on the basis of which name is +used. This flag overrides that mechanism, and forces +.I bzip2 +to decompress. +.TP +.B \-f --compress +The complement to \-d: forces compression, regardless of the invokation +name. +.TP +.B \-t --test +Check integrity of the specified file(s), but don't decompress them. +This really performs a trial decompression and throws away the result, +using the low-memory decompression algorithm (see \-s). +.TP +.B \-k --keep +Keep (don't delete) input files during compression or decompression. +.TP +.B \-s --small +Reduce memory usage, both for compression and decompression. +Files are decompressed using a modified algorithm which only +requires 2.5 bytes per block byte. This means any file can be +decompressed in 2300k of memory, albeit somewhat more slowly than +usual. + +During compression, -s selects a block size of 200k, which limits +memory use to around the same figure, at the expense of your +compression ratio. In short, if your machine is low on memory +(8 megabytes or less), use -s for everything. See +MEMORY MANAGEMENT above. + +.TP +.B \-v --verbose +Verbose mode -- show the compression ratio for each file processed. +Further \-v's increase the verbosity level, spewing out lots of +information which is primarily of interest for diagnostic purposes. +.TP +.B \-L --license +Display the software version, license terms and conditions. +.TP +.B \-V --version +Same as \-L. +.TP +.B \-1 to \-9 +Set the block size to 100 k, 200 k .. 900 k when +compressing. Has no effect when decompressing. +See MEMORY MANAGEMENT above. +.TP +.B \--repetitive-fast +.I bzip2 +injects some small pseudo-random variations +into very repetitive blocks to limit +worst-case performance during compression. +If sorting runs into difficulties, the block +is randomised, and sorting is restarted. +Very roughly, +.I bzip2 +persists for three times as long as a well-behaved input +would take before resorting to randomisation. +This flag makes it give up much sooner. + +.TP +.B \--repetitive-best +Opposite of \--repetitive-fast; try a lot harder before +resorting to randomisation. + +.SH RECOVERING DATA FROM DAMAGED FILES +.I bzip2 +compresses files in blocks, usually 900kbytes long. +Each block is handled independently. If a media or +transmission error causes a multi-block .bz2 +file to become damaged, +it may be possible to recover data from the undamaged blocks +in the file. + +The compressed representation of each block is delimited by +a 48-bit pattern, which makes it possible to find the block +boundaries with reasonable certainty. Each block also carries +its own 32-bit CRC, so damaged blocks can be +distinguished from undamaged ones. + +.I bzip2recover +is a simple program whose purpose is to search for +blocks in .bz2 files, and write each block out into +its own .bz2 file. You can then use +.I bzip2 -t +to test the integrity of the resulting files, +and decompress those which are undamaged. + +.I bzip2recover +takes a single argument, the name of the damaged file, +and writes a number of files "rec0001file.bz2", "rec0002file.bz2", +etc, containing the extracted blocks. The output filenames +are designed so that the use of wildcards in subsequent processing +-- for example, "bzip2 -dc rec*file.bz2 > recovered_data" -- +lists the files in the "right" order. + +.I bzip2recover +should be of most use dealing with large .bz2 files, as +these will contain many blocks. It is clearly futile to +use it on damaged single-block files, since a damaged +block cannot be recovered. If you wish to minimise +any potential data loss through media or transmission +errors, you might consider compressing with a smaller +block size. + +.SH PERFORMANCE NOTES +The sorting phase of compression gathers together similar strings +in the file. Because of this, files containing very long +runs of repeated symbols, like "aabaabaabaab ..." (repeated +several hundred times) may compress extraordinarily slowly. +You can use the +\-vvvvv +option to monitor progress in great detail, if you want. +Decompression speed is unaffected. + +Such pathological cases +seem rare in practice, appearing mostly in artificially-constructed +test files, and in low-level disk images. It may be inadvisable to +use +.I bzip2 +to compress the latter. +If you do get a file which causes severe slowness in compression, +try making the block size as small as possible, with flag \-1. + +Incompressible or virtually-incompressible data may decompress +rather more slowly than one would hope. This is due to +a naive implementation of the move-to-front coder. + +.I bzip2 +usually allocates several megabytes of memory to operate in, +and then charges all over it in a fairly random fashion. This +means that performance, both for compressing and decompressing, +is largely determined by the speed +at which your machine can service cache misses. +Because of this, small changes +to the code to reduce the miss rate have been observed to give +disproportionately large performance improvements. +I imagine +.I bzip2 +will perform best on machines with very large caches. + +Test mode (\-t) uses the low-memory decompression algorithm +(\-s). This means test mode does not run as fast as it could; +it could run as fast as the normal decompression machinery. +This could easily be fixed at the cost of some code bloat. + +.SH CAVEATS +I/O error messages are not as helpful as they could be. +.I Bzip2 +tries hard to detect I/O errors and exit cleanly, but the +details of what the problem is sometimes seem rather misleading. + +This manual page pertains to version 0.1 of +.I bzip2. +It may well happen that some future version will +use a different compressed file format. If you try to +decompress, using 0.1, a .bz2 file created with some +future version which uses a different compressed file format, +0.1 will complain that your file "is not a bzip2 file". +If that happens, you should obtain a more recent version +of +.I bzip2 +and use that to decompress the file. + +Wildcard expansion for Windows 95 and NT +is flaky. + +.I bzip2recover +uses 32-bit integers to represent bit positions in +compressed files, so it cannot handle compressed files +more than 512 megabytes long. This could easily be fixed. + +.I bzip2recover +sometimes reports a very small, incomplete final block. +This is spurious and can be safely ignored. + +.SH RELATIONSHIP TO bzip-0.21 +This program is a descendant of the +.I bzip +program, version 0.21, which I released in August 1996. +The primary difference of +.I bzip2 +is its avoidance of the possibly patented algorithms +which were used in 0.21. +.I bzip2 +also brings various useful refinements (\-s, \-t), +uses less memory, decompresses significantly faster, and +has support for recovering data from damaged files. + +Because +.I bzip2 +uses Huffman coding to construct the compressed bitstream, +rather than the arithmetic coding used in 0.21, +the compressed representations generated by the two programs +are incompatible, and they will not interoperate. The change +in suffix from .bz to .bz2 reflects this. It would have been +helpful to at least allow +.I bzip2 +to decompress files created by 0.21, but this would +defeat the primary aim of having a patent-free compressor. + +Huffman coding necessarily involves some coding inefficiency +compared to arithmetic coding. This means that +.I bzip2 +compresses about 1% worse than 0.21, an unfortunate but +unavoidable fact-of-life. On the other hand, decompression +is approximately 50% faster for the same reason, and the +change in file format gave an opportunity to add data-recovery +features. So it is not all bad. + +.SH AUTHOR +Julian Seward, jseward@acm.org. + +The ideas embodied in +.I bzip +and +.I bzip2 +are due to (at least) the following people: +Michael Burrows and David Wheeler (for the block sorting +transformation), David Wheeler (again, for the Huffman coder), +Peter Fenwick (for the structured coding model in 0.21, +and many refinements), +and +Alistair Moffat, Radford Neal and Ian Witten (for the arithmetic +coder in 0.21). I am much indebted for their help, support and advice. +See the file ALGORITHMS in the source distribution for pointers to +sources of documentation. +Christian von Roques encouraged me to look for faster +sorting algorithms, so as to speed up compression. +Bela Lubkin encouraged me to improve the worst-case +compression performance. +Many people sent patches, helped with portability problems, +lent machines, gave advice and were generally helpful. + diff --git a/bzip2.1.preformatted b/bzip2.1.preformatted new file mode 100644 index 0000000..947dc97 --- /dev/null +++ b/bzip2.1.preformatted @@ -0,0 +1,462 @@ + + + +bzip2(1) bzip2(1) + + +NNAAMMEE + bzip2, bunzip2 - a block-sorting file compressor, v0.1 + bzip2recover - recovers data from damaged bzip2 files + + +SSYYNNOOPPSSIISS + bbzziipp22 [ --ccddffkkssttvvVVLL112233445566778899 ] [ _f_i_l_e_n_a_m_e_s _._._. ] + bbuunnzziipp22 [ --kkvvssVVLL ] [ _f_i_l_e_n_a_m_e_s _._._. ] + bbzziipp22rreeccoovveerr _f_i_l_e_n_a_m_e + + +DDEESSCCRRIIPPTTIIOONN + _B_z_i_p_2 compresses files using the Burrows-Wheeler block- + sorting text compression algorithm, and Huffman coding. + Compression is generally considerably better than that + achieved by more conventional LZ77/LZ78-based compressors, + and approaches the performance of the PPM family of sta- + tistical compressors. + + The command-line options are deliberately very similar to + those of _G_N_U _G_z_i_p_, but they are not identical. + + _B_z_i_p_2 expects a list of file names to accompany the com- + mand-line flags. Each file is replaced by a compressed + version of itself, with the name "original_name.bz2". + Each compressed file has the same modification date and + permissions as the corresponding original, so that these + properties can be correctly restored at decompression + time. File name handling is naive in the sense that there + is no mechanism for preserving original file names, per- + missions and dates in filesystems which lack these con- + cepts, or have serious file name length restrictions, such + as MS-DOS. + + _B_z_i_p_2 and _b_u_n_z_i_p_2 will not overwrite existing files; if + you want this to happen, you should delete them first. + + If no file names are specified, _b_z_i_p_2 compresses from + standard input to standard output. In this case, _b_z_i_p_2 + will decline to write compressed output to a terminal, as + this would be entirely incomprehensible and therefore + pointless. + + _B_u_n_z_i_p_2 (or _b_z_i_p_2 _-_d ) decompresses and restores all spec- + ified files whose names end in ".bz2". Files without this + suffix are ignored. Again, supplying no filenames causes + decompression from standard input to standard output. + + You can also compress or decompress files to the standard + output by giving the -c flag. You can decompress multiple + files like this, but you may only compress a single file + this way, since it would otherwise be difficult to sepa- + rate out the compressed representations of the original + files. + + + + 1 + + + + + +bzip2(1) bzip2(1) + + + Compression is always performed, even if the compressed + file is slightly larger than the original. Files of less + than about one hundred bytes tend to get larger, since the + compression mechanism has a constant overhead in the + region of 50 bytes. Random data (including the output of + most file compressors) is coded at about 8.05 bits per + byte, giving an expansion of around 0.5%. + + As a self-check for your protection, _b_z_i_p_2 uses 32-bit + CRCs to make sure that the decompressed version of a file + is identical to the original. This guards against corrup- + tion of the compressed data, and against undetected bugs + in _b_z_i_p_2 (hopefully very unlikely). The chances of data + corruption going undetected is microscopic, about one + chance in four billion for each file processed. Be aware, + though, that the check occurs upon decompression, so it + can only tell you that that something is wrong. It can't + help you recover the original uncompressed data. You can + use _b_z_i_p_2_r_e_c_o_v_e_r to try to recover data from damaged + files. + + Return values: 0 for a normal exit, 1 for environmental + problems (file not found, invalid flags, I/O errors, &c), + 2 to indicate a corrupt compressed file, 3 for an internal + consistency error (eg, bug) which caused _b_z_i_p_2 to panic. + + +MMEEMMOORRYY MMAANNAAGGEEMMEENNTT + _B_z_i_p_2 compresses large files in blocks. The block size + affects both the compression ratio achieved, and the + amount of memory needed both for compression and decom- + pression. The flags -1 through -9 specify the block size + to be 100,000 bytes through 900,000 bytes (the default) + respectively. At decompression-time, the block size used + for compression is read from the header of the compressed + file, and _b_u_n_z_i_p_2 then allocates itself just enough memory + to decompress the file. Since block sizes are stored in + compressed files, it follows that the flags -1 to -9 are + irrelevant to and so ignored during decompression. Com- + pression and decompression requirements, in bytes, can be + estimated as: + + Compression: 400k + ( 7 x block size ) + + Decompression: 100k + ( 5 x block size ), or + 100k + ( 2.5 x block size ) + + Larger block sizes give rapidly diminishing marginal + returns; most of the compression comes from the first two + or three hundred k of block size, a fact worth bearing in + mind when using _b_z_i_p_2 on small machines. It is also + important to appreciate that the decompression memory + requirement is set at compression-time by the choice of + block size. + + + + 2 + + + + + +bzip2(1) bzip2(1) + + + For files compressed with the default 900k block size, + _b_u_n_z_i_p_2 will require about 4600 kbytes to decompress. To + support decompression of any file on a 4 megabyte machine, + _b_u_n_z_i_p_2 has an option to decompress using approximately + half this amount of memory, about 2300 kbytes. Decompres- + sion speed is also halved, so you should use this option + only where necessary. The relevant flag is -s. + + In general, try and use the largest block size memory con- + straints allow, since that maximises the compression + achieved. Compression and decompression speed are virtu- + ally unaffected by block size. + + Another significant point applies to files which fit in a + single block -- that means most files you'd encounter + using a large block size. The amount of real memory + touched is proportional to the size of the file, since the + file is smaller than a block. For example, compressing a + file 20,000 bytes long with the flag -9 will cause the + compressor to allocate around 6700k of memory, but only + touch 400k + 20000 * 7 = 540 kbytes of it. Similarly, the + decompressor will allocate 4600k but only touch 100k + + 20000 * 5 = 200 kbytes. + + Here is a table which summarises the maximum memory usage + for different block sizes. Also recorded is the total + compressed size for 14 files of the Calgary Text Compres- + sion Corpus totalling 3,141,622 bytes. This column gives + some feel for how compression varies with block size. + These figures tend to understate the advantage of larger + block sizes for larger files, since the Corpus is domi- + nated by smaller files. + + Compress Decompress Decompress Corpus + Flag usage usage -s usage Size + + -1 1100k 600k 350k 914704 + -2 1800k 1100k 600k 877703 + -3 2500k 1600k 850k 860338 + -4 3200k 2100k 1100k 846899 + -5 3900k 2600k 1350k 845160 + -6 4600k 3100k 1600k 838626 + -7 5400k 3600k 1850k 834096 + -8 6000k 4100k 2100k 828642 + -9 6700k 4600k 2350k 828642 + + +OOPPTTIIOONNSS + --cc ----ssttddoouutt + Compress or decompress to standard output. -c will + decompress multiple files to stdout, but will only + compress a single file to stdout. + + + + + + 3 + + + + + +bzip2(1) bzip2(1) + + + --dd ----ddeeccoommpprreessss + Force decompression. _B_z_i_p_2 and _b_u_n_z_i_p_2 are really + the same program, and the decision about whether to + compress or decompress is done on the basis of + which name is used. This flag overrides that mech- + anism, and forces _b_z_i_p_2 to decompress. + + --ff ----ccoommpprreessss + The complement to -d: forces compression, regard- + less of the invokation name. + + --tt ----tteesstt + Check integrity of the specified file(s), but don't + decompress them. This really performs a trial + decompression and throws away the result, using the + low-memory decompression algorithm (see -s). + + --kk ----kkeeeepp + Keep (don't delete) input files during compression + or decompression. + + --ss ----ssmmaallll + Reduce memory usage, both for compression and + decompression. Files are decompressed using a mod- + ified algorithm which only requires 2.5 bytes per + block byte. This means any file can be decom- + pressed in 2300k of memory, albeit somewhat more + slowly than usual. + + During compression, -s selects a block size of + 200k, which limits memory use to around the same + figure, at the expense of your compression ratio. + In short, if your machine is low on memory (8 + megabytes or less), use -s for everything. See + MEMORY MANAGEMENT above. + + + --vv ----vveerrbboossee + Verbose mode -- show the compression ratio for each + file processed. Further -v's increase the ver- + bosity level, spewing out lots of information which + is primarily of interest for diagnostic purposes. + + --LL ----lliicceennssee + Display the software version, license terms and + conditions. + + --VV ----vveerrssiioonn + Same as -L. + + --11 ttoo --99 + Set the block size to 100 k, 200 k .. 900 k when + compressing. Has no effect when decompressing. + See MEMORY MANAGEMENT above. + + + + 4 + + + + + +bzip2(1) bzip2(1) + + + ----rreeppeettiittiivvee--ffaasstt + _b_z_i_p_2 injects some small pseudo-random variations + into very repetitive blocks to limit worst-case + performance during compression. If sorting runs + into difficulties, the block is randomised, and + sorting is restarted. Very roughly, _b_z_i_p_2 persists + for three times as long as a well-behaved input + would take before resorting to randomisation. This + flag makes it give up much sooner. + + + ----rreeppeettiittiivvee--bbeesstt + Opposite of --repetitive-fast; try a lot harder + before resorting to randomisation. + + +RREECCOOVVEERRIINNGG DDAATTAA FFRROOMM DDAAMMAAGGEEDD FFIILLEESS + _b_z_i_p_2 compresses files in blocks, usually 900kbytes long. + Each block is handled independently. If a media or trans- + mission error causes a multi-block .bz2 file to become + damaged, it may be possible to recover data from the + undamaged blocks in the file. + + The compressed representation of each block is delimited + by a 48-bit pattern, which makes it possible to find the + block boundaries with reasonable certainty. Each block + also carries its own 32-bit CRC, so damaged blocks can be + distinguished from undamaged ones. + + _b_z_i_p_2_r_e_c_o_v_e_r is a simple program whose purpose is to + search for blocks in .bz2 files, and write each block out + into its own .bz2 file. You can then use _b_z_i_p_2 _-_t to test + the integrity of the resulting files, and decompress those + which are undamaged. + + _b_z_i_p_2_r_e_c_o_v_e_r takes a single argument, the name of the dam- + aged file, and writes a number of files "rec0001file.bz2", + "rec0002file.bz2", etc, containing the extracted blocks. + The output filenames are designed so that the use of wild- + cards in subsequent processing -- for example, "bzip2 -dc + rec*file.bz2 > recovered_data" -- lists the files in the + "right" order. + + _b_z_i_p_2_r_e_c_o_v_e_r should be of most use dealing with large .bz2 + files, as these will contain many blocks. It is clearly + futile to use it on damaged single-block files, since a + damaged block cannot be recovered. If you wish to min- + imise any potential data loss through media or transmis- + sion errors, you might consider compressing with a smaller + block size. + + +PPEERRFFOORRMMAANNCCEE NNOOTTEESS + The sorting phase of compression gathers together similar + + + + 5 + + + + + +bzip2(1) bzip2(1) + + + strings in the file. Because of this, files containing + very long runs of repeated symbols, like "aabaabaabaab + ..." (repeated several hundred times) may compress + extraordinarily slowly. You can use the -vvvvv option to + monitor progress in great detail, if you want. Decompres- + sion speed is unaffected. + + Such pathological cases seem rare in practice, appearing + mostly in artificially-constructed test files, and in low- + level disk images. It may be inadvisable to use _b_z_i_p_2 to + compress the latter. If you do get a file which causes + severe slowness in compression, try making the block size + as small as possible, with flag -1. + + Incompressible or virtually-incompressible data may decom- + press rather more slowly than one would hope. This is due + to a naive implementation of the move-to-front coder. + + _b_z_i_p_2 usually allocates several megabytes of memory to + operate in, and then charges all over it in a fairly ran- + dom fashion. This means that performance, both for com- + pressing and decompressing, is largely determined by the + speed at which your machine can service cache misses. + Because of this, small changes to the code to reduce the + miss rate have been observed to give disproportionately + large performance improvements. I imagine _b_z_i_p_2 will per- + form best on machines with very large caches. + + Test mode (-t) uses the low-memory decompression algorithm + (-s). This means test mode does not run as fast as it + could; it could run as fast as the normal decompression + machinery. This could easily be fixed at the cost of some + code bloat. + + +CCAAVVEEAATTSS + I/O error messages are not as helpful as they could be. + _B_z_i_p_2 tries hard to detect I/O errors and exit cleanly, + but the details of what the problem is sometimes seem + rather misleading. + + This manual page pertains to version 0.1 of _b_z_i_p_2_. It may + well happen that some future version will use a different + compressed file format. If you try to decompress, using + 0.1, a .bz2 file created with some future version which + uses a different compressed file format, 0.1 will complain + that your file "is not a bzip2 file". If that happens, + you should obtain a more recent version of _b_z_i_p_2 and use + that to decompress the file. + + Wildcard expansion for Windows 95 and NT is flaky. + + _b_z_i_p_2_r_e_c_o_v_e_r uses 32-bit integers to represent bit posi- + tions in compressed files, so it cannot handle compressed + + + + 6 + + + + + +bzip2(1) bzip2(1) + + + files more than 512 megabytes long. This could easily be + fixed. + + _b_z_i_p_2_r_e_c_o_v_e_r sometimes reports a very small, incomplete + final block. This is spurious and can be safely ignored. + + +RREELLAATTIIOONNSSHHIIPP TTOO bbzziipp--00..2211 + This program is a descendant of the _b_z_i_p program, version + 0.21, which I released in August 1996. The primary dif- + ference of _b_z_i_p_2 is its avoidance of the possibly patented + algorithms which were used in 0.21. _b_z_i_p_2 also brings + various useful refinements (-s, -t), uses less memory, + decompresses significantly faster, and has support for + recovering data from damaged files. + + Because _b_z_i_p_2 uses Huffman coding to construct the com- + pressed bitstream, rather than the arithmetic coding used + in 0.21, the compressed representations generated by the + two programs are incompatible, and they will not interop- + erate. The change in suffix from .bz to .bz2 reflects + this. It would have been helpful to at least allow _b_z_i_p_2 + to decompress files created by 0.21, but this would defeat + the primary aim of having a patent-free compressor. + + Huffman coding necessarily involves some coding ineffi- + ciency compared to arithmetic coding. This means that + _b_z_i_p_2 compresses about 1% worse than 0.21, an unfortunate + but unavoidable fact-of-life. On the other hand, decom- + pression is approximately 50% faster for the same reason, + and the change in file format gave an opportunity to add + data-recovery features. So it is not all bad. + + +AAUUTTHHOORR + Julian Seward, jseward@acm.org. + + The ideas embodied in _b_z_i_p and _b_z_i_p_2 are due to (at least) + the following people: Michael Burrows and David Wheeler + (for the block sorting transformation), David Wheeler + (again, for the Huffman coder), Peter Fenwick (for the + structured coding model in 0.21, and many refinements), + and Alistair Moffat, Radford Neal and Ian Witten (for the + arithmetic coder in 0.21). I am much indebted for their + help, support and advice. See the file ALGORITHMS in the + source distribution for pointers to sources of documenta- + tion. Christian von Roques encouraged me to look for + faster sorting algorithms, so as to speed up compression. + Bela Lubkin encouraged me to improve the worst-case com- + pression performance. Many people sent patches, helped + with portability problems, lent machines, gave advice and + were generally helpful. + + + + + + 7 + + diff --git a/bzip2.c b/bzip2.c new file mode 100644 index 0000000..0fb45fb --- /dev/null +++ b/bzip2.c @@ -0,0 +1,4036 @@ + +/*-----------------------------------------------------------*/ +/*--- A block-sorting, lossless compressor bzip2.c ---*/ +/*-----------------------------------------------------------*/ + +/*-- + This program is bzip2, a lossless, block-sorting data compressor, + version 0.1pl0, dated 17-Aug-1997. + + Copyright (C) 1996, 1997 by Julian Seward. + Guildford, Surrey, UK + email: jseward@acm.org + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 2 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + + The GNU General Public License is contained in the file LICENSE. + + This program is based on (at least) the work of: + Mike Burrows + David Wheeler + Peter Fenwick + Alistair Moffat + Radford Neal + Ian H. Witten + Robert Sedgewick + Jon L. Bentley + + For more information on these sources, see the file ALGORITHMS. +--*/ + +/*----------------------------------------------------*/ +/*--- IMPORTANT ---*/ +/*----------------------------------------------------*/ + +/*-- + WARNING: + This program (attempts to) compress data by performing several + non-trivial transformations on it. Unless you are 100% familiar + with *all* the algorithms contained herein, and with the + consequences of modifying them, you should NOT meddle with the + compression or decompression machinery. Incorrect changes can + and very likely *will* lead to disasterous loss of data. + + DISCLAIMER: + I TAKE NO RESPONSIBILITY FOR ANY LOSS OF DATA ARISING FROM THE + USE OF THIS PROGRAM, HOWSOEVER CAUSED. + + Every compression of a file implies an assumption that the + compressed file can be decompressed to reproduce the original. + Great efforts in design, coding and testing have been made to + ensure that this program works correctly. However, the + complexity of the algorithms, and, in particular, the presence + of various special cases in the code which occur with very low + but non-zero probability make it impossible to rule out the + possibility of bugs remaining in the program. DO NOT COMPRESS + ANY DATA WITH THIS PROGRAM UNLESS YOU ARE PREPARED TO ACCEPT THE + POSSIBILITY, HOWEVER SMALL, THAT THE DATA WILL NOT BE RECOVERABLE. + + That is not to say this program is inherently unreliable. + Indeed, I very much hope the opposite is true. bzip2 has been + carefully constructed and extensively tested. +--*/ + + + +/*----------------------------------------------------*/ +/*--- and now for something much more pleasant :-) ---*/ +/*----------------------------------------------------*/ + +/*---------------------------------------------*/ +/*-- + Place a 1 beside your platform, and 0 elsewhere. +--*/ + +/*-- + Generic 32-bit Unix. + Also works on 64-bit Unix boxes. +--*/ +#define BZ_UNIX 1 + +/*-- + Win32, as seen by Jacob Navia's excellent + port of (Chris Fraser & David Hanson)'s excellent + lcc compiler. +--*/ +#define BZ_LCCWIN32 0 + + + +/*---------------------------------------------*/ +/*-- + Some stuff for all platforms. +--*/ + +#include +#include +#if DEBUG + #include +#endif +#include +#include +#include +#include + +#define ERROR_IF_EOF(i) { if ((i) == EOF) ioError(); } +#define ERROR_IF_NOT_ZERO(i) { if ((i) != 0) ioError(); } +#define ERROR_IF_MINUS_ONE(i) { if ((i) == (-1)) ioError(); } + + +/*---------------------------------------------*/ +/*-- + Platform-specific stuff. +--*/ + +#if BZ_UNIX + #include + #include + #include + #include + #include + + #define Int32 int + #define UInt32 unsigned int + #define Char char + #define UChar unsigned char + #define Int16 short + #define UInt16 unsigned short + + #define PATH_SEP '/' + #define MY_LSTAT lstat + #define MY_S_IFREG S_ISREG + #define MY_STAT stat + + #define APPEND_FILESPEC(root, name) \ + root=snocString((root), (name)) + + #define SET_BINARY_MODE(fd) /**/ + + /*-- + You should try very hard to persuade your C compiler + to inline the bits marked INLINE. Otherwise bzip2 will + run rather slowly. gcc version 2.x is recommended. + --*/ + #ifdef __GNUC__ + #define INLINE inline + #define NORETURN __attribute__ ((noreturn)) + #else + #define INLINE /**/ + #define NORETURN /**/ + #endif +#endif + + + +#if BZ_LCCWIN32 + #include + #include + #include + + #define Int32 int + #define UInt32 unsigned int + #define Int16 short + #define UInt16 unsigned short + #define Char char + #define UChar unsigned char + + #define INLINE /**/ + #define NORETURN /**/ + #define PATH_SEP '\\' + #define MY_LSTAT _stat + #define MY_STAT _stat + #define MY_S_IFREG(x) ((x) & _S_IFREG) + + #if 0 + /*-- lcc-win32 seems to expand wildcards itself --*/ + #define APPEND_FILESPEC(root, spec) \ + do { \ + if ((spec)[0] == '-') { \ + root = snocString((root), (spec)); \ + } else { \ + struct _finddata_t c_file; \ + long hFile; \ + hFile = _findfirst((spec), &c_file); \ + if ( hFile == -1L ) { \ + root = snocString ((root), (spec)); \ + } else { \ + int anInt = 0; \ + while ( anInt == 0 ) { \ + root = snocString((root), \ + &c_file.name[0]); \ + anInt = _findnext(hFile, &c_file); \ + } \ + } \ + } \ + } while ( 0 ) + #else + #define APPEND_FILESPEC(root, name) \ + root = snocString ((root), (name)) + #endif + + #define SET_BINARY_MODE(fd) \ + do { \ + int retVal = setmode ( fileno ( fd ), \ + O_BINARY ); \ + ERROR_IF_MINUS_ONE ( retVal ); \ + } while ( 0 ) + +#endif + + +/*---------------------------------------------*/ +/*-- + Some more stuff for all platforms :-) +--*/ + +#define Bool unsigned char +#define True 1 +#define False 0 + +/*-- + IntNative is your platform's `native' int size. + Only here to avoid probs with 64-bit platforms. +--*/ +#define IntNative int + + +/*-- + change to 1, or compile with -DDEBUG=1 to debug +--*/ +#ifndef DEBUG +#define DEBUG 0 +#endif + + +/*---------------------------------------------------*/ +/*--- ---*/ +/*---------------------------------------------------*/ + +/*-- + Implementation notes, July 1997 + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + Memory allocation + ~~~~~~~~~~~~~~~~~ + All large data structures are allocated on the C heap, + for better or for worse. That includes the various + arrays of pointers, striped words, bytes, frequency + tables and buffers for compression and decompression. + + bzip2 can operate at various block-sizes, ranging from + 100k to 900k in 100k steps, and it allocates only as + much as it needs to. When compressing, we know from the + command-line options what the block-size is going to be, + so all allocation can be done at start-up; if that + succeeds, there can be no further allocation problems. + + Decompression is more complicated. Each compressed file + contains, in its header, a byte indicating the block + size used for compression. This means bzip2 potentially + needs to reallocate memory for each file it deals with, + which in turn opens the possibility for a memory allocation + failure part way through a run of files, by encountering + a file requiring a much larger block size than all the + ones preceding it. + + The policy is to simply give up if a memory allocation + failure occurs. During decompression, it would be + possible to move on to subsequent files in the hope that + some might ask for a smaller block size, but the + complications for doing this seem more trouble than they + are worth. + + + Compressed file formats + ~~~~~~~~~~~~~~~~~~~~~~~ + [This is now entirely different from both 0.21, and from + any previous Huffman-coded variant of bzip. + See the associated file bzip2.txt for details.] + + + Error conditions + ~~~~~~~~~~~~~~~~ + Dealing with error conditions is the least satisfactory + aspect of bzip2. The policy is to try and leave the + filesystem in a consistent state, then quit, even if it + means not processing some of the files mentioned in the + command line. `A consistent state' means that a file + exists either in its compressed or uncompressed form, + but not both. This boils down to the rule `delete the + output file if an error condition occurs, leaving the + input intact'. Input files are only deleted when we can + be pretty sure the output file has been written and + closed successfully. + + Errors are a dog because there's so many things to + deal with. The following can happen mid-file, and + require cleaning up. + + internal `panics' -- indicating a bug + corrupted or inconsistent compressed file + can't allocate enough memory to decompress this file + I/O error reading/writing/opening/closing + signal catches -- Control-C, SIGTERM, SIGHUP. + + Other conditions, primarily pertaining to file names, + can be checked in-between files, which makes dealing + with them easier. +--*/ + + + +/*---------------------------------------------------*/ +/*--- Misc (file handling) data decls ---*/ +/*---------------------------------------------------*/ + +UInt32 bytesIn, bytesOut; +Int32 verbosity; +Bool keepInputFiles, smallMode, testFailsExist; +UInt32 globalCrc; +Int32 numFileNames, numFilesProcessed; + + +/*-- source modes; F==file, I==stdin, O==stdout --*/ +#define SM_I2O 1 +#define SM_F2O 2 +#define SM_F2F 3 + +/*-- operation modes --*/ +#define OM_Z 1 +#define OM_UNZ 2 +#define OM_TEST 3 + +Int32 opMode; +Int32 srcMode; + + +Int32 longestFileName; +Char inName[1024]; +Char outName[1024]; +Char *progName; +Char progNameReally[1024]; +FILE *outputHandleJustInCase; + +void panic ( Char* ) NORETURN; +void ioError ( void ) NORETURN; +void compressOutOfMemory ( Int32, Int32 ) NORETURN; +void uncompressOutOfMemory ( Int32, Int32 ) NORETURN; +void blockOverrun ( void ) NORETURN; +void badBlockHeader ( void ) NORETURN; +void badBGLengths ( void ) NORETURN; +void crcError ( UInt32, UInt32 ) NORETURN; +void bitStreamEOF ( void ) NORETURN; +void cleanUpAndFail ( Int32 ) NORETURN; +void compressedStreamEOF ( void ) NORETURN; + +void* myMalloc ( Int32 ); + + + +/*---------------------------------------------------*/ +/*--- Data decls for the front end ---*/ +/*---------------------------------------------------*/ + +/*-- + The overshoot bytes allow us to avoid most of + the cost of pointer renormalisation during + comparison of rotations in sorting. + The figure of 20 is derived as follows: + qSort3 allows an overshoot of up to 10. + It then calls simpleSort, which calls + fullGtU, also with max overshoot 10. + fullGtU does up to 10 comparisons without + renormalising, giving 10+10 == 20. +--*/ +#define NUM_OVERSHOOT_BYTES 20 + +/*-- + These are the main data structures for + the Burrows-Wheeler transform. +--*/ + +/*-- + Pointers to compression and decompression + structures. Set by + allocateCompressStructures and + setDecompressStructureSizes + + The structures are always set to be suitable + for a block of size 100000 * blockSize100k. +--*/ +UChar *block; /*-- compress --*/ +UInt16 *quadrant; /*-- compress --*/ +Int32 *zptr; /*-- compress --*/ +UInt16 *szptr; /*-- overlays zptr ---*/ +Int32 *ftab; /*-- compress --*/ + +UInt16 *ll16; /*-- small decompress --*/ +UChar *ll4; /*-- small decompress --*/ + +Int32 *tt; /*-- fast decompress --*/ +UChar *ll8; /*-- fast decompress --*/ + + +/*-- + freq table collected to save a pass over the data + during decompression. +--*/ +Int32 unzftab[256]; + + +/*-- + index of the last char in the block, so + the block size == last + 1. +--*/ +Int32 last; + + +/*-- + index in zptr[] of original string after sorting. +--*/ +Int32 origPtr; + + +/*-- + always: in the range 0 .. 9. + The current block size is 100000 * this number. +--*/ +Int32 blockSize100k; + + +/*-- + Used when sorting. If too many long comparisons + happen, we stop sorting, randomise the block + slightly, and try again. +--*/ + +Int32 workFactor; +Int32 workDone; +Int32 workLimit; +Bool blockRandomised; +Bool firstAttempt; +Int32 nBlocksRandomised; + + + +/*---------------------------------------------------*/ +/*--- Data decls for the back end ---*/ +/*---------------------------------------------------*/ + +#define MAX_ALPHA_SIZE 258 +#define MAX_CODE_LEN 23 + +#define RUNA 0 +#define RUNB 1 + +#define N_GROUPS 6 +#define G_SIZE 50 +#define N_ITERS 4 + +#define MAX_SELECTORS (2 + (900000 / G_SIZE)) + +Bool inUse[256]; +Int32 nInUse; + +UChar seqToUnseq[256]; +UChar unseqToSeq[256]; + +UChar selector [MAX_SELECTORS]; +UChar selectorMtf[MAX_SELECTORS]; + +Int32 nMTF; + +Int32 mtfFreq[MAX_ALPHA_SIZE]; + +UChar len [N_GROUPS][MAX_ALPHA_SIZE]; + +/*-- decompress only --*/ +Int32 limit [N_GROUPS][MAX_ALPHA_SIZE]; +Int32 base [N_GROUPS][MAX_ALPHA_SIZE]; +Int32 perm [N_GROUPS][MAX_ALPHA_SIZE]; +Int32 minLens[N_GROUPS]; + +/*-- compress only --*/ +Int32 code [N_GROUPS][MAX_ALPHA_SIZE]; +Int32 rfreq[N_GROUPS][MAX_ALPHA_SIZE]; + + +/*---------------------------------------------------*/ +/*--- 32-bit CRC grunge ---*/ +/*---------------------------------------------------*/ + +/*-- + I think this is an implementation of the AUTODIN-II, + Ethernet & FDDI 32-bit CRC standard. Vaguely derived + from code by Rob Warnock, in Section 51 of the + comp.compression FAQ. +--*/ + +UInt32 crc32Table[256] = { + + /*-- Ugly, innit? --*/ + + 0x00000000L, 0x04c11db7L, 0x09823b6eL, 0x0d4326d9L, + 0x130476dcL, 0x17c56b6bL, 0x1a864db2L, 0x1e475005L, + 0x2608edb8L, 0x22c9f00fL, 0x2f8ad6d6L, 0x2b4bcb61L, + 0x350c9b64L, 0x31cd86d3L, 0x3c8ea00aL, 0x384fbdbdL, + 0x4c11db70L, 0x48d0c6c7L, 0x4593e01eL, 0x4152fda9L, + 0x5f15adacL, 0x5bd4b01bL, 0x569796c2L, 0x52568b75L, + 0x6a1936c8L, 0x6ed82b7fL, 0x639b0da6L, 0x675a1011L, + 0x791d4014L, 0x7ddc5da3L, 0x709f7b7aL, 0x745e66cdL, + 0x9823b6e0L, 0x9ce2ab57L, 0x91a18d8eL, 0x95609039L, + 0x8b27c03cL, 0x8fe6dd8bL, 0x82a5fb52L, 0x8664e6e5L, + 0xbe2b5b58L, 0xbaea46efL, 0xb7a96036L, 0xb3687d81L, + 0xad2f2d84L, 0xa9ee3033L, 0xa4ad16eaL, 0xa06c0b5dL, + 0xd4326d90L, 0xd0f37027L, 0xddb056feL, 0xd9714b49L, + 0xc7361b4cL, 0xc3f706fbL, 0xceb42022L, 0xca753d95L, + 0xf23a8028L, 0xf6fb9d9fL, 0xfbb8bb46L, 0xff79a6f1L, + 0xe13ef6f4L, 0xe5ffeb43L, 0xe8bccd9aL, 0xec7dd02dL, + 0x34867077L, 0x30476dc0L, 0x3d044b19L, 0x39c556aeL, + 0x278206abL, 0x23431b1cL, 0x2e003dc5L, 0x2ac12072L, + 0x128e9dcfL, 0x164f8078L, 0x1b0ca6a1L, 0x1fcdbb16L, + 0x018aeb13L, 0x054bf6a4L, 0x0808d07dL, 0x0cc9cdcaL, + 0x7897ab07L, 0x7c56b6b0L, 0x71159069L, 0x75d48ddeL, + 0x6b93dddbL, 0x6f52c06cL, 0x6211e6b5L, 0x66d0fb02L, + 0x5e9f46bfL, 0x5a5e5b08L, 0x571d7dd1L, 0x53dc6066L, + 0x4d9b3063L, 0x495a2dd4L, 0x44190b0dL, 0x40d816baL, + 0xaca5c697L, 0xa864db20L, 0xa527fdf9L, 0xa1e6e04eL, + 0xbfa1b04bL, 0xbb60adfcL, 0xb6238b25L, 0xb2e29692L, + 0x8aad2b2fL, 0x8e6c3698L, 0x832f1041L, 0x87ee0df6L, + 0x99a95df3L, 0x9d684044L, 0x902b669dL, 0x94ea7b2aL, + 0xe0b41de7L, 0xe4750050L, 0xe9362689L, 0xedf73b3eL, + 0xf3b06b3bL, 0xf771768cL, 0xfa325055L, 0xfef34de2L, + 0xc6bcf05fL, 0xc27dede8L, 0xcf3ecb31L, 0xcbffd686L, + 0xd5b88683L, 0xd1799b34L, 0xdc3abdedL, 0xd8fba05aL, + 0x690ce0eeL, 0x6dcdfd59L, 0x608edb80L, 0x644fc637L, + 0x7a089632L, 0x7ec98b85L, 0x738aad5cL, 0x774bb0ebL, + 0x4f040d56L, 0x4bc510e1L, 0x46863638L, 0x42472b8fL, + 0x5c007b8aL, 0x58c1663dL, 0x558240e4L, 0x51435d53L, + 0x251d3b9eL, 0x21dc2629L, 0x2c9f00f0L, 0x285e1d47L, + 0x36194d42L, 0x32d850f5L, 0x3f9b762cL, 0x3b5a6b9bL, + 0x0315d626L, 0x07d4cb91L, 0x0a97ed48L, 0x0e56f0ffL, + 0x1011a0faL, 0x14d0bd4dL, 0x19939b94L, 0x1d528623L, + 0xf12f560eL, 0xf5ee4bb9L, 0xf8ad6d60L, 0xfc6c70d7L, + 0xe22b20d2L, 0xe6ea3d65L, 0xeba91bbcL, 0xef68060bL, + 0xd727bbb6L, 0xd3e6a601L, 0xdea580d8L, 0xda649d6fL, + 0xc423cd6aL, 0xc0e2d0ddL, 0xcda1f604L, 0xc960ebb3L, + 0xbd3e8d7eL, 0xb9ff90c9L, 0xb4bcb610L, 0xb07daba7L, + 0xae3afba2L, 0xaafbe615L, 0xa7b8c0ccL, 0xa379dd7bL, + 0x9b3660c6L, 0x9ff77d71L, 0x92b45ba8L, 0x9675461fL, + 0x8832161aL, 0x8cf30badL, 0x81b02d74L, 0x857130c3L, + 0x5d8a9099L, 0x594b8d2eL, 0x5408abf7L, 0x50c9b640L, + 0x4e8ee645L, 0x4a4ffbf2L, 0x470cdd2bL, 0x43cdc09cL, + 0x7b827d21L, 0x7f436096L, 0x7200464fL, 0x76c15bf8L, + 0x68860bfdL, 0x6c47164aL, 0x61043093L, 0x65c52d24L, + 0x119b4be9L, 0x155a565eL, 0x18197087L, 0x1cd86d30L, + 0x029f3d35L, 0x065e2082L, 0x0b1d065bL, 0x0fdc1becL, + 0x3793a651L, 0x3352bbe6L, 0x3e119d3fL, 0x3ad08088L, + 0x2497d08dL, 0x2056cd3aL, 0x2d15ebe3L, 0x29d4f654L, + 0xc5a92679L, 0xc1683bceL, 0xcc2b1d17L, 0xc8ea00a0L, + 0xd6ad50a5L, 0xd26c4d12L, 0xdf2f6bcbL, 0xdbee767cL, + 0xe3a1cbc1L, 0xe760d676L, 0xea23f0afL, 0xeee2ed18L, + 0xf0a5bd1dL, 0xf464a0aaL, 0xf9278673L, 0xfde69bc4L, + 0x89b8fd09L, 0x8d79e0beL, 0x803ac667L, 0x84fbdbd0L, + 0x9abc8bd5L, 0x9e7d9662L, 0x933eb0bbL, 0x97ffad0cL, + 0xafb010b1L, 0xab710d06L, 0xa6322bdfL, 0xa2f33668L, + 0xbcb4666dL, 0xb8757bdaL, 0xb5365d03L, 0xb1f740b4L +}; + + +/*---------------------------------------------*/ +void initialiseCRC ( void ) +{ + globalCrc = 0xffffffffL; +} + + +/*---------------------------------------------*/ +UInt32 getFinalCRC ( void ) +{ + return ~globalCrc; +} + + +/*---------------------------------------------*/ +UInt32 getGlobalCRC ( void ) +{ + return globalCrc; +} + + +/*---------------------------------------------*/ +void setGlobalCRC ( UInt32 newCrc ) +{ + globalCrc = newCrc; +} + + +/*---------------------------------------------*/ +#define UPDATE_CRC(crcVar,cha) \ +{ \ + crcVar = (crcVar << 8) ^ \ + crc32Table[(crcVar >> 24) ^ \ + ((UChar)cha)]; \ +} + + +/*---------------------------------------------------*/ +/*--- Bit stream I/O ---*/ +/*---------------------------------------------------*/ + + +UInt32 bsBuff; +Int32 bsLive; +FILE* bsStream; +Bool bsWriting; + + +/*---------------------------------------------*/ +void bsSetStream ( FILE* f, Bool wr ) +{ + if (bsStream != NULL) panic ( "bsSetStream" ); + bsStream = f; + bsLive = 0; + bsBuff = 0; + bytesOut = 0; + bytesIn = 0; + bsWriting = wr; +} + + +/*---------------------------------------------*/ +void bsFinishedWithStream ( void ) +{ + if (bsWriting) + while (bsLive > 0) { + fputc ( (UChar)(bsBuff >> 24), bsStream ); + bsBuff <<= 8; + bsLive -= 8; + bytesOut++; + } + bsStream = NULL; +} + + +/*---------------------------------------------*/ +#define bsNEEDR(nz) \ +{ \ + while (bsLive < nz) { \ + Int32 zzi = fgetc ( bsStream ); \ + if (zzi == EOF) compressedStreamEOF(); \ + bsBuff = (bsBuff << 8) | (zzi & 0xffL); \ + bsLive += 8; \ + } \ +} + + +/*---------------------------------------------*/ +#define bsNEEDW(nz) \ +{ \ + while (bsLive >= 8) { \ + fputc ( (UChar)(bsBuff >> 24), \ + bsStream ); \ + bsBuff <<= 8; \ + bsLive -= 8; \ + bytesOut++; \ + } \ +} + + +/*---------------------------------------------*/ +#define bsR1(vz) \ +{ \ + bsNEEDR(1); \ + vz = (bsBuff >> (bsLive-1)) & 1; \ + bsLive--; \ +} + + +/*---------------------------------------------*/ +INLINE UInt32 bsR ( Int32 n ) +{ + UInt32 v; + bsNEEDR ( n ); + v = (bsBuff >> (bsLive-n)) & ((1 << n)-1); + bsLive -= n; + return v; +} + + +/*---------------------------------------------*/ +INLINE void bsW ( Int32 n, UInt32 v ) +{ + bsNEEDW ( n ); + bsBuff |= (v << (32 - bsLive - n)); + bsLive += n; +} + + +/*---------------------------------------------*/ +UChar bsGetUChar ( void ) +{ + return (UChar)bsR(8); +} + + +/*---------------------------------------------*/ +void bsPutUChar ( UChar c ) +{ + bsW(8, (UInt32)c ); +} + + +/*---------------------------------------------*/ +Int32 bsGetUInt32 ( void ) +{ + UInt32 u; + u = 0; + u = (u << 8) | bsR(8); + u = (u << 8) | bsR(8); + u = (u << 8) | bsR(8); + u = (u << 8) | bsR(8); + return u; +} + + +/*---------------------------------------------*/ +UInt32 bsGetIntVS ( UInt32 numBits ) +{ + return (UInt32)bsR(numBits); +} + + +/*---------------------------------------------*/ +UInt32 bsGetInt32 ( void ) +{ + return (Int32)bsGetUInt32(); +} + + +/*---------------------------------------------*/ +void bsPutUInt32 ( UInt32 u ) +{ + bsW ( 8, (u >> 24) & 0xffL ); + bsW ( 8, (u >> 16) & 0xffL ); + bsW ( 8, (u >> 8) & 0xffL ); + bsW ( 8, u & 0xffL ); +} + + +/*---------------------------------------------*/ +void bsPutInt32 ( Int32 c ) +{ + bsPutUInt32 ( (UInt32)c ); +} + + +/*---------------------------------------------*/ +void bsPutIntVS ( Int32 numBits, UInt32 c ) +{ + bsW ( numBits, c ); +} + + +/*---------------------------------------------------*/ +/*--- Huffman coding low-level stuff ---*/ +/*---------------------------------------------------*/ + +#define WEIGHTOF(zz0) ((zz0) & 0xffffff00) +#define DEPTHOF(zz1) ((zz1) & 0x000000ff) +#define MYMAX(zz2,zz3) ((zz2) > (zz3) ? (zz2) : (zz3)) + +#define ADDWEIGHTS(zw1,zw2) \ + (WEIGHTOF(zw1)+WEIGHTOF(zw2)) | \ + (1 + MYMAX(DEPTHOF(zw1),DEPTHOF(zw2))) + +#define UPHEAP(z) \ +{ \ + Int32 zz, tmp; \ + zz = z; tmp = heap[zz]; \ + while (weight[tmp] < weight[heap[zz >> 1]]) { \ + heap[zz] = heap[zz >> 1]; \ + zz >>= 1; \ + } \ + heap[zz] = tmp; \ +} + +#define DOWNHEAP(z) \ +{ \ + Int32 zz, yy, tmp; \ + zz = z; tmp = heap[zz]; \ + while (True) { \ + yy = zz << 1; \ + if (yy > nHeap) break; \ + if (yy < nHeap && \ + weight[heap[yy+1]] < weight[heap[yy]]) \ + yy++; \ + if (weight[tmp] < weight[heap[yy]]) break; \ + heap[zz] = heap[yy]; \ + zz = yy; \ + } \ + heap[zz] = tmp; \ +} + + +/*---------------------------------------------*/ +void hbMakeCodeLengths ( UChar *len, + Int32 *freq, + Int32 alphaSize, + Int32 maxLen ) +{ + /*-- + Nodes and heap entries run from 1. Entry 0 + for both the heap and nodes is a sentinel. + --*/ + Int32 nNodes, nHeap, n1, n2, i, j, k; + Bool tooLong; + + Int32 heap [ MAX_ALPHA_SIZE + 2 ]; + Int32 weight [ MAX_ALPHA_SIZE * 2 ]; + Int32 parent [ MAX_ALPHA_SIZE * 2 ]; + + for (i = 0; i < alphaSize; i++) + weight[i+1] = (freq[i] == 0 ? 1 : freq[i]) << 8; + + while (True) { + + nNodes = alphaSize; + nHeap = 0; + + heap[0] = 0; + weight[0] = 0; + parent[0] = -2; + + for (i = 1; i <= alphaSize; i++) { + parent[i] = -1; + nHeap++; + heap[nHeap] = i; + UPHEAP(nHeap); + } + if (!(nHeap < (MAX_ALPHA_SIZE+2))) + panic ( "hbMakeCodeLengths(1)" ); + + while (nHeap > 1) { + n1 = heap[1]; heap[1] = heap[nHeap]; nHeap--; DOWNHEAP(1); + n2 = heap[1]; heap[1] = heap[nHeap]; nHeap--; DOWNHEAP(1); + nNodes++; + parent[n1] = parent[n2] = nNodes; + weight[nNodes] = ADDWEIGHTS(weight[n1], weight[n2]); + parent[nNodes] = -1; + nHeap++; + heap[nHeap] = nNodes; + UPHEAP(nHeap); + } + if (!(nNodes < (MAX_ALPHA_SIZE * 2))) + panic ( "hbMakeCodeLengths(2)" ); + + tooLong = False; + for (i = 1; i <= alphaSize; i++) { + j = 0; + k = i; + while (parent[k] >= 0) { k = parent[k]; j++; } + len[i-1] = j; + if (j > maxLen) tooLong = True; + } + + if (! tooLong) break; + + for (i = 1; i < alphaSize; i++) { + j = weight[i] >> 8; + j = 1 + (j / 2); + weight[i] = j << 8; + } + } +} + + +/*---------------------------------------------*/ +void hbAssignCodes ( Int32 *code, + UChar *length, + Int32 minLen, + Int32 maxLen, + Int32 alphaSize ) +{ + Int32 n, vec, i; + + vec = 0; + for (n = minLen; n <= maxLen; n++) { + for (i = 0; i < alphaSize; i++) + if (length[i] == n) { code[i] = vec; vec++; }; + vec <<= 1; + } +} + + +/*---------------------------------------------*/ +void hbCreateDecodeTables ( Int32 *limit, + Int32 *base, + Int32 *perm, + UChar *length, + Int32 minLen, + Int32 maxLen, + Int32 alphaSize ) +{ + Int32 pp, i, j, vec; + + pp = 0; + for (i = minLen; i <= maxLen; i++) + for (j = 0; j < alphaSize; j++) + if (length[j] == i) { perm[pp] = j; pp++; }; + + for (i = 0; i < MAX_CODE_LEN; i++) base[i] = 0; + for (i = 0; i < alphaSize; i++) base[length[i]+1]++; + + for (i = 1; i < MAX_CODE_LEN; i++) base[i] += base[i-1]; + + for (i = 0; i < MAX_CODE_LEN; i++) limit[i] = 0; + vec = 0; + + for (i = minLen; i <= maxLen; i++) { + vec += (base[i+1] - base[i]); + limit[i] = vec-1; + vec <<= 1; + } + for (i = minLen + 1; i <= maxLen; i++) + base[i] = ((limit[i-1] + 1) << 1) - base[i]; +} + + + +/*---------------------------------------------------*/ +/*--- Undoing the reversible transformation ---*/ +/*---------------------------------------------------*/ + +/*---------------------------------------------*/ +#define SET_LL4(i,n) \ + { if (((i) & 0x1) == 0) \ + ll4[(i) >> 1] = (ll4[(i) >> 1] & 0xf0) | (n); else \ + ll4[(i) >> 1] = (ll4[(i) >> 1] & 0x0f) | ((n) << 4); \ + } + +#define GET_LL4(i) \ + (((UInt32)(ll4[(i) >> 1])) >> (((i) << 2) & 0x4) & 0xF) + +#define SET_LL(i,n) \ + { ll16[i] = (UInt16)(n & 0x0000ffff); \ + SET_LL4(i, n >> 16); \ + } + +#define GET_LL(i) \ + (((UInt32)ll16[i]) | (GET_LL4(i) << 16)) + + +/*---------------------------------------------*/ +/*-- + Manage memory for compression/decompression. + When compressing, a single block size applies to + all files processed, and that's set when the + program starts. But when decompressing, each file + processed could have been compressed with a + different block size, so we may have to free + and reallocate on a per-file basis. + + A call with argument of zero means + `free up everything.' And a value of zero for + blockSize100k means no memory is currently allocated. +--*/ + + +/*---------------------------------------------*/ +void allocateCompressStructures ( void ) +{ + Int32 n = 100000 * blockSize100k; + block = malloc ( (n + 1 + NUM_OVERSHOOT_BYTES) * sizeof(UChar) ); + quadrant = malloc ( (n + NUM_OVERSHOOT_BYTES) * sizeof(Int16) ); + zptr = malloc ( n * sizeof(Int32) ); + ftab = malloc ( 65537 * sizeof(Int32) ); + + if (block == NULL || quadrant == NULL || + zptr == NULL || ftab == NULL) { + Int32 totalDraw + = (n + 1 + NUM_OVERSHOOT_BYTES) * sizeof(UChar) + + (n + NUM_OVERSHOOT_BYTES) * sizeof(Int16) + + n * sizeof(Int32) + + 65537 * sizeof(Int32); + + compressOutOfMemory ( totalDraw, n ); + } + + /*-- + Since we want valid indexes for block of + -1 to n + NUM_OVERSHOOT_BYTES - 1 + inclusive. + --*/ + block++; + + /*-- + The back end needs a place to store the MTF values + whilst it calculates the coding tables. We could + put them in the zptr array. However, these values + will fit in a short, so we overlay szptr at the + start of zptr, in the hope of reducing the number + of cache misses induced by the multiple traversals + of the MTF values when calculating coding tables. + Seems to improve compression speed by about 1%. + --*/ + szptr = (UInt16*)zptr; +} + + +/*---------------------------------------------*/ +void setDecompressStructureSizes ( Int32 newSize100k ) +{ + if (! (0 <= newSize100k && newSize100k <= 9 && + 0 <= blockSize100k && blockSize100k <= 9)) + panic ( "setDecompressStructureSizes" ); + + if (newSize100k == blockSize100k) return; + + blockSize100k = newSize100k; + + if (ll16 != NULL) free ( ll16 ); + if (ll4 != NULL) free ( ll4 ); + if (ll8 != NULL) free ( ll8 ); + if (tt != NULL) free ( tt ); + + if (newSize100k == 0) return; + + if (smallMode) { + + Int32 n = 100000 * newSize100k; + ll16 = malloc ( n * sizeof(UInt16) ); + ll4 = malloc ( ((n+1) >> 1) * sizeof(UChar) ); + + if (ll4 == NULL || ll16 == NULL) { + Int32 totalDraw + = n * sizeof(Int16) + ((n+1) >> 1) * sizeof(UChar); + uncompressOutOfMemory ( totalDraw, n ); + } + + } else { + + Int32 n = 100000 * newSize100k; + ll8 = malloc ( n * sizeof(UChar) ); + tt = malloc ( n * sizeof(Int32) ); + + if (ll8 == NULL || tt == NULL) { + Int32 totalDraw + = n * sizeof(UChar) + n * sizeof(UInt32); + uncompressOutOfMemory ( totalDraw, n ); + } + + } +} + + + +/*---------------------------------------------------*/ +/*--- The new back end ---*/ +/*---------------------------------------------------*/ + +/*---------------------------------------------*/ +void makeMaps ( void ) +{ + Int32 i; + nInUse = 0; + for (i = 0; i < 256; i++) + if (inUse[i]) { + seqToUnseq[nInUse] = i; + unseqToSeq[i] = nInUse; + nInUse++; + } +} + + +/*---------------------------------------------*/ +void generateMTFValues ( void ) +{ + UChar yy[256]; + Int32 i, j; + UChar tmp; + UChar tmp2; + Int32 zPend; + Int32 wr; + Int32 EOB; + + makeMaps(); + EOB = nInUse+1; + + for (i = 0; i <= EOB; i++) mtfFreq[i] = 0; + + wr = 0; + zPend = 0; + for (i = 0; i < nInUse; i++) yy[i] = (UChar) i; + + + for (i = 0; i <= last; i++) { + UChar ll_i; + + #if DEBUG + assert (wr <= i); + #endif + + ll_i = unseqToSeq[block[zptr[i] - 1]]; + #if DEBUG + assert (ll_i < nInUse); + #endif + + j = 0; + tmp = yy[j]; + while ( ll_i != tmp ) { + j++; + tmp2 = tmp; + tmp = yy[j]; + yy[j] = tmp2; + }; + yy[0] = tmp; + + if (j == 0) { + zPend++; + } else { + if (zPend > 0) { + zPend--; + while (True) { + switch (zPend % 2) { + case 0: szptr[wr] = RUNA; wr++; mtfFreq[RUNA]++; break; + case 1: szptr[wr] = RUNB; wr++; mtfFreq[RUNB]++; break; + }; + if (zPend < 2) break; + zPend = (zPend - 2) / 2; + }; + zPend = 0; + } + szptr[wr] = j+1; wr++; mtfFreq[j+1]++; + } + } + + if (zPend > 0) { + zPend--; + while (True) { + switch (zPend % 2) { + case 0: szptr[wr] = RUNA; wr++; mtfFreq[RUNA]++; break; + case 1: szptr[wr] = RUNB; wr++; mtfFreq[RUNB]++; break; + }; + if (zPend < 2) break; + zPend = (zPend - 2) / 2; + }; + } + + szptr[wr] = EOB; wr++; mtfFreq[EOB]++; + + nMTF = wr; +} + + +/*---------------------------------------------*/ +#define LESSER_ICOST 0 +#define GREATER_ICOST 15 + +void sendMTFValues ( void ) +{ + Int32 v, t, i, j, gs, ge, totc, bt, bc, iter; + Int32 nSelectors, alphaSize, minLen, maxLen, selCtr; + Int32 nGroups, nBytes; + + /*-- + UChar len [N_GROUPS][MAX_ALPHA_SIZE]; + is a global since the decoder also needs it. + + Int32 code[N_GROUPS][MAX_ALPHA_SIZE]; + Int32 rfreq[N_GROUPS][MAX_ALPHA_SIZE]; + are also globals only used in this proc. + Made global to keep stack frame size small. + --*/ + + + UInt16 cost[N_GROUPS]; + Int32 fave[N_GROUPS]; + + if (verbosity >= 3) + fprintf ( stderr, + " %d in block, %d after MTF & 1-2 coding, %d+2 syms in use\n", + last+1, nMTF, nInUse ); + + alphaSize = nInUse+2; + for (t = 0; t < N_GROUPS; t++) + for (v = 0; v < alphaSize; v++) + len[t][v] = GREATER_ICOST; + + /*--- Decide how many coding tables to use ---*/ + if (nMTF <= 0) panic ( "sendMTFValues(0)" ); + if (nMTF < 200) nGroups = 2; else + if (nMTF < 800) nGroups = 4; else + nGroups = 6; + + /*--- Generate an initial set of coding tables ---*/ + { + Int32 nPart, remF, tFreq, aFreq; + + nPart = nGroups; + remF = nMTF; + gs = 0; + while (nPart > 0) { + tFreq = remF / nPart; + ge = gs-1; + aFreq = 0; + while (aFreq < tFreq && ge < alphaSize-1) { + ge++; + aFreq += mtfFreq[ge]; + } + + if (ge > gs + && nPart != nGroups && nPart != 1 + && ((nGroups-nPart) % 2 == 1)) { + aFreq -= mtfFreq[ge]; + ge--; + } + + if (verbosity >= 3) + fprintf ( stderr, + " initial group %d, [%d .. %d], has %d syms (%4.1f%%)\n", + nPart, gs, ge, aFreq, + (100.0 * (float)aFreq) / (float)nMTF ); + + for (v = 0; v < alphaSize; v++) + if (v >= gs && v <= ge) + len[nPart-1][v] = LESSER_ICOST; else + len[nPart-1][v] = GREATER_ICOST; + + nPart--; + gs = ge+1; + remF -= aFreq; + } + } + + /*--- + Iterate up to N_ITERS times to improve the tables. + ---*/ + for (iter = 0; iter < N_ITERS; iter++) { + + for (t = 0; t < nGroups; t++) fave[t] = 0; + + for (t = 0; t < nGroups; t++) + for (v = 0; v < alphaSize; v++) + rfreq[t][v] = 0; + + nSelectors = 0; + totc = 0; + gs = 0; + while (True) { + + /*--- Set group start & end marks. --*/ + if (gs >= nMTF) break; + ge = gs + G_SIZE - 1; + if (ge >= nMTF) ge = nMTF-1; + + /*-- + Calculate the cost of this group as coded + by each of the coding tables. + --*/ + for (t = 0; t < nGroups; t++) cost[t] = 0; + + if (nGroups == 6) { + register UInt16 cost0, cost1, cost2, cost3, cost4, cost5; + cost0 = cost1 = cost2 = cost3 = cost4 = cost5 = 0; + for (i = gs; i <= ge; i++) { + UInt16 icv = szptr[i]; + cost0 += len[0][icv]; + cost1 += len[1][icv]; + cost2 += len[2][icv]; + cost3 += len[3][icv]; + cost4 += len[4][icv]; + cost5 += len[5][icv]; + } + cost[0] = cost0; cost[1] = cost1; cost[2] = cost2; + cost[3] = cost3; cost[4] = cost4; cost[5] = cost5; + } else { + for (i = gs; i <= ge; i++) { + UInt16 icv = szptr[i]; + for (t = 0; t < nGroups; t++) cost[t] += len[t][icv]; + } + } + + /*-- + Find the coding table which is best for this group, + and record its identity in the selector table. + --*/ + bc = 999999999; bt = -1; + for (t = 0; t < nGroups; t++) + if (cost[t] < bc) { bc = cost[t]; bt = t; }; + totc += bc; + fave[bt]++; + selector[nSelectors] = bt; + nSelectors++; + + /*-- + Increment the symbol frequencies for the selected table. + --*/ + for (i = gs; i <= ge; i++) + rfreq[bt][ szptr[i] ]++; + + gs = ge+1; + } + if (verbosity >= 3) { + fprintf ( stderr, + " pass %d: size is %d, grp uses are ", + iter+1, totc/8 ); + for (t = 0; t < nGroups; t++) + fprintf ( stderr, "%d ", fave[t] ); + fprintf ( stderr, "\n" ); + } + + /*-- + Recompute the tables based on the accumulated frequencies. + --*/ + for (t = 0; t < nGroups; t++) + hbMakeCodeLengths ( &len[t][0], &rfreq[t][0], alphaSize, 20 ); + } + + + if (!(nGroups < 8)) panic ( "sendMTFValues(1)" ); + if (!(nSelectors < 32768 && + nSelectors <= (2 + (900000 / G_SIZE)))) + panic ( "sendMTFValues(2)" ); + + + /*--- Compute MTF values for the selectors. ---*/ + { + UChar pos[N_GROUPS], ll_i, tmp2, tmp; + for (i = 0; i < nGroups; i++) pos[i] = i; + for (i = 0; i < nSelectors; i++) { + ll_i = selector[i]; + j = 0; + tmp = pos[j]; + while ( ll_i != tmp ) { + j++; + tmp2 = tmp; + tmp = pos[j]; + pos[j] = tmp2; + }; + pos[0] = tmp; + selectorMtf[i] = j; + } + }; + + /*--- Assign actual codes for the tables. --*/ + for (t = 0; t < nGroups; t++) { + minLen = 32; + maxLen = 0; + for (i = 0; i < alphaSize; i++) { + if (len[t][i] > maxLen) maxLen = len[t][i]; + if (len[t][i] < minLen) minLen = len[t][i]; + } + if (maxLen > 20) panic ( "sendMTFValues(3)" ); + if (minLen < 1) panic ( "sendMTFValues(4)" ); + hbAssignCodes ( &code[t][0], &len[t][0], + minLen, maxLen, alphaSize ); + } + + /*--- Transmit the mapping table. ---*/ + { + Bool inUse16[16]; + for (i = 0; i < 16; i++) { + inUse16[i] = False; + for (j = 0; j < 16; j++) + if (inUse[i * 16 + j]) inUse16[i] = True; + } + + nBytes = bytesOut; + for (i = 0; i < 16; i++) + if (inUse16[i]) bsW(1,1); else bsW(1,0); + + for (i = 0; i < 16; i++) + if (inUse16[i]) + for (j = 0; j < 16; j++) + if (inUse[i * 16 + j]) bsW(1,1); else bsW(1,0); + + if (verbosity >= 3) + fprintf ( stderr, " bytes: mapping %d, ", bytesOut-nBytes ); + } + + /*--- Now the selectors. ---*/ + nBytes = bytesOut; + bsW ( 3, nGroups ); + bsW ( 15, nSelectors ); + for (i = 0; i < nSelectors; i++) { + for (j = 0; j < selectorMtf[i]; j++) bsW(1,1); + bsW(1,0); + } + if (verbosity >= 3) + fprintf ( stderr, "selectors %d, ", bytesOut-nBytes ); + + /*--- Now the coding tables. ---*/ + nBytes = bytesOut; + + for (t = 0; t < nGroups; t++) { + Int32 curr = len[t][0]; + bsW ( 5, curr ); + for (i = 0; i < alphaSize; i++) { + while (curr < len[t][i]) { bsW(2,2); curr++; /* 10 */ }; + while (curr > len[t][i]) { bsW(2,3); curr--; /* 11 */ }; + bsW ( 1, 0 ); + } + } + + if (verbosity >= 3) + fprintf ( stderr, "code lengths %d, ", bytesOut-nBytes ); + + /*--- And finally, the block data proper ---*/ + nBytes = bytesOut; + selCtr = 0; + gs = 0; + while (True) { + if (gs >= nMTF) break; + ge = gs + G_SIZE - 1; + if (ge >= nMTF) ge = nMTF-1; + for (i = gs; i <= ge; i++) { + #if DEBUG + assert (selector[selCtr] < nGroups); + #endif + bsW ( len [selector[selCtr]] [szptr[i]], + code [selector[selCtr]] [szptr[i]] ); + } + + gs = ge+1; + selCtr++; + } + if (!(selCtr == nSelectors)) panic ( "sendMTFValues(5)" ); + + if (verbosity >= 3) + fprintf ( stderr, "codes %d\n", bytesOut-nBytes ); +} + + +/*---------------------------------------------*/ +void moveToFrontCodeAndSend ( void ) +{ + bsPutIntVS ( 24, origPtr ); + generateMTFValues(); + sendMTFValues(); +} + + +/*---------------------------------------------*/ +void recvDecodingTables ( void ) +{ + Int32 i, j, t, nGroups, nSelectors, alphaSize; + Int32 minLen, maxLen; + Bool inUse16[16]; + + /*--- Receive the mapping table ---*/ + for (i = 0; i < 16; i++) + if (bsR(1) == 1) + inUse16[i] = True; else + inUse16[i] = False; + + for (i = 0; i < 256; i++) inUse[i] = False; + + for (i = 0; i < 16; i++) + if (inUse16[i]) + for (j = 0; j < 16; j++) + if (bsR(1) == 1) inUse[i * 16 + j] = True; + + makeMaps(); + alphaSize = nInUse+2; + + /*--- Now the selectors ---*/ + nGroups = bsR ( 3 ); + nSelectors = bsR ( 15 ); + for (i = 0; i < nSelectors; i++) { + j = 0; + while (bsR(1) == 1) j++; + selectorMtf[i] = j; + } + + /*--- Undo the MTF values for the selectors. ---*/ + { + UChar pos[N_GROUPS], tmp, v; + for (v = 0; v < nGroups; v++) pos[v] = v; + + for (i = 0; i < nSelectors; i++) { + v = selectorMtf[i]; + tmp = pos[v]; + while (v > 0) { pos[v] = pos[v-1]; v--; } + pos[0] = tmp; + selector[i] = tmp; + } + } + + /*--- Now the coding tables ---*/ + for (t = 0; t < nGroups; t++) { + Int32 curr = bsR ( 5 ); + for (i = 0; i < alphaSize; i++) { + while (bsR(1) == 1) { + if (bsR(1) == 0) curr++; else curr--; + } + len[t][i] = curr; + } + } + + /*--- Create the Huffman decoding tables ---*/ + for (t = 0; t < nGroups; t++) { + minLen = 32; + maxLen = 0; + for (i = 0; i < alphaSize; i++) { + if (len[t][i] > maxLen) maxLen = len[t][i]; + if (len[t][i] < minLen) minLen = len[t][i]; + } + hbCreateDecodeTables ( + &limit[t][0], &base[t][0], &perm[t][0], &len[t][0], + minLen, maxLen, alphaSize + ); + minLens[t] = minLen; + } +} + + +/*---------------------------------------------*/ +#define GET_MTF_VAL(lval) \ +{ \ + Int32 zt, zn, zvec, zj; \ + if (groupPos == 0) { \ + groupNo++; \ + groupPos = G_SIZE; \ + } \ + groupPos--; \ + zt = selector[groupNo]; \ + zn = minLens[zt]; \ + zvec = bsR ( zn ); \ + while (zvec > limit[zt][zn]) { \ + zn++; bsR1(zj); \ + zvec = (zvec << 1) | zj; \ + }; \ + lval = perm[zt][zvec - base[zt][zn]]; \ +} + + +/*---------------------------------------------*/ +void getAndMoveToFrontDecode ( void ) +{ + UChar yy[256]; + Int32 i, j, nextSym, limitLast; + Int32 EOB, groupNo, groupPos; + + limitLast = 100000 * blockSize100k; + origPtr = bsGetIntVS ( 24 ); + + recvDecodingTables(); + EOB = nInUse+1; + groupNo = -1; + groupPos = 0; + + /*-- + Setting up the unzftab entries here is not strictly + necessary, but it does save having to do it later + in a separate pass, and so saves a block's worth of + cache misses. + --*/ + for (i = 0; i <= 255; i++) unzftab[i] = 0; + + for (i = 0; i <= 255; i++) yy[i] = (UChar) i; + + last = -1; + + GET_MTF_VAL(nextSym); + + while (True) { + + if (nextSym == EOB) break; + + if (nextSym == RUNA || nextSym == RUNB) { + UChar ch; + Int32 s = -1; + Int32 N = 1; + do { + if (nextSym == RUNA) s = s + (0+1) * N; else + if (nextSym == RUNB) s = s + (1+1) * N; + N = N * 2; + GET_MTF_VAL(nextSym); + } + while (nextSym == RUNA || nextSym == RUNB); + + s++; + ch = seqToUnseq[yy[0]]; + unzftab[ch] += s; + + if (smallMode) + while (s > 0) { + last++; + ll16[last] = ch; + s--; + } + else + while (s > 0) { + last++; + ll8[last] = ch; + s--; + }; + + if (last >= limitLast) blockOverrun(); + continue; + + } else { + + UChar tmp; + last++; if (last >= limitLast) blockOverrun(); + + tmp = yy[nextSym-1]; + unzftab[seqToUnseq[tmp]]++; + if (smallMode) + ll16[last] = seqToUnseq[tmp]; else + ll8[last] = seqToUnseq[tmp]; + + /*-- + This loop is hammered during decompression, + hence the unrolling. + + for (j = nextSym-1; j > 0; j--) yy[j] = yy[j-1]; + --*/ + + j = nextSym-1; + for (; j > 3; j -= 4) { + yy[j] = yy[j-1]; + yy[j-1] = yy[j-2]; + yy[j-2] = yy[j-3]; + yy[j-3] = yy[j-4]; + } + for (; j > 0; j--) yy[j] = yy[j-1]; + + yy[0] = tmp; + GET_MTF_VAL(nextSym); + continue; + } + } +} + + +/*---------------------------------------------------*/ +/*--- Block-sorting machinery ---*/ +/*---------------------------------------------------*/ + +/*---------------------------------------------*/ +/*-- + Compare two strings in block. We assume (see + discussion above) that i1 and i2 have a max + offset of 10 on entry, and that the first + bytes of both block and quadrant have been + copied into the "overshoot area", ie + into the subscript range + [last+1 .. last+NUM_OVERSHOOT_BYTES]. +--*/ +INLINE Bool fullGtU ( Int32 i1, Int32 i2 ) +{ + Int32 k; + UChar c1, c2; + UInt16 s1, s2; + + #if DEBUG + /*-- + shellsort shouldn't ask to compare + something with itself. + --*/ + assert (i1 != i2); + #endif + + c1 = block[i1]; + c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + i1++; i2++; + + c1 = block[i1]; + c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + i1++; i2++; + + c1 = block[i1]; + c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + i1++; i2++; + + c1 = block[i1]; + c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + i1++; i2++; + + c1 = block[i1]; + c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + i1++; i2++; + + c1 = block[i1]; + c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + i1++; i2++; + + k = last + 1; + + do { + + c1 = block[i1]; + c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + s1 = quadrant[i1]; + s2 = quadrant[i2]; + if (s1 != s2) return (s1 > s2); + i1++; i2++; + + c1 = block[i1]; + c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + s1 = quadrant[i1]; + s2 = quadrant[i2]; + if (s1 != s2) return (s1 > s2); + i1++; i2++; + + c1 = block[i1]; + c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + s1 = quadrant[i1]; + s2 = quadrant[i2]; + if (s1 != s2) return (s1 > s2); + i1++; i2++; + + c1 = block[i1]; + c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + s1 = quadrant[i1]; + s2 = quadrant[i2]; + if (s1 != s2) return (s1 > s2); + i1++; i2++; + + if (i1 > last) { i1 -= last; i1--; }; + if (i2 > last) { i2 -= last; i2--; }; + + k -= 4; + workDone++; + } + while (k >= 0); + + return False; +} + +/*---------------------------------------------*/ +/*-- + Knuth's increments seem to work better + than Incerpi-Sedgewick here. Possibly + because the number of elems to sort is + usually small, typically <= 20. +--*/ +Int32 incs[14] = { 1, 4, 13, 40, 121, 364, 1093, 3280, + 9841, 29524, 88573, 265720, + 797161, 2391484 }; + +void simpleSort ( Int32 lo, Int32 hi, Int32 d ) +{ + Int32 i, j, h, bigN, hp; + Int32 v; + + bigN = hi - lo + 1; + if (bigN < 2) return; + + hp = 0; + while (incs[hp] < bigN) hp++; + hp--; + + for (; hp >= 0; hp--) { + h = incs[hp]; + if (verbosity >= 5) + fprintf ( stderr, " shell increment %d\n", h ); + + i = lo + h; + while (True) { + + /*-- copy 1 --*/ + if (i > hi) break; + v = zptr[i]; + j = i; + while ( fullGtU ( zptr[j-h]+d, v+d ) ) { + zptr[j] = zptr[j-h]; + j = j - h; + if (j <= (lo + h - 1)) break; + } + zptr[j] = v; + i++; + + /*-- copy 2 --*/ + if (i > hi) break; + v = zptr[i]; + j = i; + while ( fullGtU ( zptr[j-h]+d, v+d ) ) { + zptr[j] = zptr[j-h]; + j = j - h; + if (j <= (lo + h - 1)) break; + } + zptr[j] = v; + i++; + + /*-- copy 3 --*/ + if (i > hi) break; + v = zptr[i]; + j = i; + while ( fullGtU ( zptr[j-h]+d, v+d ) ) { + zptr[j] = zptr[j-h]; + j = j - h; + if (j <= (lo + h - 1)) break; + } + zptr[j] = v; + i++; + + if (workDone > workLimit && firstAttempt) return; + } + } +} + + +/*---------------------------------------------*/ +/*-- + The following is an implementation of + an elegant 3-way quicksort for strings, + described in a paper "Fast Algorithms for + Sorting and Searching Strings", by Robert + Sedgewick and Jon L. Bentley. +--*/ + +#define swap(lv1, lv2) \ + { Int32 tmp = lv1; lv1 = lv2; lv2 = tmp; } + +INLINE void vswap ( Int32 p1, Int32 p2, Int32 n ) +{ + while (n > 0) { + swap(zptr[p1], zptr[p2]); + p1++; p2++; n--; + } +} + +INLINE UChar med3 ( UChar a, UChar b, UChar c ) +{ + UChar t; + if (a > b) { t = a; a = b; b = t; }; + if (b > c) { t = b; b = c; c = t; }; + if (a > b) b = a; + return b; +} + + +#define min(a,b) ((a) < (b)) ? (a) : (b) + +typedef + struct { Int32 ll; Int32 hh; Int32 dd; } + StackElem; + +#define push(lz,hz,dz) { stack[sp].ll = lz; \ + stack[sp].hh = hz; \ + stack[sp].dd = dz; \ + sp++; } + +#define pop(lz,hz,dz) { sp--; \ + lz = stack[sp].ll; \ + hz = stack[sp].hh; \ + dz = stack[sp].dd; } + +#define SMALL_THRESH 20 +#define DEPTH_THRESH 10 + +/*-- + If you are ever unlucky/improbable enough + to get a stack overflow whilst sorting, + increase the following constant and try + again. In practice I have never seen the + stack go above 27 elems, so the following + limit seems very generous. +--*/ +#define QSORT_STACK_SIZE 1000 + + +void qSort3 ( Int32 loSt, Int32 hiSt, Int32 dSt ) +{ + Int32 unLo, unHi, ltLo, gtHi, med, n, m; + Int32 sp, lo, hi, d; + StackElem stack[QSORT_STACK_SIZE]; + + sp = 0; + push ( loSt, hiSt, dSt ); + + while (sp > 0) { + + if (sp >= QSORT_STACK_SIZE) panic ( "stack overflow in qSort3" ); + + pop ( lo, hi, d ); + + if (hi - lo < SMALL_THRESH || d > DEPTH_THRESH) { + simpleSort ( lo, hi, d ); + if (workDone > workLimit && firstAttempt) return; + continue; + } + + med = med3 ( block[zptr[ lo ]+d], + block[zptr[ hi ]+d], + block[zptr[ (lo+hi)>>1 ]+d] ); + + unLo = ltLo = lo; + unHi = gtHi = hi; + + while (True) { + while (True) { + if (unLo > unHi) break; + n = ((Int32)block[zptr[unLo]+d]) - med; + if (n == 0) { swap(zptr[unLo], zptr[ltLo]); ltLo++; unLo++; continue; }; + if (n > 0) break; + unLo++; + } + while (True) { + if (unLo > unHi) break; + n = ((Int32)block[zptr[unHi]+d]) - med; + if (n == 0) { swap(zptr[unHi], zptr[gtHi]); gtHi--; unHi--; continue; }; + if (n < 0) break; + unHi--; + } + if (unLo > unHi) break; + swap(zptr[unLo], zptr[unHi]); unLo++; unHi--; + } + #if DEBUG + assert (unHi == unLo-1); + #endif + + if (gtHi < ltLo) { + push(lo, hi, d+1 ); + continue; + } + + n = min(ltLo-lo, unLo-ltLo); vswap(lo, unLo-n, n); + m = min(hi-gtHi, gtHi-unHi); vswap(unLo, hi-m+1, m); + + n = lo + unLo - ltLo - 1; + m = hi - (gtHi - unHi) + 1; + + push ( lo, n, d ); + push ( n+1, m-1, d+1 ); + push ( m, hi, d ); + } +} + + +/*---------------------------------------------*/ + +#define BIGFREQ(b) (ftab[((b)+1) << 8] - ftab[(b) << 8]) + +#define SETMASK (1 << 21) +#define CLEARMASK (~(SETMASK)) + +void sortIt ( void ) +{ + Int32 i, j, ss, sb; + Int32 runningOrder[256]; + Int32 copy[256]; + Bool bigDone[256]; + UChar c1, c2; + Int32 numQSorted; + + /*-- + In the various block-sized structures, live data runs + from 0 to last+NUM_OVERSHOOT_BYTES inclusive. First, + set up the overshoot area for block. + --*/ + + if (verbosity >= 4) fprintf ( stderr, " sort initialise ...\n" ); + for (i = 0; i < NUM_OVERSHOOT_BYTES; i++) + block[last+i+1] = block[i % (last+1)]; + for (i = 0; i <= last+NUM_OVERSHOOT_BYTES; i++) + quadrant[i] = 0; + + block[-1] = block[last]; + + if (last < 4000) { + + /*-- + Use simpleSort(), since the full sorting mechanism + has quite a large constant overhead. + --*/ + if (verbosity >= 4) fprintf ( stderr, " simpleSort ...\n" ); + for (i = 0; i <= last; i++) zptr[i] = i; + firstAttempt = False; + workDone = workLimit = 0; + simpleSort ( 0, last, 0 ); + if (verbosity >= 4) fprintf ( stderr, " simpleSort done.\n" ); + + } else { + + numQSorted = 0; + for (i = 0; i <= 255; i++) bigDone[i] = False; + + if (verbosity >= 4) fprintf ( stderr, " bucket sorting ...\n" ); + + for (i = 0; i <= 65536; i++) ftab[i] = 0; + + c1 = block[-1]; + for (i = 0; i <= last; i++) { + c2 = block[i]; + ftab[(c1 << 8) + c2]++; + c1 = c2; + } + + for (i = 1; i <= 65536; i++) ftab[i] += ftab[i-1]; + + c1 = block[0]; + for (i = 0; i < last; i++) { + c2 = block[i+1]; + j = (c1 << 8) + c2; + c1 = c2; + ftab[j]--; + zptr[ftab[j]] = i; + } + j = (block[last] << 8) + block[0]; + ftab[j]--; + zptr[ftab[j]] = last; + + /*-- + Now ftab contains the first loc of every small bucket. + Calculate the running order, from smallest to largest + big bucket. + --*/ + + for (i = 0; i <= 255; i++) runningOrder[i] = i; + + { + Int32 vv; + Int32 h = 1; + do h = 3 * h + 1; while (h <= 256); + do { + h = h / 3; + for (i = h; i <= 255; i++) { + vv = runningOrder[i]; + j = i; + while ( BIGFREQ(runningOrder[j-h]) > BIGFREQ(vv) ) { + runningOrder[j] = runningOrder[j-h]; + j = j - h; + if (j <= (h - 1)) goto zero; + } + zero: + runningOrder[j] = vv; + } + } while (h != 1); + } + + /*-- + The main sorting loop. + --*/ + + for (i = 0; i <= 255; i++) { + + /*-- + Process big buckets, starting with the least full. + --*/ + ss = runningOrder[i]; + + /*-- + Complete the big bucket [ss] by quicksorting + any unsorted small buckets [ss, j]. Hopefully + previous pointer-scanning phases have already + completed many of the small buckets [ss, j], so + we don't have to sort them at all. + --*/ + for (j = 0; j <= 255; j++) { + sb = (ss << 8) + j; + if ( ! (ftab[sb] & SETMASK) ) { + Int32 lo = ftab[sb] & CLEARMASK; + Int32 hi = (ftab[sb+1] & CLEARMASK) - 1; + if (hi > lo) { + if (verbosity >= 4) + fprintf ( stderr, + " qsort [0x%x, 0x%x] done %d this %d\n", + ss, j, numQSorted, hi - lo + 1 ); + qSort3 ( lo, hi, 2 ); + numQSorted += ( hi - lo + 1 ); + if (workDone > workLimit && firstAttempt) return; + } + ftab[sb] |= SETMASK; + } + } + + /*-- + The ss big bucket is now done. Record this fact, + and update the quadrant descriptors. Remember to + update quadrants in the overshoot area too, if + necessary. The "if (i < 255)" test merely skips + this updating for the last bucket processed, since + updating for the last bucket is pointless. + --*/ + bigDone[ss] = True; + + if (i < 255) { + Int32 bbStart = ftab[ss << 8] & CLEARMASK; + Int32 bbSize = (ftab[(ss+1) << 8] & CLEARMASK) - bbStart; + Int32 shifts = 0; + + while ((bbSize >> shifts) > 65534) shifts++; + + for (j = 0; j < bbSize; j++) { + Int32 a2update = zptr[bbStart + j]; + UInt16 qVal = (UInt16)(j >> shifts); + quadrant[a2update] = qVal; + if (a2update < NUM_OVERSHOOT_BYTES) + quadrant[a2update + last + 1] = qVal; + } + + if (! ( ((bbSize-1) >> shifts) <= 65535 )) panic ( "sortIt" ); + } + + /*-- + Now scan this big bucket so as to synthesise the + sorted order for small buckets [t, ss] for all t != ss. + --*/ + for (j = 0; j <= 255; j++) + copy[j] = ftab[(j << 8) + ss] & CLEARMASK; + + for (j = ftab[ss << 8] & CLEARMASK; + j < (ftab[(ss+1) << 8] & CLEARMASK); + j++) { + c1 = block[zptr[j]-1]; + if ( ! bigDone[c1] ) { + zptr[copy[c1]] = zptr[j] == 0 ? last : zptr[j] - 1; + copy[c1] ++; + } + } + + for (j = 0; j <= 255; j++) ftab[(j << 8) + ss] |= SETMASK; + } + if (verbosity >= 4) + fprintf ( stderr, " %d pointers, %d sorted, %d scanned\n", + last+1, numQSorted, (last+1) - numQSorted ); + } +} + + +/*---------------------------------------------------*/ +/*--- Stuff for randomising repetitive blocks ---*/ +/*---------------------------------------------------*/ + +/*---------------------------------------------*/ +Int32 rNums[512] = { + 619, 720, 127, 481, 931, 816, 813, 233, 566, 247, + 985, 724, 205, 454, 863, 491, 741, 242, 949, 214, + 733, 859, 335, 708, 621, 574, 73, 654, 730, 472, + 419, 436, 278, 496, 867, 210, 399, 680, 480, 51, + 878, 465, 811, 169, 869, 675, 611, 697, 867, 561, + 862, 687, 507, 283, 482, 129, 807, 591, 733, 623, + 150, 238, 59, 379, 684, 877, 625, 169, 643, 105, + 170, 607, 520, 932, 727, 476, 693, 425, 174, 647, + 73, 122, 335, 530, 442, 853, 695, 249, 445, 515, + 909, 545, 703, 919, 874, 474, 882, 500, 594, 612, + 641, 801, 220, 162, 819, 984, 589, 513, 495, 799, + 161, 604, 958, 533, 221, 400, 386, 867, 600, 782, + 382, 596, 414, 171, 516, 375, 682, 485, 911, 276, + 98, 553, 163, 354, 666, 933, 424, 341, 533, 870, + 227, 730, 475, 186, 263, 647, 537, 686, 600, 224, + 469, 68, 770, 919, 190, 373, 294, 822, 808, 206, + 184, 943, 795, 384, 383, 461, 404, 758, 839, 887, + 715, 67, 618, 276, 204, 918, 873, 777, 604, 560, + 951, 160, 578, 722, 79, 804, 96, 409, 713, 940, + 652, 934, 970, 447, 318, 353, 859, 672, 112, 785, + 645, 863, 803, 350, 139, 93, 354, 99, 820, 908, + 609, 772, 154, 274, 580, 184, 79, 626, 630, 742, + 653, 282, 762, 623, 680, 81, 927, 626, 789, 125, + 411, 521, 938, 300, 821, 78, 343, 175, 128, 250, + 170, 774, 972, 275, 999, 639, 495, 78, 352, 126, + 857, 956, 358, 619, 580, 124, 737, 594, 701, 612, + 669, 112, 134, 694, 363, 992, 809, 743, 168, 974, + 944, 375, 748, 52, 600, 747, 642, 182, 862, 81, + 344, 805, 988, 739, 511, 655, 814, 334, 249, 515, + 897, 955, 664, 981, 649, 113, 974, 459, 893, 228, + 433, 837, 553, 268, 926, 240, 102, 654, 459, 51, + 686, 754, 806, 760, 493, 403, 415, 394, 687, 700, + 946, 670, 656, 610, 738, 392, 760, 799, 887, 653, + 978, 321, 576, 617, 626, 502, 894, 679, 243, 440, + 680, 879, 194, 572, 640, 724, 926, 56, 204, 700, + 707, 151, 457, 449, 797, 195, 791, 558, 945, 679, + 297, 59, 87, 824, 713, 663, 412, 693, 342, 606, + 134, 108, 571, 364, 631, 212, 174, 643, 304, 329, + 343, 97, 430, 751, 497, 314, 983, 374, 822, 928, + 140, 206, 73, 263, 980, 736, 876, 478, 430, 305, + 170, 514, 364, 692, 829, 82, 855, 953, 676, 246, + 369, 970, 294, 750, 807, 827, 150, 790, 288, 923, + 804, 378, 215, 828, 592, 281, 565, 555, 710, 82, + 896, 831, 547, 261, 524, 462, 293, 465, 502, 56, + 661, 821, 976, 991, 658, 869, 905, 758, 745, 193, + 768, 550, 608, 933, 378, 286, 215, 979, 792, 961, + 61, 688, 793, 644, 986, 403, 106, 366, 905, 644, + 372, 567, 466, 434, 645, 210, 389, 550, 919, 135, + 780, 773, 635, 389, 707, 100, 626, 958, 165, 504, + 920, 176, 193, 713, 857, 265, 203, 50, 668, 108, + 645, 990, 626, 197, 510, 357, 358, 850, 858, 364, + 936, 638 +}; + + +#define RAND_DECLS \ + Int32 rNToGo = 0; \ + Int32 rTPos = 0; \ + +#define RAND_MASK ((rNToGo == 1) ? 1 : 0) + +#define RAND_UPD_MASK \ + if (rNToGo == 0) { \ + rNToGo = rNums[rTPos]; \ + rTPos++; if (rTPos == 512) rTPos = 0; \ + } \ + rNToGo--; + + + +/*---------------------------------------------------*/ +/*--- The Reversible Transformation (tm) ---*/ +/*---------------------------------------------------*/ + +/*---------------------------------------------*/ +void randomiseBlock ( void ) +{ + Int32 i; + RAND_DECLS; + for (i = 0; i < 256; i++) inUse[i] = False; + + for (i = 0; i <= last; i++) { + RAND_UPD_MASK; + block[i] ^= RAND_MASK; + inUse[block[i]] = True; + } +} + + +/*---------------------------------------------*/ +void doReversibleTransformation ( void ) +{ + Int32 i; + + if (verbosity >= 2) fprintf ( stderr, "\n" ); + + workLimit = workFactor * last; + workDone = 0; + blockRandomised = False; + firstAttempt = True; + + sortIt (); + + if (verbosity >= 3) + fprintf ( stderr, " %d work, %d block, ratio %5.2f\n", + workDone, last, (float)workDone / (float)(last) ); + + if (workDone > workLimit && firstAttempt) { + if (verbosity >= 2) + fprintf ( stderr, " sorting aborted; randomising block\n" ); + randomiseBlock (); + workLimit = workDone = 0; + blockRandomised = True; + firstAttempt = False; + sortIt(); + if (verbosity >= 3) + fprintf ( stderr, " %d work, %d block, ratio %f\n", + workDone, last, (float)workDone / (float)(last) ); + } + + origPtr = -1; + for (i = 0; i <= last; i++) + if (zptr[i] == 0) + { origPtr = i; break; }; + + if (origPtr == -1) panic ( "doReversibleTransformation" ); +} + + +/*---------------------------------------------*/ + +INLINE Int32 indexIntoF ( Int32 indx, Int32 *cftab ) +{ + Int32 nb, na, mid; + nb = 0; + na = 256; + do { + mid = (nb + na) >> 1; + if (indx >= cftab[mid]) nb = mid; else na = mid; + } + while (na - nb != 1); + return nb; +} + + +#define GET_SMALL(cccc) \ + \ + cccc = indexIntoF ( tPos, cftab ); \ + tPos = GET_LL(tPos); + + +void undoReversibleTransformation_small ( FILE* dst ) +{ + Int32 cftab[257], cftabAlso[257]; + Int32 i, j, tmp, tPos; + UChar ch; + + /*-- + We assume here that the global array unzftab will + already be holding the frequency counts for + ll8[0 .. last]. + --*/ + + /*-- Set up cftab to facilitate generation of indexIntoF --*/ + cftab[0] = 0; + for (i = 1; i <= 256; i++) cftab[i] = unzftab[i-1]; + for (i = 1; i <= 256; i++) cftab[i] += cftab[i-1]; + + /*-- Make a copy of it, used in generation of T --*/ + for (i = 0; i <= 256; i++) cftabAlso[i] = cftab[i]; + + /*-- compute the T vector --*/ + for (i = 0; i <= last; i++) { + ch = (UChar)ll16[i]; + SET_LL(i, cftabAlso[ch]); + cftabAlso[ch]++; + } + + /*-- + Compute T^(-1) by pointer reversal on T. This is rather + subtle, in that, if the original block was two or more + (in general, N) concatenated copies of the same thing, + the T vector will consist of N cycles, each of length + blocksize / N, and decoding will involve traversing one + of these cycles N times. Which particular cycle doesn't + matter -- they are all equivalent. The tricky part is to + make sure that the pointer reversal creates a correct + reversed cycle for us to traverse. So, the code below + simply reverses whatever cycle origPtr happens to fall into, + without regard to the cycle length. That gives one reversed + cycle, which for normal blocks, is the entire block-size long. + For repeated blocks, it will be interspersed with the other + N-1 non-reversed cycles. Providing that the F-subscripting + phase which follows starts at origPtr, all then works ok. + --*/ + i = origPtr; + j = GET_LL(i); + do { + tmp = GET_LL(j); + SET_LL(j, i); + i = j; + j = tmp; + } + while (i != origPtr); + + /*-- + We recreate the original by subscripting F through T^(-1). + The run-length-decoder below requires characters incrementally, + so tPos is set to a starting value, and is updated by + the GET_SMALL macro. + --*/ + tPos = origPtr; + + /*-------------------------------------------------*/ + /*-- + This is pretty much a verbatim copy of the + run-length decoder present in the distribution + bzip-0.21; it has to be here to avoid creating + block[] as an intermediary structure. As in 0.21, + this code derives from some sent to me by + Christian von Roques. + + It allows dst==NULL, so as to support the test (-t) + option without slowing down the fast decompression + code. + --*/ + { + IntNative retVal; + Int32 i2, count, chPrev, ch2; + UInt32 localCrc; + + count = 0; + i2 = 0; + ch2 = 256; /*-- not a char and not EOF --*/ + localCrc = getGlobalCRC(); + + { + RAND_DECLS; + while ( i2 <= last ) { + chPrev = ch2; + GET_SMALL(ch2); + if (blockRandomised) { + RAND_UPD_MASK; + ch2 ^= (UInt32)RAND_MASK; + } + i2++; + + if (dst) + retVal = putc ( ch2, dst ); + + UPDATE_CRC ( localCrc, (UChar)ch2 ); + + if (ch2 != chPrev) { + count = 1; + } else { + count++; + if (count >= 4) { + Int32 j2; + UChar z; + GET_SMALL(z); + if (blockRandomised) { + RAND_UPD_MASK; + z ^= RAND_MASK; + } + for (j2 = 0; j2 < (Int32)z; j2++) { + if (dst) retVal = putc (ch2, dst); + UPDATE_CRC ( localCrc, (UChar)ch2 ); + } + i2++; + count = 0; + } + } + } + } + + setGlobalCRC ( localCrc ); + } + /*-- end of the in-line run-length-decoder. --*/ +} +#undef GET_SMALL + + +/*---------------------------------------------*/ + +#define GET_FAST(cccc) \ + \ + cccc = ll8[tPos]; \ + tPos = tt[tPos]; + + +void undoReversibleTransformation_fast ( FILE* dst ) +{ + Int32 cftab[257]; + Int32 i, tPos; + UChar ch; + + /*-- + We assume here that the global array unzftab will + already be holding the frequency counts for + ll8[0 .. last]. + --*/ + + /*-- Set up cftab to facilitate generation of T^(-1) --*/ + cftab[0] = 0; + for (i = 1; i <= 256; i++) cftab[i] = unzftab[i-1]; + for (i = 1; i <= 256; i++) cftab[i] += cftab[i-1]; + + /*-- compute the T^(-1) vector --*/ + for (i = 0; i <= last; i++) { + ch = (UChar)ll8[i]; + tt[cftab[ch]] = i; + cftab[ch]++; + } + + /*-- + We recreate the original by subscripting L through T^(-1). + The run-length-decoder below requires characters incrementally, + so tPos is set to a starting value, and is updated by + the GET_FAST macro. + --*/ + tPos = tt[origPtr]; + + /*-------------------------------------------------*/ + /*-- + This is pretty much a verbatim copy of the + run-length decoder present in the distribution + bzip-0.21; it has to be here to avoid creating + block[] as an intermediary structure. As in 0.21, + this code derives from some sent to me by + Christian von Roques. + --*/ + { + IntNative retVal; + Int32 i2, count, chPrev, ch2; + UInt32 localCrc; + + count = 0; + i2 = 0; + ch2 = 256; /*-- not a char and not EOF --*/ + localCrc = getGlobalCRC(); + + if (blockRandomised) { + RAND_DECLS; + while ( i2 <= last ) { + chPrev = ch2; + GET_FAST(ch2); + RAND_UPD_MASK; + ch2 ^= (UInt32)RAND_MASK; + i2++; + + retVal = putc ( ch2, dst ); + UPDATE_CRC ( localCrc, (UChar)ch2 ); + + if (ch2 != chPrev) { + count = 1; + } else { + count++; + if (count >= 4) { + Int32 j2; + UChar z; + GET_FAST(z); + RAND_UPD_MASK; + z ^= RAND_MASK; + for (j2 = 0; j2 < (Int32)z; j2++) { + retVal = putc (ch2, dst); + UPDATE_CRC ( localCrc, (UChar)ch2 ); + } + i2++; + count = 0; + } + } + } + + } else { + + while ( i2 <= last ) { + chPrev = ch2; + GET_FAST(ch2); + i2++; + + retVal = putc ( ch2, dst ); + UPDATE_CRC ( localCrc, (UChar)ch2 ); + + if (ch2 != chPrev) { + count = 1; + } else { + count++; + if (count >= 4) { + Int32 j2; + UChar z; + GET_FAST(z); + for (j2 = 0; j2 < (Int32)z; j2++) { + retVal = putc (ch2, dst); + UPDATE_CRC ( localCrc, (UChar)ch2 ); + } + i2++; + count = 0; + } + } + } + + } /*-- if (blockRandomised) --*/ + + setGlobalCRC ( localCrc ); + } + /*-- end of the in-line run-length-decoder. --*/ +} +#undef GET_FAST + + +/*---------------------------------------------------*/ +/*--- The block loader and RLEr ---*/ +/*---------------------------------------------------*/ + +/*---------------------------------------------*/ +/* Top 16: run length, 1 to 255. +* Lower 16: the char, or MY_EOF for EOF. +*/ + +#define MY_EOF 257 + +INLINE Int32 getRLEpair ( FILE* src ) +{ + Int32 runLength; + IntNative ch, chLatest; + + ch = getc ( src ); + + /*--- Because I have no idea what kind of a value EOF is. ---*/ + if (ch == EOF) { + ERROR_IF_NOT_ZERO ( errno ); + return (1 << 16) | MY_EOF; + } + + runLength = 0; + do { + chLatest = getc ( src ); + runLength++; + bytesIn++; + } + while (ch == chLatest && runLength < 255); + + if ( chLatest != EOF ) { + if ( ungetc ( chLatest, src ) == EOF ) + panic ( "getRLEpair: ungetc failed" ); + } else { + ERROR_IF_NOT_ZERO ( errno ); + } + + /*--- Conditional is just a speedup hack. ---*/ + if (runLength == 1) { + UPDATE_CRC ( globalCrc, (UChar)ch ); + return (1 << 16) | ch; + } else { + Int32 i; + for (i = 1; i <= runLength; i++) + UPDATE_CRC ( globalCrc, (UChar)ch ); + return (runLength << 16) | ch; + } +} + + +/*---------------------------------------------*/ +void loadAndRLEsource ( FILE* src ) +{ + Int32 ch, allowableBlockSize, i; + + last = -1; + ch = 0; + + for (i = 0; i < 256; i++) inUse[i] = False; + + /*--- 20 is just a paranoia constant ---*/ + allowableBlockSize = 100000 * blockSize100k - 20; + + while (last < allowableBlockSize && ch != MY_EOF) { + Int32 rlePair, runLen; + rlePair = getRLEpair ( src ); + ch = rlePair & 0xFFFF; + runLen = (UInt32)rlePair >> 16; + + #if DEBUG + assert (runLen >= 1 && runLen <= 255); + #endif + + if (ch != MY_EOF) { + inUse[ch] = True; + switch (runLen) { + case 1: + last++; block[last] = (UChar)ch; break; + case 2: + last++; block[last] = (UChar)ch; + last++; block[last] = (UChar)ch; break; + case 3: + last++; block[last] = (UChar)ch; + last++; block[last] = (UChar)ch; + last++; block[last] = (UChar)ch; break; + default: + inUse[runLen-4] = True; + last++; block[last] = (UChar)ch; + last++; block[last] = (UChar)ch; + last++; block[last] = (UChar)ch; + last++; block[last] = (UChar)ch; + last++; block[last] = (UChar)(runLen-4); break; + } + } + } +} + + +/*---------------------------------------------------*/ +/*--- Processing of complete files and streams ---*/ +/*---------------------------------------------------*/ + +/*---------------------------------------------*/ +void compressStream ( FILE *stream, FILE *zStream ) +{ + IntNative retVal; + UInt32 blockCRC, combinedCRC; + Int32 blockNo; + + blockNo = 0; + bytesIn = 0; + bytesOut = 0; + nBlocksRandomised = 0; + + SET_BINARY_MODE(stream); + SET_BINARY_MODE(zStream); + + ERROR_IF_NOT_ZERO ( ferror(stream) ); + ERROR_IF_NOT_ZERO ( ferror(zStream) ); + + bsSetStream ( zStream, True ); + + /*--- Write `magic' bytes B and Z, + then h indicating file-format == huffmanised, + followed by a digit indicating blockSize100k. + ---*/ + bsPutUChar ( 'B' ); + bsPutUChar ( 'Z' ); + bsPutUChar ( 'h' ); + bsPutUChar ( '0' + blockSize100k ); + + combinedCRC = 0; + + if (verbosity >= 2) fprintf ( stderr, "\n" ); + + while (True) { + + blockNo++; + initialiseCRC (); + loadAndRLEsource ( stream ); + ERROR_IF_NOT_ZERO ( ferror(stream) ); + if (last == -1) break; + + blockCRC = getFinalCRC (); + combinedCRC = (combinedCRC << 1) | (combinedCRC >> 31); + combinedCRC ^= blockCRC; + + if (verbosity >= 2) + fprintf ( stderr, " block %d: crc = 0x%8x, combined CRC = 0x%8x, size = %d", + blockNo, blockCRC, combinedCRC, last+1 ); + + /*-- sort the block and establish posn of original string --*/ + doReversibleTransformation (); + + /*-- + A 6-byte block header, the value chosen arbitrarily + as 0x314159265359 :-). A 32 bit value does not really + give a strong enough guarantee that the value will not + appear by chance in the compressed datastream. Worst-case + probability of this event, for a 900k block, is about + 2.0e-3 for 32 bits, 1.0e-5 for 40 bits and 4.0e-8 for 48 bits. + For a compressed file of size 100Gb -- about 100000 blocks -- + only a 48-bit marker will do. NB: normal compression/ + decompression do *not* rely on these statistical properties. + They are only important when trying to recover blocks from + damaged files. + --*/ + bsPutUChar ( 0x31 ); bsPutUChar ( 0x41 ); + bsPutUChar ( 0x59 ); bsPutUChar ( 0x26 ); + bsPutUChar ( 0x53 ); bsPutUChar ( 0x59 ); + + /*-- Now the block's CRC, so it is in a known place. --*/ + bsPutUInt32 ( blockCRC ); + + /*-- Now a single bit indicating randomisation. --*/ + if (blockRandomised) { + bsW(1,1); nBlocksRandomised++; + } else + bsW(1,0); + + /*-- Finally, block's contents proper. --*/ + moveToFrontCodeAndSend (); + + ERROR_IF_NOT_ZERO ( ferror(zStream) ); + } + + if (verbosity >= 2 && nBlocksRandomised > 0) + fprintf ( stderr, " %d block%s needed randomisation\n", + nBlocksRandomised, + nBlocksRandomised == 1 ? "" : "s" ); + + /*-- + Now another magic 48-bit number, 0x177245385090, to + indicate the end of the last block. (sqrt(pi), if + you want to know. I did want to use e, but it contains + too much repetition -- 27 18 28 18 28 46 -- for me + to feel statistically comfortable. Call me paranoid.) + --*/ + + bsPutUChar ( 0x17 ); bsPutUChar ( 0x72 ); + bsPutUChar ( 0x45 ); bsPutUChar ( 0x38 ); + bsPutUChar ( 0x50 ); bsPutUChar ( 0x90 ); + + bsPutUInt32 ( combinedCRC ); + if (verbosity >= 2) + fprintf ( stderr, " final combined CRC = 0x%x\n ", combinedCRC ); + + /*-- Close the files in an utterly paranoid way. --*/ + bsFinishedWithStream (); + + ERROR_IF_NOT_ZERO ( ferror(zStream) ); + retVal = fflush ( zStream ); + ERROR_IF_EOF ( retVal ); + retVal = fclose ( zStream ); + ERROR_IF_EOF ( retVal ); + + ERROR_IF_NOT_ZERO ( ferror(stream) ); + retVal = fclose ( stream ); + ERROR_IF_EOF ( retVal ); + + if (bytesIn == 0) bytesIn = 1; + if (bytesOut == 0) bytesOut = 1; + + if (verbosity >= 1) + fprintf ( stderr, "%6.3f:1, %6.3f bits/byte, " + "%5.2f%% saved, %d in, %d out.\n", + (float)bytesIn / (float)bytesOut, + (8.0 * (float)bytesOut) / (float)bytesIn, + 100.0 * (1.0 - (float)bytesOut / (float)bytesIn), + bytesIn, + bytesOut + ); +} + + +/*---------------------------------------------*/ +Bool uncompressStream ( FILE *zStream, FILE *stream ) +{ + UChar magic1, magic2, magic3, magic4; + UChar magic5, magic6; + UInt32 storedBlockCRC, storedCombinedCRC; + UInt32 computedBlockCRC, computedCombinedCRC; + Int32 currBlockNo; + IntNative retVal; + + SET_BINARY_MODE(stream); + SET_BINARY_MODE(zStream); + + ERROR_IF_NOT_ZERO ( ferror(stream) ); + ERROR_IF_NOT_ZERO ( ferror(zStream) ); + + bsSetStream ( zStream, False ); + + /*-- + A bad magic number is `recoverable from'; + return with False so the caller skips the file. + --*/ + magic1 = bsGetUChar (); + magic2 = bsGetUChar (); + magic3 = bsGetUChar (); + magic4 = bsGetUChar (); + if (magic1 != 'B' || + magic2 != 'Z' || + magic3 != 'h' || + magic4 < '1' || + magic4 > '9') { + bsFinishedWithStream(); + retVal = fclose ( stream ); + ERROR_IF_EOF ( retVal ); + return False; + } + + setDecompressStructureSizes ( magic4 - '0' ); + computedCombinedCRC = 0; + + if (verbosity >= 2) fprintf ( stderr, "\n " ); + currBlockNo = 0; + + while (True) { + magic1 = bsGetUChar (); + magic2 = bsGetUChar (); + magic3 = bsGetUChar (); + magic4 = bsGetUChar (); + magic5 = bsGetUChar (); + magic6 = bsGetUChar (); + if (magic1 == 0x17 && magic2 == 0x72 && + magic3 == 0x45 && magic4 == 0x38 && + magic5 == 0x50 && magic6 == 0x90) break; + + if (magic1 != 0x31 || magic2 != 0x41 || + magic3 != 0x59 || magic4 != 0x26 || + magic5 != 0x53 || magic6 != 0x59) badBlockHeader(); + + storedBlockCRC = bsGetUInt32 (); + + if (bsR(1) == 1) + blockRandomised = True; else + blockRandomised = False; + + currBlockNo++; + if (verbosity >= 2) + fprintf ( stderr, "[%d: huff+mtf ", currBlockNo ); + getAndMoveToFrontDecode (); + ERROR_IF_NOT_ZERO ( ferror(zStream) ); + + initialiseCRC(); + if (verbosity >= 2) fprintf ( stderr, "rt+rld" ); + if (smallMode) + undoReversibleTransformation_small ( stream ); + else + undoReversibleTransformation_fast ( stream ); + + ERROR_IF_NOT_ZERO ( ferror(stream) ); + + computedBlockCRC = getFinalCRC(); + if (verbosity >= 3) + fprintf ( stderr, " {0x%x, 0x%x}", storedBlockCRC, computedBlockCRC ); + if (verbosity >= 2) fprintf ( stderr, "] " ); + + /*-- A bad CRC is considered a fatal error. --*/ + if (storedBlockCRC != computedBlockCRC) + crcError ( storedBlockCRC, computedBlockCRC ); + + computedCombinedCRC = (computedCombinedCRC << 1) | (computedCombinedCRC >> 31); + computedCombinedCRC ^= computedBlockCRC; + }; + + if (verbosity >= 2) fprintf ( stderr, "\n " ); + + storedCombinedCRC = bsGetUInt32 (); + if (verbosity >= 2) + fprintf ( stderr, + "combined CRCs: stored = 0x%x, computed = 0x%x\n ", + storedCombinedCRC, computedCombinedCRC ); + if (storedCombinedCRC != computedCombinedCRC) + crcError ( storedCombinedCRC, computedCombinedCRC ); + + + bsFinishedWithStream (); + ERROR_IF_NOT_ZERO ( ferror(zStream) ); + retVal = fclose ( zStream ); + ERROR_IF_EOF ( retVal ); + + ERROR_IF_NOT_ZERO ( ferror(stream) ); + retVal = fflush ( stream ); + ERROR_IF_NOT_ZERO ( retVal ); + if (stream != stdout) { + retVal = fclose ( stream ); + ERROR_IF_EOF ( retVal ); + } + return True; +} + + +/*---------------------------------------------*/ +Bool testStream ( FILE *zStream ) +{ + UChar magic1, magic2, magic3, magic4; + UChar magic5, magic6; + UInt32 storedBlockCRC, storedCombinedCRC; + UInt32 computedBlockCRC, computedCombinedCRC; + Int32 currBlockNo; + IntNative retVal; + + SET_BINARY_MODE(zStream); + ERROR_IF_NOT_ZERO ( ferror(zStream) ); + + bsSetStream ( zStream, False ); + + magic1 = bsGetUChar (); + magic2 = bsGetUChar (); + magic3 = bsGetUChar (); + magic4 = bsGetUChar (); + if (magic1 != 'B' || + magic2 != 'Z' || + magic3 != 'h' || + magic4 < '1' || + magic4 > '9') { + bsFinishedWithStream(); + fclose ( zStream ); + fprintf ( stderr, "\n%s: bad magic number (ie, not created by bzip2)\n", + inName ); + return False; + } + + smallMode = True; + setDecompressStructureSizes ( magic4 - '0' ); + computedCombinedCRC = 0; + + if (verbosity >= 2) fprintf ( stderr, "\n" ); + currBlockNo = 0; + + while (True) { + magic1 = bsGetUChar (); + magic2 = bsGetUChar (); + magic3 = bsGetUChar (); + magic4 = bsGetUChar (); + magic5 = bsGetUChar (); + magic6 = bsGetUChar (); + if (magic1 == 0x17 && magic2 == 0x72 && + magic3 == 0x45 && magic4 == 0x38 && + magic5 == 0x50 && magic6 == 0x90) break; + + currBlockNo++; + if (magic1 != 0x31 || magic2 != 0x41 || + magic3 != 0x59 || magic4 != 0x26 || + magic5 != 0x53 || magic6 != 0x59) { + bsFinishedWithStream(); + fclose ( zStream ); + fprintf ( stderr, + "\n%s, block %d: bad header (not == 0x314159265359)\n", + inName, currBlockNo ); + return False; + } + storedBlockCRC = bsGetUInt32 (); + + if (bsR(1) == 1) + blockRandomised = True; else + blockRandomised = False; + + if (verbosity >= 2) + fprintf ( stderr, " block [%d: huff+mtf ", currBlockNo ); + getAndMoveToFrontDecode (); + ERROR_IF_NOT_ZERO ( ferror(zStream) ); + + initialiseCRC(); + if (verbosity >= 2) fprintf ( stderr, "rt+rld" ); + undoReversibleTransformation_small ( NULL ); + + computedBlockCRC = getFinalCRC(); + if (verbosity >= 3) + fprintf ( stderr, " {0x%x, 0x%x}", storedBlockCRC, computedBlockCRC ); + if (verbosity >= 2) fprintf ( stderr, "] " ); + + if (storedBlockCRC != computedBlockCRC) { + bsFinishedWithStream(); + fclose ( zStream ); + fprintf ( stderr, "\n%s, block %d: computed CRC does not match stored one\n", + inName, currBlockNo ); + return False; + } + + if (verbosity >= 2) fprintf ( stderr, "ok\n" ); + computedCombinedCRC = (computedCombinedCRC << 1) | (computedCombinedCRC >> 31); + computedCombinedCRC ^= computedBlockCRC; + }; + + storedCombinedCRC = bsGetUInt32 (); + if (verbosity >= 2) + fprintf ( stderr, + " combined CRCs: stored = 0x%x, computed = 0x%x\n ", + storedCombinedCRC, computedCombinedCRC ); + if (storedCombinedCRC != computedCombinedCRC) { + bsFinishedWithStream(); + fclose ( zStream ); + fprintf ( stderr, "\n%s: computed CRC does not match stored one\n", + inName ); + return False; + } + + bsFinishedWithStream (); + ERROR_IF_NOT_ZERO ( ferror(zStream) ); + retVal = fclose ( zStream ); + ERROR_IF_EOF ( retVal ); + return True; +} + + + +/*---------------------------------------------------*/ +/*--- Error [non-] handling grunge ---*/ +/*---------------------------------------------------*/ + +/*---------------------------------------------*/ +void cadvise ( void ) +{ + fprintf ( + stderr, + "\nIt is possible that the compressed file(s) have become corrupted.\n" + "You can use the -tvv option to test integrity of such files.\n\n" + "You can use the `bzip2recover' program to *attempt* to recover\n" + "data from undamaged sections of corrupted files.\n\n" + ); +} + + +/*---------------------------------------------*/ +void showFileNames ( void ) +{ + fprintf ( + stderr, + "\tInput file = %s, output file = %s\n", + inName==NULL ? "(null)" : inName, + outName==NULL ? "(null)" : outName + ); +} + + +/*---------------------------------------------*/ +void cleanUpAndFail ( Int32 ec ) +{ + IntNative retVal; + + if ( srcMode == SM_F2F && opMode != OM_TEST ) { + fprintf ( stderr, "%s: Deleting output file %s, if it exists.\n", + progName, + outName==NULL ? "(null)" : outName ); + if (outputHandleJustInCase != NULL) + fclose ( outputHandleJustInCase ); + retVal = remove ( outName ); + if (retVal != 0) + fprintf ( stderr, + "%s: WARNING: deletion of output file (apparently) failed.\n", + progName ); + } + if (numFileNames > 0 && numFilesProcessed < numFileNames) { + fprintf ( stderr, + "%s: WARNING: some files have not been processed:\n" + "\t%d specified on command line, %d not processed yet.\n\n", + progName, numFileNames, + numFileNames - numFilesProcessed ); + } + exit ( ec ); +} + + +/*---------------------------------------------*/ +void panic ( Char* s ) +{ + fprintf ( stderr, + "\n%s: PANIC -- internal consistency error:\n" + "\t%s\n" + "\tThis is a BUG. Please report it to me at:\n" + "\tjseward@acm.org\n", + progName, s ); + showFileNames(); + cleanUpAndFail( 3 ); +} + + +/*---------------------------------------------*/ +void badBGLengths ( void ) +{ + fprintf ( stderr, + "\n%s: error when reading background model code lengths,\n" + "\twhich probably means the compressed file is corrupted.\n", + progName ); + showFileNames(); + cadvise(); + cleanUpAndFail( 2 ); +} + + +/*---------------------------------------------*/ +void crcError ( UInt32 crcStored, UInt32 crcComputed ) +{ + fprintf ( stderr, + "\n%s: Data integrity error when decompressing.\n" + "\tStored CRC = 0x%x, computed CRC = 0x%x\n", + progName, crcStored, crcComputed ); + showFileNames(); + cadvise(); + cleanUpAndFail( 2 ); +} + + +/*---------------------------------------------*/ +void compressedStreamEOF ( void ) +{ + fprintf ( stderr, + "\n%s: Compressed file ends unexpectedly;\n\t" + "perhaps it is corrupted? *Possible* reason follows.\n", + progName ); + perror ( progName ); + showFileNames(); + cadvise(); + cleanUpAndFail( 2 ); +} + + +/*---------------------------------------------*/ +void ioError ( ) +{ + fprintf ( stderr, + "\n%s: I/O or other error, bailing out. Possible reason follows.\n", + progName ); + perror ( progName ); + showFileNames(); + cleanUpAndFail( 1 ); +} + + +/*---------------------------------------------*/ +void blockOverrun () +{ + fprintf ( stderr, + "\n%s: block overrun during decompression,\n" + "\twhich probably means the compressed file\n" + "\tis corrupted.\n", + progName ); + showFileNames(); + cadvise(); + cleanUpAndFail( 2 ); +} + + +/*---------------------------------------------*/ +void badBlockHeader () +{ + fprintf ( stderr, + "\n%s: bad block header in the compressed file,\n" + "\twhich probably means it is corrupted.\n", + progName ); + showFileNames(); + cadvise(); + cleanUpAndFail( 2 ); +} + + +/*---------------------------------------------*/ +void bitStreamEOF () +{ + fprintf ( stderr, + "\n%s: read past the end of compressed data,\n" + "\twhich probably means it is corrupted.\n", + progName ); + showFileNames(); + cadvise(); + cleanUpAndFail( 2 ); +} + + +/*---------------------------------------------*/ +void mySignalCatcher ( IntNative n ) +{ + fprintf ( stderr, + "\n%s: Control-C (or similar) caught, quitting.\n", + progName ); + cleanUpAndFail(1); +} + + +/*---------------------------------------------*/ +void mySIGSEGVorSIGBUScatcher ( IntNative n ) +{ + if (opMode == OM_Z) + fprintf ( stderr, + "\n%s: Caught a SIGSEGV or SIGBUS whilst compressing,\n" + "\twhich probably indicates a bug in bzip2. Please\n" + "\treport it to me at: jseward@acm.org\n", + progName ); + else + fprintf ( stderr, + "\n%s: Caught a SIGSEGV or SIGBUS whilst decompressing,\n" + "\twhich probably indicates that the compressed data\n" + "\tis corrupted.\n", + progName ); + + showFileNames(); + if (opMode == OM_Z) + cleanUpAndFail( 3 ); else + { cadvise(); cleanUpAndFail( 2 ); } +} + + +/*---------------------------------------------*/ +void uncompressOutOfMemory ( Int32 draw, Int32 blockSize ) +{ + fprintf ( stderr, + "\n%s: Can't allocate enough memory for decompression.\n" + "\tRequested %d bytes for a block size of %d.\n" + "\tTry selecting space-economic decompress (with flag -s)\n" + "\tand failing that, find a machine with more memory.\n", + progName, draw, blockSize ); + showFileNames(); + cleanUpAndFail(1); +} + + +/*---------------------------------------------*/ +void compressOutOfMemory ( Int32 draw, Int32 blockSize ) +{ + fprintf ( stderr, + "\n%s: Can't allocate enough memory for compression.\n" + "\tRequested %d bytes for a block size of %d.\n" + "\tTry selecting a small block size (with flag -s).\n", + progName, draw, blockSize ); + showFileNames(); + cleanUpAndFail(1); +} + + +/*---------------------------------------------------*/ +/*--- The main driver machinery ---*/ +/*---------------------------------------------------*/ + +/*---------------------------------------------*/ +void pad ( Char *s ) +{ + Int32 i; + if ( (Int32)strlen(s) >= longestFileName ) return; + for (i = 1; i <= longestFileName - (Int32)strlen(s); i++) + fprintf ( stderr, " " ); +} + + +/*---------------------------------------------*/ +Bool fileExists ( Char* name ) +{ + FILE *tmp = fopen ( name, "rb" ); + Bool exists = (tmp != NULL); + if (tmp != NULL) fclose ( tmp ); + return exists; +} + + +/*---------------------------------------------*/ +/*-- + if in doubt, return True +--*/ +Bool notABogStandardFile ( Char* name ) +{ + IntNative i; + struct MY_STAT statBuf; + + i = MY_LSTAT ( name, &statBuf ); + if (i != 0) return True; + if (MY_S_IFREG(statBuf.st_mode)) return False; + return True; +} + + +/*---------------------------------------------*/ +void copyDateAndPermissions ( Char *srcName, Char *dstName ) +{ + #if BZ_UNIX + IntNative retVal; + struct MY_STAT statBuf; + struct utimbuf uTimBuf; + + retVal = MY_LSTAT ( srcName, &statBuf ); + ERROR_IF_NOT_ZERO ( retVal ); + uTimBuf.actime = statBuf.st_atime; + uTimBuf.modtime = statBuf.st_mtime; + + retVal = chmod ( dstName, statBuf.st_mode ); + ERROR_IF_NOT_ZERO ( retVal ); + retVal = utime ( dstName, &uTimBuf ); + ERROR_IF_NOT_ZERO ( retVal ); + #endif +} + + +/*---------------------------------------------*/ +Bool endsInBz2 ( Char* name ) +{ + Int32 n = strlen ( name ); + if (n <= 4) return False; + return + (name[n-4] == '.' && + name[n-3] == 'b' && + name[n-2] == 'z' && + name[n-1] == '2'); +} + + +/*---------------------------------------------*/ +Bool containsDubiousChars ( Char* name ) +{ + Bool cdc = False; + for (; *name != '\0'; name++) + if (*name == '?' || *name == '*') cdc = True; + return cdc; +} + + +/*---------------------------------------------*/ +void compress ( Char *name ) +{ + FILE *inStr; + FILE *outStr; + + if (name == NULL && srcMode != SM_I2O) + panic ( "compress: bad modes\n" ); + + switch (srcMode) { + case SM_I2O: strcpy ( inName, "(stdin)" ); + strcpy ( outName, "(stdout)" ); break; + case SM_F2F: strcpy ( inName, name ); + strcpy ( outName, name ); + strcat ( outName, ".bz2" ); break; + case SM_F2O: strcpy ( inName, name ); + strcpy ( outName, "(stdout)" ); break; + } + + if ( srcMode != SM_I2O && containsDubiousChars ( inName ) ) { + fprintf ( stderr, "%s: There are no files matching `%s'.\n", + progName, inName ); + return; + } + if ( srcMode != SM_I2O && !fileExists ( inName ) ) { + fprintf ( stderr, "%s: Input file %s doesn't exist, skipping.\n", + progName, inName ); + return; + } + if ( srcMode != SM_I2O && endsInBz2 ( inName )) { + fprintf ( stderr, "%s: Input file name %s ends in `.bz2', skipping.\n", + progName, inName ); + return; + } + if ( srcMode != SM_I2O && notABogStandardFile ( inName )) { + fprintf ( stderr, "%s: Input file %s is not a normal file, skipping.\n", + progName, inName ); + return; + } + if ( srcMode == SM_F2F && fileExists ( outName ) ) { + fprintf ( stderr, "%s: Output file %s already exists, skipping.\n", + progName, outName ); + return; + } + + switch ( srcMode ) { + + case SM_I2O: + inStr = stdin; + outStr = stdout; + if ( isatty ( fileno ( stdout ) ) ) { + fprintf ( stderr, + "%s: I won't write compressed data to a terminal.\n", + progName ); + fprintf ( stderr, "%s: For help, type: `%s --help'.\n", + progName, progName ); + return; + }; + break; + + case SM_F2O: + inStr = fopen ( inName, "rb" ); + outStr = stdout; + if ( isatty ( fileno ( stdout ) ) ) { + fprintf ( stderr, + "%s: I won't write compressed data to a terminal.\n", + progName ); + fprintf ( stderr, "%s: For help, type: `%s --help'.\n", + progName, progName ); + return; + }; + if ( inStr == NULL ) { + fprintf ( stderr, "%s: Can't open input file %s, skipping.\n", + progName, inName ); + return; + }; + break; + + case SM_F2F: + inStr = fopen ( inName, "rb" ); + outStr = fopen ( outName, "wb" ); + if ( outStr == NULL) { + fprintf ( stderr, "%s: Can't create output file %s, skipping.\n", + progName, outName ); + return; + } + if ( inStr == NULL ) { + fprintf ( stderr, "%s: Can't open input file %s, skipping.\n", + progName, inName ); + return; + }; + break; + + default: + panic ( "compress: bad srcMode" ); + break; + } + + if (verbosity >= 1) { + fprintf ( stderr, " %s: ", inName ); + pad ( inName ); + fflush ( stderr ); + } + + /*--- Now the input and output handles are sane. Do the Biz. ---*/ + errno = 0; + outputHandleJustInCase = outStr; + compressStream ( inStr, outStr ); + outputHandleJustInCase = NULL; + + /*--- If there was an I/O error, we won't get here. ---*/ + if ( srcMode == SM_F2F ) { + copyDateAndPermissions ( inName, outName ); + if ( !keepInputFiles ) { + IntNative retVal = remove ( inName ); + ERROR_IF_NOT_ZERO ( retVal ); + } + } +} + + +/*---------------------------------------------*/ +void uncompress ( Char *name ) +{ + FILE *inStr; + FILE *outStr; + Bool magicNumberOK; + + if (name == NULL && srcMode != SM_I2O) + panic ( "uncompress: bad modes\n" ); + + switch (srcMode) { + case SM_I2O: strcpy ( inName, "(stdin)" ); + strcpy ( outName, "(stdout)" ); break; + case SM_F2F: strcpy ( inName, name ); + strcpy ( outName, name ); + if (endsInBz2 ( outName )) + outName [ strlen ( outName ) - 4 ] = '\0'; + break; + case SM_F2O: strcpy ( inName, name ); + strcpy ( outName, "(stdout)" ); break; + } + + if ( srcMode != SM_I2O && containsDubiousChars ( inName ) ) { + fprintf ( stderr, "%s: There are no files matching `%s'.\n", + progName, inName ); + return; + } + if ( srcMode != SM_I2O && !fileExists ( inName ) ) { + fprintf ( stderr, "%s: Input file %s doesn't exist, skipping.\n", + progName, inName ); + return; + } + if ( srcMode != SM_I2O && !endsInBz2 ( inName )) { + fprintf ( stderr, + "%s: Input file name %s doesn't end in `.bz2', skipping.\n", + progName, inName ); + return; + } + if ( srcMode != SM_I2O && notABogStandardFile ( inName )) { + fprintf ( stderr, "%s: Input file %s is not a normal file, skipping.\n", + progName, inName ); + return; + } + if ( srcMode == SM_F2F && fileExists ( outName ) ) { + fprintf ( stderr, "%s: Output file %s already exists, skipping.\n", + progName, outName ); + return; + } + + switch ( srcMode ) { + + case SM_I2O: + inStr = stdin; + outStr = stdout; + if ( isatty ( fileno ( stdin ) ) ) { + fprintf ( stderr, + "%s: I won't read compressed data from a terminal.\n", + progName ); + fprintf ( stderr, "%s: For help, type: `%s --help'.\n", + progName, progName ); + return; + }; + break; + + case SM_F2O: + inStr = fopen ( inName, "rb" ); + outStr = stdout; + if ( inStr == NULL ) { + fprintf ( stderr, "%s: Can't open input file %s, skipping.\n", + progName, inName ); + return; + }; + break; + + case SM_F2F: + inStr = fopen ( inName, "rb" ); + outStr = fopen ( outName, "wb" ); + if ( outStr == NULL) { + fprintf ( stderr, "%s: Can't create output file %s, skipping.\n", + progName, outName ); + return; + } + if ( inStr == NULL ) { + fprintf ( stderr, "%s: Can't open input file %s, skipping.\n", + progName, inName ); + return; + }; + break; + + default: + panic ( "uncompress: bad srcMode" ); + break; + } + + if (verbosity >= 1) { + fprintf ( stderr, " %s: ", inName ); + pad ( inName ); + fflush ( stderr ); + } + + /*--- Now the input and output handles are sane. Do the Biz. ---*/ + errno = 0; + outputHandleJustInCase = outStr; + magicNumberOK = uncompressStream ( inStr, outStr ); + outputHandleJustInCase = NULL; + + /*--- If there was an I/O error, we won't get here. ---*/ + if ( magicNumberOK ) { + if ( srcMode == SM_F2F ) { + copyDateAndPermissions ( inName, outName ); + if ( !keepInputFiles ) { + IntNative retVal = remove ( inName ); + ERROR_IF_NOT_ZERO ( retVal ); + } + } + } else { + if ( srcMode == SM_F2F ) { + IntNative retVal = remove ( outName ); + ERROR_IF_NOT_ZERO ( retVal ); + } + } + + if ( magicNumberOK ) { + if (verbosity >= 1) + fprintf ( stderr, "done\n" ); + } else { + if (verbosity >= 1) + fprintf ( stderr, "not a bzip2 file, skipping.\n" ); else + fprintf ( stderr, + "%s: %s is not a bzip2 file, skipping.\n", + progName, inName ); + } + +} + + +/*---------------------------------------------*/ +void testf ( Char *name ) +{ + FILE *inStr; + Bool allOK; + + if (name == NULL && srcMode != SM_I2O) + panic ( "testf: bad modes\n" ); + + strcpy ( outName, "(none)" ); + switch (srcMode) { + case SM_I2O: strcpy ( inName, "(stdin)" ); break; + case SM_F2F: strcpy ( inName, name ); break; + case SM_F2O: strcpy ( inName, name ); break; + } + + if ( srcMode != SM_I2O && containsDubiousChars ( inName ) ) { + fprintf ( stderr, "%s: There are no files matching `%s'.\n", + progName, inName ); + return; + } + if ( srcMode != SM_I2O && !fileExists ( inName ) ) { + fprintf ( stderr, "%s: Input file %s doesn't exist, skipping.\n", + progName, inName ); + return; + } + if ( srcMode != SM_I2O && !endsInBz2 ( inName )) { + fprintf ( stderr, + "%s: Input file name %s doesn't end in `.bz2', skipping.\n", + progName, inName ); + return; + } + if ( srcMode != SM_I2O && notABogStandardFile ( inName )) { + fprintf ( stderr, "%s: Input file %s is not a normal file, skipping.\n", + progName, inName ); + return; + } + + switch ( srcMode ) { + + case SM_I2O: + if ( isatty ( fileno ( stdin ) ) ) { + fprintf ( stderr, + "%s: I won't read compressed data from a terminal.\n", + progName ); + fprintf ( stderr, "%s: For help, type: `%s --help'.\n", + progName, progName ); + return; + }; + inStr = stdin; + break; + + case SM_F2O: case SM_F2F: + inStr = fopen ( inName, "rb" ); + if ( inStr == NULL ) { + fprintf ( stderr, "%s: Can't open input file %s, skipping.\n", + progName, inName ); + return; + }; + break; + + default: + panic ( "testf: bad srcMode" ); + break; + } + + if (verbosity >= 1) { + fprintf ( stderr, " %s: ", inName ); + pad ( inName ); + fflush ( stderr ); + } + + /*--- Now the input handle is sane. Do the Biz. ---*/ + errno = 0; + allOK = testStream ( inStr ); + + if (allOK && verbosity >= 1) fprintf ( stderr, "ok\n" ); + if (!allOK) testFailsExist = True; +} + + +/*---------------------------------------------*/ +void license ( void ) +{ + fprintf ( stderr, + + "bzip2, a block-sorting file compressor. " + "Version 0.1pl0, 17-Aug-97.\n" + " \n" + " Copyright (C) 1996, 1997 by Julian Seward.\n" + " \n" + " This program is free software; you can redistribute it and/or modify\n" + " it under the terms of the GNU General Public License as published by\n" + " the Free Software Foundation; either version 2 of the License, or\n" + " (at your option) any later version.\n" + " \n" + " This program is distributed in the hope that it will be useful,\n" + " but WITHOUT ANY WARRANTY; without even the implied warranty of\n" + " MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\n" + " GNU General Public License for more details.\n" + " \n" + " You should have received a copy of the GNU General Public License\n" + " along with this program; if not, write to the Free Software\n" + " Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.\n" + " \n" + " The GNU General Public License is contained in the file LICENSE.\n" + " \n" + ); +} + + +/*---------------------------------------------*/ +void usage ( Char *fullProgName ) +{ + fprintf ( + stderr, + "bzip2, a block-sorting file compressor. " + "Version 0.1pl0, 17-Aug-97.\n" + "\n usage: %s [flags and input files in any order]\n" + "\n" + " -h --help print this message\n" + " -d --decompress force decompression\n" + " -f --compress force compression\n" + " -t --test test compressed file integrity\n" + " -c --stdout output to standard out\n" + " -v --verbose be verbose (a 2nd -v gives more)\n" + " -k --keep keep (don't delete) input files\n" + " -L --license display software version & license\n" + " -V --version display software version & license\n" + " -s --small use less memory (at most 2500k)\n" + " -1 .. -9 set block size to 100k .. 900k\n" + " --repetitive-fast compress repetitive blocks faster\n" + " --repetitive-best compress repetitive blocks better\n" + "\n" + " If invoked as `bzip2', the default action is to compress.\n" + " as `bunzip2', the default action is to decompress.\n" + "\n" + " If no file names are given, bzip2 compresses or decompresses\n" + " from standard input to standard output. You can combine\n" + " flags, so `-v -4' means the same as -v4 or -4v, &c.\n" + #if BZ_UNIX + "\n" + #endif + , + + fullProgName + ); +} + + +/*---------------------------------------------*/ +/*-- + All the garbage from here to main() is purely to + implement a linked list of command-line arguments, + into which main() copies argv[1 .. argc-1]. + + The purpose of this ridiculous exercise is to + facilitate the expansion of wildcard characters + * and ? in filenames for halfwitted OSs like + MSDOS, Windows 95 and NT. + + The actual Dirty Work is done by the platform-specific + macro APPEND_FILESPEC. +--*/ + +typedef + struct zzzz { + Char *name; + struct zzzz *link; + } + Cell; + + +/*---------------------------------------------*/ +void *myMalloc ( Int32 n ) +{ + void* p; + + p = malloc ( (size_t)n ); + if (p == NULL) { + fprintf ( + stderr, + "%s: `malloc' failed on request for %d bytes.\n", + progName, n + ); + exit ( 1 ); + } + return p; +} + + +/*---------------------------------------------*/ +Cell *mkCell ( void ) +{ + Cell *c; + + c = (Cell*) myMalloc ( sizeof ( Cell ) ); + c->name = NULL; + c->link = NULL; + return c; +} + + +/*---------------------------------------------*/ +Cell *snocString ( Cell *root, Char *name ) +{ + if (root == NULL) { + Cell *tmp = mkCell(); + tmp->name = (Char*) myMalloc ( 5 + strlen(name) ); + strcpy ( tmp->name, name ); + return tmp; + } else { + Cell *tmp = root; + while (tmp->link != NULL) tmp = tmp->link; + tmp->link = snocString ( tmp->link, name ); + return root; + } +} + + + +/*---------------------------------------------*/ +#define ISFLAG(s) (strcmp(aa->name, (s))==0) + + +IntNative main ( IntNative argc, Char *argv[] ) +{ + Int32 i, j; + Char *tmp; + Cell *argList; + Cell *aa; + + + #if DEBUG + fprintf ( stderr, "bzip2: *** compiled with debugging ON ***\n" ); + #endif + + /*-- Be really really really paranoid :-) --*/ + if (sizeof(Int32) != 4 || sizeof(UInt32) != 4 || + sizeof(Int16) != 2 || sizeof(UInt16) != 2 || + sizeof(Char) != 1 || sizeof(UChar) != 1) { + fprintf ( stderr, + "bzip2: I'm not configured correctly for this platform!\n" + "\tI require Int32, Int16 and Char to have sizes\n" + "\tof 4, 2 and 1 bytes to run properly, and they don't.\n" + "\tProbably you can fix this by defining them correctly,\n" + "\tand recompiling. Bye!\n" ); + exit(1); + } + + + /*-- Set up signal handlers --*/ + signal (SIGINT, mySignalCatcher); + signal (SIGTERM, mySignalCatcher); + signal (SIGSEGV, mySIGSEGVorSIGBUScatcher); + #if BZ_UNIX + signal (SIGHUP, mySignalCatcher); + signal (SIGBUS, mySIGSEGVorSIGBUScatcher); + #endif + + + /*-- Initialise --*/ + outputHandleJustInCase = NULL; + ftab = NULL; + ll4 = NULL; + ll16 = NULL; + ll8 = NULL; + tt = NULL; + block = NULL; + zptr = NULL; + errno = 0; + smallMode = False; + keepInputFiles = False; + verbosity = 0; + blockSize100k = 9; + testFailsExist = False; + bsStream = NULL; + numFileNames = 0; + numFilesProcessed = 0; + workFactor = 30; + + strcpy ( inName, "(none)" ); + strcpy ( outName, "(none)" ); + + strcpy ( progNameReally, argv[0] ); + progName = &progNameReally[0]; + for (tmp = &progNameReally[0]; *tmp != '\0'; tmp++) + if (*tmp == PATH_SEP) progName = tmp + 1; + + + /*-- Expand filename wildcards in arg list --*/ + argList = NULL; + for (i = 1; i <= argc-1; i++) + APPEND_FILESPEC(argList, argv[i]); + + + /*-- Find the length of the longest filename --*/ + longestFileName = 7; + numFileNames = 0; + for (aa = argList; aa != NULL; aa = aa->link) + if (aa->name[0] != '-') { + numFileNames++; + if (longestFileName < (Int32)strlen(aa->name) ) + longestFileName = (Int32)strlen(aa->name); + } + + + /*-- Determine what to do (compress/uncompress/test). --*/ + /*-- Note that subsequent flag handling may change this. --*/ + opMode = OM_Z; + + if ( (strcmp ( "bunzip2", progName ) == 0) || + (strcmp ( "BUNZIP2", progName ) == 0) || + (strcmp ( "bunzip2.exe", progName ) == 0) || + (strcmp ( "BUNZIP2.EXE", progName ) == 0) ) + opMode = OM_UNZ; + + + /*-- Determine source modes; flag handling may change this too. --*/ + if (numFileNames == 0) + srcMode = SM_I2O; else srcMode = SM_F2F; + + + /*-- Look at the flags. --*/ + for (aa = argList; aa != NULL; aa = aa->link) + if (aa->name[0] == '-' && aa->name[1] != '-') + for (j = 1; aa->name[j] != '\0'; j++) + switch (aa->name[j]) { + case 'c': srcMode = SM_F2O; break; + case 'd': opMode = OM_UNZ; break; + case 'f': opMode = OM_Z; break; + case 't': opMode = OM_TEST; break; + case 'k': keepInputFiles = True; break; + case 's': smallMode = True; break; + case '1': blockSize100k = 1; break; + case '2': blockSize100k = 2; break; + case '3': blockSize100k = 3; break; + case '4': blockSize100k = 4; break; + case '5': blockSize100k = 5; break; + case '6': blockSize100k = 6; break; + case '7': blockSize100k = 7; break; + case '8': blockSize100k = 8; break; + case '9': blockSize100k = 9; break; + case 'V': + case 'L': license(); break; + case 'v': verbosity++; break; + case 'h': usage ( progName ); + exit ( 1 ); + break; + default: fprintf ( stderr, "%s: Bad flag `%s'\n", + progName, aa->name ); + usage ( progName ); + exit ( 1 ); + break; + } + + /*-- And again ... --*/ + for (aa = argList; aa != NULL; aa = aa->link) { + if (ISFLAG("--stdout")) srcMode = SM_F2O; else + if (ISFLAG("--decompress")) opMode = OM_UNZ; else + if (ISFLAG("--compress")) opMode = OM_Z; else + if (ISFLAG("--test")) opMode = OM_TEST; else + if (ISFLAG("--keep")) keepInputFiles = True; else + if (ISFLAG("--small")) smallMode = True; else + if (ISFLAG("--version")) license(); else + if (ISFLAG("--license")) license(); else + if (ISFLAG("--repetitive-fast")) workFactor = 5; else + if (ISFLAG("--repetitive-best")) workFactor = 150; else + if (ISFLAG("--verbose")) verbosity++; else + if (ISFLAG("--help")) { usage ( progName ); exit ( 1 ); } + else + if (strncmp ( aa->name, "--", 2) == 0) { + fprintf ( stderr, "%s: Bad flag `%s'\n", progName, aa->name ); + usage ( progName ); + exit ( 1 ); + } + } + + if (opMode == OM_Z && smallMode) blockSize100k = 2; + + if (opMode == OM_Z && srcMode == SM_F2O && numFileNames > 1) { + fprintf ( stderr, "%s: I won't compress multiple files to stdout.\n", + progName ); + exit ( 1 ); + } + + if (opMode == OM_TEST && srcMode == SM_F2O) { + fprintf ( stderr, "%s: -c and -t cannot be used together.\n", + progName ); + exit ( 1 ); + } + + if (opMode != OM_Z) blockSize100k = 0; + + if (opMode == OM_Z) { + allocateCompressStructures(); + if (srcMode == SM_I2O) + compress ( NULL ); + else + for (aa = argList; aa != NULL; aa = aa->link) + if (aa->name[0] != '-') { + numFilesProcessed++; + compress ( aa->name ); + } + } else + if (opMode == OM_UNZ) { + if (srcMode == SM_I2O) + uncompress ( NULL ); + else + for (aa = argList; aa != NULL; aa = aa->link) + if (aa->name[0] != '-') { + numFilesProcessed++; + uncompress ( aa->name ); + } + } else { + testFailsExist = False; + if (srcMode == SM_I2O) + testf ( NULL ); + else + for (aa = argList; aa != NULL; aa = aa->link) + if (aa->name[0] != '-') { + numFilesProcessed++; + testf ( aa->name ); + } + if (testFailsExist) { + fprintf ( stderr, + "\n" + "You can use the `bzip2recover' program to *attempt* to recover\n" + "data from undamaged sections of corrupted files.\n\n" + ); + exit(2); + } + } + return 0; +} + + +/*-----------------------------------------------------------*/ +/*--- end bzip2.c ---*/ +/*-----------------------------------------------------------*/ diff --git a/bzip2.exe b/bzip2.exe new file mode 100644 index 0000000..4b3c4c1 Binary files /dev/null and b/bzip2.exe differ diff --git a/bzip2.txt b/bzip2.txt new file mode 100644 index 0000000..83366bc --- /dev/null +++ b/bzip2.txt @@ -0,0 +1,462 @@ + + + +bzip2(1) bzip2(1) + + +NAME + bzip2, bunzip2 - a block-sorting file compressor, v0.1 + bzip2recover - recovers data from damaged bzip2 files + + +SYNOPSIS + bzip2 [ -cdfkstvVL123456789 ] [ filenames ... ] + bunzip2 [ -kvsVL ] [ filenames ... ] + bzip2recover filename + + +DESCRIPTION + Bzip2 compresses files using the Burrows-Wheeler block- + sorting text compression algorithm, and Huffman coding. + Compression is generally considerably better than that + achieved by more conventional LZ77/LZ78-based compressors, + and approaches the performance of the PPM family of sta- + tistical compressors. + + The command-line options are deliberately very similar to + those of GNU Gzip, but they are not identical. + + Bzip2 expects a list of file names to accompany the com- + mand-line flags. Each file is replaced by a compressed + version of itself, with the name "original_name.bz2". + Each compressed file has the same modification date and + permissions as the corresponding original, so that these + properties can be correctly restored at decompression + time. File name handling is naive in the sense that there + is no mechanism for preserving original file names, per- + missions and dates in filesystems which lack these con- + cepts, or have serious file name length restrictions, such + as MS-DOS. + + Bzip2 and bunzip2 will not overwrite existing files; if + you want this to happen, you should delete them first. + + If no file names are specified, bzip2 compresses from + standard input to standard output. In this case, bzip2 + will decline to write compressed output to a terminal, as + this would be entirely incomprehensible and therefore + pointless. + + Bunzip2 (or bzip2 -d ) decompresses and restores all spec- + ified files whose names end in ".bz2". Files without this + suffix are ignored. Again, supplying no filenames causes + decompression from standard input to standard output. + + You can also compress or decompress files to the standard + output by giving the -c flag. You can decompress multiple + files like this, but you may only compress a single file + this way, since it would otherwise be difficult to sepa- + rate out the compressed representations of the original + files. + + + + 1 + + + + + +bzip2(1) bzip2(1) + + + Compression is always performed, even if the compressed + file is slightly larger than the original. Files of less + than about one hundred bytes tend to get larger, since the + compression mechanism has a constant overhead in the + region of 50 bytes. Random data (including the output of + most file compressors) is coded at about 8.05 bits per + byte, giving an expansion of around 0.5%. + + As a self-check for your protection, bzip2 uses 32-bit + CRCs to make sure that the decompressed version of a file + is identical to the original. This guards against corrup- + tion of the compressed data, and against undetected bugs + in bzip2 (hopefully very unlikely). The chances of data + corruption going undetected is microscopic, about one + chance in four billion for each file processed. Be aware, + though, that the check occurs upon decompression, so it + can only tell you that that something is wrong. It can't + help you recover the original uncompressed data. You can + use bzip2recover to try to recover data from damaged + files. + + Return values: 0 for a normal exit, 1 for environmental + problems (file not found, invalid flags, I/O errors, &c), + 2 to indicate a corrupt compressed file, 3 for an internal + consistency error (eg, bug) which caused bzip2 to panic. + + +MEMORY MANAGEMENT + Bzip2 compresses large files in blocks. The block size + affects both the compression ratio achieved, and the + amount of memory needed both for compression and decom- + pression. The flags -1 through -9 specify the block size + to be 100,000 bytes through 900,000 bytes (the default) + respectively. At decompression-time, the block size used + for compression is read from the header of the compressed + file, and bunzip2 then allocates itself just enough memory + to decompress the file. Since block sizes are stored in + compressed files, it follows that the flags -1 to -9 are + irrelevant to and so ignored during decompression. Com- + pression and decompression requirements, in bytes, can be + estimated as: + + Compression: 400k + ( 7 x block size ) + + Decompression: 100k + ( 5 x block size ), or + 100k + ( 2.5 x block size ) + + Larger block sizes give rapidly diminishing marginal + returns; most of the compression comes from the first two + or three hundred k of block size, a fact worth bearing in + mind when using bzip2 on small machines. It is also + important to appreciate that the decompression memory + requirement is set at compression-time by the choice of + block size. + + + + 2 + + + + + +bzip2(1) bzip2(1) + + + For files compressed with the default 900k block size, + bunzip2 will require about 4600 kbytes to decompress. To + support decompression of any file on a 4 megabyte machine, + bunzip2 has an option to decompress using approximately + half this amount of memory, about 2300 kbytes. Decompres- + sion speed is also halved, so you should use this option + only where necessary. The relevant flag is -s. + + In general, try and use the largest block size memory con- + straints allow, since that maximises the compression + achieved. Compression and decompression speed are virtu- + ally unaffected by block size. + + Another significant point applies to files which fit in a + single block -- that means most files you'd encounter + using a large block size. The amount of real memory + touched is proportional to the size of the file, since the + file is smaller than a block. For example, compressing a + file 20,000 bytes long with the flag -9 will cause the + compressor to allocate around 6700k of memory, but only + touch 400k + 20000 * 7 = 540 kbytes of it. Similarly, the + decompressor will allocate 4600k but only touch 100k + + 20000 * 5 = 200 kbytes. + + Here is a table which summarises the maximum memory usage + for different block sizes. Also recorded is the total + compressed size for 14 files of the Calgary Text Compres- + sion Corpus totalling 3,141,622 bytes. This column gives + some feel for how compression varies with block size. + These figures tend to understate the advantage of larger + block sizes for larger files, since the Corpus is domi- + nated by smaller files. + + Compress Decompress Decompress Corpus + Flag usage usage -s usage Size + + -1 1100k 600k 350k 914704 + -2 1800k 1100k 600k 877703 + -3 2500k 1600k 850k 860338 + -4 3200k 2100k 1100k 846899 + -5 3900k 2600k 1350k 845160 + -6 4600k 3100k 1600k 838626 + -7 5400k 3600k 1850k 834096 + -8 6000k 4100k 2100k 828642 + -9 6700k 4600k 2350k 828642 + + +OPTIONS + -c --stdout + Compress or decompress to standard output. -c will + decompress multiple files to stdout, but will only + compress a single file to stdout. + + + + + + 3 + + + + + +bzip2(1) bzip2(1) + + + -d --decompress + Force decompression. Bzip2 and bunzip2 are really + the same program, and the decision about whether to + compress or decompress is done on the basis of + which name is used. This flag overrides that mech- + anism, and forces bzip2 to decompress. + + -f --compress + The complement to -d: forces compression, regard- + less of the invokation name. + + -t --test + Check integrity of the specified file(s), but don't + decompress them. This really performs a trial + decompression and throws away the result, using the + low-memory decompression algorithm (see -s). + + -k --keep + Keep (don't delete) input files during compression + or decompression. + + -s --small + Reduce memory usage, both for compression and + decompression. Files are decompressed using a mod- + ified algorithm which only requires 2.5 bytes per + block byte. This means any file can be decom- + pressed in 2300k of memory, albeit somewhat more + slowly than usual. + + During compression, -s selects a block size of + 200k, which limits memory use to around the same + figure, at the expense of your compression ratio. + In short, if your machine is low on memory (8 + megabytes or less), use -s for everything. See + MEMORY MANAGEMENT above. + + + -v --verbose + Verbose mode -- show the compression ratio for each + file processed. Further -v's increase the ver- + bosity level, spewing out lots of information which + is primarily of interest for diagnostic purposes. + + -L --license + Display the software version, license terms and + conditions. + + -V --version + Same as -L. + + -1 to -9 + Set the block size to 100 k, 200 k .. 900 k when + compressing. Has no effect when decompressing. + See MEMORY MANAGEMENT above. + + + + 4 + + + + + +bzip2(1) bzip2(1) + + + --repetitive-fast + bzip2 injects some small pseudo-random variations + into very repetitive blocks to limit worst-case + performance during compression. If sorting runs + into difficulties, the block is randomised, and + sorting is restarted. Very roughly, bzip2 persists + for three times as long as a well-behaved input + would take before resorting to randomisation. This + flag makes it give up much sooner. + + + --repetitive-best + Opposite of --repetitive-fast; try a lot harder + before resorting to randomisation. + + +RECOVERING DATA FROM DAMAGED FILES + bzip2 compresses files in blocks, usually 900kbytes long. + Each block is handled independently. If a media or trans- + mission error causes a multi-block .bz2 file to become + damaged, it may be possible to recover data from the + undamaged blocks in the file. + + The compressed representation of each block is delimited + by a 48-bit pattern, which makes it possible to find the + block boundaries with reasonable certainty. Each block + also carries its own 32-bit CRC, so damaged blocks can be + distinguished from undamaged ones. + + bzip2recover is a simple program whose purpose is to + search for blocks in .bz2 files, and write each block out + into its own .bz2 file. You can then use bzip2 -t to test + the integrity of the resulting files, and decompress those + which are undamaged. + + bzip2recover takes a single argument, the name of the dam- + aged file, and writes a number of files "rec0001file.bz2", + "rec0002file.bz2", etc, containing the extracted blocks. + The output filenames are designed so that the use of wild- + cards in subsequent processing -- for example, "bzip2 -dc + rec*file.bz2 > recovered_data" -- lists the files in the + "right" order. + + bzip2recover should be of most use dealing with large .bz2 + files, as these will contain many blocks. It is clearly + futile to use it on damaged single-block files, since a + damaged block cannot be recovered. If you wish to min- + imise any potential data loss through media or transmis- + sion errors, you might consider compressing with a smaller + block size. + + +PERFORMANCE NOTES + The sorting phase of compression gathers together similar + + + + 5 + + + + + +bzip2(1) bzip2(1) + + + strings in the file. Because of this, files containing + very long runs of repeated symbols, like "aabaabaabaab + ..." (repeated several hundred times) may compress + extraordinarily slowly. You can use the -vvvvv option to + monitor progress in great detail, if you want. Decompres- + sion speed is unaffected. + + Such pathological cases seem rare in practice, appearing + mostly in artificially-constructed test files, and in low- + level disk images. It may be inadvisable to use bzip2 to + compress the latter. If you do get a file which causes + severe slowness in compression, try making the block size + as small as possible, with flag -1. + + Incompressible or virtually-incompressible data may decom- + press rather more slowly than one would hope. This is due + to a naive implementation of the move-to-front coder. + + bzip2 usually allocates several megabytes of memory to + operate in, and then charges all over it in a fairly ran- + dom fashion. This means that performance, both for com- + pressing and decompressing, is largely determined by the + speed at which your machine can service cache misses. + Because of this, small changes to the code to reduce the + miss rate have been observed to give disproportionately + large performance improvements. I imagine bzip2 will per- + form best on machines with very large caches. + + Test mode (-t) uses the low-memory decompression algorithm + (-s). This means test mode does not run as fast as it + could; it could run as fast as the normal decompression + machinery. This could easily be fixed at the cost of some + code bloat. + + +CAVEATS + I/O error messages are not as helpful as they could be. + Bzip2 tries hard to detect I/O errors and exit cleanly, + but the details of what the problem is sometimes seem + rather misleading. + + This manual page pertains to version 0.1 of bzip2. It may + well happen that some future version will use a different + compressed file format. If you try to decompress, using + 0.1, a .bz2 file created with some future version which + uses a different compressed file format, 0.1 will complain + that your file "is not a bzip2 file". If that happens, + you should obtain a more recent version of bzip2 and use + that to decompress the file. + + Wildcard expansion for Windows 95 and NT is flaky. + + bzip2recover uses 32-bit integers to represent bit posi- + tions in compressed files, so it cannot handle compressed + + + + 6 + + + + + +bzip2(1) bzip2(1) + + + files more than 512 megabytes long. This could easily be + fixed. + + bzip2recover sometimes reports a very small, incomplete + final block. This is spurious and can be safely ignored. + + +RELATIONSHIP TO bzip-0.21 + This program is a descendant of the bzip program, version + 0.21, which I released in August 1996. The primary dif- + ference of bzip2 is its avoidance of the possibly patented + algorithms which were used in 0.21. bzip2 also brings + various useful refinements (-s, -t), uses less memory, + decompresses significantly faster, and has support for + recovering data from damaged files. + + Because bzip2 uses Huffman coding to construct the com- + pressed bitstream, rather than the arithmetic coding used + in 0.21, the compressed representations generated by the + two programs are incompatible, and they will not interop- + erate. The change in suffix from .bz to .bz2 reflects + this. It would have been helpful to at least allow bzip2 + to decompress files created by 0.21, but this would defeat + the primary aim of having a patent-free compressor. + + Huffman coding necessarily involves some coding ineffi- + ciency compared to arithmetic coding. This means that + bzip2 compresses about 1% worse than 0.21, an unfortunate + but unavoidable fact-of-life. On the other hand, decom- + pression is approximately 50% faster for the same reason, + and the change in file format gave an opportunity to add + data-recovery features. So it is not all bad. + + +AUTHOR + Julian Seward, jseward@acm.org. + + The ideas embodied in bzip and bzip2 are due to (at least) + the following people: Michael Burrows and David Wheeler + (for the block sorting transformation), David Wheeler + (again, for the Huffman coder), Peter Fenwick (for the + structured coding model in 0.21, and many refinements), + and Alistair Moffat, Radford Neal and Ian Witten (for the + arithmetic coder in 0.21). I am much indebted for their + help, support and advice. See the file ALGORITHMS in the + source distribution for pointers to sources of documenta- + tion. Christian von Roques encouraged me to look for + faster sorting algorithms, so as to speed up compression. + Bela Lubkin encouraged me to improve the worst-case com- + pression performance. Many people sent patches, helped + with portability problems, lent machines, gave advice and + were generally helpful. + + + + + + 7 + + diff --git a/bzip2recover.c b/bzip2recover.c new file mode 100644 index 0000000..efdfb3c --- /dev/null +++ b/bzip2recover.c @@ -0,0 +1,399 @@ + +/*-----------------------------------------------------------*/ +/*--- Block recoverer program for bzip2 ---*/ +/*--- bzip2recover.c ---*/ +/*-----------------------------------------------------------*/ + +/*-- + This program is bzip2recover, a program to attempt data + salvage from damaged files created by the accompanying + bzip2 program. + + Copyright (C) 1996, 1997 by Julian Seward. + Guildford, Surrey, UK + email: jseward@acm.org + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 2 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + + The GNU General Public License is contained in the file LICENSE. +--*/ + + +#include +#include +#include +#include +#include /*-- or try string.h --*/ + +#define UInt32 unsigned int +#define Int32 int +#define UChar unsigned char +#define Char char +#define Bool unsigned char +#define True 1 +#define False 0 + + +Char inFileName[2000]; +Char outFileName[2000]; +Char progName[2000]; + +UInt32 bytesOut = 0; +UInt32 bytesIn = 0; + + +/*---------------------------------------------------*/ +/*--- I/O errors ---*/ +/*---------------------------------------------------*/ + +/*---------------------------------------------*/ +void readError ( void ) +{ + fprintf ( stderr, + "%s: I/O error reading `%s', possible reason follows.\n", + progName, inFileName ); + perror ( progName ); + fprintf ( stderr, "%s: warning: output file(s) may be incomplete.\n", + progName ); + exit ( 1 ); +} + + +/*---------------------------------------------*/ +void writeError ( void ) +{ + fprintf ( stderr, + "%s: I/O error reading `%s', possible reason follows.\n", + progName, inFileName ); + perror ( progName ); + fprintf ( stderr, "%s: warning: output file(s) may be incomplete.\n", + progName ); + exit ( 1 ); +} + + +/*---------------------------------------------*/ +void mallocFail ( Int32 n ) +{ + fprintf ( stderr, + "%s: malloc failed on request for %d bytes.\n", + progName, n ); + fprintf ( stderr, "%s: warning: output file(s) may be incomplete.\n", + progName ); + exit ( 1 ); +} + + +/*---------------------------------------------------*/ +/*--- Bit stream I/O ---*/ +/*---------------------------------------------------*/ + +typedef + struct { + FILE* handle; + Int32 buffer; + Int32 buffLive; + Char mode; + } + BitStream; + + +/*---------------------------------------------*/ +BitStream* bsOpenReadStream ( FILE* stream ) +{ + BitStream *bs = malloc ( sizeof(BitStream) ); + if (bs == NULL) mallocFail ( sizeof(BitStream) ); + bs->handle = stream; + bs->buffer = 0; + bs->buffLive = 0; + bs->mode = 'r'; + return bs; +} + + +/*---------------------------------------------*/ +BitStream* bsOpenWriteStream ( FILE* stream ) +{ + BitStream *bs = malloc ( sizeof(BitStream) ); + if (bs == NULL) mallocFail ( sizeof(BitStream) ); + bs->handle = stream; + bs->buffer = 0; + bs->buffLive = 0; + bs->mode = 'w'; + return bs; +} + + +/*---------------------------------------------*/ +void bsPutBit ( BitStream* bs, Int32 bit ) +{ + if (bs->buffLive == 8) { + Int32 retVal = putc ( (UChar) bs->buffer, bs->handle ); + if (retVal == EOF) writeError(); + bytesOut++; + bs->buffLive = 1; + bs->buffer = bit & 0x1; + } else { + bs->buffer = ( (bs->buffer << 1) | (bit & 0x1) ); + bs->buffLive++; + }; +} + + +/*---------------------------------------------*/ +/*-- + Returns 0 or 1, or 2 to indicate EOF. +--*/ +Int32 bsGetBit ( BitStream* bs ) +{ + if (bs->buffLive > 0) { + bs->buffLive --; + return ( ((bs->buffer) >> (bs->buffLive)) & 0x1 ); + } else { + Int32 retVal = getc ( bs->handle ); + if ( retVal == EOF ) { + if (errno != 0) readError(); + return 2; + } + bs->buffLive = 7; + bs->buffer = retVal; + return ( ((bs->buffer) >> 7) & 0x1 ); + } +} + + +/*---------------------------------------------*/ +void bsClose ( BitStream* bs ) +{ + Int32 retVal; + + if ( bs->mode == 'w' ) { + while ( bs->buffLive < 8 ) { + bs->buffLive++; + bs->buffer <<= 1; + }; + retVal = putc ( (UChar) (bs->buffer), bs->handle ); + if (retVal == EOF) writeError(); + bytesOut++; + retVal = fflush ( bs->handle ); + if (retVal == EOF) writeError(); + } + retVal = fclose ( bs->handle ); + if (retVal == EOF) + if (bs->mode == 'w') writeError(); else readError(); + free ( bs ); +} + + +/*---------------------------------------------*/ +void bsPutUChar ( BitStream* bs, UChar c ) +{ + Int32 i; + for (i = 7; i >= 0; i--) + bsPutBit ( bs, (((UInt32) c) >> i) & 0x1 ); +} + + +/*---------------------------------------------*/ +void bsPutUInt32 ( BitStream* bs, UInt32 c ) +{ + Int32 i; + + for (i = 31; i >= 0; i--) + bsPutBit ( bs, (c >> i) & 0x1 ); +} + + +/*---------------------------------------------*/ +Bool endsInBz2 ( Char* name ) +{ + Int32 n = strlen ( name ); + if (n <= 4) return False; + return + (name[n-4] == '.' && + name[n-3] == 'b' && + name[n-2] == 'z' && + name[n-1] == '2'); +} + + +/*---------------------------------------------------*/ +/*--- ---*/ +/*---------------------------------------------------*/ + +#define BLOCK_HEADER_HI 0x00003141UL +#define BLOCK_HEADER_LO 0x59265359UL + +#define BLOCK_ENDMARK_HI 0x00001772UL +#define BLOCK_ENDMARK_LO 0x45385090UL + +Int32 main ( Int32 argc, Char** argv ) +{ + FILE* inFile; + FILE* outFile; + BitStream* bsIn, *bsWr; + Int32 currBlock, b, wrBlock; + UInt32 bitsRead; + UInt32 bStart[20000]; + UInt32 bEnd[20000]; + UInt32 buffHi, buffLo, blockCRC; + Char* p; + + strcpy ( progName, argv[0] ); + inFileName[0] = outFileName[0] = 0; + + fprintf ( stderr, "bzip2recover: extracts blocks from damaged .bz2 files.\n" ); + + if (argc != 2) { + fprintf ( stderr, "%s: usage is `%s damaged_file_name'.\n", + progName, progName ); + exit(1); + } + + strcpy ( inFileName, argv[1] ); + + inFile = fopen ( inFileName, "rb" ); + if (inFile == NULL) { + fprintf ( stderr, "%s: can't read `%s'\n", progName, inFileName ); + exit(1); + } + + bsIn = bsOpenReadStream ( inFile ); + fprintf ( stderr, "%s: searching for block boundaries ...\n", progName ); + + bitsRead = 0; + buffHi = buffLo = 0; + currBlock = 0; + bStart[currBlock] = 0; + + while (True) { + b = bsGetBit ( bsIn ); + bitsRead++; + if (b == 2) { + if (bitsRead >= bStart[currBlock] && + (bitsRead - bStart[currBlock]) >= 40) { + bEnd[currBlock] = bitsRead-1; + if (currBlock > 0) + fprintf ( stderr, " block %d runs from %d to %d (incomplete)\n", + currBlock, bStart[currBlock], bEnd[currBlock] ); + } else + currBlock--; + break; + } + buffHi = (buffHi << 1) | (buffLo >> 31); + buffLo = (buffLo << 1) | (b & 1); + if ( ( (buffHi & 0x0000ffff) == BLOCK_HEADER_HI + && buffLo == BLOCK_HEADER_LO) + || + ( (buffHi & 0x0000ffff) == BLOCK_ENDMARK_HI + && buffLo == BLOCK_ENDMARK_LO) + ) { + if (bitsRead > 49) + bEnd[currBlock] = bitsRead-49; else + bEnd[currBlock] = 0; + if (currBlock > 0) + fprintf ( stderr, " block %d runs from %d to %d\n", + currBlock, bStart[currBlock], bEnd[currBlock] ); + currBlock++; + bStart[currBlock] = bitsRead; + } + } + + bsClose ( bsIn ); + + /*-- identified blocks run from 1 to currBlock inclusive. --*/ + + if (currBlock < 1) { + fprintf ( stderr, + "%s: sorry, I couldn't find any block boundaries.\n", + progName ); + exit(1); + }; + + fprintf ( stderr, "%s: splitting into blocks\n", progName ); + + inFile = fopen ( inFileName, "rb" ); + if (inFile == NULL) { + fprintf ( stderr, "%s: can't open `%s'\n", progName, inFileName ); + exit(1); + } + bsIn = bsOpenReadStream ( inFile ); + + /*-- placate gcc's dataflow analyser --*/ + blockCRC = 0; bsWr = 0; + + bitsRead = 0; + outFile = NULL; + wrBlock = 1; + while (True) { + b = bsGetBit(bsIn); + if (b == 2) break; + buffHi = (buffHi << 1) | (buffLo >> 31); + buffLo = (buffLo << 1) | (b & 1); + if (bitsRead == 47+bStart[wrBlock]) + blockCRC = (buffHi << 16) | (buffLo >> 16); + + if (outFile != NULL && bitsRead >= bStart[wrBlock] + && bitsRead <= bEnd[wrBlock]) { + bsPutBit ( bsWr, b ); + } + + bitsRead++; + + if (bitsRead == bEnd[wrBlock]+1) { + if (outFile != NULL) { + bsPutUChar ( bsWr, 0x17 ); bsPutUChar ( bsWr, 0x72 ); + bsPutUChar ( bsWr, 0x45 ); bsPutUChar ( bsWr, 0x38 ); + bsPutUChar ( bsWr, 0x50 ); bsPutUChar ( bsWr, 0x90 ); + bsPutUInt32 ( bsWr, blockCRC ); + bsClose ( bsWr ); + } + if (wrBlock >= currBlock) break; + wrBlock++; + } else + if (bitsRead == bStart[wrBlock]) { + outFileName[0] = 0; + sprintf ( outFileName, "rec%4d", wrBlock ); + for (p = outFileName; *p != 0; p++) if (*p == ' ') *p = '0'; + strcat ( outFileName, inFileName ); + if ( !endsInBz2(outFileName)) strcat ( outFileName, ".bz2" ); + + fprintf ( stderr, " writing block %d to `%s' ...\n", + wrBlock, outFileName ); + + outFile = fopen ( outFileName, "wb" ); + if (outFile == NULL) { + fprintf ( stderr, "%s: can't write `%s'\n", + progName, outFileName ); + exit(1); + } + bsWr = bsOpenWriteStream ( outFile ); + bsPutUChar ( bsWr, 'B' ); bsPutUChar ( bsWr, 'Z' ); + bsPutUChar ( bsWr, 'h' ); bsPutUChar ( bsWr, '9' ); + bsPutUChar ( bsWr, 0x31 ); bsPutUChar ( bsWr, 0x41 ); + bsPutUChar ( bsWr, 0x59 ); bsPutUChar ( bsWr, 0x26 ); + bsPutUChar ( bsWr, 0x53 ); bsPutUChar ( bsWr, 0x59 ); + } + } + + fprintf ( stderr, "%s: finished\n", progName ); + return 0; +} + + + +/*-----------------------------------------------------------*/ +/*--- end bzip2recover.c ---*/ +/*-----------------------------------------------------------*/ diff --git a/sample1.bz2 b/sample1.bz2 new file mode 100644 index 0000000..18dea60 Binary files /dev/null and b/sample1.bz2 differ diff --git a/sample1.ref b/sample1.ref new file mode 100644 index 0000000..a56e52b Binary files /dev/null and b/sample1.ref differ diff --git a/sample2.bz2 b/sample2.bz2 new file mode 100644 index 0000000..d5a6160 Binary files /dev/null and b/sample2.bz2 differ diff --git a/sample2.ref b/sample2.ref new file mode 100644 index 0000000..34af958 Binary files /dev/null and b/sample2.ref differ diff --git a/test.bat b/test.bat new file mode 100644 index 0000000..30b747d --- /dev/null +++ b/test.bat @@ -0,0 +1,9 @@ +@rem +@rem MSDOS test driver for bzip2 +@rem +type words1 +.\bzip2 -1 < sample1.ref > sample1.rbz +.\bzip2 -2 < sample2.ref > sample2.rbz +.\bzip2 -dvv < sample1.bz2 > sample1.tst +.\bzip2 -dvv < sample2.bz2 > sample2.tst +type words3sh \ No newline at end of file diff --git a/test.cmd b/test.cmd new file mode 100644 index 0000000..f7bc866 --- /dev/null +++ b/test.cmd @@ -0,0 +1,9 @@ +@rem +@rem OS/2 test driver for bzip2 +@rem +type words1 +.\bzip2 -1 < sample1.ref > sample1.rbz +.\bzip2 -2 < sample2.ref > sample2.rbz +.\bzip2 -dvv < sample1.bz2 > sample1.tst +.\bzip2 -dvv < sample2.bz2 > sample2.tst +type words3sh \ No newline at end of file diff --git a/words0 b/words0 new file mode 100644 index 0000000..527fb43 --- /dev/null +++ b/words0 @@ -0,0 +1,7 @@ +***-------------------------------------------------*** +***--------- IMPORTANT: READ WHAT FOLLOWS! ---------*** +***--------- viz: pay attention :-) ---------*** +***-------------------------------------------------*** + +Compiling bzip2 ... + diff --git a/words1 b/words1 new file mode 100644 index 0000000..c75293b --- /dev/null +++ b/words1 @@ -0,0 +1,5 @@ + + +Doing 4 tests (2 compress, 2 uncompress) ... +If there's a problem, things might stop at this point. + diff --git a/words2 b/words2 new file mode 100644 index 0000000..d3cafb9 --- /dev/null +++ b/words2 @@ -0,0 +1,6 @@ + + +Checking test results. If any of the four "cmp"s which follow +report any differences, something is wrong. If you can't easily +figure out what, please let me know (jseward@acm.org). + diff --git a/words3 b/words3 new file mode 100644 index 0000000..5739d18 --- /dev/null +++ b/words3 @@ -0,0 +1,23 @@ + + +If you got this far and the "cmp"s didn't find anything amiss, looks +like you're in business. You should install bzip2 and bunzip2: + + copy bzip2 to a public place, maybe /usr/bin. + In that public place, make bunzip2 a symbolic link + to the bzip2 you just copied there. + Put the manual page, bzip2.1, somewhere appropriate; + perhaps in /usr/man/man1. + +Complete instructions for use are in the preformatted +manual page, in the file bzip2.1.preformatted. + +You can also do "bzip2 --help" to see some helpful information. + +"bzip2 -L" displays the software license. + +Please read the README file carefully. +Finally, note that bzip2 comes with ABSOLUTELY NO WARRANTY. + +Happy compressing! + diff --git a/words3sh b/words3sh new file mode 100644 index 0000000..1139177 --- /dev/null +++ b/words3sh @@ -0,0 +1,12 @@ +If you got this far and the "bzip2 -dvv"s give identical +stored vs computed CRCs, you're probably in business. +Complete instructions for use are in the preformatted manual page, +in the file bzip2.txt. + +You can also do "bzip2 --help" to see some helpful information. +"bzip2 -L" displays the software license. + +Please read the README file carefully. +Finally, note that bzip2 comes with ABSOLUTELY NO WARRANTY. + +Happy compressing! \ No newline at end of file -- cgit v1.2.3 From 1eb67a9d8f7f05ae310bc9ef297d176f3a3f8a37 Mon Sep 17 00:00:00 2001 From: Julian Seward Date: Fri, 29 Aug 1997 22:13:13 +0200 Subject: bzip2-0.1pl2 --- Makefile | 2 +- README | 16 +++++- README.DOS | 26 ++++----- bzip2.1 | 3 + bzip2.1.preformatted | 3 + bzip2.c | 158 +++++++++++++++++++++++++++------------------------ bzip2.exe | Bin 45716 -> 0 bytes bzip2.txt | 3 + bzip2recover.c | 2 +- 9 files changed, 118 insertions(+), 95 deletions(-) delete mode 100644 bzip2.exe diff --git a/Makefile b/Makefile index d124743..9d35b43 100644 --- a/Makefile +++ b/Makefile @@ -2,7 +2,7 @@ CC = gcc SH = /bin/sh -CFLAGS = -O3 -fomit-frame-pointer -funroll-loops -Wall -Winline -W +CFLAGS = -O3 -fomit-frame-pointer -funroll-loops diff --git a/README b/README index d77830f..d58bb49 100644 --- a/README +++ b/README @@ -8,7 +8,7 @@ GREETINGS! for details, see the file LICENSE. Pointers to the algorithms used are in ALGORITHMS. Instructions for use are in bzip2.1.preformatted. - Please read this file carefully. + Please read all of this file carefully. @@ -223,7 +223,15 @@ DISCLAIMER: I very much hope the opposite is true. bzip2 has been carefully constructed and extensively tested. -End of nasty legalities. + +PATENTS: + + To the best of my knowledge, bzip2 does not use any patented + algorithms. However, I do not have the resources available to + carry out a full patent search. Therefore I cannot give any + guarantee of the above statement. + +End of legalities. I hope you find bzip2 useful. Feel free to contact me at @@ -240,4 +248,6 @@ Manchester, UK 25 August 1996 (version 0.21) Guildford, Surrey, UK -7 August 1997 (bzip2, version 0.0) \ No newline at end of file +7 August 1997 (bzip2, version 0.1) +29 August 1997 (bzip2, version 0.1pl2) + diff --git a/README.DOS b/README.DOS index d522b81..048de8c 100644 --- a/README.DOS +++ b/README.DOS @@ -1,20 +1,16 @@ -Windows 95 & Windows NT users: +As of today (3 March 1998) I've removed the +Win95/NT executables from this distribution, sorry. -1. There's a pre-built executable, bzip2.exe, which - should work. You don't need to compile anything. - You can run the `test.bat' batch file to check - the executable is working ok, if you want. +You can still get an executable from +http://www.muraroa.demon.co.uk, or (as a last +resort) by mailing me at jseward@acm.org. -2. The control-C signal catcher seems pretty dodgy - under Windows, at least for the executable supplied. - When it catches a control-C, bzip2 tries to delete - its output file, so you don't get left with a half- - baked file. But this sometimes seems to fail - under Windows. Caveat Emptor! I think I am doing - something not-quite-right in the signal catching. - Windows-&-C gurus got any suggestions? +The reason for this change of packaging is that it +makes it easier for me to fix problems with specific +executables if they are not included in the main +distribution. - Control-C handling all seems to work fine under Unix. +J -7 Aug 97 + diff --git a/bzip2.1 b/bzip2.1 index 9094c7c..489668f 100644 --- a/bzip2.1 +++ b/bzip2.1 @@ -406,6 +406,9 @@ helpful to at least allow to decompress files created by 0.21, but this would defeat the primary aim of having a patent-free compressor. +For a more precise statement about patent issues in +bzip2, please see the README file in the distribution. + Huffman coding necessarily involves some coding inefficiency compared to arithmetic coding. This means that .I bzip2 diff --git a/bzip2.1.preformatted b/bzip2.1.preformatted index 947dc97..5206e05 100644 --- a/bzip2.1.preformatted +++ b/bzip2.1.preformatted @@ -425,6 +425,9 @@ RREELLAATTIIOONNSSHHIIPP TTOO bbzziipp--00..2211 to decompress files created by 0.21, but this would defeat the primary aim of having a patent-free compressor. + For a more precise statement about patent issues in bzip2, + please see the README file in the distribution. + Huffman coding necessarily involves some coding ineffi- ciency compared to arithmetic coding. This means that _b_z_i_p_2 compresses about 1% worse than 0.21, an unfortunate diff --git a/bzip2.c b/bzip2.c index 0fb45fb..53ce10d 100644 --- a/bzip2.c +++ b/bzip2.c @@ -5,7 +5,7 @@ /*-- This program is bzip2, a lossless, block-sorting data compressor, - version 0.1pl0, dated 17-Aug-1997. + version 0.1pl2, dated 29-Aug-1997. Copyright (C) 1996, 1997 by Julian Seward. Guildford, Surrey, UK @@ -71,6 +71,12 @@ That is not to say this program is inherently unreliable. Indeed, I very much hope the opposite is true. bzip2 has been carefully constructed and extensively tested. + + PATENTS: + To the best of my knowledge, bzip2 does not use any patented + algorithms. However, I do not have the resources available to + carry out a full patent search. Therefore I cannot give any + guarantee of the above statement. --*/ @@ -111,7 +117,6 @@ #endif #include #include -#include #include #define ERROR_IF_EOF(i) { if ((i) == EOF) ioError(); } @@ -125,6 +130,7 @@ --*/ #if BZ_UNIX + #include #include #include #include @@ -512,77 +518,77 @@ UInt32 crc32Table[256] = { /*-- Ugly, innit? --*/ - 0x00000000L, 0x04c11db7L, 0x09823b6eL, 0x0d4326d9L, - 0x130476dcL, 0x17c56b6bL, 0x1a864db2L, 0x1e475005L, - 0x2608edb8L, 0x22c9f00fL, 0x2f8ad6d6L, 0x2b4bcb61L, - 0x350c9b64L, 0x31cd86d3L, 0x3c8ea00aL, 0x384fbdbdL, - 0x4c11db70L, 0x48d0c6c7L, 0x4593e01eL, 0x4152fda9L, - 0x5f15adacL, 0x5bd4b01bL, 0x569796c2L, 0x52568b75L, - 0x6a1936c8L, 0x6ed82b7fL, 0x639b0da6L, 0x675a1011L, - 0x791d4014L, 0x7ddc5da3L, 0x709f7b7aL, 0x745e66cdL, - 0x9823b6e0L, 0x9ce2ab57L, 0x91a18d8eL, 0x95609039L, - 0x8b27c03cL, 0x8fe6dd8bL, 0x82a5fb52L, 0x8664e6e5L, - 0xbe2b5b58L, 0xbaea46efL, 0xb7a96036L, 0xb3687d81L, - 0xad2f2d84L, 0xa9ee3033L, 0xa4ad16eaL, 0xa06c0b5dL, - 0xd4326d90L, 0xd0f37027L, 0xddb056feL, 0xd9714b49L, - 0xc7361b4cL, 0xc3f706fbL, 0xceb42022L, 0xca753d95L, - 0xf23a8028L, 0xf6fb9d9fL, 0xfbb8bb46L, 0xff79a6f1L, - 0xe13ef6f4L, 0xe5ffeb43L, 0xe8bccd9aL, 0xec7dd02dL, - 0x34867077L, 0x30476dc0L, 0x3d044b19L, 0x39c556aeL, - 0x278206abL, 0x23431b1cL, 0x2e003dc5L, 0x2ac12072L, - 0x128e9dcfL, 0x164f8078L, 0x1b0ca6a1L, 0x1fcdbb16L, - 0x018aeb13L, 0x054bf6a4L, 0x0808d07dL, 0x0cc9cdcaL, - 0x7897ab07L, 0x7c56b6b0L, 0x71159069L, 0x75d48ddeL, - 0x6b93dddbL, 0x6f52c06cL, 0x6211e6b5L, 0x66d0fb02L, - 0x5e9f46bfL, 0x5a5e5b08L, 0x571d7dd1L, 0x53dc6066L, - 0x4d9b3063L, 0x495a2dd4L, 0x44190b0dL, 0x40d816baL, - 0xaca5c697L, 0xa864db20L, 0xa527fdf9L, 0xa1e6e04eL, - 0xbfa1b04bL, 0xbb60adfcL, 0xb6238b25L, 0xb2e29692L, - 0x8aad2b2fL, 0x8e6c3698L, 0x832f1041L, 0x87ee0df6L, - 0x99a95df3L, 0x9d684044L, 0x902b669dL, 0x94ea7b2aL, - 0xe0b41de7L, 0xe4750050L, 0xe9362689L, 0xedf73b3eL, - 0xf3b06b3bL, 0xf771768cL, 0xfa325055L, 0xfef34de2L, - 0xc6bcf05fL, 0xc27dede8L, 0xcf3ecb31L, 0xcbffd686L, - 0xd5b88683L, 0xd1799b34L, 0xdc3abdedL, 0xd8fba05aL, - 0x690ce0eeL, 0x6dcdfd59L, 0x608edb80L, 0x644fc637L, - 0x7a089632L, 0x7ec98b85L, 0x738aad5cL, 0x774bb0ebL, - 0x4f040d56L, 0x4bc510e1L, 0x46863638L, 0x42472b8fL, - 0x5c007b8aL, 0x58c1663dL, 0x558240e4L, 0x51435d53L, - 0x251d3b9eL, 0x21dc2629L, 0x2c9f00f0L, 0x285e1d47L, - 0x36194d42L, 0x32d850f5L, 0x3f9b762cL, 0x3b5a6b9bL, - 0x0315d626L, 0x07d4cb91L, 0x0a97ed48L, 0x0e56f0ffL, - 0x1011a0faL, 0x14d0bd4dL, 0x19939b94L, 0x1d528623L, - 0xf12f560eL, 0xf5ee4bb9L, 0xf8ad6d60L, 0xfc6c70d7L, - 0xe22b20d2L, 0xe6ea3d65L, 0xeba91bbcL, 0xef68060bL, - 0xd727bbb6L, 0xd3e6a601L, 0xdea580d8L, 0xda649d6fL, - 0xc423cd6aL, 0xc0e2d0ddL, 0xcda1f604L, 0xc960ebb3L, - 0xbd3e8d7eL, 0xb9ff90c9L, 0xb4bcb610L, 0xb07daba7L, - 0xae3afba2L, 0xaafbe615L, 0xa7b8c0ccL, 0xa379dd7bL, - 0x9b3660c6L, 0x9ff77d71L, 0x92b45ba8L, 0x9675461fL, - 0x8832161aL, 0x8cf30badL, 0x81b02d74L, 0x857130c3L, - 0x5d8a9099L, 0x594b8d2eL, 0x5408abf7L, 0x50c9b640L, - 0x4e8ee645L, 0x4a4ffbf2L, 0x470cdd2bL, 0x43cdc09cL, - 0x7b827d21L, 0x7f436096L, 0x7200464fL, 0x76c15bf8L, - 0x68860bfdL, 0x6c47164aL, 0x61043093L, 0x65c52d24L, - 0x119b4be9L, 0x155a565eL, 0x18197087L, 0x1cd86d30L, - 0x029f3d35L, 0x065e2082L, 0x0b1d065bL, 0x0fdc1becL, - 0x3793a651L, 0x3352bbe6L, 0x3e119d3fL, 0x3ad08088L, - 0x2497d08dL, 0x2056cd3aL, 0x2d15ebe3L, 0x29d4f654L, - 0xc5a92679L, 0xc1683bceL, 0xcc2b1d17L, 0xc8ea00a0L, - 0xd6ad50a5L, 0xd26c4d12L, 0xdf2f6bcbL, 0xdbee767cL, - 0xe3a1cbc1L, 0xe760d676L, 0xea23f0afL, 0xeee2ed18L, - 0xf0a5bd1dL, 0xf464a0aaL, 0xf9278673L, 0xfde69bc4L, - 0x89b8fd09L, 0x8d79e0beL, 0x803ac667L, 0x84fbdbd0L, - 0x9abc8bd5L, 0x9e7d9662L, 0x933eb0bbL, 0x97ffad0cL, - 0xafb010b1L, 0xab710d06L, 0xa6322bdfL, 0xa2f33668L, - 0xbcb4666dL, 0xb8757bdaL, 0xb5365d03L, 0xb1f740b4L + 0x00000000UL, 0x04c11db7UL, 0x09823b6eUL, 0x0d4326d9UL, + 0x130476dcUL, 0x17c56b6bUL, 0x1a864db2UL, 0x1e475005UL, + 0x2608edb8UL, 0x22c9f00fUL, 0x2f8ad6d6UL, 0x2b4bcb61UL, + 0x350c9b64UL, 0x31cd86d3UL, 0x3c8ea00aUL, 0x384fbdbdUL, + 0x4c11db70UL, 0x48d0c6c7UL, 0x4593e01eUL, 0x4152fda9UL, + 0x5f15adacUL, 0x5bd4b01bUL, 0x569796c2UL, 0x52568b75UL, + 0x6a1936c8UL, 0x6ed82b7fUL, 0x639b0da6UL, 0x675a1011UL, + 0x791d4014UL, 0x7ddc5da3UL, 0x709f7b7aUL, 0x745e66cdUL, + 0x9823b6e0UL, 0x9ce2ab57UL, 0x91a18d8eUL, 0x95609039UL, + 0x8b27c03cUL, 0x8fe6dd8bUL, 0x82a5fb52UL, 0x8664e6e5UL, + 0xbe2b5b58UL, 0xbaea46efUL, 0xb7a96036UL, 0xb3687d81UL, + 0xad2f2d84UL, 0xa9ee3033UL, 0xa4ad16eaUL, 0xa06c0b5dUL, + 0xd4326d90UL, 0xd0f37027UL, 0xddb056feUL, 0xd9714b49UL, + 0xc7361b4cUL, 0xc3f706fbUL, 0xceb42022UL, 0xca753d95UL, + 0xf23a8028UL, 0xf6fb9d9fUL, 0xfbb8bb46UL, 0xff79a6f1UL, + 0xe13ef6f4UL, 0xe5ffeb43UL, 0xe8bccd9aUL, 0xec7dd02dUL, + 0x34867077UL, 0x30476dc0UL, 0x3d044b19UL, 0x39c556aeUL, + 0x278206abUL, 0x23431b1cUL, 0x2e003dc5UL, 0x2ac12072UL, + 0x128e9dcfUL, 0x164f8078UL, 0x1b0ca6a1UL, 0x1fcdbb16UL, + 0x018aeb13UL, 0x054bf6a4UL, 0x0808d07dUL, 0x0cc9cdcaUL, + 0x7897ab07UL, 0x7c56b6b0UL, 0x71159069UL, 0x75d48ddeUL, + 0x6b93dddbUL, 0x6f52c06cUL, 0x6211e6b5UL, 0x66d0fb02UL, + 0x5e9f46bfUL, 0x5a5e5b08UL, 0x571d7dd1UL, 0x53dc6066UL, + 0x4d9b3063UL, 0x495a2dd4UL, 0x44190b0dUL, 0x40d816baUL, + 0xaca5c697UL, 0xa864db20UL, 0xa527fdf9UL, 0xa1e6e04eUL, + 0xbfa1b04bUL, 0xbb60adfcUL, 0xb6238b25UL, 0xb2e29692UL, + 0x8aad2b2fUL, 0x8e6c3698UL, 0x832f1041UL, 0x87ee0df6UL, + 0x99a95df3UL, 0x9d684044UL, 0x902b669dUL, 0x94ea7b2aUL, + 0xe0b41de7UL, 0xe4750050UL, 0xe9362689UL, 0xedf73b3eUL, + 0xf3b06b3bUL, 0xf771768cUL, 0xfa325055UL, 0xfef34de2UL, + 0xc6bcf05fUL, 0xc27dede8UL, 0xcf3ecb31UL, 0xcbffd686UL, + 0xd5b88683UL, 0xd1799b34UL, 0xdc3abdedUL, 0xd8fba05aUL, + 0x690ce0eeUL, 0x6dcdfd59UL, 0x608edb80UL, 0x644fc637UL, + 0x7a089632UL, 0x7ec98b85UL, 0x738aad5cUL, 0x774bb0ebUL, + 0x4f040d56UL, 0x4bc510e1UL, 0x46863638UL, 0x42472b8fUL, + 0x5c007b8aUL, 0x58c1663dUL, 0x558240e4UL, 0x51435d53UL, + 0x251d3b9eUL, 0x21dc2629UL, 0x2c9f00f0UL, 0x285e1d47UL, + 0x36194d42UL, 0x32d850f5UL, 0x3f9b762cUL, 0x3b5a6b9bUL, + 0x0315d626UL, 0x07d4cb91UL, 0x0a97ed48UL, 0x0e56f0ffUL, + 0x1011a0faUL, 0x14d0bd4dUL, 0x19939b94UL, 0x1d528623UL, + 0xf12f560eUL, 0xf5ee4bb9UL, 0xf8ad6d60UL, 0xfc6c70d7UL, + 0xe22b20d2UL, 0xe6ea3d65UL, 0xeba91bbcUL, 0xef68060bUL, + 0xd727bbb6UL, 0xd3e6a601UL, 0xdea580d8UL, 0xda649d6fUL, + 0xc423cd6aUL, 0xc0e2d0ddUL, 0xcda1f604UL, 0xc960ebb3UL, + 0xbd3e8d7eUL, 0xb9ff90c9UL, 0xb4bcb610UL, 0xb07daba7UL, + 0xae3afba2UL, 0xaafbe615UL, 0xa7b8c0ccUL, 0xa379dd7bUL, + 0x9b3660c6UL, 0x9ff77d71UL, 0x92b45ba8UL, 0x9675461fUL, + 0x8832161aUL, 0x8cf30badUL, 0x81b02d74UL, 0x857130c3UL, + 0x5d8a9099UL, 0x594b8d2eUL, 0x5408abf7UL, 0x50c9b640UL, + 0x4e8ee645UL, 0x4a4ffbf2UL, 0x470cdd2bUL, 0x43cdc09cUL, + 0x7b827d21UL, 0x7f436096UL, 0x7200464fUL, 0x76c15bf8UL, + 0x68860bfdUL, 0x6c47164aUL, 0x61043093UL, 0x65c52d24UL, + 0x119b4be9UL, 0x155a565eUL, 0x18197087UL, 0x1cd86d30UL, + 0x029f3d35UL, 0x065e2082UL, 0x0b1d065bUL, 0x0fdc1becUL, + 0x3793a651UL, 0x3352bbe6UL, 0x3e119d3fUL, 0x3ad08088UL, + 0x2497d08dUL, 0x2056cd3aUL, 0x2d15ebe3UL, 0x29d4f654UL, + 0xc5a92679UL, 0xc1683bceUL, 0xcc2b1d17UL, 0xc8ea00a0UL, + 0xd6ad50a5UL, 0xd26c4d12UL, 0xdf2f6bcbUL, 0xdbee767cUL, + 0xe3a1cbc1UL, 0xe760d676UL, 0xea23f0afUL, 0xeee2ed18UL, + 0xf0a5bd1dUL, 0xf464a0aaUL, 0xf9278673UL, 0xfde69bc4UL, + 0x89b8fd09UL, 0x8d79e0beUL, 0x803ac667UL, 0x84fbdbd0UL, + 0x9abc8bd5UL, 0x9e7d9662UL, 0x933eb0bbUL, 0x97ffad0cUL, + 0xafb010b1UL, 0xab710d06UL, 0xa6322bdfUL, 0xa2f33668UL, + 0xbcb4666dUL, 0xb8757bdaUL, 0xb5365d03UL, 0xb1f740b4UL }; /*---------------------------------------------*/ void initialiseCRC ( void ) { - globalCrc = 0xffffffffL; + globalCrc = 0xffffffffUL; } @@ -2579,7 +2585,7 @@ INLINE Int32 getRLEpair ( FILE* src ) /*--- Because I have no idea what kind of a value EOF is. ---*/ if (ch == EOF) { - ERROR_IF_NOT_ZERO ( errno ); + ERROR_IF_NOT_ZERO ( ferror(src)); return (1 << 16) | MY_EOF; } @@ -2595,7 +2601,7 @@ INLINE Int32 getRLEpair ( FILE* src ) if ( ungetc ( chLatest, src ) == EOF ) panic ( "getRLEpair: ungetc failed" ); } else { - ERROR_IF_NOT_ZERO ( errno ); + ERROR_IF_NOT_ZERO ( ferror(src) ); } /*--- Conditional is just a speedup hack. ---*/ @@ -3442,7 +3448,6 @@ void compress ( Char *name ) } /*--- Now the input and output handles are sane. Do the Biz. ---*/ - errno = 0; outputHandleJustInCase = outStr; compressStream ( inStr, outStr ); outputHandleJustInCase = NULL; @@ -3559,7 +3564,6 @@ void uncompress ( Char *name ) } /*--- Now the input and output handles are sane. Do the Biz. ---*/ - errno = 0; outputHandleJustInCase = outStr; magicNumberOK = uncompressStream ( inStr, outStr ); outputHandleJustInCase = NULL; @@ -3667,7 +3671,6 @@ void testf ( Char *name ) } /*--- Now the input handle is sane. Do the Biz. ---*/ - errno = 0; allOK = testStream ( inStr ); if (allOK && verbosity >= 1) fprintf ( stderr, "ok\n" ); @@ -3681,7 +3684,7 @@ void license ( void ) fprintf ( stderr, "bzip2, a block-sorting file compressor. " - "Version 0.1pl0, 17-Aug-97.\n" + "Version 0.1pl2, 29-Aug-97.\n" " \n" " Copyright (C) 1996, 1997 by Julian Seward.\n" " \n" @@ -3711,7 +3714,7 @@ void usage ( Char *fullProgName ) fprintf ( stderr, "bzip2, a block-sorting file compressor. " - "Version 0.1pl0, 17-Aug-97.\n" + "Version 0.1pl2, 29-Aug-97.\n" "\n usage: %s [flags and input files in any order]\n" "\n" " -h --help print this message\n" @@ -3864,7 +3867,6 @@ IntNative main ( IntNative argc, Char *argv[] ) tt = NULL; block = NULL; zptr = NULL; - errno = 0; smallMode = False; keepInputFiles = False; verbosity = 0; @@ -3980,6 +3982,12 @@ IntNative main ( IntNative argc, Char *argv[] ) exit ( 1 ); } + if (srcMode == SM_F2O && numFileNames == 0) { + fprintf ( stderr, "%s: -c expects at least one filename.\n", + progName ); + exit ( 1 ); + } + if (opMode == OM_TEST && srcMode == SM_F2O) { fprintf ( stderr, "%s: -c and -t cannot be used together.\n", progName ); diff --git a/bzip2.exe b/bzip2.exe deleted file mode 100644 index 4b3c4c1..0000000 Binary files a/bzip2.exe and /dev/null differ diff --git a/bzip2.txt b/bzip2.txt index 83366bc..aee8e2b 100644 --- a/bzip2.txt +++ b/bzip2.txt @@ -425,6 +425,9 @@ RELATIONSHIP TO bzip-0.21 to decompress files created by 0.21, but this would defeat the primary aim of having a patent-free compressor. + For a more precise statement about patent issues in bzip2, + please see the README file in the distribution. + Huffman coding necessarily involves some coding ineffi- ciency compared to arithmetic coding. This means that bzip2 compresses about 1% worse than 0.21, an unfortunate diff --git a/bzip2recover.c b/bzip2recover.c index efdfb3c..0eef0e6 100644 --- a/bzip2recover.c +++ b/bzip2recover.c @@ -7,7 +7,7 @@ /*-- This program is bzip2recover, a program to attempt data salvage from damaged files created by the accompanying - bzip2 program. + bzip2-0.1 program. Copyright (C) 1996, 1997 by Julian Seward. Guildford, Surrey, UK -- cgit v1.2.3 From 977101ad5f833f5c0a574bfeea408e5301a6b052 Mon Sep 17 00:00:00 2001 From: Julian Seward Date: Sun, 23 Aug 1998 22:13:13 +0200 Subject: bzip2-0.9.0c --- ALGORITHMS | 47 - CHANGES | 45 + LICENSE | 360 +----- Makefile | 52 +- README | 230 +--- README.DOS | 16 - blocksort.c | 709 ++++++++++ bzip2.1 | 191 ++- bzip2.1.preformatted | 318 +++-- bzip2.c | 3521 +++++++------------------------------------------- bzip2.txt | 292 ++--- bzip2recover.c | 125 +- bzlib.c | 1512 ++++++++++++++++++++++ bzlib.h | 299 +++++ bzlib_private.h | 523 ++++++++ compress.c | 588 +++++++++ crctable.c | 144 +++ decompress.c | 636 +++++++++ dlltest.c | 163 +++ dlltest.dsp | 93 ++ howbig.c | 37 + huffman.c | 228 ++++ libbz2.def | 25 + libbz2.dsp | 130 ++ manual.texi | 2100 ++++++++++++++++++++++++++++++ randtable.c | 124 ++ test.bat | 9 - test.cmd | 9 - words0 | 7 - words1 | 1 - words2 | 1 - words3 | 21 +- words3sh | 12 - 33 files changed, 8398 insertions(+), 4170 deletions(-) delete mode 100644 ALGORITHMS create mode 100644 CHANGES delete mode 100644 README.DOS create mode 100644 blocksort.c create mode 100644 bzlib.c create mode 100644 bzlib.h create mode 100644 bzlib_private.h create mode 100644 compress.c create mode 100644 crctable.c create mode 100644 decompress.c create mode 100644 dlltest.c create mode 100644 dlltest.dsp create mode 100644 howbig.c create mode 100644 huffman.c create mode 100644 libbz2.def create mode 100644 libbz2.dsp create mode 100644 manual.texi create mode 100644 randtable.c delete mode 100644 test.bat delete mode 100644 test.cmd delete mode 100644 words0 delete mode 100644 words3sh diff --git a/ALGORITHMS b/ALGORITHMS deleted file mode 100644 index 7c7d2ca..0000000 --- a/ALGORITHMS +++ /dev/null @@ -1,47 +0,0 @@ - -Bzip2 is not research work, in the sense that it doesn't present any -new ideas. Rather, it's an engineering exercise based on existing -ideas. - -Four documents describe essentially all the ideas behind bzip2: - - Michael Burrows and D. J. Wheeler: - "A block-sorting lossless data compression algorithm" - 10th May 1994. - Digital SRC Research Report 124. - ftp://ftp.digital.com/pub/DEC/SRC/research-reports/SRC-124.ps.gz - - Daniel S. Hirschberg and Debra A. LeLewer - "Efficient Decoding of Prefix Codes" - Communications of the ACM, April 1990, Vol 33, Number 4. - You might be able to get an electronic copy of this - from the ACM Digital Library. - - David J. Wheeler - Program bred3.c and accompanying document bred3.ps. - This contains the idea behind the multi-table Huffman - coding scheme. - ftp://ftp.cl.cam.ac.uk/pub/user/djw3/ - - Jon L. Bentley and Robert Sedgewick - "Fast Algorithms for Sorting and Searching Strings" - Available from Sedgewick's web page, - www.cs.princeton.edu/~rs - -The following paper gives valuable additional insights into the -algorithm, but is not immediately the basis of any code -used in bzip2. - - Peter Fenwick: - Block Sorting Text Compression - Proceedings of the 19th Australasian Computer Science Conference, - Melbourne, Australia. Jan 31 - Feb 2, 1996. - ftp://ftp.cs.auckland.ac.nz/pub/peter-f/ACSC96paper.ps - -All three are well written, and make fascinating reading. If you want -to modify bzip2 in any non-trivial way, I strongly suggest you obtain, -read and understand these papers. - -I am much indebted to the various authors for their help, support and -advice. - diff --git a/CHANGES b/CHANGES new file mode 100644 index 0000000..ac00f3a --- /dev/null +++ b/CHANGES @@ -0,0 +1,45 @@ + + +0.9.0 +~~~~~ +First version. + + +0.9.0a +~~~~~~ +Removed 'ranlib' from Makefile, since most modern Unix-es +don't need it, or even know about it. + + +0.9.0b +~~~~~~ +Fixed a problem with error reporting in bzip2.c. This does not effect +the library in any way. Problem is: versions 0.9.0 and 0.9.0a (of the +program proper) compress and decompress correctly, but give misleading +error messages (internal panics) when an I/O error occurs, instead of +reporting the problem correctly. This shouldn't give any data loss +(as far as I can see), but is confusing. + +Made the inline declarations disappear for non-GCC compilers. + + +0.9.0c +~~~~~~ +Fixed some problems in the library pertaining to some boundary cases. +This makes the library behave more correctly in those situations. The +fixes apply only to features (calls and parameters) not used by +bzip2.c, so the non-fixedness of them in previous versions has no +effect on reliability of bzip2.c. + +In bzlib.c: + * made zero-length BZ_FLUSH work correctly in bzCompress(). + * fixed bzWrite/bzRead to ignore zero-length requests. + * fixed bzread to correctly handle read requests after EOF. + * wrong parameter order in call to bzDecompressInit in + bzBuffToBuffDecompress. Fixed. + +In compress.c: + * changed setting of nGroups in sendMTFValues() so as to + do a bit better on small files. This _does_ effect + bzip2.c. + diff --git a/LICENSE b/LICENSE index a43ea21..3de0301 100644 --- a/LICENSE +++ b/LICENSE @@ -1,339 +1,39 @@ - GNU GENERAL PUBLIC LICENSE - Version 2, June 1991 - Copyright (C) 1989, 1991 Free Software Foundation, Inc. - 675 Mass Ave, Cambridge, MA 02139, USA - Everyone is permitted to copy and distribute verbatim copies - of this license document, but changing it is not allowed. +This program, "bzip2" and associated library "libbzip2", are +copyright (C) 1996-1998 Julian R Seward. All rights reserved. - Preamble +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: - The licenses for most software are designed to take away your -freedom to share and change it. By contrast, the GNU General Public -License is intended to guarantee your freedom to share and change free -software--to make sure the software is free for all its users. This -General Public License applies to most of the Free Software -Foundation's software and to any other program whose authors commit to -using it. (Some other Free Software Foundation software is covered by -the GNU Library General Public License instead.) You can apply it to -your programs, too. +1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. - When we speak of free software, we are referring to freedom, not -price. Our General Public Licenses are designed to make sure that you -have the freedom to distribute copies of free software (and charge for -this service if you wish), that you receive source code or can get it -if you want it, that you can change the software or use pieces of it -in new free programs; and that you know you can do these things. +2. The origin of this software must not be misrepresented; you must + not claim that you wrote the original software. If you use this + software in a product, an acknowledgment in the product + documentation would be appreciated but is not required. - To protect your rights, we need to make restrictions that forbid -anyone to deny you these rights or to ask you to surrender the rights. -These restrictions translate to certain responsibilities for you if you -distribute copies of the software, or if you modify it. +3. Altered source versions must be plainly marked as such, and must + not be misrepresented as being the original software. - For example, if you distribute copies of such a program, whether -gratis or for a fee, you must give the recipients all the rights that -you have. You must make sure that they, too, receive or can get the -source code. And you must show them these terms so they know their -rights. +4. The name of the author may not be used to endorse or promote + products derived from this software without specific prior written + permission. - We protect your rights with two steps: (1) copyright the software, and -(2) offer you this license which gives you legal permission to copy, -distribute and/or modify the software. +THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS +OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY +DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE +GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING +NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - Also, for each author's protection and ours, we want to make certain -that everyone understands that there is no warranty for this free -software. If the software is modified by someone else and passed on, we -want its recipients to know that what they have is not the original, so -that any problems introduced by others will not reflect on the original -authors' reputations. +Julian Seward, Guildford, Surrey, UK. +jseward@acm.org +bzip2/libbzip2 version 0.9.0 of 28 June 1998 - Finally, any free program is threatened constantly by software -patents. We wish to avoid the danger that redistributors of a free -program will individually obtain patent licenses, in effect making the -program proprietary. To prevent this, we have made it clear that any -patent must be licensed for everyone's free use or not licensed at all. - - The precise terms and conditions for copying, distribution and -modification follow. - - GNU GENERAL PUBLIC LICENSE - TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION - - 0. This License applies to any program or other work which contains -a notice placed by the copyright holder saying it may be distributed -under the terms of this General Public License. The "Program", below, -refers to any such program or work, and a "work based on the Program" -means either the Program or any derivative work under copyright law: -that is to say, a work containing the Program or a portion of it, -either verbatim or with modifications and/or translated into another -language. (Hereinafter, translation is included without limitation in -the term "modification".) Each licensee is addressed as "you". - -Activities other than copying, distribution and modification are not -covered by this License; they are outside its scope. The act of -running the Program is not restricted, and the output from the Program -is covered only if its contents constitute a work based on the -Program (independent of having been made by running the Program). -Whether that is true depends on what the Program does. - - 1. You may copy and distribute verbatim copies of the Program's -source code as you receive it, in any medium, provided that you -conspicuously and appropriately publish on each copy an appropriate -copyright notice and disclaimer of warranty; keep intact all the -notices that refer to this License and to the absence of any warranty; -and give any other recipients of the Program a copy of this License -along with the Program. - -You may charge a fee for the physical act of transferring a copy, and -you may at your option offer warranty protection in exchange for a fee. - - 2. You may modify your copy or copies of the Program or any portion -of it, thus forming a work based on the Program, and copy and -distribute such modifications or work under the terms of Section 1 -above, provided that you also meet all of these conditions: - - a) You must cause the modified files to carry prominent notices - stating that you changed the files and the date of any change. - - b) You must cause any work that you distribute or publish, that in - whole or in part contains or is derived from the Program or any - part thereof, to be licensed as a whole at no charge to all third - parties under the terms of this License. - - c) If the modified program normally reads commands interactively - when run, you must cause it, when started running for such - interactive use in the most ordinary way, to print or display an - announcement including an appropriate copyright notice and a - notice that there is no warranty (or else, saying that you provide - a warranty) and that users may redistribute the program under - these conditions, and telling the user how to view a copy of this - License. (Exception: if the Program itself is interactive but - does not normally print such an announcement, your work based on - the Program is not required to print an announcement.) - -These requirements apply to the modified work as a whole. If -identifiable sections of that work are not derived from the Program, -and can be reasonably considered independent and separate works in -themselves, then this License, and its terms, do not apply to those -sections when you distribute them as separate works. But when you -distribute the same sections as part of a whole which is a work based -on the Program, the distribution of the whole must be on the terms of -this License, whose permissions for other licensees extend to the -entire whole, and thus to each and every part regardless of who wrote it. - -Thus, it is not the intent of this section to claim rights or contest -your rights to work written entirely by you; rather, the intent is to -exercise the right to control the distribution of derivative or -collective works based on the Program. - -In addition, mere aggregation of another work not based on the Program -with the Program (or with a work based on the Program) on a volume of -a storage or distribution medium does not bring the other work under -the scope of this License. - - 3. You may copy and distribute the Program (or a work based on it, -under Section 2) in object code or executable form under the terms of -Sections 1 and 2 above provided that you also do one of the following: - - a) Accompany it with the complete corresponding machine-readable - source code, which must be distributed under the terms of Sections - 1 and 2 above on a medium customarily used for software interchange; or, - - b) Accompany it with a written offer, valid for at least three - years, to give any third party, for a charge no more than your - cost of physically performing source distribution, a complete - machine-readable copy of the corresponding source code, to be - distributed under the terms of Sections 1 and 2 above on a medium - customarily used for software interchange; or, - - c) Accompany it with the information you received as to the offer - to distribute corresponding source code. (This alternative is - allowed only for noncommercial distribution and only if you - received the program in object code or executable form with such - an offer, in accord with Subsection b above.) - -The source code for a work means the preferred form of the work for -making modifications to it. For an executable work, complete source -code means all the source code for all modules it contains, plus any -associated interface definition files, plus the scripts used to -control compilation and installation of the executable. However, as a -special exception, the source code distributed need not include -anything that is normally distributed (in either source or binary -form) with the major components (compiler, kernel, and so on) of the -operating system on which the executable runs, unless that component -itself accompanies the executable. - -If distribution of executable or object code is made by offering -access to copy from a designated place, then offering equivalent -access to copy the source code from the same place counts as -distribution of the source code, even though third parties are not -compelled to copy the source along with the object code. - - 4. You may not copy, modify, sublicense, or distribute the Program -except as expressly provided under this License. Any attempt -otherwise to copy, modify, sublicense or distribute the Program is -void, and will automatically terminate your rights under this License. -However, parties who have received copies, or rights, from you under -this License will not have their licenses terminated so long as such -parties remain in full compliance. - - 5. You are not required to accept this License, since you have not -signed it. However, nothing else grants you permission to modify or -distribute the Program or its derivative works. These actions are -prohibited by law if you do not accept this License. Therefore, by -modifying or distributing the Program (or any work based on the -Program), you indicate your acceptance of this License to do so, and -all its terms and conditions for copying, distributing or modifying -the Program or works based on it. - - 6. Each time you redistribute the Program (or any work based on the -Program), the recipient automatically receives a license from the -original licensor to copy, distribute or modify the Program subject to -these terms and conditions. You may not impose any further -restrictions on the recipients' exercise of the rights granted herein. -You are not responsible for enforcing compliance by third parties to -this License. - - 7. If, as a consequence of a court judgment or allegation of patent -infringement or for any other reason (not limited to patent issues), -conditions are imposed on you (whether by court order, agreement or -otherwise) that contradict the conditions of this License, they do not -excuse you from the conditions of this License. If you cannot -distribute so as to satisfy simultaneously your obligations under this -License and any other pertinent obligations, then as a consequence you -may not distribute the Program at all. For example, if a patent -license would not permit royalty-free redistribution of the Program by -all those who receive copies directly or indirectly through you, then -the only way you could satisfy both it and this License would be to -refrain entirely from distribution of the Program. - -If any portion of this section is held invalid or unenforceable under -any particular circumstance, the balance of the section is intended to -apply and the section as a whole is intended to apply in other -circumstances. - -It is not the purpose of this section to induce you to infringe any -patents or other property right claims or to contest validity of any -such claims; this section has the sole purpose of protecting the -integrity of the free software distribution system, which is -implemented by public license practices. Many people have made -generous contributions to the wide range of software distributed -through that system in reliance on consistent application of that -system; it is up to the author/donor to decide if he or she is willing -to distribute software through any other system and a licensee cannot -impose that choice. - -This section is intended to make thoroughly clear what is believed to -be a consequence of the rest of this License. - - 8. If the distribution and/or use of the Program is restricted in -certain countries either by patents or by copyrighted interfaces, the -original copyright holder who places the Program under this License -may add an explicit geographical distribution limitation excluding -those countries, so that distribution is permitted only in or among -countries not thus excluded. In such case, this License incorporates -the limitation as if written in the body of this License. - - 9. The Free Software Foundation may publish revised and/or new versions -of the General Public License from time to time. Such new versions will -be similar in spirit to the present version, but may differ in detail to -address new problems or concerns. - -Each version is given a distinguishing version number. If the Program -specifies a version number of this License which applies to it and "any -later version", you have the option of following the terms and conditions -either of that version or of any later version published by the Free -Software Foundation. If the Program does not specify a version number of -this License, you may choose any version ever published by the Free Software -Foundation. - - 10. If you wish to incorporate parts of the Program into other free -programs whose distribution conditions are different, write to the author -to ask for permission. For software which is copyrighted by the Free -Software Foundation, write to the Free Software Foundation; we sometimes -make exceptions for this. Our decision will be guided by the two goals -of preserving the free status of all derivatives of our free software and -of promoting the sharing and reuse of software generally. - - NO WARRANTY - - 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY -FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN -OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES -PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED -OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF -MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS -TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE -PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, -REPAIR OR CORRECTION. - - 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING -WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR -REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, -INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING -OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED -TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY -YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER -PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE -POSSIBILITY OF SUCH DAMAGES. - - END OF TERMS AND CONDITIONS - - Appendix: How to Apply These Terms to Your New Programs - - If you develop a new program, and you want it to be of the greatest -possible use to the public, the best way to achieve this is to make it -free software which everyone can redistribute and change under these terms. - - To do so, attach the following notices to the program. It is safest -to attach them to the start of each source file to most effectively -convey the exclusion of warranty; and each file should have at least -the "copyright" line and a pointer to where the full notice is found. - - - Copyright (C) 19yy - - This program is free software; you can redistribute it and/or modify - it under the terms of the GNU General Public License as published by - the Free Software Foundation; either version 2 of the License, or - (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -Also add information on how to contact you by electronic and paper mail. - -If the program is interactive, make it output a short notice like this -when it starts in an interactive mode: - - Gnomovision version 69, Copyright (C) 19yy name of author - Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. - This is free software, and you are welcome to redistribute it - under certain conditions; type `show c' for details. - -The hypothetical commands `show w' and `show c' should show the appropriate -parts of the General Public License. Of course, the commands you use may -be called something other than `show w' and `show c'; they could even be -mouse-clicks or menu items--whatever suits your program. - -You should also get your employer (if you work as a programmer) or your -school, if any, to sign a "copyright disclaimer" for the program, if -necessary. Here is a sample; alter the names: - - Yoyodyne, Inc., hereby disclaims all copyright interest in the program - `Gnomovision' (which makes passes at compilers) written by James Hacker. - - , 1 April 1989 - Ty Coon, President of Vice - -This General Public License does not permit incorporating your program into -proprietary programs. If your program is a subroutine library, you may -consider it more useful to permit linking proprietary applications with the -library. If this is what you want to do, use the GNU Library General -Public License instead of this License. diff --git a/Makefile b/Makefile index 9d35b43..8ebea66 100644 --- a/Makefile +++ b/Makefile @@ -1,30 +1,46 @@ -CC = gcc -SH = /bin/sh - -CFLAGS = -O3 -fomit-frame-pointer -funroll-loops - +CC=gcc +CFLAGS=-Wall -O2 -fomit-frame-pointer -fno-strength-reduce + +OBJS= blocksort.o \ + huffman.o \ + crctable.o \ + randtable.o \ + compress.o \ + decompress.o \ + bzlib.o + +all: lib bzip2 test + +bzip2: lib + $(CC) $(CFLAGS) -c bzip2.c + $(CC) $(CFLAGS) -o bzip2 bzip2.o -L. -lbz2 + $(CC) $(CFLAGS) -o bzip2recover bzip2recover.c +lib: $(OBJS) + rm -f libbz2.a + ar clq libbz2.a $(OBJS) -all: - cat words0 - $(CC) $(CFLAGS) -o bzip2 bzip2.c - $(CC) $(CFLAGS) -o bzip2recover bzip2recover.c - rm -f bunzip2 - ln -s ./bzip2 ./bunzip2 - cat words1 +test: bzip2 + @cat words1 ./bzip2 -1 < sample1.ref > sample1.rb2 ./bzip2 -2 < sample2.ref > sample2.rb2 - ./bunzip2 < sample1.bz2 > sample1.tst - ./bunzip2 < sample2.bz2 > sample2.tst - cat words2 + ./bzip2 -d < sample1.bz2 > sample1.tst + ./bzip2 -d < sample2.bz2 > sample2.tst + @cat words2 cmp sample1.bz2 sample1.rb2 cmp sample2.bz2 sample2.rb2 cmp sample1.tst sample1.ref cmp sample2.tst sample2.ref - cat words3 + @cat words3 + + +clean: + rm -f *.o libbz2.a bzip2 bzip2recover sample1.rb2 sample2.rb2 sample1.tst sample2.tst +.c.o: $*.o bzlib.h bzlib_private.h + $(CC) $(CFLAGS) -c $*.c -o $*.o -clean: - rm -f bzip2 bunzip2 bzip2recover sample*.tst sample*.rb2 +tarfile: + tar cvf interim.tar *.c *.h Makefile manual.texi manual.ps LICENSE bzip2.1 bzip2.1.preformatted bzip2.txt words1 words2 words3 sample1.ref sample2.ref sample1.bz2 sample2.bz2 *.html README CHANGES libbz2.def libbz2.dsp dlltest.dsp diff --git a/README b/README index d58bb49..2f59ef7 100644 --- a/README +++ b/README @@ -1,194 +1,61 @@ -GREETINGS! - This is the README for bzip2, my block-sorting file compressor, - version 0.1. +This is the README for bzip2, a block-sorting file compressor, version +0.9.0. This version is fully compatible with the previous public +release, bzip2-0.1pl2. - bzip2 is distributed under the GNU General Public License version 2; - for details, see the file LICENSE. Pointers to the algorithms used - are in ALGORITHMS. Instructions for use are in bzip2.1.preformatted. +bzip2-0.9.0 is distributed under a BSD-style license. For details, +see the file LICENSE. - Please read all of this file carefully. +Complete documentation is available in Postscript form (manual.ps) +or html (manual_toc.html). A plain-text version of the manual page is +available as bzip2.txt. +HOW TO BUILD -- UNIX -HOW TO BUILD +Type `make'. - -- for UNIX: +This creates binaries "bzip2" and "bzip2recover". - Type `make'. (tough, huh? :-) +It also runs four compress-decompress tests to make sure things are +working properly. If all goes well, you should be up & running. +Please be sure to read the output from `make' just to be sure that the +tests went ok. - This creates binaries "bzip2", and "bunzip2", - which is a symbolic link to "bzip2". +To install bzip2 properly: - It also runs four compress-decompress tests to make sure - things are working properly. If all goes well, you should be up & - running. Please be sure to read the output from `make' - just to be sure that the tests went ok. +* Copy the binaries "bzip2" and "bzip2recover" to a publically visible + place, possibly /usr/bin or /usr/local/bin. - To install bzip2 properly: +* In that directory, make "bunzip2" and "bzcat" be symbolic links + to "bzip2". - -- Copy the binary "bzip2" to a publically visible place, - possibly /usr/bin, /usr/common/bin or /usr/local/bin. - - -- In that directory, make "bunzip2" be a symbolic link - to "bzip2". - - -- Copy the manual page, bzip2.1, to the relevant place. - Probably the right place is /usr/man/man1/. - - -- for Windows 95 and NT: +* Copy the manual page, bzip2.1, to the relevant place. + Probably the right place is /usr/man/man1/. - For a start, do you *really* want to recompile bzip2? - The standard distribution includes a pre-compiled version - for Windows 95 and NT, `bzip2.exe'. +If you want to program with the library, you'll need to copy libbz2.a +and bzlib.h to /usr/lib and /usr/include respectively. + - This executable was created with Jacob Navia's excellent - port to Win32 of Chris Fraser & David Hanson's excellent - ANSI C compiler, "lcc". You can get to it at the pages - of the CS department of Princeton University, - www.cs.princeton.edu. - I have not tried to compile this version of bzip2 with - a commercial C compiler such as MS Visual C, as I don't - have one available. - - Note that lcc is designed primarily to be portable and - fast. Code quality is a secondary aim, so bzip2.exe - runs perhaps 40% slower than it could if compiled with - a good optimising compiler. - - I compiled a previous version of bzip (0.21) with Borland - C 5.0, which worked fine, and with MS VC++ 2.0, which - didn't. Here is an comment from the README for bzip-0.21. - - MS VC++ 2.0's optimising compiler has a bug which, at - maximum optimisation, gives an executable which produces - garbage compressed files. Proceed with caution. - I do not know whether or not this happens with later - versions of VC++. - - Edit the defines starting at line 86 of bzip.c to - select your platform/compiler combination, and then compile. - Then check that the resulting executable (assumed to be - called bzip.exe) works correctly, using the SELFTEST.BAT file. - Bearing in mind the previous paragraph, the self-test is - important. - - Note that the defines which bzip-0.21 had, to support - compilation with VC 2.0 and BC 5.0, are gone. Windows - is not my preferred operating system, and I am, for the - moment, content with the modestly fast executable created - by lcc-win32. - - A manual page is supplied, unformatted (bzip2.1), - preformatted (bzip2.1.preformatted), and preformatted - and sanitised for MS-DOS (bzip2.txt). - - - -COMPILATION NOTES - - bzip2 should work on any 32 or 64-bit machine. It is known to work - [meaning: it has compiled and passed self-tests] on the - following platform-os combinations: - - Intel i386/i486 running Linux 2.0.21 - Sun Sparcs (various) running SunOS 4.1.4 and Solaris 2.5 - Intel i386/i486 running Windows 95 and NT - DEC Alpha running Digital Unix 4.0 - - Following the release of bzip-0.21, many people mailed me - from around the world to say they had made it work on all sorts - of weird and wonderful machines. Chances are, if you have - a reasonable ANSI C compiler and a 32-bit machine, you can - get it to work. - - The #defines starting at around line 82 of bzip2.c supply some - degree of platform-independance. If you configure bzip2 for some - new far-out platform which is not covered by the existing definitions, - please send me the relevant definitions. - - I recommend GNU C for compilation. The code is standard ANSI C, - except for the Unix-specific file handling, so any ANSI C compiler - should work. Note however that the many routines marked INLINE - should be inlined by your compiler, else performance will be very - poor. Asking your compiler to unroll loops gives some - small improvement too; for gcc, the relevant flag is - -funroll-loops. - - On a 386/486 machines, I'd recommend giving gcc the - -fomit-frame-pointer flag; this liberates another register for - allocation, which measurably improves performance. - - I used the abovementioned lcc compiler to develop bzip2. - I would highly recommend this compiler for day-to-day development; - it is fast, reliable, lightweight, has an excellent profiler, - and is generally excellent. And it's fun to retarget, if you're - into that kind of thing. - - If you compile bzip2 on a new platform or with a new compiler, - please be sure to run the four compress-decompress tests, either - using the Makefile, or with the test.bat (MSDOS) or test.cmd (OS/2) - files. Some compilers have been seen to introduce subtle bugs - when optimising, so this check is important. Ideally you should - then go on to test bzip2 on a file several megabytes or even - tens of megabytes long, just to be 110% sure. ``Professional - programmers are paranoid programmers.'' (anon). +HOW TO BUILD -- Windows 95, NT, DOS, Mac, etc. +It's difficult for me to support compilation on all these platforms. +My approach is to collect binaries for these platforms, and put them +on my web page (http://www.muraroa.demon.co.uk). Look there. VALIDATION - Correct operation, in the sense that a compressed file can always be - decompressed to reproduce the original, is obviously of paramount - importance. To validate bzip2, I used a modified version of - Mark Nelson's churn program. Churn is an automated test driver - which recursively traverses a directory structure, using bzip2 to - compress and then decompress each file it encounters, and checking - that the decompressed data is the same as the original. As test - material, I used several runs over several filesystems of differing - sizes. - - One set of tests was done on my base Linux filesystem, - 410 megabytes in 23,000 files. There were several runs over - this filesystem, in various configurations designed to break bzip2. - That filesystem also contained some specially constructed test - files designed to exercise boundary cases in the code. - This included files of zero length, various long, highly repetitive - files, and some files which generate blocks with all values the same. +Correct operation, in the sense that a compressed file can always be +decompressed to reproduce the original, is obviously of paramount +importance. To validate bzip2, I used a modified version of Mark +Nelson's churn program. Churn is an automated test driver which +recursively traverses a directory structure, using bzip2 to compress +and then decompress each file it encounters, and checking that the +decompressed data is the same as the original. There are more details +in Section 4 of the user guide. - The other set of tests was done just with the "normal" configuration, - but on a much larger quantity of data. - - Tests are: - - Linux FS, 410M, 23000 files - - As above, with --repetitive-fast - - As above, with -1 - - Low level disk image of a disk containing - Windows NT4.0; 420M in a single huge file - - Linux distribution, incl Slackware, - all GNU sources. 1900M in 2300 files. - - Approx ~100M compiler sources and related - programming tools, running under Purify. - - About 500M of data in 120 files of around - 4 M each. This is raw data from a - biomagnetometer (SQUID-based thing). - - Overall, total volume of test data is about - 3300 megabytes in 25000 files. - - The distribution does four tests after building bzip. These tests - include test decompressions of pre-supplied compressed files, so - they not only test that bzip works correctly on the machine it was - built on, but can also decompress files compressed on a different - machine. This guards against unforseen interoperability problems. Please read and be aware of the following: @@ -234,14 +101,30 @@ PATENTS: End of legalities. +WHAT'S NEW IN 0.9.0 (as compared to 0.1pl2) ? + + * Approx 10% faster compression, 30% faster decompression + * -t (test mode) is a lot quicker + * Can decompress concatenated compressed files + * Programming interface, so programs can directly read/write .bz2 files + * Less restrictive (BSD-style) licensing + * Flag handling more compatible with GNU gzip + * Much more documentation, i.e., a proper user manual + * Hopefully, improved portability (at least of the library) + + I hope you find bzip2 useful. Feel free to contact me at jseward@acm.org if you have any suggestions or queries. Many people mailed me with -comments, suggestions and patches after the releases of 0.15 and 0.21, -and the changes in bzip2 are largely a result of this feedback. -I thank you for your comments. +comments, suggestions and patches after the releases of bzip-0.15, +bzip-0.21 and bzip2-0.1pl2, and the changes in bzip2 are largely a +result of this feedback. I thank you for your comments. + +At least for the time being, bzip2's "home" is +http://www.muraroa.demon.co.uk. Julian Seward +jseward@acm.org Manchester, UK 18 July 1996 (version 0.15) @@ -250,4 +133,5 @@ Manchester, UK Guildford, Surrey, UK 7 August 1997 (bzip2, version 0.1) 29 August 1997 (bzip2, version 0.1pl2) +23 August 1998 (bzip2, version 0.9.0) diff --git a/README.DOS b/README.DOS deleted file mode 100644 index 048de8c..0000000 --- a/README.DOS +++ /dev/null @@ -1,16 +0,0 @@ - -As of today (3 March 1998) I've removed the -Win95/NT executables from this distribution, sorry. - -You can still get an executable from -http://www.muraroa.demon.co.uk, or (as a last -resort) by mailing me at jseward@acm.org. - -The reason for this change of packaging is that it -makes it easier for me to fix problems with specific -executables if they are not included in the main -distribution. - -J - - diff --git a/blocksort.c b/blocksort.c new file mode 100644 index 0000000..d8bb26a --- /dev/null +++ b/blocksort.c @@ -0,0 +1,709 @@ + +/*-------------------------------------------------------------*/ +/*--- Block sorting machinery ---*/ +/*--- blocksort.c ---*/ +/*-------------------------------------------------------------*/ + +/*-- + This file is a part of bzip2 and/or libbzip2, a program and + library for lossless, block-sorting data compression. + + Copyright (C) 1996-1998 Julian R Seward. All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + 1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + 2. The origin of this software must not be misrepresented; you must + not claim that you wrote the original software. If you use this + software in a product, an acknowledgment in the product + documentation would be appreciated but is not required. + + 3. Altered source versions must be plainly marked as such, and must + not be misrepresented as being the original software. + + 4. The name of the author may not be used to endorse or promote + products derived from this software without specific prior written + permission. + + THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS + OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED + WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY + DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE + GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS + INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, + WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING + NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS + SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + Julian Seward, Guildford, Surrey, UK. + jseward@acm.org + bzip2/libbzip2 version 0.9.0c of 18 October 1998 + + This program is based on (at least) the work of: + Mike Burrows + David Wheeler + Peter Fenwick + Alistair Moffat + Radford Neal + Ian H. Witten + Robert Sedgewick + Jon L. Bentley + + For more information on these sources, see the manual. +--*/ + + +#include "bzlib_private.h" + +/*---------------------------------------------*/ +/*-- + Compare two strings in block. We assume (see + discussion above) that i1 and i2 have a max + offset of 10 on entry, and that the first + bytes of both block and quadrant have been + copied into the "overshoot area", ie + into the subscript range + [nblock .. nblock+NUM_OVERSHOOT_BYTES-1]. +--*/ +static __inline__ Bool fullGtU ( UChar* block, + UInt16* quadrant, + UInt32 nblock, + Int32* workDone, + Int32 i1, + Int32 i2 + ) +{ + Int32 k; + UChar c1, c2; + UInt16 s1, s2; + + AssertD ( i1 != i2, "fullGtU(1)" ); + + c1 = block[i1]; + c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + i1++; i2++; + + c1 = block[i1]; + c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + i1++; i2++; + + c1 = block[i1]; + c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + i1++; i2++; + + c1 = block[i1]; + c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + i1++; i2++; + + c1 = block[i1]; + c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + i1++; i2++; + + c1 = block[i1]; + c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + i1++; i2++; + + k = nblock; + + do { + + c1 = block[i1]; + c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + s1 = quadrant[i1]; + s2 = quadrant[i2]; + if (s1 != s2) return (s1 > s2); + i1++; i2++; + + c1 = block[i1]; + c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + s1 = quadrant[i1]; + s2 = quadrant[i2]; + if (s1 != s2) return (s1 > s2); + i1++; i2++; + + c1 = block[i1]; + c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + s1 = quadrant[i1]; + s2 = quadrant[i2]; + if (s1 != s2) return (s1 > s2); + i1++; i2++; + + c1 = block[i1]; + c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + s1 = quadrant[i1]; + s2 = quadrant[i2]; + if (s1 != s2) return (s1 > s2); + i1++; i2++; + + if (i1 >= nblock) i1 -= nblock; + if (i2 >= nblock) i2 -= nblock; + + k -= 4; + (*workDone)++; + } + while (k >= 0); + + return False; +} + +/*---------------------------------------------*/ +/*-- + Knuth's increments seem to work better + than Incerpi-Sedgewick here. Possibly + because the number of elems to sort is + usually small, typically <= 20. +--*/ +static Int32 incs[14] = { 1, 4, 13, 40, 121, 364, 1093, 3280, + 9841, 29524, 88573, 265720, + 797161, 2391484 }; + +static void simpleSort ( EState* s, Int32 lo, Int32 hi, Int32 d ) +{ + Int32 i, j, h, bigN, hp; + Int32 v; + + UChar* block = s->block; + UInt32* zptr = s->zptr; + UInt16* quadrant = s->quadrant; + Int32* workDone = &(s->workDone); + Int32 nblock = s->nblock; + Int32 workLimit = s->workLimit; + Bool firstAttempt = s->firstAttempt; + + bigN = hi - lo + 1; + if (bigN < 2) return; + + hp = 0; + while (incs[hp] < bigN) hp++; + hp--; + + for (; hp >= 0; hp--) { + h = incs[hp]; + i = lo + h; + while (True) { + + /*-- copy 1 --*/ + if (i > hi) break; + v = zptr[i]; + j = i; + while ( fullGtU ( block, quadrant, nblock, workDone, + zptr[j-h]+d, v+d ) ) { + zptr[j] = zptr[j-h]; + j = j - h; + if (j <= (lo + h - 1)) break; + } + zptr[j] = v; + i++; + + /*-- copy 2 --*/ + if (i > hi) break; + v = zptr[i]; + j = i; + while ( fullGtU ( block, quadrant, nblock, workDone, + zptr[j-h]+d, v+d ) ) { + zptr[j] = zptr[j-h]; + j = j - h; + if (j <= (lo + h - 1)) break; + } + zptr[j] = v; + i++; + + /*-- copy 3 --*/ + if (i > hi) break; + v = zptr[i]; + j = i; + while ( fullGtU ( block, quadrant, nblock, workDone, + zptr[j-h]+d, v+d ) ) { + zptr[j] = zptr[j-h]; + j = j - h; + if (j <= (lo + h - 1)) break; + } + zptr[j] = v; + i++; + + if (*workDone > workLimit && firstAttempt) return; + } + } +} + + +/*---------------------------------------------*/ +/*-- + The following is an implementation of + an elegant 3-way quicksort for strings, + described in a paper "Fast Algorithms for + Sorting and Searching Strings", by Robert + Sedgewick and Jon L. Bentley. +--*/ + +#define swap(lv1, lv2) \ + { Int32 tmp = lv1; lv1 = lv2; lv2 = tmp; } + +static void vswap ( UInt32* zptr, Int32 p1, Int32 p2, Int32 n ) +{ + while (n > 0) { + swap(zptr[p1], zptr[p2]); + p1++; p2++; n--; + } +} + +static UChar med3 ( UChar a, UChar b, UChar c ) +{ + UChar t; + if (a > b) { t = a; a = b; b = t; }; + if (b > c) { t = b; b = c; c = t; }; + if (a > b) b = a; + return b; +} + + +#define min(a,b) ((a) < (b)) ? (a) : (b) + +typedef + struct { Int32 ll; Int32 hh; Int32 dd; } + StackElem; + +#define push(lz,hz,dz) { stack[sp].ll = lz; \ + stack[sp].hh = hz; \ + stack[sp].dd = dz; \ + sp++; } + +#define pop(lz,hz,dz) { sp--; \ + lz = stack[sp].ll; \ + hz = stack[sp].hh; \ + dz = stack[sp].dd; } + +#define SMALL_THRESH 20 +#define DEPTH_THRESH 10 + +/*-- + If you are ever unlucky/improbable enough + to get a stack overflow whilst sorting, + increase the following constant and try + again. In practice I have never seen the + stack go above 27 elems, so the following + limit seems very generous. +--*/ +#define QSORT_STACK_SIZE 1000 + + +static void qSort3 ( EState* s, Int32 loSt, Int32 hiSt, Int32 dSt ) +{ + Int32 unLo, unHi, ltLo, gtHi, med, n, m; + Int32 sp, lo, hi, d; + StackElem stack[QSORT_STACK_SIZE]; + + UChar* block = s->block; + UInt32* zptr = s->zptr; + Int32* workDone = &(s->workDone); + Int32 workLimit = s->workLimit; + Bool firstAttempt = s->firstAttempt; + + sp = 0; + push ( loSt, hiSt, dSt ); + + while (sp > 0) { + + AssertH ( sp < QSORT_STACK_SIZE, 1001 ); + + pop ( lo, hi, d ); + + if (hi - lo < SMALL_THRESH || d > DEPTH_THRESH) { + simpleSort ( s, lo, hi, d ); + if (*workDone > workLimit && firstAttempt) return; + continue; + } + + med = med3 ( block[zptr[ lo ]+d], + block[zptr[ hi ]+d], + block[zptr[ (lo+hi)>>1 ]+d] ); + + unLo = ltLo = lo; + unHi = gtHi = hi; + + while (True) { + while (True) { + if (unLo > unHi) break; + n = ((Int32)block[zptr[unLo]+d]) - med; + if (n == 0) { swap(zptr[unLo], zptr[ltLo]); ltLo++; unLo++; continue; }; + if (n > 0) break; + unLo++; + } + while (True) { + if (unLo > unHi) break; + n = ((Int32)block[zptr[unHi]+d]) - med; + if (n == 0) { swap(zptr[unHi], zptr[gtHi]); gtHi--; unHi--; continue; }; + if (n < 0) break; + unHi--; + } + if (unLo > unHi) break; + swap(zptr[unLo], zptr[unHi]); unLo++; unHi--; + } + + AssertD ( unHi == unLo-1, "bad termination in qSort3" ); + + if (gtHi < ltLo) { + push(lo, hi, d+1 ); + continue; + } + + n = min(ltLo-lo, unLo-ltLo); vswap(zptr, lo, unLo-n, n); + m = min(hi-gtHi, gtHi-unHi); vswap(zptr, unLo, hi-m+1, m); + + n = lo + unLo - ltLo - 1; + m = hi - (gtHi - unHi) + 1; + + push ( lo, n, d ); + push ( n+1, m-1, d+1 ); + push ( m, hi, d ); + } +} + + +/*---------------------------------------------*/ + +#define BIGFREQ(b) (ftab[((b)+1) << 8] - ftab[(b) << 8]) + +#define SETMASK (1 << 21) +#define CLEARMASK (~(SETMASK)) + +static void sortMain ( EState* s ) +{ + Int32 i, j, k, ss, sb; + Int32 runningOrder[256]; + Int32 copy[256]; + Bool bigDone[256]; + UChar c1, c2; + Int32 numQSorted; + + UChar* block = s->block; + UInt32* zptr = s->zptr; + UInt16* quadrant = s->quadrant; + Int32* ftab = s->ftab; + Int32* workDone = &(s->workDone); + Int32 nblock = s->nblock; + Int32 workLimit = s->workLimit; + Bool firstAttempt = s->firstAttempt; + + /*-- + In the various block-sized structures, live data runs + from 0 to last+NUM_OVERSHOOT_BYTES inclusive. First, + set up the overshoot area for block. + --*/ + + if (s->verbosity >= 4) + VPrintf0( " sort initialise ...\n" ); + + for (i = 0; i < BZ_NUM_OVERSHOOT_BYTES; i++) + block[nblock+i] = block[i % nblock]; + for (i = 0; i < nblock+BZ_NUM_OVERSHOOT_BYTES; i++) + quadrant[i] = 0; + + + if (nblock <= 4000) { + + /*-- + Use simpleSort(), since the full sorting mechanism + has quite a large constant overhead. + --*/ + if (s->verbosity >= 4) VPrintf0( " simpleSort ...\n" ); + for (i = 0; i < nblock; i++) zptr[i] = i; + firstAttempt = False; + *workDone = workLimit = 0; + simpleSort ( s, 0, nblock-1, 0 ); + if (s->verbosity >= 4) VPrintf0( " simpleSort done.\n" ); + + } else { + + numQSorted = 0; + for (i = 0; i <= 255; i++) bigDone[i] = False; + + if (s->verbosity >= 4) VPrintf0( " bucket sorting ...\n" ); + + for (i = 0; i <= 65536; i++) ftab[i] = 0; + + c1 = block[nblock-1]; + for (i = 0; i < nblock; i++) { + c2 = block[i]; + ftab[(c1 << 8) + c2]++; + c1 = c2; + } + + for (i = 1; i <= 65536; i++) ftab[i] += ftab[i-1]; + + c1 = block[0]; + for (i = 0; i < nblock-1; i++) { + c2 = block[i+1]; + j = (c1 << 8) + c2; + c1 = c2; + ftab[j]--; + zptr[ftab[j]] = i; + } + j = (block[nblock-1] << 8) + block[0]; + ftab[j]--; + zptr[ftab[j]] = nblock-1; + + /*-- + Now ftab contains the first loc of every small bucket. + Calculate the running order, from smallest to largest + big bucket. + --*/ + + for (i = 0; i <= 255; i++) runningOrder[i] = i; + + { + Int32 vv; + Int32 h = 1; + do h = 3 * h + 1; while (h <= 256); + do { + h = h / 3; + for (i = h; i <= 255; i++) { + vv = runningOrder[i]; + j = i; + while ( BIGFREQ(runningOrder[j-h]) > BIGFREQ(vv) ) { + runningOrder[j] = runningOrder[j-h]; + j = j - h; + if (j <= (h - 1)) goto zero; + } + zero: + runningOrder[j] = vv; + } + } while (h != 1); + } + + /*-- + The main sorting loop. + --*/ + + for (i = 0; i <= 255; i++) { + + /*-- + Process big buckets, starting with the least full. + Basically this is a 4-step process in which we call + qSort3 to sort the small buckets [ss, j], but + also make a big effort to avoid the calls if we can. + --*/ + ss = runningOrder[i]; + + /*-- + Step 1: + Complete the big bucket [ss] by quicksorting + any unsorted small buckets [ss, j], for j != ss. + Hopefully previous pointer-scanning phases have already + completed many of the small buckets [ss, j], so + we don't have to sort them at all. + --*/ + for (j = 0; j <= 255; j++) { + if (j != ss) { + sb = (ss << 8) + j; + if ( ! (ftab[sb] & SETMASK) ) { + Int32 lo = ftab[sb] & CLEARMASK; + Int32 hi = (ftab[sb+1] & CLEARMASK) - 1; + if (hi > lo) { + if (s->verbosity >= 4) + VPrintf4( " qsort [0x%x, 0x%x] done %d this %d\n", + ss, j, numQSorted, hi - lo + 1 ); + qSort3 ( s, lo, hi, 2 ); + numQSorted += ( hi - lo + 1 ); + if (*workDone > workLimit && firstAttempt) return; + } + } + ftab[sb] |= SETMASK; + } + } + + /*-- + Step 2: + Deal specially with case [ss, ss]. This establishes the + sorted order for [ss, ss] without any comparisons. + A clever trick, cryptically described as steps Q6b and Q6c + in SRC-124 (aka BW94). This makes it entirely practical to + not use a preliminary run-length coder, but unfortunately + we are now stuck with the .bz2 file format. + --*/ + { + Int32 put0, get0, put1, get1; + Int32 sbn = (ss << 8) + ss; + Int32 lo = ftab[sbn] & CLEARMASK; + Int32 hi = (ftab[sbn+1] & CLEARMASK) - 1; + UChar ssc = (UChar)ss; + put0 = lo; + get0 = ftab[ss << 8] & CLEARMASK; + put1 = hi; + get1 = (ftab[(ss+1) << 8] & CLEARMASK) - 1; + while (get0 < put0) { + j = zptr[get0]-1; if (j < 0) j += nblock; + c1 = block[j]; + if (c1 == ssc) { zptr[put0] = j; put0++; }; + get0++; + } + while (get1 > put1) { + j = zptr[get1]-1; if (j < 0) j += nblock; + c1 = block[j]; + if (c1 == ssc) { zptr[put1] = j; put1--; }; + get1--; + } + ftab[sbn] |= SETMASK; + } + + /*-- + Step 3: + The [ss] big bucket is now done. Record this fact, + and update the quadrant descriptors. Remember to + update quadrants in the overshoot area too, if + necessary. The "if (i < 255)" test merely skips + this updating for the last bucket processed, since + updating for the last bucket is pointless. + + The quadrant array provides a way to incrementally + cache sort orderings, as they appear, so as to + make subsequent comparisons in fullGtU() complete + faster. For repetitive blocks this makes a big + difference (but not big enough to be able to avoid + randomisation for very repetitive data.) + + The precise meaning is: at all times: + + for 0 <= i < nblock and 0 <= j <= nblock + + if block[i] != block[j], + + then the relative values of quadrant[i] and + quadrant[j] are meaningless. + + else { + if quadrant[i] < quadrant[j] + then the string starting at i lexicographically + precedes the string starting at j + + else if quadrant[i] > quadrant[j] + then the string starting at j lexicographically + precedes the string starting at i + + else + the relative ordering of the strings starting + at i and j has not yet been determined. + } + --*/ + bigDone[ss] = True; + + if (i < 255) { + Int32 bbStart = ftab[ss << 8] & CLEARMASK; + Int32 bbSize = (ftab[(ss+1) << 8] & CLEARMASK) - bbStart; + Int32 shifts = 0; + + while ((bbSize >> shifts) > 65534) shifts++; + + for (j = 0; j < bbSize; j++) { + Int32 a2update = zptr[bbStart + j]; + UInt16 qVal = (UInt16)(j >> shifts); + quadrant[a2update] = qVal; + if (a2update < BZ_NUM_OVERSHOOT_BYTES) + quadrant[a2update + nblock] = qVal; + } + + AssertH ( ( ((bbSize-1) >> shifts) <= 65535 ), 1002 ); + } + + /*-- + Step 4: + Now scan this big bucket [ss] so as to synthesise the + sorted order for small buckets [t, ss] for all t != ss. + This will avoid doing Real Work in subsequent Step 1's. + --*/ + for (j = 0; j <= 255; j++) + copy[j] = ftab[(j << 8) + ss] & CLEARMASK; + + for (j = ftab[ss << 8] & CLEARMASK; + j < (ftab[(ss+1) << 8] & CLEARMASK); + j++) { + k = zptr[j]-1; if (k < 0) k += nblock; + c1 = block[k]; + if ( ! bigDone[c1] ) { + zptr[copy[c1]] = k; + copy[c1] ++; + } + } + + for (j = 0; j <= 255; j++) ftab[(j << 8) + ss] |= SETMASK; + } + if (s->verbosity >= 4) + VPrintf3( " %d pointers, %d sorted, %d scanned\n", + nblock, numQSorted, nblock - numQSorted ); + } +} + + +/*---------------------------------------------*/ +static void randomiseBlock ( EState* s ) +{ + Int32 i; + BZ_RAND_INIT_MASK; + for (i = 0; i < 256; i++) s->inUse[i] = False; + + for (i = 0; i < s->nblock; i++) { + BZ_RAND_UPD_MASK; + s->block[i] ^= BZ_RAND_MASK; + s->inUse[s->block[i]] = True; + } +} + + +/*---------------------------------------------*/ +void blockSort ( EState* s ) +{ + Int32 i; + + s->workLimit = s->workFactor * (s->nblock - 1); + s->workDone = 0; + s->blockRandomised = False; + s->firstAttempt = True; + + sortMain ( s ); + + if (s->verbosity >= 3) + VPrintf3( " %d work, %d block, ratio %5.2f\n", + s->workDone, s->nblock-1, + (float)(s->workDone) / (float)(s->nblock-1) ); + + if (s->workDone > s->workLimit && s->firstAttempt) { + if (s->verbosity >= 2) + VPrintf0( " sorting aborted; randomising block\n" ); + randomiseBlock ( s ); + s->workLimit = s->workDone = 0; + s->blockRandomised = True; + s->firstAttempt = False; + sortMain ( s ); + if (s->verbosity >= 3) + VPrintf3( " %d work, %d block, ratio %f\n", + s->workDone, s->nblock-1, + (float)(s->workDone) / (float)(s->nblock-1) ); + } + + s->origPtr = -1; + for (i = 0; i < s->nblock; i++) + if (s->zptr[i] == 0) + { s->origPtr = i; break; }; + + AssertH( s->origPtr != -1, 1003 ); +} + +/*-------------------------------------------------------------*/ +/*--- end blocksort.c ---*/ +/*-------------------------------------------------------------*/ diff --git a/bzip2.1 b/bzip2.1 index 489668f..a6789a4 100644 --- a/bzip2.1 +++ b/bzip2.1 @@ -1,21 +1,29 @@ .PU .TH bzip2 1 .SH NAME -bzip2, bunzip2 \- a block-sorting file compressor, v0.1 +bzip2, bunzip2 \- a block-sorting file compressor, v0.9.0 +.br +bzcat \- decompresses files to stdout .br bzip2recover \- recovers data from damaged bzip2 files .SH SYNOPSIS .ll +8 .B bzip2 -.RB [ " \-cdfkstvVL123456789 " ] +.RB [ " \-cdfkstvzVL123456789 " ] [ .I "filenames \&..." ] .ll -8 .br .B bunzip2 -.RB [ " \-kvsVL " ] +.RB [ " \-fkvsVL " ] +[ +.I "filenames \&..." +] +.br +.B bzcat +.RB [ " \-s " ] [ .I "filenames \&..." ] @@ -24,7 +32,7 @@ bzip2recover \- recovers data from damaged bzip2 files .I "filename" .SH DESCRIPTION -.I Bzip2 +.I bzip2 compresses files using the Burrows-Wheeler block-sorting text compression algorithm, and Huffman coding. Compression is generally considerably @@ -38,7 +46,7 @@ those of .I GNU Gzip, but they are not identical. -.I Bzip2 +.I bzip2 expects a list of file names to accompany the command-line flags. Each file is replaced by a compressed version of itself, with the name "original_name.bz2". @@ -50,11 +58,11 @@ original file names, permissions and dates in filesystems which lack these concepts, or have serious file name length restrictions, such as MS-DOS. -.I Bzip2 +.I bzip2 and .I bunzip2 -will not overwrite existing files; if you want this to happen, -you should delete them first. +will by default not overwrite existing files; +if you want this to happen, specify the \-f flag. If no file names are specified, .I bzip2 @@ -64,7 +72,7 @@ In this case, will decline to write compressed output to a terminal, as this would be entirely incomprehensible and therefore pointless. -.I Bunzip2 +.I bunzip2 (or .I bzip2 \-d ) decompresses and restores all specified files whose names @@ -73,12 +81,28 @@ Files without this suffix are ignored. Again, supplying no filenames causes decompression from standard input to standard output. +.I bunzip2 +will correctly decompress a file which is the concatenation +of two or more compressed files. The result is the concatenation +of the corresponding uncompressed files. Integrity testing +(\-t) of concatenated compressed files is also supported. + You can also compress or decompress files to the standard output by giving the \-c flag. -You can decompress multiple files like this, but you may -only compress a single file this way, since it would otherwise -be difficult to separate out the compressed representations of -the original files. +Multiple files may be compressed and decompressed like this. +The resulting outputs are fed sequentially to stdout. +Compression of multiple files in this manner generates +a stream containing multiple compressed file representations. +Such a stream can be decompressed correctly only by +.I bzip2 +version 0.9.0 or later. Earlier versions of +.I bzip2 +will stop after decompressing the first file in the stream. + +.I bzcat +(or +.I bzip2 \-dc +) decompresses all specified files to the standard output. Compression is always performed, even if the compressed file is slightly larger than the original. Files of less than about @@ -132,7 +156,7 @@ Compression and decompression requirements, in bytes, can be estimated as: Compression: 400k + ( 7 x block size ) - Decompression: 100k + ( 5 x block size ), or + Decompression: 100k + ( 4 x block size ), or .br 100k + ( 2.5 x block size ) @@ -147,7 +171,7 @@ choice of block size. For files compressed with the default 900k block size, .I bunzip2 -will require about 4600 kbytes to decompress. +will require about 3700 kbytes to decompress. To support decompression of any file on a 4 megabyte machine, .I bunzip2 has an option to decompress using approximately half this @@ -168,8 +192,8 @@ For example, compressing a file 20,000 bytes long with the flag \-9 will cause the compressor to allocate around 6700k of memory, but only touch 400k + 20000 * 7 = 540 -kbytes of it. Similarly, the decompressor will allocate 4600k but -only touch 100k + 20000 * 5 = 200 kbytes. +kbytes of it. Similarly, the decompressor will allocate 3700k but +only touch 100k + 20000 * 4 = 180 kbytes. Here is a table which summarises the maximum memory usage for different block sizes. Also recorded is the total compressed @@ -182,71 +206,73 @@ Corpus is dominated by smaller files. Compress Decompress Decompress Corpus Flag usage usage -s usage Size - -1 1100k 600k 350k 914704 - -2 1800k 1100k 600k 877703 - -3 2500k 1600k 850k 860338 - -4 3200k 2100k 1100k 846899 - -5 3900k 2600k 1350k 845160 - -6 4600k 3100k 1600k 838626 - -7 5400k 3600k 1850k 834096 - -8 6000k 4100k 2100k 828642 - -9 6700k 4600k 2350k 828642 + -1 1100k 500k 350k 914704 + -2 1800k 900k 600k 877703 + -3 2500k 1300k 850k 860338 + -4 3200k 1700k 1100k 846899 + -5 3900k 2100k 1350k 845160 + -6 4600k 2500k 1600k 838626 + -7 5400k 2900k 1850k 834096 + -8 6000k 3300k 2100k 828642 + -9 6700k 3700k 2350k 828642 .SH OPTIONS .TP -.B \-c --stdout +.B \-c --stdout Compress or decompress to standard output. \-c will decompress multiple files to stdout, but will only compress a single file to stdout. .TP .B \-d --decompress Force decompression. -.I Bzip2 -and +.I bzip2, .I bunzip2 -are really the same program, and the decision about whether to -compress or decompress is done on the basis of which name is +and +.I bzcat +are really the same program, and the decision about what actions +to take is done on the basis of which name is used. This flag overrides that mechanism, and forces .I bzip2 to decompress. .TP -.B \-f --compress +.B \-z --compress The complement to \-d: forces compression, regardless of the invokation name. .TP .B \-t --test Check integrity of the specified file(s), but don't decompress them. -This really performs a trial decompression and throws away the result, -using the low-memory decompression algorithm (see \-s). +This really performs a trial decompression and throws away the result. +.TP +.B \-f --force +Force overwrite of output files. Normally, +.I bzip2 +will not overwrite existing output files. .TP .B \-k --keep Keep (don't delete) input files during compression or decompression. .TP .B \-s --small -Reduce memory usage, both for compression and decompression. -Files are decompressed using a modified algorithm which only +Reduce memory usage, for compression, decompression and +testing. +Files are decompressed and tested using a modified algorithm which only requires 2.5 bytes per block byte. This means any file can be -decompressed in 2300k of memory, albeit somewhat more slowly than -usual. +decompressed in 2300k of memory, albeit at about half the normal +speed. During compression, -s selects a block size of 200k, which limits memory use to around the same figure, at the expense of your compression ratio. In short, if your machine is low on memory (8 megabytes or less), use -s for everything. See MEMORY MANAGEMENT above. - .TP .B \-v --verbose Verbose mode -- show the compression ratio for each file processed. Further \-v's increase the verbosity level, spewing out lots of information which is primarily of interest for diagnostic purposes. .TP -.B \-L --license +.B \-L --license -V --version Display the software version, license terms and conditions. .TP -.B \-V --version -Same as \-L. -.TP .B \-1 to \-9 Set the block size to 100 k, 200 k .. 900 k when compressing. Has no effect when decompressing. @@ -329,10 +355,6 @@ to compress the latter. If you do get a file which causes severe slowness in compression, try making the block size as small as possible, with flag \-1. -Incompressible or virtually-incompressible data may decompress -rather more slowly than one would hope. This is due to -a naive implementation of the move-to-front coder. - .I bzip2 usually allocates several megabytes of memory to operate in, and then charges all over it in a fairly random fashion. This @@ -346,28 +368,19 @@ I imagine .I bzip2 will perform best on machines with very large caches. -Test mode (\-t) uses the low-memory decompression algorithm -(\-s). This means test mode does not run as fast as it could; -it could run as fast as the normal decompression machinery. -This could easily be fixed at the cost of some code bloat. - .SH CAVEATS I/O error messages are not as helpful as they could be. .I Bzip2 tries hard to detect I/O errors and exit cleanly, but the details of what the problem is sometimes seem rather misleading. -This manual page pertains to version 0.1 of +This manual page pertains to version 0.9.0 of .I bzip2. -It may well happen that some future version will -use a different compressed file format. If you try to -decompress, using 0.1, a .bz2 file created with some -future version which uses a different compressed file format, -0.1 will complain that your file "is not a bzip2 file". -If that happens, you should obtain a more recent version -of -.I bzip2 -and use that to decompress the file. +Compressed data created by this version is entirely forwards and +backwards compatible with the previous public release, version 0.1pl2, +but with the following exception: 0.9.0 can correctly decompress +multiple concatenated compressed files. 0.1pl2 cannot do this; it +will stop after decompressing just the first file in the stream. Wildcard expansion for Windows 95 and NT is flaky. @@ -377,63 +390,25 @@ uses 32-bit integers to represent bit positions in compressed files, so it cannot handle compressed files more than 512 megabytes long. This could easily be fixed. -.I bzip2recover -sometimes reports a very small, incomplete final block. -This is spurious and can be safely ignored. - -.SH RELATIONSHIP TO bzip-0.21 -This program is a descendant of the -.I bzip -program, version 0.21, which I released in August 1996. -The primary difference of -.I bzip2 -is its avoidance of the possibly patented algorithms -which were used in 0.21. -.I bzip2 -also brings various useful refinements (\-s, \-t), -uses less memory, decompresses significantly faster, and -has support for recovering data from damaged files. - -Because -.I bzip2 -uses Huffman coding to construct the compressed bitstream, -rather than the arithmetic coding used in 0.21, -the compressed representations generated by the two programs -are incompatible, and they will not interoperate. The change -in suffix from .bz to .bz2 reflects this. It would have been -helpful to at least allow -.I bzip2 -to decompress files created by 0.21, but this would -defeat the primary aim of having a patent-free compressor. - -For a more precise statement about patent issues in -bzip2, please see the README file in the distribution. - -Huffman coding necessarily involves some coding inefficiency -compared to arithmetic coding. This means that -.I bzip2 -compresses about 1% worse than 0.21, an unfortunate but -unavoidable fact-of-life. On the other hand, decompression -is approximately 50% faster for the same reason, and the -change in file format gave an opportunity to add data-recovery -features. So it is not all bad. - .SH AUTHOR Julian Seward, jseward@acm.org. +http://www.muraroa.demon.co.uk + The ideas embodied in -.I bzip -and .I bzip2 are due to (at least) the following people: Michael Burrows and David Wheeler (for the block sorting transformation), David Wheeler (again, for the Huffman coder), -Peter Fenwick (for the structured coding model in 0.21, +Peter Fenwick (for the structured coding model in the original +.I bzip, and many refinements), and Alistair Moffat, Radford Neal and Ian Witten (for the arithmetic -coder in 0.21). I am much indebted for their help, support and advice. -See the file ALGORITHMS in the source distribution for pointers to +coder in the original +.I bzip). +I am much indebted for their help, support and advice. +See the manual in the source distribution for pointers to sources of documentation. Christian von Roques encouraged me to look for faster sorting algorithms, so as to speed up compression. diff --git a/bzip2.1.preformatted b/bzip2.1.preformatted index 5206e05..8c4fab1 100644 --- a/bzip2.1.preformatted +++ b/bzip2.1.preformatted @@ -5,18 +5,20 @@ bzip2(1) bzip2(1) NNAAMMEE - bzip2, bunzip2 - a block-sorting file compressor, v0.1 + bzip2, bunzip2 - a block-sorting file compressor, v0.9.0 + bzcat - decompresses files to stdout bzip2recover - recovers data from damaged bzip2 files SSYYNNOOPPSSIISS - bbzziipp22 [ --ccddffkkssttvvVVLL112233445566778899 ] [ _f_i_l_e_n_a_m_e_s _._._. ] - bbuunnzziipp22 [ --kkvvssVVLL ] [ _f_i_l_e_n_a_m_e_s _._._. ] + bbzziipp22 [ --ccddffkkssttvvzzVVLL112233445566778899 ] [ _f_i_l_e_n_a_m_e_s _._._. ] + bbuunnzziipp22 [ --ffkkvvssVVLL ] [ _f_i_l_e_n_a_m_e_s _._._. ] + bbzzccaatt [ --ss ] [ _f_i_l_e_n_a_m_e_s _._._. ] bbzziipp22rreeccoovveerr _f_i_l_e_n_a_m_e DDEESSCCRRIIPPTTIIOONN - _B_z_i_p_2 compresses files using the Burrows-Wheeler block- + _b_z_i_p_2 compresses files using the Burrows-Wheeler block- sorting text compression algorithm, and Huffman coding. Compression is generally considerably better than that achieved by more conventional LZ77/LZ78-based compressors, @@ -26,7 +28,7 @@ DDEESSCCRRIIPPTTIIOONN The command-line options are deliberately very similar to those of _G_N_U _G_z_i_p_, but they are not identical. - _B_z_i_p_2 expects a list of file names to accompany the com- + _b_z_i_p_2 expects a list of file names to accompany the com- mand-line flags. Each file is replaced by a compressed version of itself, with the name "original_name.bz2". Each compressed file has the same modification date and @@ -38,8 +40,8 @@ DDEESSCCRRIIPPTTIIOONN cepts, or have serious file name length restrictions, such as MS-DOS. - _B_z_i_p_2 and _b_u_n_z_i_p_2 will not overwrite existing files; if - you want this to happen, you should delete them first. + _b_z_i_p_2 and _b_u_n_z_i_p_2 will by default not overwrite existing + files; if you want this to happen, specify the -f flag. If no file names are specified, _b_z_i_p_2 compresses from standard input to standard output. In this case, _b_z_i_p_2 @@ -47,17 +49,15 @@ DDEESSCCRRIIPPTTIIOONN this would be entirely incomprehensible and therefore pointless. - _B_u_n_z_i_p_2 (or _b_z_i_p_2 _-_d ) decompresses and restores all spec- + _b_u_n_z_i_p_2 (or _b_z_i_p_2 _-_d ) decompresses and restores all spec- ified files whose names end in ".bz2". Files without this suffix are ignored. Again, supplying no filenames causes decompression from standard input to standard output. - You can also compress or decompress files to the standard - output by giving the -c flag. You can decompress multiple - files like this, but you may only compress a single file - this way, since it would otherwise be difficult to sepa- - rate out the compressed representations of the original - files. + _b_u_n_z_i_p_2 will correctly decompress a file which is the con- + catenation of two or more compressed files. The result is + the concatenation of the corresponding uncompressed files. + Integrity testing (-t) of concatenated compressed files is @@ -70,6 +70,21 @@ DDEESSCCRRIIPPTTIIOONN bzip2(1) bzip2(1) + also supported. + + You can also compress or decompress files to the standard + output by giving the -c flag. Multiple files may be com- + pressed and decompressed like this. The resulting outputs + are fed sequentially to stdout. Compression of multiple + files in this manner generates a stream containing multi- + ple compressed file representations. Such a stream can be + decompressed correctly only by _b_z_i_p_2 version 0.9.0 or + later. Earlier versions of _b_z_i_p_2 will stop after decom- + pressing the first file in the stream. + + _b_z_c_a_t (or _b_z_i_p_2 _-_d_c ) decompresses all specified files to + the standard output. + Compression is always performed, even if the compressed file is slightly larger than the original. Files of less than about one hundred bytes tend to get larger, since the @@ -108,36 +123,37 @@ MMEEMMOORRYY MMAANNAAGGEEMMEENNTT file, and _b_u_n_z_i_p_2 then allocates itself just enough memory to decompress the file. Since block sizes are stored in compressed files, it follows that the flags -1 to -9 are - irrelevant to and so ignored during decompression. Com- - pression and decompression requirements, in bytes, can be - estimated as: + irrelevant to and so ignored during decompression. - Compression: 400k + ( 7 x block size ) - Decompression: 100k + ( 5 x block size ), or - 100k + ( 2.5 x block size ) - Larger block sizes give rapidly diminishing marginal - returns; most of the compression comes from the first two - or three hundred k of block size, a fact worth bearing in - mind when using _b_z_i_p_2 on small machines. It is also - important to appreciate that the decompression memory - requirement is set at compression-time by the choice of - block size. + 2 - 2 +bzip2(1) bzip2(1) + Compression and decompression requirements, in bytes, can + be estimated as: -bzip2(1) bzip2(1) + Compression: 400k + ( 7 x block size ) + Decompression: 100k + ( 4 x block size ), or + 100k + ( 2.5 x block size ) + + Larger block sizes give rapidly diminishing marginal + returns; most of the compression comes from the first two + or three hundred k of block size, a fact worth bearing in + mind when using _b_z_i_p_2 on small machines. It is also + important to appreciate that the decompression memory + requirement is set at compression-time by the choice of + block size. For files compressed with the default 900k block size, - _b_u_n_z_i_p_2 will require about 4600 kbytes to decompress. To + _b_u_n_z_i_p_2 will require about 3700 kbytes to decompress. To support decompression of any file on a 4 megabyte machine, _b_u_n_z_i_p_2 has an option to decompress using approximately half this amount of memory, about 2300 kbytes. Decompres- @@ -157,8 +173,8 @@ bzip2(1) bzip2(1) file 20,000 bytes long with the flag -9 will cause the compressor to allocate around 6700k of memory, but only touch 400k + 20000 * 7 = 540 kbytes of it. Similarly, the - decompressor will allocate 4600k but only touch 100k + - 20000 * 5 = 200 kbytes. + decompressor will allocate 3700k but only touch 100k + + 20000 * 4 = 180 kbytes. Here is a table which summarises the maximum memory usage for different block sizes. Also recorded is the total @@ -172,64 +188,66 @@ bzip2(1) bzip2(1) Compress Decompress Decompress Corpus Flag usage usage -s usage Size - -1 1100k 600k 350k 914704 - -2 1800k 1100k 600k 877703 - -3 2500k 1600k 850k 860338 - -4 3200k 2100k 1100k 846899 - -5 3900k 2600k 1350k 845160 - -6 4600k 3100k 1600k 838626 - -7 5400k 3600k 1850k 834096 - -8 6000k 4100k 2100k 828642 - -9 6700k 4600k 2350k 828642 + -1 1100k 500k 350k 914704 + -2 1800k 900k 600k 877703 -OOPPTTIIOONNSS - --cc ----ssttddoouutt - Compress or decompress to standard output. -c will - decompress multiple files to stdout, but will only - compress a single file to stdout. - + 3 - 3 +bzip2(1) bzip2(1) + -3 2500k 1300k 850k 860338 + -4 3200k 1700k 1100k 846899 + -5 3900k 2100k 1350k 845160 + -6 4600k 2500k 1600k 838626 + -7 5400k 2900k 1850k 834096 + -8 6000k 3300k 2100k 828642 + -9 6700k 3700k 2350k 828642 -bzip2(1) bzip2(1) +OOPPTTIIOONNSS + --cc ----ssttddoouutt + Compress or decompress to standard output. -c will + decompress multiple files to stdout, but will only + compress a single file to stdout. --dd ----ddeeccoommpprreessss - Force decompression. _B_z_i_p_2 and _b_u_n_z_i_p_2 are really - the same program, and the decision about whether to - compress or decompress is done on the basis of - which name is used. This flag overrides that mech- - anism, and forces _b_z_i_p_2 to decompress. + Force decompression. _b_z_i_p_2_, _b_u_n_z_i_p_2 and _b_z_c_a_t are + really the same program, and the decision about + what actions to take is done on the basis of which + name is used. This flag overrides that mechanism, + and forces _b_z_i_p_2 to decompress. - --ff ----ccoommpprreessss + --zz ----ccoommpprreessss The complement to -d: forces compression, regard- less of the invokation name. --tt ----tteesstt Check integrity of the specified file(s), but don't decompress them. This really performs a trial - decompression and throws away the result, using the - low-memory decompression algorithm (see -s). + decompression and throws away the result. + + --ff ----ffoorrccee + Force overwrite of output files. Normally, _b_z_i_p_2 + will not overwrite existing output files. --kk ----kkeeeepp Keep (don't delete) input files during compression or decompression. --ss ----ssmmaallll - Reduce memory usage, both for compression and - decompression. Files are decompressed using a mod- - ified algorithm which only requires 2.5 bytes per - block byte. This means any file can be decom- - pressed in 2300k of memory, albeit somewhat more - slowly than usual. + Reduce memory usage, for compression, decompression + and testing. Files are decompressed and tested + using a modified algorithm which only requires 2.5 + bytes per block byte. This means any file can be + decompressed in 2300k of memory, albeit at about + half the normal speed. During compression, -s selects a block size of 200k, which limits memory use to around the same @@ -239,35 +257,32 @@ bzip2(1) bzip2(1) MEMORY MANAGEMENT above. + + + 4 + + + + + +bzip2(1) bzip2(1) + + --vv ----vveerrbboossee Verbose mode -- show the compression ratio for each file processed. Further -v's increase the ver- bosity level, spewing out lots of information which is primarily of interest for diagnostic purposes. - --LL ----lliicceennssee + --LL ----lliicceennssee --VV ----vveerrssiioonn Display the software version, license terms and conditions. - --VV ----vveerrssiioonn - Same as -L. - --11 ttoo --99 Set the block size to 100 k, 200 k .. 900 k when compressing. Has no effect when decompressing. See MEMORY MANAGEMENT above. - - - 4 - - - - - -bzip2(1) bzip2(1) - - ----rreeppeettiittiivvee--ffaasstt _b_z_i_p_2 injects some small pseudo-random variations into very repetitive blocks to limit worst-case @@ -306,34 +321,34 @@ RREECCOOVVEERRIINNGG DDAATTAA FFRROOMM DDAAMMAAGGEEDD F _b_z_i_p_2_r_e_c_o_v_e_r takes a single argument, the name of the dam- aged file, and writes a number of files "rec0001file.bz2", "rec0002file.bz2", etc, containing the extracted blocks. - The output filenames are designed so that the use of wild- - cards in subsequent processing -- for example, "bzip2 -dc - rec*file.bz2 > recovered_data" -- lists the files in the - "right" order. + The output filenames are designed so that the use of - _b_z_i_p_2_r_e_c_o_v_e_r should be of most use dealing with large .bz2 - files, as these will contain many blocks. It is clearly - futile to use it on damaged single-block files, since a - damaged block cannot be recovered. If you wish to min- - imise any potential data loss through media or transmis- - sion errors, you might consider compressing with a smaller - block size. -PPEERRFFOORRMMAANNCCEE NNOOTTEESS - The sorting phase of compression gathers together similar + 5 - 5 +bzip2(1) bzip2(1) + wildcards in subsequent processing -- for example, "bzip2 + -dc rec*file.bz2 > recovered_data" -- lists the files in + the "right" order. -bzip2(1) bzip2(1) + _b_z_i_p_2_r_e_c_o_v_e_r should be of most use dealing with large .bz2 + files, as these will contain many blocks. It is clearly + futile to use it on damaged single-block files, since a + damaged block cannot be recovered. If you wish to min- + imise any potential data loss through media or transmis- + sion errors, you might consider compressing with a smaller + block size. +PPEERRFFOORRMMAANNCCEE NNOOTTEESS + The sorting phase of compression gathers together similar strings in the file. Because of this, files containing very long runs of repeated symbols, like "aabaabaabaab ..." (repeated several hundred times) may compress @@ -348,10 +363,6 @@ bzip2(1) bzip2(1) severe slowness in compression, try making the block size as small as possible, with flag -1. - Incompressible or virtually-incompressible data may decom- - press rather more slowly than one would hope. This is due - to a naive implementation of the move-to-front coder. - _b_z_i_p_2 usually allocates several megabytes of memory to operate in, and then charges all over it in a fairly ran- dom fashion. This means that performance, both for com- @@ -362,12 +373,6 @@ bzip2(1) bzip2(1) large performance improvements. I imagine _b_z_i_p_2 will per- form best on machines with very large caches. - Test mode (-t) uses the low-memory decompression algorithm - (-s). This means test mode does not run as fast as it - could; it could run as fast as the normal decompression - machinery. This could easily be fixed at the cost of some - code bloat. - CCAAVVEEAATTSS I/O error messages are not as helpful as they could be. @@ -375,19 +380,14 @@ CCAAVVEEAATTSS but the details of what the problem is sometimes seem rather misleading. - This manual page pertains to version 0.1 of _b_z_i_p_2_. It may - well happen that some future version will use a different - compressed file format. If you try to decompress, using - 0.1, a .bz2 file created with some future version which - uses a different compressed file format, 0.1 will complain - that your file "is not a bzip2 file". If that happens, - you should obtain a more recent version of _b_z_i_p_2 and use - that to decompress the file. + This manual page pertains to version 0.9.0 of _b_z_i_p_2_. Com- + pressed data created by this version is entirely forwards + and backwards compatible with the previous public release, + version 0.1pl2, but with the following exception: 0.9.0 + can correctly decompress multiple concatenated compressed + files. 0.1pl2 cannot do this; it will stop after decom- + pressing just the first file in the stream. - Wildcard expansion for Windows 95 and NT is flaky. - - _b_z_i_p_2_r_e_c_o_v_e_r uses 32-bit integers to represent bit posi- - tions in compressed files, so it cannot handle compressed @@ -400,61 +400,59 @@ CCAAVVEEAATTSS bzip2(1) bzip2(1) - files more than 512 megabytes long. This could easily be + Wildcard expansion for Windows 95 and NT is flaky. + + _b_z_i_p_2_r_e_c_o_v_e_r uses 32-bit integers to represent bit posi- + tions in compressed files, so it cannot handle compressed + files more than 512 megabytes long. This could easily be fixed. - _b_z_i_p_2_r_e_c_o_v_e_r sometimes reports a very small, incomplete - final block. This is spurious and can be safely ignored. + +AAUUTTHHOORR + Julian Seward, jseward@acm.org. + http://www.muraroa.demon.co.uk + + The ideas embodied in _b_z_i_p_2 are due to (at least) the fol- + lowing people: Michael Burrows and David Wheeler (for the + block sorting transformation), David Wheeler (again, for + the Huffman coder), Peter Fenwick (for the structured cod- + ing model in the original _b_z_i_p_, and many refinements), and + Alistair Moffat, Radford Neal and Ian Witten (for the + arithmetic coder in the original _b_z_i_p_)_. I am much + indebted for their help, support and advice. See the man- + ual in the source distribution for pointers to sources of + documentation. Christian von Roques encouraged me to look + for faster sorting algorithms, so as to speed up compres- + sion. Bela Lubkin encouraged me to improve the worst-case + compression performance. Many people sent patches, helped + with portability problems, lent machines, gave advice and + were generally helpful. + + + + + + + + + + + + + + + + + + -RREELLAATTIIOONNSSHHIIPP TTOO bbzziipp--00..2211 - This program is a descendant of the _b_z_i_p program, version - 0.21, which I released in August 1996. The primary dif- - ference of _b_z_i_p_2 is its avoidance of the possibly patented - algorithms which were used in 0.21. _b_z_i_p_2 also brings - various useful refinements (-s, -t), uses less memory, - decompresses significantly faster, and has support for - recovering data from damaged files. - Because _b_z_i_p_2 uses Huffman coding to construct the com- - pressed bitstream, rather than the arithmetic coding used - in 0.21, the compressed representations generated by the - two programs are incompatible, and they will not interop- - erate. The change in suffix from .bz to .bz2 reflects - this. It would have been helpful to at least allow _b_z_i_p_2 - to decompress files created by 0.21, but this would defeat - the primary aim of having a patent-free compressor. - For a more precise statement about patent issues in bzip2, - please see the README file in the distribution. - Huffman coding necessarily involves some coding ineffi- - ciency compared to arithmetic coding. This means that - _b_z_i_p_2 compresses about 1% worse than 0.21, an unfortunate - but unavoidable fact-of-life. On the other hand, decom- - pression is approximately 50% faster for the same reason, - and the change in file format gave an opportunity to add - data-recovery features. So it is not all bad. -AAUUTTHHOORR - Julian Seward, jseward@acm.org. - The ideas embodied in _b_z_i_p and _b_z_i_p_2 are due to (at least) - the following people: Michael Burrows and David Wheeler - (for the block sorting transformation), David Wheeler - (again, for the Huffman coder), Peter Fenwick (for the - structured coding model in 0.21, and many refinements), - and Alistair Moffat, Radford Neal and Ian Witten (for the - arithmetic coder in 0.21). I am much indebted for their - help, support and advice. See the file ALGORITHMS in the - source distribution for pointers to sources of documenta- - tion. Christian von Roques encouraged me to look for - faster sorting algorithms, so as to speed up compression. - Bela Lubkin encouraged me to improve the worst-case com- - pression performance. Many people sent patches, helped - with portability problems, lent machines, gave advice and - were generally helpful. diff --git a/bzip2.c b/bzip2.c index 53ce10d..6a3ab95 100644 --- a/bzip2.c +++ b/bzip2.c @@ -4,28 +4,45 @@ /*-----------------------------------------------------------*/ /*-- - This program is bzip2, a lossless, block-sorting data compressor, - version 0.1pl2, dated 29-Aug-1997. - - Copyright (C) 1996, 1997 by Julian Seward. - Guildford, Surrey, UK - email: jseward@acm.org - - This program is free software; you can redistribute it and/or modify - it under the terms of the GNU General Public License as published by - the Free Software Foundation; either version 2 of the License, or - (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - - The GNU General Public License is contained in the file LICENSE. + This file is a part of bzip2 and/or libbzip2, a program and + library for lossless, block-sorting data compression. + + Copyright (C) 1996-1998 Julian R Seward. All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + 1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + 2. The origin of this software must not be misrepresented; you must + not claim that you wrote the original software. If you use this + software in a product, an acknowledgment in the product + documentation would be appreciated but is not required. + + 3. Altered source versions must be plainly marked as such, and must + not be misrepresented as being the original software. + + 4. The name of the author may not be used to endorse or promote + products derived from this software without specific prior written + permission. + + THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS + OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED + WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY + DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE + GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS + INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, + WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING + NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS + SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + Julian Seward, Guildford, Surrey, UK. + jseward@acm.org + bzip2/libbzip2 version 0.9.0c of 18 October 1998 This program is based on (at least) the work of: Mike Burrows @@ -37,21 +54,23 @@ Robert Sedgewick Jon L. Bentley - For more information on these sources, see the file ALGORITHMS. + For more information on these sources, see the manual. --*/ + /*----------------------------------------------------*/ /*--- IMPORTANT ---*/ /*----------------------------------------------------*/ /*-- WARNING: - This program (attempts to) compress data by performing several - non-trivial transformations on it. Unless you are 100% familiar - with *all* the algorithms contained herein, and with the - consequences of modifying them, you should NOT meddle with the - compression or decompression machinery. Incorrect changes can - and very likely *will* lead to disasterous loss of data. + This program and library (attempts to) compress data by + performing several non-trivial transformations on it. + Unless you are 100% familiar with *all* the algorithms + contained herein, and with the consequences of modifying them, + you should NOT meddle with the compression or decompression + machinery. Incorrect changes can and very likely *will* + lead to disasterous loss of data. DISCLAIMER: I TAKE NO RESPONSIBILITY FOR ANY LOSS OF DATA ARISING FROM THE @@ -65,18 +84,19 @@ of various special cases in the code which occur with very low but non-zero probability make it impossible to rule out the possibility of bugs remaining in the program. DO NOT COMPRESS - ANY DATA WITH THIS PROGRAM UNLESS YOU ARE PREPARED TO ACCEPT THE - POSSIBILITY, HOWEVER SMALL, THAT THE DATA WILL NOT BE RECOVERABLE. + ANY DATA WITH THIS PROGRAM AND/OR LIBRARY UNLESS YOU ARE PREPARED + TO ACCEPT THE POSSIBILITY, HOWEVER SMALL, THAT THE DATA WILL + NOT BE RECOVERABLE. That is not to say this program is inherently unreliable. - Indeed, I very much hope the opposite is true. bzip2 has been - carefully constructed and extensively tested. + Indeed, I very much hope the opposite is true. bzip2/libbzip2 + has been carefully constructed and extensively tested. PATENTS: - To the best of my knowledge, bzip2 does not use any patented - algorithms. However, I do not have the resources available to - carry out a full patent search. Therefore I cannot give any - guarantee of the above statement. + To the best of my knowledge, bzip2/libbzip2 does not use any + patented algorithms. However, I do not have the resources + available to carry out a full patent search. Therefore I cannot + give any guarantee of the above statement. --*/ @@ -103,6 +123,10 @@ --*/ #define BZ_LCCWIN32 0 +#ifdef _WIN32 +#define BZ_LCCWIN32 1 +#define BZ_UNIX 0 +#endif /*---------------------------------------------*/ @@ -112,12 +136,10 @@ #include #include -#if DEBUG - #include -#endif #include #include #include +#include "bzlib.h" #define ERROR_IF_EOF(i) { if ((i) == EOF) ioError(); } #define ERROR_IF_NOT_ZERO(i) { if ((i) != 0) ioError(); } @@ -130,2910 +152,421 @@ --*/ #if BZ_UNIX - #include - #include - #include - #include - #include - #include - - #define Int32 int - #define UInt32 unsigned int - #define Char char - #define UChar unsigned char - #define Int16 short - #define UInt16 unsigned short - - #define PATH_SEP '/' - #define MY_LSTAT lstat - #define MY_S_IFREG S_ISREG - #define MY_STAT stat - - #define APPEND_FILESPEC(root, name) \ +# include +# include +# include +# include +# include + +# define PATH_SEP '/' +# define MY_LSTAT lstat +# define MY_S_IFREG S_ISREG +# define MY_STAT stat + +# define APPEND_FILESPEC(root, name) \ root=snocString((root), (name)) - #define SET_BINARY_MODE(fd) /**/ - - /*-- - You should try very hard to persuade your C compiler - to inline the bits marked INLINE. Otherwise bzip2 will - run rather slowly. gcc version 2.x is recommended. - --*/ - #ifdef __GNUC__ - #define INLINE inline - #define NORETURN __attribute__ ((noreturn)) - #else - #define INLINE /**/ - #define NORETURN /**/ - #endif -#endif - - - -#if BZ_LCCWIN32 - #include - #include - #include - - #define Int32 int - #define UInt32 unsigned int - #define Int16 short - #define UInt16 unsigned short - #define Char char - #define UChar unsigned char - - #define INLINE /**/ - #define NORETURN /**/ - #define PATH_SEP '\\' - #define MY_LSTAT _stat - #define MY_STAT _stat - #define MY_S_IFREG(x) ((x) & _S_IFREG) - - #if 0 - /*-- lcc-win32 seems to expand wildcards itself --*/ - #define APPEND_FILESPEC(root, spec) \ - do { \ - if ((spec)[0] == '-') { \ - root = snocString((root), (spec)); \ - } else { \ - struct _finddata_t c_file; \ - long hFile; \ - hFile = _findfirst((spec), &c_file); \ - if ( hFile == -1L ) { \ - root = snocString ((root), (spec)); \ - } else { \ - int anInt = 0; \ - while ( anInt == 0 ) { \ - root = snocString((root), \ - &c_file.name[0]); \ - anInt = _findnext(hFile, &c_file); \ - } \ - } \ - } \ - } while ( 0 ) - #else - #define APPEND_FILESPEC(root, name) \ - root = snocString ((root), (name)) - #endif - - #define SET_BINARY_MODE(fd) \ - do { \ - int retVal = setmode ( fileno ( fd ), \ - O_BINARY ); \ - ERROR_IF_MINUS_ONE ( retVal ); \ - } while ( 0 ) - -#endif - - -/*---------------------------------------------*/ -/*-- - Some more stuff for all platforms :-) ---*/ - -#define Bool unsigned char -#define True 1 -#define False 0 - -/*-- - IntNative is your platform's `native' int size. - Only here to avoid probs with 64-bit platforms. ---*/ -#define IntNative int - - -/*-- - change to 1, or compile with -DDEBUG=1 to debug ---*/ -#ifndef DEBUG -#define DEBUG 0 -#endif - - -/*---------------------------------------------------*/ -/*--- ---*/ -/*---------------------------------------------------*/ - -/*-- - Implementation notes, July 1997 - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - Memory allocation - ~~~~~~~~~~~~~~~~~ - All large data structures are allocated on the C heap, - for better or for worse. That includes the various - arrays of pointers, striped words, bytes, frequency - tables and buffers for compression and decompression. - - bzip2 can operate at various block-sizes, ranging from - 100k to 900k in 100k steps, and it allocates only as - much as it needs to. When compressing, we know from the - command-line options what the block-size is going to be, - so all allocation can be done at start-up; if that - succeeds, there can be no further allocation problems. - - Decompression is more complicated. Each compressed file - contains, in its header, a byte indicating the block - size used for compression. This means bzip2 potentially - needs to reallocate memory for each file it deals with, - which in turn opens the possibility for a memory allocation - failure part way through a run of files, by encountering - a file requiring a much larger block size than all the - ones preceding it. - - The policy is to simply give up if a memory allocation - failure occurs. During decompression, it would be - possible to move on to subsequent files in the hope that - some might ask for a smaller block size, but the - complications for doing this seem more trouble than they - are worth. - - - Compressed file formats - ~~~~~~~~~~~~~~~~~~~~~~~ - [This is now entirely different from both 0.21, and from - any previous Huffman-coded variant of bzip. - See the associated file bzip2.txt for details.] - - - Error conditions - ~~~~~~~~~~~~~~~~ - Dealing with error conditions is the least satisfactory - aspect of bzip2. The policy is to try and leave the - filesystem in a consistent state, then quit, even if it - means not processing some of the files mentioned in the - command line. `A consistent state' means that a file - exists either in its compressed or uncompressed form, - but not both. This boils down to the rule `delete the - output file if an error condition occurs, leaving the - input intact'. Input files are only deleted when we can - be pretty sure the output file has been written and - closed successfully. - - Errors are a dog because there's so many things to - deal with. The following can happen mid-file, and - require cleaning up. - - internal `panics' -- indicating a bug - corrupted or inconsistent compressed file - can't allocate enough memory to decompress this file - I/O error reading/writing/opening/closing - signal catches -- Control-C, SIGTERM, SIGHUP. - - Other conditions, primarily pertaining to file names, - can be checked in-between files, which makes dealing - with them easier. ---*/ - - - -/*---------------------------------------------------*/ -/*--- Misc (file handling) data decls ---*/ -/*---------------------------------------------------*/ - -UInt32 bytesIn, bytesOut; -Int32 verbosity; -Bool keepInputFiles, smallMode, testFailsExist; -UInt32 globalCrc; -Int32 numFileNames, numFilesProcessed; - - -/*-- source modes; F==file, I==stdin, O==stdout --*/ -#define SM_I2O 1 -#define SM_F2O 2 -#define SM_F2F 3 - -/*-- operation modes --*/ -#define OM_Z 1 -#define OM_UNZ 2 -#define OM_TEST 3 - -Int32 opMode; -Int32 srcMode; - - -Int32 longestFileName; -Char inName[1024]; -Char outName[1024]; -Char *progName; -Char progNameReally[1024]; -FILE *outputHandleJustInCase; - -void panic ( Char* ) NORETURN; -void ioError ( void ) NORETURN; -void compressOutOfMemory ( Int32, Int32 ) NORETURN; -void uncompressOutOfMemory ( Int32, Int32 ) NORETURN; -void blockOverrun ( void ) NORETURN; -void badBlockHeader ( void ) NORETURN; -void badBGLengths ( void ) NORETURN; -void crcError ( UInt32, UInt32 ) NORETURN; -void bitStreamEOF ( void ) NORETURN; -void cleanUpAndFail ( Int32 ) NORETURN; -void compressedStreamEOF ( void ) NORETURN; - -void* myMalloc ( Int32 ); - - - -/*---------------------------------------------------*/ -/*--- Data decls for the front end ---*/ -/*---------------------------------------------------*/ - -/*-- - The overshoot bytes allow us to avoid most of - the cost of pointer renormalisation during - comparison of rotations in sorting. - The figure of 20 is derived as follows: - qSort3 allows an overshoot of up to 10. - It then calls simpleSort, which calls - fullGtU, also with max overshoot 10. - fullGtU does up to 10 comparisons without - renormalising, giving 10+10 == 20. ---*/ -#define NUM_OVERSHOOT_BYTES 20 - -/*-- - These are the main data structures for - the Burrows-Wheeler transform. ---*/ - -/*-- - Pointers to compression and decompression - structures. Set by - allocateCompressStructures and - setDecompressStructureSizes - - The structures are always set to be suitable - for a block of size 100000 * blockSize100k. ---*/ -UChar *block; /*-- compress --*/ -UInt16 *quadrant; /*-- compress --*/ -Int32 *zptr; /*-- compress --*/ -UInt16 *szptr; /*-- overlays zptr ---*/ -Int32 *ftab; /*-- compress --*/ - -UInt16 *ll16; /*-- small decompress --*/ -UChar *ll4; /*-- small decompress --*/ - -Int32 *tt; /*-- fast decompress --*/ -UChar *ll8; /*-- fast decompress --*/ - - -/*-- - freq table collected to save a pass over the data - during decompression. ---*/ -Int32 unzftab[256]; - - -/*-- - index of the last char in the block, so - the block size == last + 1. ---*/ -Int32 last; - - -/*-- - index in zptr[] of original string after sorting. ---*/ -Int32 origPtr; - - -/*-- - always: in the range 0 .. 9. - The current block size is 100000 * this number. ---*/ -Int32 blockSize100k; - - -/*-- - Used when sorting. If too many long comparisons - happen, we stop sorting, randomise the block - slightly, and try again. ---*/ - -Int32 workFactor; -Int32 workDone; -Int32 workLimit; -Bool blockRandomised; -Bool firstAttempt; -Int32 nBlocksRandomised; - - - -/*---------------------------------------------------*/ -/*--- Data decls for the back end ---*/ -/*---------------------------------------------------*/ - -#define MAX_ALPHA_SIZE 258 -#define MAX_CODE_LEN 23 - -#define RUNA 0 -#define RUNB 1 - -#define N_GROUPS 6 -#define G_SIZE 50 -#define N_ITERS 4 - -#define MAX_SELECTORS (2 + (900000 / G_SIZE)) - -Bool inUse[256]; -Int32 nInUse; - -UChar seqToUnseq[256]; -UChar unseqToSeq[256]; - -UChar selector [MAX_SELECTORS]; -UChar selectorMtf[MAX_SELECTORS]; - -Int32 nMTF; - -Int32 mtfFreq[MAX_ALPHA_SIZE]; - -UChar len [N_GROUPS][MAX_ALPHA_SIZE]; - -/*-- decompress only --*/ -Int32 limit [N_GROUPS][MAX_ALPHA_SIZE]; -Int32 base [N_GROUPS][MAX_ALPHA_SIZE]; -Int32 perm [N_GROUPS][MAX_ALPHA_SIZE]; -Int32 minLens[N_GROUPS]; - -/*-- compress only --*/ -Int32 code [N_GROUPS][MAX_ALPHA_SIZE]; -Int32 rfreq[N_GROUPS][MAX_ALPHA_SIZE]; - - -/*---------------------------------------------------*/ -/*--- 32-bit CRC grunge ---*/ -/*---------------------------------------------------*/ - -/*-- - I think this is an implementation of the AUTODIN-II, - Ethernet & FDDI 32-bit CRC standard. Vaguely derived - from code by Rob Warnock, in Section 51 of the - comp.compression FAQ. ---*/ - -UInt32 crc32Table[256] = { - - /*-- Ugly, innit? --*/ - - 0x00000000UL, 0x04c11db7UL, 0x09823b6eUL, 0x0d4326d9UL, - 0x130476dcUL, 0x17c56b6bUL, 0x1a864db2UL, 0x1e475005UL, - 0x2608edb8UL, 0x22c9f00fUL, 0x2f8ad6d6UL, 0x2b4bcb61UL, - 0x350c9b64UL, 0x31cd86d3UL, 0x3c8ea00aUL, 0x384fbdbdUL, - 0x4c11db70UL, 0x48d0c6c7UL, 0x4593e01eUL, 0x4152fda9UL, - 0x5f15adacUL, 0x5bd4b01bUL, 0x569796c2UL, 0x52568b75UL, - 0x6a1936c8UL, 0x6ed82b7fUL, 0x639b0da6UL, 0x675a1011UL, - 0x791d4014UL, 0x7ddc5da3UL, 0x709f7b7aUL, 0x745e66cdUL, - 0x9823b6e0UL, 0x9ce2ab57UL, 0x91a18d8eUL, 0x95609039UL, - 0x8b27c03cUL, 0x8fe6dd8bUL, 0x82a5fb52UL, 0x8664e6e5UL, - 0xbe2b5b58UL, 0xbaea46efUL, 0xb7a96036UL, 0xb3687d81UL, - 0xad2f2d84UL, 0xa9ee3033UL, 0xa4ad16eaUL, 0xa06c0b5dUL, - 0xd4326d90UL, 0xd0f37027UL, 0xddb056feUL, 0xd9714b49UL, - 0xc7361b4cUL, 0xc3f706fbUL, 0xceb42022UL, 0xca753d95UL, - 0xf23a8028UL, 0xf6fb9d9fUL, 0xfbb8bb46UL, 0xff79a6f1UL, - 0xe13ef6f4UL, 0xe5ffeb43UL, 0xe8bccd9aUL, 0xec7dd02dUL, - 0x34867077UL, 0x30476dc0UL, 0x3d044b19UL, 0x39c556aeUL, - 0x278206abUL, 0x23431b1cUL, 0x2e003dc5UL, 0x2ac12072UL, - 0x128e9dcfUL, 0x164f8078UL, 0x1b0ca6a1UL, 0x1fcdbb16UL, - 0x018aeb13UL, 0x054bf6a4UL, 0x0808d07dUL, 0x0cc9cdcaUL, - 0x7897ab07UL, 0x7c56b6b0UL, 0x71159069UL, 0x75d48ddeUL, - 0x6b93dddbUL, 0x6f52c06cUL, 0x6211e6b5UL, 0x66d0fb02UL, - 0x5e9f46bfUL, 0x5a5e5b08UL, 0x571d7dd1UL, 0x53dc6066UL, - 0x4d9b3063UL, 0x495a2dd4UL, 0x44190b0dUL, 0x40d816baUL, - 0xaca5c697UL, 0xa864db20UL, 0xa527fdf9UL, 0xa1e6e04eUL, - 0xbfa1b04bUL, 0xbb60adfcUL, 0xb6238b25UL, 0xb2e29692UL, - 0x8aad2b2fUL, 0x8e6c3698UL, 0x832f1041UL, 0x87ee0df6UL, - 0x99a95df3UL, 0x9d684044UL, 0x902b669dUL, 0x94ea7b2aUL, - 0xe0b41de7UL, 0xe4750050UL, 0xe9362689UL, 0xedf73b3eUL, - 0xf3b06b3bUL, 0xf771768cUL, 0xfa325055UL, 0xfef34de2UL, - 0xc6bcf05fUL, 0xc27dede8UL, 0xcf3ecb31UL, 0xcbffd686UL, - 0xd5b88683UL, 0xd1799b34UL, 0xdc3abdedUL, 0xd8fba05aUL, - 0x690ce0eeUL, 0x6dcdfd59UL, 0x608edb80UL, 0x644fc637UL, - 0x7a089632UL, 0x7ec98b85UL, 0x738aad5cUL, 0x774bb0ebUL, - 0x4f040d56UL, 0x4bc510e1UL, 0x46863638UL, 0x42472b8fUL, - 0x5c007b8aUL, 0x58c1663dUL, 0x558240e4UL, 0x51435d53UL, - 0x251d3b9eUL, 0x21dc2629UL, 0x2c9f00f0UL, 0x285e1d47UL, - 0x36194d42UL, 0x32d850f5UL, 0x3f9b762cUL, 0x3b5a6b9bUL, - 0x0315d626UL, 0x07d4cb91UL, 0x0a97ed48UL, 0x0e56f0ffUL, - 0x1011a0faUL, 0x14d0bd4dUL, 0x19939b94UL, 0x1d528623UL, - 0xf12f560eUL, 0xf5ee4bb9UL, 0xf8ad6d60UL, 0xfc6c70d7UL, - 0xe22b20d2UL, 0xe6ea3d65UL, 0xeba91bbcUL, 0xef68060bUL, - 0xd727bbb6UL, 0xd3e6a601UL, 0xdea580d8UL, 0xda649d6fUL, - 0xc423cd6aUL, 0xc0e2d0ddUL, 0xcda1f604UL, 0xc960ebb3UL, - 0xbd3e8d7eUL, 0xb9ff90c9UL, 0xb4bcb610UL, 0xb07daba7UL, - 0xae3afba2UL, 0xaafbe615UL, 0xa7b8c0ccUL, 0xa379dd7bUL, - 0x9b3660c6UL, 0x9ff77d71UL, 0x92b45ba8UL, 0x9675461fUL, - 0x8832161aUL, 0x8cf30badUL, 0x81b02d74UL, 0x857130c3UL, - 0x5d8a9099UL, 0x594b8d2eUL, 0x5408abf7UL, 0x50c9b640UL, - 0x4e8ee645UL, 0x4a4ffbf2UL, 0x470cdd2bUL, 0x43cdc09cUL, - 0x7b827d21UL, 0x7f436096UL, 0x7200464fUL, 0x76c15bf8UL, - 0x68860bfdUL, 0x6c47164aUL, 0x61043093UL, 0x65c52d24UL, - 0x119b4be9UL, 0x155a565eUL, 0x18197087UL, 0x1cd86d30UL, - 0x029f3d35UL, 0x065e2082UL, 0x0b1d065bUL, 0x0fdc1becUL, - 0x3793a651UL, 0x3352bbe6UL, 0x3e119d3fUL, 0x3ad08088UL, - 0x2497d08dUL, 0x2056cd3aUL, 0x2d15ebe3UL, 0x29d4f654UL, - 0xc5a92679UL, 0xc1683bceUL, 0xcc2b1d17UL, 0xc8ea00a0UL, - 0xd6ad50a5UL, 0xd26c4d12UL, 0xdf2f6bcbUL, 0xdbee767cUL, - 0xe3a1cbc1UL, 0xe760d676UL, 0xea23f0afUL, 0xeee2ed18UL, - 0xf0a5bd1dUL, 0xf464a0aaUL, 0xf9278673UL, 0xfde69bc4UL, - 0x89b8fd09UL, 0x8d79e0beUL, 0x803ac667UL, 0x84fbdbd0UL, - 0x9abc8bd5UL, 0x9e7d9662UL, 0x933eb0bbUL, 0x97ffad0cUL, - 0xafb010b1UL, 0xab710d06UL, 0xa6322bdfUL, 0xa2f33668UL, - 0xbcb4666dUL, 0xb8757bdaUL, 0xb5365d03UL, 0xb1f740b4UL -}; - - -/*---------------------------------------------*/ -void initialiseCRC ( void ) -{ - globalCrc = 0xffffffffUL; -} - - -/*---------------------------------------------*/ -UInt32 getFinalCRC ( void ) -{ - return ~globalCrc; -} - - -/*---------------------------------------------*/ -UInt32 getGlobalCRC ( void ) -{ - return globalCrc; -} - - -/*---------------------------------------------*/ -void setGlobalCRC ( UInt32 newCrc ) -{ - globalCrc = newCrc; -} - - -/*---------------------------------------------*/ -#define UPDATE_CRC(crcVar,cha) \ -{ \ - crcVar = (crcVar << 8) ^ \ - crc32Table[(crcVar >> 24) ^ \ - ((UChar)cha)]; \ -} - - -/*---------------------------------------------------*/ -/*--- Bit stream I/O ---*/ -/*---------------------------------------------------*/ - - -UInt32 bsBuff; -Int32 bsLive; -FILE* bsStream; -Bool bsWriting; - - -/*---------------------------------------------*/ -void bsSetStream ( FILE* f, Bool wr ) -{ - if (bsStream != NULL) panic ( "bsSetStream" ); - bsStream = f; - bsLive = 0; - bsBuff = 0; - bytesOut = 0; - bytesIn = 0; - bsWriting = wr; -} - - -/*---------------------------------------------*/ -void bsFinishedWithStream ( void ) -{ - if (bsWriting) - while (bsLive > 0) { - fputc ( (UChar)(bsBuff >> 24), bsStream ); - bsBuff <<= 8; - bsLive -= 8; - bytesOut++; - } - bsStream = NULL; -} - - -/*---------------------------------------------*/ -#define bsNEEDR(nz) \ -{ \ - while (bsLive < nz) { \ - Int32 zzi = fgetc ( bsStream ); \ - if (zzi == EOF) compressedStreamEOF(); \ - bsBuff = (bsBuff << 8) | (zzi & 0xffL); \ - bsLive += 8; \ - } \ -} - - -/*---------------------------------------------*/ -#define bsNEEDW(nz) \ -{ \ - while (bsLive >= 8) { \ - fputc ( (UChar)(bsBuff >> 24), \ - bsStream ); \ - bsBuff <<= 8; \ - bsLive -= 8; \ - bytesOut++; \ - } \ -} - - -/*---------------------------------------------*/ -#define bsR1(vz) \ -{ \ - bsNEEDR(1); \ - vz = (bsBuff >> (bsLive-1)) & 1; \ - bsLive--; \ -} - - -/*---------------------------------------------*/ -INLINE UInt32 bsR ( Int32 n ) -{ - UInt32 v; - bsNEEDR ( n ); - v = (bsBuff >> (bsLive-n)) & ((1 << n)-1); - bsLive -= n; - return v; -} - - -/*---------------------------------------------*/ -INLINE void bsW ( Int32 n, UInt32 v ) -{ - bsNEEDW ( n ); - bsBuff |= (v << (32 - bsLive - n)); - bsLive += n; -} - - -/*---------------------------------------------*/ -UChar bsGetUChar ( void ) -{ - return (UChar)bsR(8); -} - - -/*---------------------------------------------*/ -void bsPutUChar ( UChar c ) -{ - bsW(8, (UInt32)c ); -} - - -/*---------------------------------------------*/ -Int32 bsGetUInt32 ( void ) -{ - UInt32 u; - u = 0; - u = (u << 8) | bsR(8); - u = (u << 8) | bsR(8); - u = (u << 8) | bsR(8); - u = (u << 8) | bsR(8); - return u; -} - - -/*---------------------------------------------*/ -UInt32 bsGetIntVS ( UInt32 numBits ) -{ - return (UInt32)bsR(numBits); -} - - -/*---------------------------------------------*/ -UInt32 bsGetInt32 ( void ) -{ - return (Int32)bsGetUInt32(); -} - - -/*---------------------------------------------*/ -void bsPutUInt32 ( UInt32 u ) -{ - bsW ( 8, (u >> 24) & 0xffL ); - bsW ( 8, (u >> 16) & 0xffL ); - bsW ( 8, (u >> 8) & 0xffL ); - bsW ( 8, u & 0xffL ); -} - - -/*---------------------------------------------*/ -void bsPutInt32 ( Int32 c ) -{ - bsPutUInt32 ( (UInt32)c ); -} - - -/*---------------------------------------------*/ -void bsPutIntVS ( Int32 numBits, UInt32 c ) -{ - bsW ( numBits, c ); -} - - -/*---------------------------------------------------*/ -/*--- Huffman coding low-level stuff ---*/ -/*---------------------------------------------------*/ - -#define WEIGHTOF(zz0) ((zz0) & 0xffffff00) -#define DEPTHOF(zz1) ((zz1) & 0x000000ff) -#define MYMAX(zz2,zz3) ((zz2) > (zz3) ? (zz2) : (zz3)) - -#define ADDWEIGHTS(zw1,zw2) \ - (WEIGHTOF(zw1)+WEIGHTOF(zw2)) | \ - (1 + MYMAX(DEPTHOF(zw1),DEPTHOF(zw2))) - -#define UPHEAP(z) \ -{ \ - Int32 zz, tmp; \ - zz = z; tmp = heap[zz]; \ - while (weight[tmp] < weight[heap[zz >> 1]]) { \ - heap[zz] = heap[zz >> 1]; \ - zz >>= 1; \ - } \ - heap[zz] = tmp; \ -} - -#define DOWNHEAP(z) \ -{ \ - Int32 zz, yy, tmp; \ - zz = z; tmp = heap[zz]; \ - while (True) { \ - yy = zz << 1; \ - if (yy > nHeap) break; \ - if (yy < nHeap && \ - weight[heap[yy+1]] < weight[heap[yy]]) \ - yy++; \ - if (weight[tmp] < weight[heap[yy]]) break; \ - heap[zz] = heap[yy]; \ - zz = yy; \ - } \ - heap[zz] = tmp; \ -} - - -/*---------------------------------------------*/ -void hbMakeCodeLengths ( UChar *len, - Int32 *freq, - Int32 alphaSize, - Int32 maxLen ) -{ - /*-- - Nodes and heap entries run from 1. Entry 0 - for both the heap and nodes is a sentinel. - --*/ - Int32 nNodes, nHeap, n1, n2, i, j, k; - Bool tooLong; - - Int32 heap [ MAX_ALPHA_SIZE + 2 ]; - Int32 weight [ MAX_ALPHA_SIZE * 2 ]; - Int32 parent [ MAX_ALPHA_SIZE * 2 ]; - - for (i = 0; i < alphaSize; i++) - weight[i+1] = (freq[i] == 0 ? 1 : freq[i]) << 8; - - while (True) { - - nNodes = alphaSize; - nHeap = 0; - - heap[0] = 0; - weight[0] = 0; - parent[0] = -2; - - for (i = 1; i <= alphaSize; i++) { - parent[i] = -1; - nHeap++; - heap[nHeap] = i; - UPHEAP(nHeap); - } - if (!(nHeap < (MAX_ALPHA_SIZE+2))) - panic ( "hbMakeCodeLengths(1)" ); - - while (nHeap > 1) { - n1 = heap[1]; heap[1] = heap[nHeap]; nHeap--; DOWNHEAP(1); - n2 = heap[1]; heap[1] = heap[nHeap]; nHeap--; DOWNHEAP(1); - nNodes++; - parent[n1] = parent[n2] = nNodes; - weight[nNodes] = ADDWEIGHTS(weight[n1], weight[n2]); - parent[nNodes] = -1; - nHeap++; - heap[nHeap] = nNodes; - UPHEAP(nHeap); - } - if (!(nNodes < (MAX_ALPHA_SIZE * 2))) - panic ( "hbMakeCodeLengths(2)" ); - - tooLong = False; - for (i = 1; i <= alphaSize; i++) { - j = 0; - k = i; - while (parent[k] >= 0) { k = parent[k]; j++; } - len[i-1] = j; - if (j > maxLen) tooLong = True; - } - - if (! tooLong) break; - - for (i = 1; i < alphaSize; i++) { - j = weight[i] >> 8; - j = 1 + (j / 2); - weight[i] = j << 8; - } - } -} - - -/*---------------------------------------------*/ -void hbAssignCodes ( Int32 *code, - UChar *length, - Int32 minLen, - Int32 maxLen, - Int32 alphaSize ) -{ - Int32 n, vec, i; - - vec = 0; - for (n = minLen; n <= maxLen; n++) { - for (i = 0; i < alphaSize; i++) - if (length[i] == n) { code[i] = vec; vec++; }; - vec <<= 1; - } -} - - -/*---------------------------------------------*/ -void hbCreateDecodeTables ( Int32 *limit, - Int32 *base, - Int32 *perm, - UChar *length, - Int32 minLen, - Int32 maxLen, - Int32 alphaSize ) -{ - Int32 pp, i, j, vec; - - pp = 0; - for (i = minLen; i <= maxLen; i++) - for (j = 0; j < alphaSize; j++) - if (length[j] == i) { perm[pp] = j; pp++; }; - - for (i = 0; i < MAX_CODE_LEN; i++) base[i] = 0; - for (i = 0; i < alphaSize; i++) base[length[i]+1]++; - - for (i = 1; i < MAX_CODE_LEN; i++) base[i] += base[i-1]; - - for (i = 0; i < MAX_CODE_LEN; i++) limit[i] = 0; - vec = 0; - - for (i = minLen; i <= maxLen; i++) { - vec += (base[i+1] - base[i]); - limit[i] = vec-1; - vec <<= 1; - } - for (i = minLen + 1; i <= maxLen; i++) - base[i] = ((limit[i-1] + 1) << 1) - base[i]; -} - - - -/*---------------------------------------------------*/ -/*--- Undoing the reversible transformation ---*/ -/*---------------------------------------------------*/ - -/*---------------------------------------------*/ -#define SET_LL4(i,n) \ - { if (((i) & 0x1) == 0) \ - ll4[(i) >> 1] = (ll4[(i) >> 1] & 0xf0) | (n); else \ - ll4[(i) >> 1] = (ll4[(i) >> 1] & 0x0f) | ((n) << 4); \ - } - -#define GET_LL4(i) \ - (((UInt32)(ll4[(i) >> 1])) >> (((i) << 2) & 0x4) & 0xF) - -#define SET_LL(i,n) \ - { ll16[i] = (UInt16)(n & 0x0000ffff); \ - SET_LL4(i, n >> 16); \ - } - -#define GET_LL(i) \ - (((UInt32)ll16[i]) | (GET_LL4(i) << 16)) - - -/*---------------------------------------------*/ -/*-- - Manage memory for compression/decompression. - When compressing, a single block size applies to - all files processed, and that's set when the - program starts. But when decompressing, each file - processed could have been compressed with a - different block size, so we may have to free - and reallocate on a per-file basis. - - A call with argument of zero means - `free up everything.' And a value of zero for - blockSize100k means no memory is currently allocated. ---*/ - - -/*---------------------------------------------*/ -void allocateCompressStructures ( void ) -{ - Int32 n = 100000 * blockSize100k; - block = malloc ( (n + 1 + NUM_OVERSHOOT_BYTES) * sizeof(UChar) ); - quadrant = malloc ( (n + NUM_OVERSHOOT_BYTES) * sizeof(Int16) ); - zptr = malloc ( n * sizeof(Int32) ); - ftab = malloc ( 65537 * sizeof(Int32) ); - - if (block == NULL || quadrant == NULL || - zptr == NULL || ftab == NULL) { - Int32 totalDraw - = (n + 1 + NUM_OVERSHOOT_BYTES) * sizeof(UChar) + - (n + NUM_OVERSHOOT_BYTES) * sizeof(Int16) + - n * sizeof(Int32) + - 65537 * sizeof(Int32); - - compressOutOfMemory ( totalDraw, n ); - } - - /*-- - Since we want valid indexes for block of - -1 to n + NUM_OVERSHOOT_BYTES - 1 - inclusive. - --*/ - block++; - - /*-- - The back end needs a place to store the MTF values - whilst it calculates the coding tables. We could - put them in the zptr array. However, these values - will fit in a short, so we overlay szptr at the - start of zptr, in the hope of reducing the number - of cache misses induced by the multiple traversals - of the MTF values when calculating coding tables. - Seems to improve compression speed by about 1%. - --*/ - szptr = (UInt16*)zptr; -} - - -/*---------------------------------------------*/ -void setDecompressStructureSizes ( Int32 newSize100k ) -{ - if (! (0 <= newSize100k && newSize100k <= 9 && - 0 <= blockSize100k && blockSize100k <= 9)) - panic ( "setDecompressStructureSizes" ); - - if (newSize100k == blockSize100k) return; - - blockSize100k = newSize100k; - - if (ll16 != NULL) free ( ll16 ); - if (ll4 != NULL) free ( ll4 ); - if (ll8 != NULL) free ( ll8 ); - if (tt != NULL) free ( tt ); - - if (newSize100k == 0) return; - - if (smallMode) { - - Int32 n = 100000 * newSize100k; - ll16 = malloc ( n * sizeof(UInt16) ); - ll4 = malloc ( ((n+1) >> 1) * sizeof(UChar) ); - - if (ll4 == NULL || ll16 == NULL) { - Int32 totalDraw - = n * sizeof(Int16) + ((n+1) >> 1) * sizeof(UChar); - uncompressOutOfMemory ( totalDraw, n ); - } - - } else { - - Int32 n = 100000 * newSize100k; - ll8 = malloc ( n * sizeof(UChar) ); - tt = malloc ( n * sizeof(Int32) ); - - if (ll8 == NULL || tt == NULL) { - Int32 totalDraw - = n * sizeof(UChar) + n * sizeof(UInt32); - uncompressOutOfMemory ( totalDraw, n ); - } - - } -} - - - -/*---------------------------------------------------*/ -/*--- The new back end ---*/ -/*---------------------------------------------------*/ - -/*---------------------------------------------*/ -void makeMaps ( void ) -{ - Int32 i; - nInUse = 0; - for (i = 0; i < 256; i++) - if (inUse[i]) { - seqToUnseq[nInUse] = i; - unseqToSeq[i] = nInUse; - nInUse++; - } -} - - -/*---------------------------------------------*/ -void generateMTFValues ( void ) -{ - UChar yy[256]; - Int32 i, j; - UChar tmp; - UChar tmp2; - Int32 zPend; - Int32 wr; - Int32 EOB; - - makeMaps(); - EOB = nInUse+1; - - for (i = 0; i <= EOB; i++) mtfFreq[i] = 0; - - wr = 0; - zPend = 0; - for (i = 0; i < nInUse; i++) yy[i] = (UChar) i; - - - for (i = 0; i <= last; i++) { - UChar ll_i; - - #if DEBUG - assert (wr <= i); - #endif - - ll_i = unseqToSeq[block[zptr[i] - 1]]; - #if DEBUG - assert (ll_i < nInUse); - #endif - - j = 0; - tmp = yy[j]; - while ( ll_i != tmp ) { - j++; - tmp2 = tmp; - tmp = yy[j]; - yy[j] = tmp2; - }; - yy[0] = tmp; - - if (j == 0) { - zPend++; - } else { - if (zPend > 0) { - zPend--; - while (True) { - switch (zPend % 2) { - case 0: szptr[wr] = RUNA; wr++; mtfFreq[RUNA]++; break; - case 1: szptr[wr] = RUNB; wr++; mtfFreq[RUNB]++; break; - }; - if (zPend < 2) break; - zPend = (zPend - 2) / 2; - }; - zPend = 0; - } - szptr[wr] = j+1; wr++; mtfFreq[j+1]++; - } - } - - if (zPend > 0) { - zPend--; - while (True) { - switch (zPend % 2) { - case 0: szptr[wr] = RUNA; wr++; mtfFreq[RUNA]++; break; - case 1: szptr[wr] = RUNB; wr++; mtfFreq[RUNB]++; break; - }; - if (zPend < 2) break; - zPend = (zPend - 2) / 2; - }; - } - - szptr[wr] = EOB; wr++; mtfFreq[EOB]++; - - nMTF = wr; -} - - -/*---------------------------------------------*/ -#define LESSER_ICOST 0 -#define GREATER_ICOST 15 - -void sendMTFValues ( void ) -{ - Int32 v, t, i, j, gs, ge, totc, bt, bc, iter; - Int32 nSelectors, alphaSize, minLen, maxLen, selCtr; - Int32 nGroups, nBytes; - - /*-- - UChar len [N_GROUPS][MAX_ALPHA_SIZE]; - is a global since the decoder also needs it. - - Int32 code[N_GROUPS][MAX_ALPHA_SIZE]; - Int32 rfreq[N_GROUPS][MAX_ALPHA_SIZE]; - are also globals only used in this proc. - Made global to keep stack frame size small. - --*/ - - - UInt16 cost[N_GROUPS]; - Int32 fave[N_GROUPS]; - - if (verbosity >= 3) - fprintf ( stderr, - " %d in block, %d after MTF & 1-2 coding, %d+2 syms in use\n", - last+1, nMTF, nInUse ); - - alphaSize = nInUse+2; - for (t = 0; t < N_GROUPS; t++) - for (v = 0; v < alphaSize; v++) - len[t][v] = GREATER_ICOST; - - /*--- Decide how many coding tables to use ---*/ - if (nMTF <= 0) panic ( "sendMTFValues(0)" ); - if (nMTF < 200) nGroups = 2; else - if (nMTF < 800) nGroups = 4; else - nGroups = 6; - - /*--- Generate an initial set of coding tables ---*/ - { - Int32 nPart, remF, tFreq, aFreq; - - nPart = nGroups; - remF = nMTF; - gs = 0; - while (nPart > 0) { - tFreq = remF / nPart; - ge = gs-1; - aFreq = 0; - while (aFreq < tFreq && ge < alphaSize-1) { - ge++; - aFreq += mtfFreq[ge]; - } - - if (ge > gs - && nPart != nGroups && nPart != 1 - && ((nGroups-nPart) % 2 == 1)) { - aFreq -= mtfFreq[ge]; - ge--; - } - - if (verbosity >= 3) - fprintf ( stderr, - " initial group %d, [%d .. %d], has %d syms (%4.1f%%)\n", - nPart, gs, ge, aFreq, - (100.0 * (float)aFreq) / (float)nMTF ); - - for (v = 0; v < alphaSize; v++) - if (v >= gs && v <= ge) - len[nPart-1][v] = LESSER_ICOST; else - len[nPart-1][v] = GREATER_ICOST; - - nPart--; - gs = ge+1; - remF -= aFreq; - } - } - - /*--- - Iterate up to N_ITERS times to improve the tables. - ---*/ - for (iter = 0; iter < N_ITERS; iter++) { - - for (t = 0; t < nGroups; t++) fave[t] = 0; - - for (t = 0; t < nGroups; t++) - for (v = 0; v < alphaSize; v++) - rfreq[t][v] = 0; - - nSelectors = 0; - totc = 0; - gs = 0; - while (True) { - - /*--- Set group start & end marks. --*/ - if (gs >= nMTF) break; - ge = gs + G_SIZE - 1; - if (ge >= nMTF) ge = nMTF-1; - - /*-- - Calculate the cost of this group as coded - by each of the coding tables. - --*/ - for (t = 0; t < nGroups; t++) cost[t] = 0; - - if (nGroups == 6) { - register UInt16 cost0, cost1, cost2, cost3, cost4, cost5; - cost0 = cost1 = cost2 = cost3 = cost4 = cost5 = 0; - for (i = gs; i <= ge; i++) { - UInt16 icv = szptr[i]; - cost0 += len[0][icv]; - cost1 += len[1][icv]; - cost2 += len[2][icv]; - cost3 += len[3][icv]; - cost4 += len[4][icv]; - cost5 += len[5][icv]; - } - cost[0] = cost0; cost[1] = cost1; cost[2] = cost2; - cost[3] = cost3; cost[4] = cost4; cost[5] = cost5; - } else { - for (i = gs; i <= ge; i++) { - UInt16 icv = szptr[i]; - for (t = 0; t < nGroups; t++) cost[t] += len[t][icv]; - } - } - - /*-- - Find the coding table which is best for this group, - and record its identity in the selector table. - --*/ - bc = 999999999; bt = -1; - for (t = 0; t < nGroups; t++) - if (cost[t] < bc) { bc = cost[t]; bt = t; }; - totc += bc; - fave[bt]++; - selector[nSelectors] = bt; - nSelectors++; - - /*-- - Increment the symbol frequencies for the selected table. - --*/ - for (i = gs; i <= ge; i++) - rfreq[bt][ szptr[i] ]++; - - gs = ge+1; - } - if (verbosity >= 3) { - fprintf ( stderr, - " pass %d: size is %d, grp uses are ", - iter+1, totc/8 ); - for (t = 0; t < nGroups; t++) - fprintf ( stderr, "%d ", fave[t] ); - fprintf ( stderr, "\n" ); - } - - /*-- - Recompute the tables based on the accumulated frequencies. - --*/ - for (t = 0; t < nGroups; t++) - hbMakeCodeLengths ( &len[t][0], &rfreq[t][0], alphaSize, 20 ); - } - - - if (!(nGroups < 8)) panic ( "sendMTFValues(1)" ); - if (!(nSelectors < 32768 && - nSelectors <= (2 + (900000 / G_SIZE)))) - panic ( "sendMTFValues(2)" ); - - - /*--- Compute MTF values for the selectors. ---*/ - { - UChar pos[N_GROUPS], ll_i, tmp2, tmp; - for (i = 0; i < nGroups; i++) pos[i] = i; - for (i = 0; i < nSelectors; i++) { - ll_i = selector[i]; - j = 0; - tmp = pos[j]; - while ( ll_i != tmp ) { - j++; - tmp2 = tmp; - tmp = pos[j]; - pos[j] = tmp2; - }; - pos[0] = tmp; - selectorMtf[i] = j; - } - }; - - /*--- Assign actual codes for the tables. --*/ - for (t = 0; t < nGroups; t++) { - minLen = 32; - maxLen = 0; - for (i = 0; i < alphaSize; i++) { - if (len[t][i] > maxLen) maxLen = len[t][i]; - if (len[t][i] < minLen) minLen = len[t][i]; - } - if (maxLen > 20) panic ( "sendMTFValues(3)" ); - if (minLen < 1) panic ( "sendMTFValues(4)" ); - hbAssignCodes ( &code[t][0], &len[t][0], - minLen, maxLen, alphaSize ); - } - - /*--- Transmit the mapping table. ---*/ - { - Bool inUse16[16]; - for (i = 0; i < 16; i++) { - inUse16[i] = False; - for (j = 0; j < 16; j++) - if (inUse[i * 16 + j]) inUse16[i] = True; - } - - nBytes = bytesOut; - for (i = 0; i < 16; i++) - if (inUse16[i]) bsW(1,1); else bsW(1,0); - - for (i = 0; i < 16; i++) - if (inUse16[i]) - for (j = 0; j < 16; j++) - if (inUse[i * 16 + j]) bsW(1,1); else bsW(1,0); - - if (verbosity >= 3) - fprintf ( stderr, " bytes: mapping %d, ", bytesOut-nBytes ); - } - - /*--- Now the selectors. ---*/ - nBytes = bytesOut; - bsW ( 3, nGroups ); - bsW ( 15, nSelectors ); - for (i = 0; i < nSelectors; i++) { - for (j = 0; j < selectorMtf[i]; j++) bsW(1,1); - bsW(1,0); - } - if (verbosity >= 3) - fprintf ( stderr, "selectors %d, ", bytesOut-nBytes ); - - /*--- Now the coding tables. ---*/ - nBytes = bytesOut; - - for (t = 0; t < nGroups; t++) { - Int32 curr = len[t][0]; - bsW ( 5, curr ); - for (i = 0; i < alphaSize; i++) { - while (curr < len[t][i]) { bsW(2,2); curr++; /* 10 */ }; - while (curr > len[t][i]) { bsW(2,3); curr--; /* 11 */ }; - bsW ( 1, 0 ); - } - } - - if (verbosity >= 3) - fprintf ( stderr, "code lengths %d, ", bytesOut-nBytes ); - - /*--- And finally, the block data proper ---*/ - nBytes = bytesOut; - selCtr = 0; - gs = 0; - while (True) { - if (gs >= nMTF) break; - ge = gs + G_SIZE - 1; - if (ge >= nMTF) ge = nMTF-1; - for (i = gs; i <= ge; i++) { - #if DEBUG - assert (selector[selCtr] < nGroups); - #endif - bsW ( len [selector[selCtr]] [szptr[i]], - code [selector[selCtr]] [szptr[i]] ); - } - - gs = ge+1; - selCtr++; - } - if (!(selCtr == nSelectors)) panic ( "sendMTFValues(5)" ); - - if (verbosity >= 3) - fprintf ( stderr, "codes %d\n", bytesOut-nBytes ); -} - - -/*---------------------------------------------*/ -void moveToFrontCodeAndSend ( void ) -{ - bsPutIntVS ( 24, origPtr ); - generateMTFValues(); - sendMTFValues(); -} - - -/*---------------------------------------------*/ -void recvDecodingTables ( void ) -{ - Int32 i, j, t, nGroups, nSelectors, alphaSize; - Int32 minLen, maxLen; - Bool inUse16[16]; - - /*--- Receive the mapping table ---*/ - for (i = 0; i < 16; i++) - if (bsR(1) == 1) - inUse16[i] = True; else - inUse16[i] = False; - - for (i = 0; i < 256; i++) inUse[i] = False; - - for (i = 0; i < 16; i++) - if (inUse16[i]) - for (j = 0; j < 16; j++) - if (bsR(1) == 1) inUse[i * 16 + j] = True; - - makeMaps(); - alphaSize = nInUse+2; - - /*--- Now the selectors ---*/ - nGroups = bsR ( 3 ); - nSelectors = bsR ( 15 ); - for (i = 0; i < nSelectors; i++) { - j = 0; - while (bsR(1) == 1) j++; - selectorMtf[i] = j; - } - - /*--- Undo the MTF values for the selectors. ---*/ - { - UChar pos[N_GROUPS], tmp, v; - for (v = 0; v < nGroups; v++) pos[v] = v; - - for (i = 0; i < nSelectors; i++) { - v = selectorMtf[i]; - tmp = pos[v]; - while (v > 0) { pos[v] = pos[v-1]; v--; } - pos[0] = tmp; - selector[i] = tmp; - } - } - - /*--- Now the coding tables ---*/ - for (t = 0; t < nGroups; t++) { - Int32 curr = bsR ( 5 ); - for (i = 0; i < alphaSize; i++) { - while (bsR(1) == 1) { - if (bsR(1) == 0) curr++; else curr--; - } - len[t][i] = curr; - } - } - - /*--- Create the Huffman decoding tables ---*/ - for (t = 0; t < nGroups; t++) { - minLen = 32; - maxLen = 0; - for (i = 0; i < alphaSize; i++) { - if (len[t][i] > maxLen) maxLen = len[t][i]; - if (len[t][i] < minLen) minLen = len[t][i]; - } - hbCreateDecodeTables ( - &limit[t][0], &base[t][0], &perm[t][0], &len[t][0], - minLen, maxLen, alphaSize - ); - minLens[t] = minLen; - } -} - - -/*---------------------------------------------*/ -#define GET_MTF_VAL(lval) \ -{ \ - Int32 zt, zn, zvec, zj; \ - if (groupPos == 0) { \ - groupNo++; \ - groupPos = G_SIZE; \ - } \ - groupPos--; \ - zt = selector[groupNo]; \ - zn = minLens[zt]; \ - zvec = bsR ( zn ); \ - while (zvec > limit[zt][zn]) { \ - zn++; bsR1(zj); \ - zvec = (zvec << 1) | zj; \ - }; \ - lval = perm[zt][zvec - base[zt][zn]]; \ -} - - -/*---------------------------------------------*/ -void getAndMoveToFrontDecode ( void ) -{ - UChar yy[256]; - Int32 i, j, nextSym, limitLast; - Int32 EOB, groupNo, groupPos; - - limitLast = 100000 * blockSize100k; - origPtr = bsGetIntVS ( 24 ); - - recvDecodingTables(); - EOB = nInUse+1; - groupNo = -1; - groupPos = 0; - - /*-- - Setting up the unzftab entries here is not strictly - necessary, but it does save having to do it later - in a separate pass, and so saves a block's worth of - cache misses. - --*/ - for (i = 0; i <= 255; i++) unzftab[i] = 0; - - for (i = 0; i <= 255; i++) yy[i] = (UChar) i; - - last = -1; - - GET_MTF_VAL(nextSym); - - while (True) { - - if (nextSym == EOB) break; - - if (nextSym == RUNA || nextSym == RUNB) { - UChar ch; - Int32 s = -1; - Int32 N = 1; - do { - if (nextSym == RUNA) s = s + (0+1) * N; else - if (nextSym == RUNB) s = s + (1+1) * N; - N = N * 2; - GET_MTF_VAL(nextSym); - } - while (nextSym == RUNA || nextSym == RUNB); - - s++; - ch = seqToUnseq[yy[0]]; - unzftab[ch] += s; - - if (smallMode) - while (s > 0) { - last++; - ll16[last] = ch; - s--; - } - else - while (s > 0) { - last++; - ll8[last] = ch; - s--; - }; - - if (last >= limitLast) blockOverrun(); - continue; - - } else { - - UChar tmp; - last++; if (last >= limitLast) blockOverrun(); - - tmp = yy[nextSym-1]; - unzftab[seqToUnseq[tmp]]++; - if (smallMode) - ll16[last] = seqToUnseq[tmp]; else - ll8[last] = seqToUnseq[tmp]; - - /*-- - This loop is hammered during decompression, - hence the unrolling. - - for (j = nextSym-1; j > 0; j--) yy[j] = yy[j-1]; - --*/ - - j = nextSym-1; - for (; j > 3; j -= 4) { - yy[j] = yy[j-1]; - yy[j-1] = yy[j-2]; - yy[j-2] = yy[j-3]; - yy[j-3] = yy[j-4]; - } - for (; j > 0; j--) yy[j] = yy[j-1]; - - yy[0] = tmp; - GET_MTF_VAL(nextSym); - continue; - } - } -} - - -/*---------------------------------------------------*/ -/*--- Block-sorting machinery ---*/ -/*---------------------------------------------------*/ - -/*---------------------------------------------*/ -/*-- - Compare two strings in block. We assume (see - discussion above) that i1 and i2 have a max - offset of 10 on entry, and that the first - bytes of both block and quadrant have been - copied into the "overshoot area", ie - into the subscript range - [last+1 .. last+NUM_OVERSHOOT_BYTES]. ---*/ -INLINE Bool fullGtU ( Int32 i1, Int32 i2 ) -{ - Int32 k; - UChar c1, c2; - UInt16 s1, s2; - - #if DEBUG - /*-- - shellsort shouldn't ask to compare - something with itself. - --*/ - assert (i1 != i2); - #endif - - c1 = block[i1]; - c2 = block[i2]; - if (c1 != c2) return (c1 > c2); - i1++; i2++; - - c1 = block[i1]; - c2 = block[i2]; - if (c1 != c2) return (c1 > c2); - i1++; i2++; - - c1 = block[i1]; - c2 = block[i2]; - if (c1 != c2) return (c1 > c2); - i1++; i2++; - - c1 = block[i1]; - c2 = block[i2]; - if (c1 != c2) return (c1 > c2); - i1++; i2++; - - c1 = block[i1]; - c2 = block[i2]; - if (c1 != c2) return (c1 > c2); - i1++; i2++; - - c1 = block[i1]; - c2 = block[i2]; - if (c1 != c2) return (c1 > c2); - i1++; i2++; - - k = last + 1; - - do { - - c1 = block[i1]; - c2 = block[i2]; - if (c1 != c2) return (c1 > c2); - s1 = quadrant[i1]; - s2 = quadrant[i2]; - if (s1 != s2) return (s1 > s2); - i1++; i2++; - - c1 = block[i1]; - c2 = block[i2]; - if (c1 != c2) return (c1 > c2); - s1 = quadrant[i1]; - s2 = quadrant[i2]; - if (s1 != s2) return (s1 > s2); - i1++; i2++; - - c1 = block[i1]; - c2 = block[i2]; - if (c1 != c2) return (c1 > c2); - s1 = quadrant[i1]; - s2 = quadrant[i2]; - if (s1 != s2) return (s1 > s2); - i1++; i2++; - - c1 = block[i1]; - c2 = block[i2]; - if (c1 != c2) return (c1 > c2); - s1 = quadrant[i1]; - s2 = quadrant[i2]; - if (s1 != s2) return (s1 > s2); - i1++; i2++; - - if (i1 > last) { i1 -= last; i1--; }; - if (i2 > last) { i2 -= last; i2--; }; - - k -= 4; - workDone++; - } - while (k >= 0); - - return False; -} - -/*---------------------------------------------*/ -/*-- - Knuth's increments seem to work better - than Incerpi-Sedgewick here. Possibly - because the number of elems to sort is - usually small, typically <= 20. ---*/ -Int32 incs[14] = { 1, 4, 13, 40, 121, 364, 1093, 3280, - 9841, 29524, 88573, 265720, - 797161, 2391484 }; - -void simpleSort ( Int32 lo, Int32 hi, Int32 d ) -{ - Int32 i, j, h, bigN, hp; - Int32 v; - - bigN = hi - lo + 1; - if (bigN < 2) return; - - hp = 0; - while (incs[hp] < bigN) hp++; - hp--; - - for (; hp >= 0; hp--) { - h = incs[hp]; - if (verbosity >= 5) - fprintf ( stderr, " shell increment %d\n", h ); - - i = lo + h; - while (True) { - - /*-- copy 1 --*/ - if (i > hi) break; - v = zptr[i]; - j = i; - while ( fullGtU ( zptr[j-h]+d, v+d ) ) { - zptr[j] = zptr[j-h]; - j = j - h; - if (j <= (lo + h - 1)) break; - } - zptr[j] = v; - i++; - - /*-- copy 2 --*/ - if (i > hi) break; - v = zptr[i]; - j = i; - while ( fullGtU ( zptr[j-h]+d, v+d ) ) { - zptr[j] = zptr[j-h]; - j = j - h; - if (j <= (lo + h - 1)) break; - } - zptr[j] = v; - i++; - - /*-- copy 3 --*/ - if (i > hi) break; - v = zptr[i]; - j = i; - while ( fullGtU ( zptr[j-h]+d, v+d ) ) { - zptr[j] = zptr[j-h]; - j = j - h; - if (j <= (lo + h - 1)) break; - } - zptr[j] = v; - i++; - - if (workDone > workLimit && firstAttempt) return; - } - } -} - - -/*---------------------------------------------*/ -/*-- - The following is an implementation of - an elegant 3-way quicksort for strings, - described in a paper "Fast Algorithms for - Sorting and Searching Strings", by Robert - Sedgewick and Jon L. Bentley. ---*/ - -#define swap(lv1, lv2) \ - { Int32 tmp = lv1; lv1 = lv2; lv2 = tmp; } - -INLINE void vswap ( Int32 p1, Int32 p2, Int32 n ) -{ - while (n > 0) { - swap(zptr[p1], zptr[p2]); - p1++; p2++; n--; - } -} - -INLINE UChar med3 ( UChar a, UChar b, UChar c ) -{ - UChar t; - if (a > b) { t = a; a = b; b = t; }; - if (b > c) { t = b; b = c; c = t; }; - if (a > b) b = a; - return b; -} - - -#define min(a,b) ((a) < (b)) ? (a) : (b) - -typedef - struct { Int32 ll; Int32 hh; Int32 dd; } - StackElem; - -#define push(lz,hz,dz) { stack[sp].ll = lz; \ - stack[sp].hh = hz; \ - stack[sp].dd = dz; \ - sp++; } - -#define pop(lz,hz,dz) { sp--; \ - lz = stack[sp].ll; \ - hz = stack[sp].hh; \ - dz = stack[sp].dd; } - -#define SMALL_THRESH 20 -#define DEPTH_THRESH 10 - -/*-- - If you are ever unlucky/improbable enough - to get a stack overflow whilst sorting, - increase the following constant and try - again. In practice I have never seen the - stack go above 27 elems, so the following - limit seems very generous. ---*/ -#define QSORT_STACK_SIZE 1000 - - -void qSort3 ( Int32 loSt, Int32 hiSt, Int32 dSt ) -{ - Int32 unLo, unHi, ltLo, gtHi, med, n, m; - Int32 sp, lo, hi, d; - StackElem stack[QSORT_STACK_SIZE]; - - sp = 0; - push ( loSt, hiSt, dSt ); - - while (sp > 0) { - - if (sp >= QSORT_STACK_SIZE) panic ( "stack overflow in qSort3" ); - - pop ( lo, hi, d ); - - if (hi - lo < SMALL_THRESH || d > DEPTH_THRESH) { - simpleSort ( lo, hi, d ); - if (workDone > workLimit && firstAttempt) return; - continue; - } - - med = med3 ( block[zptr[ lo ]+d], - block[zptr[ hi ]+d], - block[zptr[ (lo+hi)>>1 ]+d] ); - - unLo = ltLo = lo; - unHi = gtHi = hi; - - while (True) { - while (True) { - if (unLo > unHi) break; - n = ((Int32)block[zptr[unLo]+d]) - med; - if (n == 0) { swap(zptr[unLo], zptr[ltLo]); ltLo++; unLo++; continue; }; - if (n > 0) break; - unLo++; - } - while (True) { - if (unLo > unHi) break; - n = ((Int32)block[zptr[unHi]+d]) - med; - if (n == 0) { swap(zptr[unHi], zptr[gtHi]); gtHi--; unHi--; continue; }; - if (n < 0) break; - unHi--; - } - if (unLo > unHi) break; - swap(zptr[unLo], zptr[unHi]); unLo++; unHi--; - } - #if DEBUG - assert (unHi == unLo-1); - #endif - - if (gtHi < ltLo) { - push(lo, hi, d+1 ); - continue; - } - - n = min(ltLo-lo, unLo-ltLo); vswap(lo, unLo-n, n); - m = min(hi-gtHi, gtHi-unHi); vswap(unLo, hi-m+1, m); - - n = lo + unLo - ltLo - 1; - m = hi - (gtHi - unHi) + 1; - - push ( lo, n, d ); - push ( n+1, m-1, d+1 ); - push ( m, hi, d ); - } -} - - -/*---------------------------------------------*/ - -#define BIGFREQ(b) (ftab[((b)+1) << 8] - ftab[(b) << 8]) - -#define SETMASK (1 << 21) -#define CLEARMASK (~(SETMASK)) - -void sortIt ( void ) -{ - Int32 i, j, ss, sb; - Int32 runningOrder[256]; - Int32 copy[256]; - Bool bigDone[256]; - UChar c1, c2; - Int32 numQSorted; - - /*-- - In the various block-sized structures, live data runs - from 0 to last+NUM_OVERSHOOT_BYTES inclusive. First, - set up the overshoot area for block. - --*/ - - if (verbosity >= 4) fprintf ( stderr, " sort initialise ...\n" ); - for (i = 0; i < NUM_OVERSHOOT_BYTES; i++) - block[last+i+1] = block[i % (last+1)]; - for (i = 0; i <= last+NUM_OVERSHOOT_BYTES; i++) - quadrant[i] = 0; - - block[-1] = block[last]; - - if (last < 4000) { - - /*-- - Use simpleSort(), since the full sorting mechanism - has quite a large constant overhead. - --*/ - if (verbosity >= 4) fprintf ( stderr, " simpleSort ...\n" ); - for (i = 0; i <= last; i++) zptr[i] = i; - firstAttempt = False; - workDone = workLimit = 0; - simpleSort ( 0, last, 0 ); - if (verbosity >= 4) fprintf ( stderr, " simpleSort done.\n" ); - - } else { - - numQSorted = 0; - for (i = 0; i <= 255; i++) bigDone[i] = False; - - if (verbosity >= 4) fprintf ( stderr, " bucket sorting ...\n" ); - - for (i = 0; i <= 65536; i++) ftab[i] = 0; - - c1 = block[-1]; - for (i = 0; i <= last; i++) { - c2 = block[i]; - ftab[(c1 << 8) + c2]++; - c1 = c2; - } - - for (i = 1; i <= 65536; i++) ftab[i] += ftab[i-1]; - - c1 = block[0]; - for (i = 0; i < last; i++) { - c2 = block[i+1]; - j = (c1 << 8) + c2; - c1 = c2; - ftab[j]--; - zptr[ftab[j]] = i; - } - j = (block[last] << 8) + block[0]; - ftab[j]--; - zptr[ftab[j]] = last; - - /*-- - Now ftab contains the first loc of every small bucket. - Calculate the running order, from smallest to largest - big bucket. - --*/ - - for (i = 0; i <= 255; i++) runningOrder[i] = i; - - { - Int32 vv; - Int32 h = 1; - do h = 3 * h + 1; while (h <= 256); - do { - h = h / 3; - for (i = h; i <= 255; i++) { - vv = runningOrder[i]; - j = i; - while ( BIGFREQ(runningOrder[j-h]) > BIGFREQ(vv) ) { - runningOrder[j] = runningOrder[j-h]; - j = j - h; - if (j <= (h - 1)) goto zero; - } - zero: - runningOrder[j] = vv; - } - } while (h != 1); - } - - /*-- - The main sorting loop. - --*/ - - for (i = 0; i <= 255; i++) { - - /*-- - Process big buckets, starting with the least full. - --*/ - ss = runningOrder[i]; - - /*-- - Complete the big bucket [ss] by quicksorting - any unsorted small buckets [ss, j]. Hopefully - previous pointer-scanning phases have already - completed many of the small buckets [ss, j], so - we don't have to sort them at all. - --*/ - for (j = 0; j <= 255; j++) { - sb = (ss << 8) + j; - if ( ! (ftab[sb] & SETMASK) ) { - Int32 lo = ftab[sb] & CLEARMASK; - Int32 hi = (ftab[sb+1] & CLEARMASK) - 1; - if (hi > lo) { - if (verbosity >= 4) - fprintf ( stderr, - " qsort [0x%x, 0x%x] done %d this %d\n", - ss, j, numQSorted, hi - lo + 1 ); - qSort3 ( lo, hi, 2 ); - numQSorted += ( hi - lo + 1 ); - if (workDone > workLimit && firstAttempt) return; - } - ftab[sb] |= SETMASK; - } - } - - /*-- - The ss big bucket is now done. Record this fact, - and update the quadrant descriptors. Remember to - update quadrants in the overshoot area too, if - necessary. The "if (i < 255)" test merely skips - this updating for the last bucket processed, since - updating for the last bucket is pointless. - --*/ - bigDone[ss] = True; - - if (i < 255) { - Int32 bbStart = ftab[ss << 8] & CLEARMASK; - Int32 bbSize = (ftab[(ss+1) << 8] & CLEARMASK) - bbStart; - Int32 shifts = 0; - - while ((bbSize >> shifts) > 65534) shifts++; - - for (j = 0; j < bbSize; j++) { - Int32 a2update = zptr[bbStart + j]; - UInt16 qVal = (UInt16)(j >> shifts); - quadrant[a2update] = qVal; - if (a2update < NUM_OVERSHOOT_BYTES) - quadrant[a2update + last + 1] = qVal; - } - - if (! ( ((bbSize-1) >> shifts) <= 65535 )) panic ( "sortIt" ); - } - - /*-- - Now scan this big bucket so as to synthesise the - sorted order for small buckets [t, ss] for all t != ss. - --*/ - for (j = 0; j <= 255; j++) - copy[j] = ftab[(j << 8) + ss] & CLEARMASK; - - for (j = ftab[ss << 8] & CLEARMASK; - j < (ftab[(ss+1) << 8] & CLEARMASK); - j++) { - c1 = block[zptr[j]-1]; - if ( ! bigDone[c1] ) { - zptr[copy[c1]] = zptr[j] == 0 ? last : zptr[j] - 1; - copy[c1] ++; - } - } - - for (j = 0; j <= 255; j++) ftab[(j << 8) + ss] |= SETMASK; - } - if (verbosity >= 4) - fprintf ( stderr, " %d pointers, %d sorted, %d scanned\n", - last+1, numQSorted, (last+1) - numQSorted ); - } -} - - -/*---------------------------------------------------*/ -/*--- Stuff for randomising repetitive blocks ---*/ -/*---------------------------------------------------*/ - -/*---------------------------------------------*/ -Int32 rNums[512] = { - 619, 720, 127, 481, 931, 816, 813, 233, 566, 247, - 985, 724, 205, 454, 863, 491, 741, 242, 949, 214, - 733, 859, 335, 708, 621, 574, 73, 654, 730, 472, - 419, 436, 278, 496, 867, 210, 399, 680, 480, 51, - 878, 465, 811, 169, 869, 675, 611, 697, 867, 561, - 862, 687, 507, 283, 482, 129, 807, 591, 733, 623, - 150, 238, 59, 379, 684, 877, 625, 169, 643, 105, - 170, 607, 520, 932, 727, 476, 693, 425, 174, 647, - 73, 122, 335, 530, 442, 853, 695, 249, 445, 515, - 909, 545, 703, 919, 874, 474, 882, 500, 594, 612, - 641, 801, 220, 162, 819, 984, 589, 513, 495, 799, - 161, 604, 958, 533, 221, 400, 386, 867, 600, 782, - 382, 596, 414, 171, 516, 375, 682, 485, 911, 276, - 98, 553, 163, 354, 666, 933, 424, 341, 533, 870, - 227, 730, 475, 186, 263, 647, 537, 686, 600, 224, - 469, 68, 770, 919, 190, 373, 294, 822, 808, 206, - 184, 943, 795, 384, 383, 461, 404, 758, 839, 887, - 715, 67, 618, 276, 204, 918, 873, 777, 604, 560, - 951, 160, 578, 722, 79, 804, 96, 409, 713, 940, - 652, 934, 970, 447, 318, 353, 859, 672, 112, 785, - 645, 863, 803, 350, 139, 93, 354, 99, 820, 908, - 609, 772, 154, 274, 580, 184, 79, 626, 630, 742, - 653, 282, 762, 623, 680, 81, 927, 626, 789, 125, - 411, 521, 938, 300, 821, 78, 343, 175, 128, 250, - 170, 774, 972, 275, 999, 639, 495, 78, 352, 126, - 857, 956, 358, 619, 580, 124, 737, 594, 701, 612, - 669, 112, 134, 694, 363, 992, 809, 743, 168, 974, - 944, 375, 748, 52, 600, 747, 642, 182, 862, 81, - 344, 805, 988, 739, 511, 655, 814, 334, 249, 515, - 897, 955, 664, 981, 649, 113, 974, 459, 893, 228, - 433, 837, 553, 268, 926, 240, 102, 654, 459, 51, - 686, 754, 806, 760, 493, 403, 415, 394, 687, 700, - 946, 670, 656, 610, 738, 392, 760, 799, 887, 653, - 978, 321, 576, 617, 626, 502, 894, 679, 243, 440, - 680, 879, 194, 572, 640, 724, 926, 56, 204, 700, - 707, 151, 457, 449, 797, 195, 791, 558, 945, 679, - 297, 59, 87, 824, 713, 663, 412, 693, 342, 606, - 134, 108, 571, 364, 631, 212, 174, 643, 304, 329, - 343, 97, 430, 751, 497, 314, 983, 374, 822, 928, - 140, 206, 73, 263, 980, 736, 876, 478, 430, 305, - 170, 514, 364, 692, 829, 82, 855, 953, 676, 246, - 369, 970, 294, 750, 807, 827, 150, 790, 288, 923, - 804, 378, 215, 828, 592, 281, 565, 555, 710, 82, - 896, 831, 547, 261, 524, 462, 293, 465, 502, 56, - 661, 821, 976, 991, 658, 869, 905, 758, 745, 193, - 768, 550, 608, 933, 378, 286, 215, 979, 792, 961, - 61, 688, 793, 644, 986, 403, 106, 366, 905, 644, - 372, 567, 466, 434, 645, 210, 389, 550, 919, 135, - 780, 773, 635, 389, 707, 100, 626, 958, 165, 504, - 920, 176, 193, 713, 857, 265, 203, 50, 668, 108, - 645, 990, 626, 197, 510, 357, 358, 850, 858, 364, - 936, 638 -}; - - -#define RAND_DECLS \ - Int32 rNToGo = 0; \ - Int32 rTPos = 0; \ - -#define RAND_MASK ((rNToGo == 1) ? 1 : 0) - -#define RAND_UPD_MASK \ - if (rNToGo == 0) { \ - rNToGo = rNums[rTPos]; \ - rTPos++; if (rTPos == 512) rTPos = 0; \ - } \ - rNToGo--; - - - -/*---------------------------------------------------*/ -/*--- The Reversible Transformation (tm) ---*/ -/*---------------------------------------------------*/ - -/*---------------------------------------------*/ -void randomiseBlock ( void ) -{ - Int32 i; - RAND_DECLS; - for (i = 0; i < 256; i++) inUse[i] = False; - - for (i = 0; i <= last; i++) { - RAND_UPD_MASK; - block[i] ^= RAND_MASK; - inUse[block[i]] = True; - } -} - - -/*---------------------------------------------*/ -void doReversibleTransformation ( void ) -{ - Int32 i; - - if (verbosity >= 2) fprintf ( stderr, "\n" ); - - workLimit = workFactor * last; - workDone = 0; - blockRandomised = False; - firstAttempt = True; - - sortIt (); - - if (verbosity >= 3) - fprintf ( stderr, " %d work, %d block, ratio %5.2f\n", - workDone, last, (float)workDone / (float)(last) ); - - if (workDone > workLimit && firstAttempt) { - if (verbosity >= 2) - fprintf ( stderr, " sorting aborted; randomising block\n" ); - randomiseBlock (); - workLimit = workDone = 0; - blockRandomised = True; - firstAttempt = False; - sortIt(); - if (verbosity >= 3) - fprintf ( stderr, " %d work, %d block, ratio %f\n", - workDone, last, (float)workDone / (float)(last) ); - } - - origPtr = -1; - for (i = 0; i <= last; i++) - if (zptr[i] == 0) - { origPtr = i; break; }; - - if (origPtr == -1) panic ( "doReversibleTransformation" ); -} - - -/*---------------------------------------------*/ - -INLINE Int32 indexIntoF ( Int32 indx, Int32 *cftab ) -{ - Int32 nb, na, mid; - nb = 0; - na = 256; - do { - mid = (nb + na) >> 1; - if (indx >= cftab[mid]) nb = mid; else na = mid; - } - while (na - nb != 1); - return nb; -} - - -#define GET_SMALL(cccc) \ - \ - cccc = indexIntoF ( tPos, cftab ); \ - tPos = GET_LL(tPos); - - -void undoReversibleTransformation_small ( FILE* dst ) -{ - Int32 cftab[257], cftabAlso[257]; - Int32 i, j, tmp, tPos; - UChar ch; - - /*-- - We assume here that the global array unzftab will - already be holding the frequency counts for - ll8[0 .. last]. - --*/ - - /*-- Set up cftab to facilitate generation of indexIntoF --*/ - cftab[0] = 0; - for (i = 1; i <= 256; i++) cftab[i] = unzftab[i-1]; - for (i = 1; i <= 256; i++) cftab[i] += cftab[i-1]; - - /*-- Make a copy of it, used in generation of T --*/ - for (i = 0; i <= 256; i++) cftabAlso[i] = cftab[i]; - - /*-- compute the T vector --*/ - for (i = 0; i <= last; i++) { - ch = (UChar)ll16[i]; - SET_LL(i, cftabAlso[ch]); - cftabAlso[ch]++; - } - - /*-- - Compute T^(-1) by pointer reversal on T. This is rather - subtle, in that, if the original block was two or more - (in general, N) concatenated copies of the same thing, - the T vector will consist of N cycles, each of length - blocksize / N, and decoding will involve traversing one - of these cycles N times. Which particular cycle doesn't - matter -- they are all equivalent. The tricky part is to - make sure that the pointer reversal creates a correct - reversed cycle for us to traverse. So, the code below - simply reverses whatever cycle origPtr happens to fall into, - without regard to the cycle length. That gives one reversed - cycle, which for normal blocks, is the entire block-size long. - For repeated blocks, it will be interspersed with the other - N-1 non-reversed cycles. Providing that the F-subscripting - phase which follows starts at origPtr, all then works ok. - --*/ - i = origPtr; - j = GET_LL(i); - do { - tmp = GET_LL(j); - SET_LL(j, i); - i = j; - j = tmp; - } - while (i != origPtr); - - /*-- - We recreate the original by subscripting F through T^(-1). - The run-length-decoder below requires characters incrementally, - so tPos is set to a starting value, and is updated by - the GET_SMALL macro. - --*/ - tPos = origPtr; - - /*-------------------------------------------------*/ - /*-- - This is pretty much a verbatim copy of the - run-length decoder present in the distribution - bzip-0.21; it has to be here to avoid creating - block[] as an intermediary structure. As in 0.21, - this code derives from some sent to me by - Christian von Roques. - - It allows dst==NULL, so as to support the test (-t) - option without slowing down the fast decompression - code. - --*/ - { - IntNative retVal; - Int32 i2, count, chPrev, ch2; - UInt32 localCrc; - - count = 0; - i2 = 0; - ch2 = 256; /*-- not a char and not EOF --*/ - localCrc = getGlobalCRC(); - - { - RAND_DECLS; - while ( i2 <= last ) { - chPrev = ch2; - GET_SMALL(ch2); - if (blockRandomised) { - RAND_UPD_MASK; - ch2 ^= (UInt32)RAND_MASK; - } - i2++; - - if (dst) - retVal = putc ( ch2, dst ); - - UPDATE_CRC ( localCrc, (UChar)ch2 ); - - if (ch2 != chPrev) { - count = 1; - } else { - count++; - if (count >= 4) { - Int32 j2; - UChar z; - GET_SMALL(z); - if (blockRandomised) { - RAND_UPD_MASK; - z ^= RAND_MASK; - } - for (j2 = 0; j2 < (Int32)z; j2++) { - if (dst) retVal = putc (ch2, dst); - UPDATE_CRC ( localCrc, (UChar)ch2 ); - } - i2++; - count = 0; - } - } - } - } +# define SET_BINARY_MODE(fd) /**/ - setGlobalCRC ( localCrc ); - } - /*-- end of the in-line run-length-decoder. --*/ -} -#undef GET_SMALL +# ifdef __GNUC__ +# define NORETURN __attribute__ ((noreturn)) +# else +# define NORETURN /**/ +# endif +#endif -/*---------------------------------------------*/ -#define GET_FAST(cccc) \ - \ - cccc = ll8[tPos]; \ - tPos = tt[tPos]; +#if BZ_LCCWIN32 +# include +# include +# include +# define NORETURN /**/ +# define PATH_SEP '\\' +# define MY_LSTAT _stat +# define MY_STAT _stat +# define MY_S_IFREG(x) ((x) & _S_IFREG) -void undoReversibleTransformation_fast ( FILE* dst ) -{ - Int32 cftab[257]; - Int32 i, tPos; - UChar ch; - - /*-- - We assume here that the global array unzftab will - already be holding the frequency counts for - ll8[0 .. last]. - --*/ - - /*-- Set up cftab to facilitate generation of T^(-1) --*/ - cftab[0] = 0; - for (i = 1; i <= 256; i++) cftab[i] = unzftab[i-1]; - for (i = 1; i <= 256; i++) cftab[i] += cftab[i-1]; - - /*-- compute the T^(-1) vector --*/ - for (i = 0; i <= last; i++) { - ch = (UChar)ll8[i]; - tt[cftab[ch]] = i; - cftab[ch]++; - } +# if 0 + /*-- lcc-win32 seems to expand wildcards itself --*/ +# define APPEND_FILESPEC(root, spec) \ + do { \ + if ((spec)[0] == '-') { \ + root = snocString((root), (spec)); \ + } else { \ + struct _finddata_t c_file; \ + long hFile; \ + hFile = _findfirst((spec), &c_file); \ + if ( hFile == -1L ) { \ + root = snocString ((root), (spec)); \ + } else { \ + int anInt = 0; \ + while ( anInt == 0 ) { \ + root = snocString((root), \ + &c_file.name[0]); \ + anInt = _findnext(hFile, &c_file); \ + } \ + } \ + } \ + } while ( 0 ) +# else +# define APPEND_FILESPEC(root, name) \ + root = snocString ((root), (name)) +# endif - /*-- - We recreate the original by subscripting L through T^(-1). - The run-length-decoder below requires characters incrementally, - so tPos is set to a starting value, and is updated by - the GET_FAST macro. - --*/ - tPos = tt[origPtr]; - - /*-------------------------------------------------*/ - /*-- - This is pretty much a verbatim copy of the - run-length decoder present in the distribution - bzip-0.21; it has to be here to avoid creating - block[] as an intermediary structure. As in 0.21, - this code derives from some sent to me by - Christian von Roques. - --*/ - { - IntNative retVal; - Int32 i2, count, chPrev, ch2; - UInt32 localCrc; - - count = 0; - i2 = 0; - ch2 = 256; /*-- not a char and not EOF --*/ - localCrc = getGlobalCRC(); - - if (blockRandomised) { - RAND_DECLS; - while ( i2 <= last ) { - chPrev = ch2; - GET_FAST(ch2); - RAND_UPD_MASK; - ch2 ^= (UInt32)RAND_MASK; - i2++; - - retVal = putc ( ch2, dst ); - UPDATE_CRC ( localCrc, (UChar)ch2 ); - - if (ch2 != chPrev) { - count = 1; - } else { - count++; - if (count >= 4) { - Int32 j2; - UChar z; - GET_FAST(z); - RAND_UPD_MASK; - z ^= RAND_MASK; - for (j2 = 0; j2 < (Int32)z; j2++) { - retVal = putc (ch2, dst); - UPDATE_CRC ( localCrc, (UChar)ch2 ); - } - i2++; - count = 0; - } - } - } +# define SET_BINARY_MODE(fd) \ + do { \ + int retVal = setmode ( fileno ( fd ), \ + O_BINARY ); \ + ERROR_IF_MINUS_ONE ( retVal ); \ + } while ( 0 ) - } else { - - while ( i2 <= last ) { - chPrev = ch2; - GET_FAST(ch2); - i2++; - - retVal = putc ( ch2, dst ); - UPDATE_CRC ( localCrc, (UChar)ch2 ); - - if (ch2 != chPrev) { - count = 1; - } else { - count++; - if (count >= 4) { - Int32 j2; - UChar z; - GET_FAST(z); - for (j2 = 0; j2 < (Int32)z; j2++) { - retVal = putc (ch2, dst); - UPDATE_CRC ( localCrc, (UChar)ch2 ); - } - i2++; - count = 0; - } - } - } +#endif - } /*-- if (blockRandomised) --*/ - setGlobalCRC ( localCrc ); - } - /*-- end of the in-line run-length-decoder. --*/ -} -#undef GET_FAST +/*---------------------------------------------*/ +/*-- + Some more stuff for all platforms :-) +--*/ + +typedef char Char; +typedef unsigned char Bool; +typedef unsigned char UChar; +typedef int Int32; +typedef unsigned int UInt32; +typedef short Int16; +typedef unsigned short UInt16; + +#define True ((Bool)1) +#define False ((Bool)0) + +/*-- + IntNative is your platform's `native' int size. + Only here to avoid probs with 64-bit platforms. +--*/ +typedef int IntNative; /*---------------------------------------------------*/ -/*--- The block loader and RLEr ---*/ +/*--- Misc (file handling) data decls ---*/ /*---------------------------------------------------*/ -/*---------------------------------------------*/ -/* Top 16: run length, 1 to 255. -* Lower 16: the char, or MY_EOF for EOF. -*/ +Int32 verbosity; +Bool keepInputFiles, smallMode; +Bool forceOverwrite, testFailsExist; +Int32 numFileNames, numFilesProcessed, blockSize100k; -#define MY_EOF 257 -INLINE Int32 getRLEpair ( FILE* src ) -{ - Int32 runLength; - IntNative ch, chLatest; +/*-- source modes; F==file, I==stdin, O==stdout --*/ +#define SM_I2O 1 +#define SM_F2O 2 +#define SM_F2F 3 - ch = getc ( src ); +/*-- operation modes --*/ +#define OM_Z 1 +#define OM_UNZ 2 +#define OM_TEST 3 - /*--- Because I have no idea what kind of a value EOF is. ---*/ - if (ch == EOF) { - ERROR_IF_NOT_ZERO ( ferror(src)); - return (1 << 16) | MY_EOF; - } +Int32 opMode; +Int32 srcMode; - runLength = 0; - do { - chLatest = getc ( src ); - runLength++; - bytesIn++; - } - while (ch == chLatest && runLength < 255); +#define FILE_NAME_LEN 1034 + +Int32 longestFileName; +Char inName[FILE_NAME_LEN]; +Char outName[FILE_NAME_LEN]; +Char *progName; +Char progNameReally[FILE_NAME_LEN]; +FILE *outputHandleJustInCase; +Int32 workFactor; + +void panic ( Char* ) NORETURN; +void ioError ( void ) NORETURN; +void outOfMemory ( void ) NORETURN; +void blockOverrun ( void ) NORETURN; +void badBlockHeader ( void ) NORETURN; +void badBGLengths ( void ) NORETURN; +void crcError ( void ) NORETURN; +void bitStreamEOF ( void ) NORETURN; +void cleanUpAndFail ( Int32 ) NORETURN; +void compressedStreamEOF ( void ) NORETURN; + +void copyFileName ( Char*, Char* ); +void* myMalloc ( Int32 ); - if ( chLatest != EOF ) { - if ( ungetc ( chLatest, src ) == EOF ) - panic ( "getRLEpair: ungetc failed" ); - } else { - ERROR_IF_NOT_ZERO ( ferror(src) ); - } - /*--- Conditional is just a speedup hack. ---*/ - if (runLength == 1) { - UPDATE_CRC ( globalCrc, (UChar)ch ); - return (1 << 16) | ch; - } else { - Int32 i; - for (i = 1; i <= runLength; i++) - UPDATE_CRC ( globalCrc, (UChar)ch ); - return (runLength << 16) | ch; - } -} +/*---------------------------------------------------*/ +/*--- Processing of complete files and streams ---*/ +/*---------------------------------------------------*/ /*---------------------------------------------*/ -void loadAndRLEsource ( FILE* src ) +Bool myfeof ( FILE* f ) { - Int32 ch, allowableBlockSize, i; - - last = -1; - ch = 0; - - for (i = 0; i < 256; i++) inUse[i] = False; - - /*--- 20 is just a paranoia constant ---*/ - allowableBlockSize = 100000 * blockSize100k - 20; - - while (last < allowableBlockSize && ch != MY_EOF) { - Int32 rlePair, runLen; - rlePair = getRLEpair ( src ); - ch = rlePair & 0xFFFF; - runLen = (UInt32)rlePair >> 16; - - #if DEBUG - assert (runLen >= 1 && runLen <= 255); - #endif - - if (ch != MY_EOF) { - inUse[ch] = True; - switch (runLen) { - case 1: - last++; block[last] = (UChar)ch; break; - case 2: - last++; block[last] = (UChar)ch; - last++; block[last] = (UChar)ch; break; - case 3: - last++; block[last] = (UChar)ch; - last++; block[last] = (UChar)ch; - last++; block[last] = (UChar)ch; break; - default: - inUse[runLen-4] = True; - last++; block[last] = (UChar)ch; - last++; block[last] = (UChar)ch; - last++; block[last] = (UChar)ch; - last++; block[last] = (UChar)ch; - last++; block[last] = (UChar)(runLen-4); break; - } - } - } + Int32 c = fgetc ( f ); + if (c == EOF) return True; + ungetc ( c, f ); + return False; } -/*---------------------------------------------------*/ -/*--- Processing of complete files and streams ---*/ -/*---------------------------------------------------*/ - /*---------------------------------------------*/ void compressStream ( FILE *stream, FILE *zStream ) { - IntNative retVal; - UInt32 blockCRC, combinedCRC; - Int32 blockNo; - - blockNo = 0; - bytesIn = 0; - bytesOut = 0; - nBlocksRandomised = 0; + BZFILE* bzf = NULL; + UChar ibuf[5000]; + Int32 nIbuf; + UInt32 nbytes_in, nbytes_out; + Int32 bzerr, bzerr_dummy, ret; SET_BINARY_MODE(stream); SET_BINARY_MODE(zStream); - ERROR_IF_NOT_ZERO ( ferror(stream) ); - ERROR_IF_NOT_ZERO ( ferror(zStream) ); + if (ferror(stream)) goto errhandler_io; + if (ferror(zStream)) goto errhandler_io; - bsSetStream ( zStream, True ); - - /*--- Write `magic' bytes B and Z, - then h indicating file-format == huffmanised, - followed by a digit indicating blockSize100k. - ---*/ - bsPutUChar ( 'B' ); - bsPutUChar ( 'Z' ); - bsPutUChar ( 'h' ); - bsPutUChar ( '0' + blockSize100k ); - - combinedCRC = 0; + bzf = bzWriteOpen ( &bzerr, zStream, + blockSize100k, verbosity, workFactor ); + if (bzerr != BZ_OK) goto errhandler; if (verbosity >= 2) fprintf ( stderr, "\n" ); while (True) { - blockNo++; - initialiseCRC (); - loadAndRLEsource ( stream ); - ERROR_IF_NOT_ZERO ( ferror(stream) ); - if (last == -1) break; - - blockCRC = getFinalCRC (); - combinedCRC = (combinedCRC << 1) | (combinedCRC >> 31); - combinedCRC ^= blockCRC; - - if (verbosity >= 2) - fprintf ( stderr, " block %d: crc = 0x%8x, combined CRC = 0x%8x, size = %d", - blockNo, blockCRC, combinedCRC, last+1 ); - - /*-- sort the block and establish posn of original string --*/ - doReversibleTransformation (); - - /*-- - A 6-byte block header, the value chosen arbitrarily - as 0x314159265359 :-). A 32 bit value does not really - give a strong enough guarantee that the value will not - appear by chance in the compressed datastream. Worst-case - probability of this event, for a 900k block, is about - 2.0e-3 for 32 bits, 1.0e-5 for 40 bits and 4.0e-8 for 48 bits. - For a compressed file of size 100Gb -- about 100000 blocks -- - only a 48-bit marker will do. NB: normal compression/ - decompression do *not* rely on these statistical properties. - They are only important when trying to recover blocks from - damaged files. - --*/ - bsPutUChar ( 0x31 ); bsPutUChar ( 0x41 ); - bsPutUChar ( 0x59 ); bsPutUChar ( 0x26 ); - bsPutUChar ( 0x53 ); bsPutUChar ( 0x59 ); - - /*-- Now the block's CRC, so it is in a known place. --*/ - bsPutUInt32 ( blockCRC ); - - /*-- Now a single bit indicating randomisation. --*/ - if (blockRandomised) { - bsW(1,1); nBlocksRandomised++; - } else - bsW(1,0); - - /*-- Finally, block's contents proper. --*/ - moveToFrontCodeAndSend (); - - ERROR_IF_NOT_ZERO ( ferror(zStream) ); - } - - if (verbosity >= 2 && nBlocksRandomised > 0) - fprintf ( stderr, " %d block%s needed randomisation\n", - nBlocksRandomised, - nBlocksRandomised == 1 ? "" : "s" ); + if (myfeof(stream)) break; + nIbuf = fread ( ibuf, sizeof(UChar), 5000, stream ); + if (ferror(stream)) goto errhandler_io; + if (nIbuf > 0) bzWrite ( &bzerr, bzf, (void*)ibuf, nIbuf ); + if (bzerr != BZ_OK) goto errhandler; - /*-- - Now another magic 48-bit number, 0x177245385090, to - indicate the end of the last block. (sqrt(pi), if - you want to know. I did want to use e, but it contains - too much repetition -- 27 18 28 18 28 46 -- for me - to feel statistically comfortable. Call me paranoid.) - --*/ - - bsPutUChar ( 0x17 ); bsPutUChar ( 0x72 ); - bsPutUChar ( 0x45 ); bsPutUChar ( 0x38 ); - bsPutUChar ( 0x50 ); bsPutUChar ( 0x90 ); - - bsPutUInt32 ( combinedCRC ); - if (verbosity >= 2) - fprintf ( stderr, " final combined CRC = 0x%x\n ", combinedCRC ); - - /*-- Close the files in an utterly paranoid way. --*/ - bsFinishedWithStream (); + } - ERROR_IF_NOT_ZERO ( ferror(zStream) ); - retVal = fflush ( zStream ); - ERROR_IF_EOF ( retVal ); - retVal = fclose ( zStream ); - ERROR_IF_EOF ( retVal ); + bzWriteClose ( &bzerr, bzf, 0, &nbytes_in, &nbytes_out ); + if (bzerr != BZ_OK) goto errhandler; - ERROR_IF_NOT_ZERO ( ferror(stream) ); - retVal = fclose ( stream ); - ERROR_IF_EOF ( retVal ); + if (ferror(zStream)) goto errhandler_io; + ret = fflush ( zStream ); + if (ret == EOF) goto errhandler_io; + if (zStream != stdout) { + ret = fclose ( zStream ); + if (ret == EOF) goto errhandler_io; + } + if (ferror(stream)) goto errhandler_io; + ret = fclose ( stream ); + if (ret == EOF) goto errhandler_io; - if (bytesIn == 0) bytesIn = 1; - if (bytesOut == 0) bytesOut = 1; + if (nbytes_in == 0) nbytes_in = 1; if (verbosity >= 1) fprintf ( stderr, "%6.3f:1, %6.3f bits/byte, " "%5.2f%% saved, %d in, %d out.\n", - (float)bytesIn / (float)bytesOut, - (8.0 * (float)bytesOut) / (float)bytesIn, - 100.0 * (1.0 - (float)bytesOut / (float)bytesIn), - bytesIn, - bytesOut + (float)nbytes_in / (float)nbytes_out, + (8.0 * (float)nbytes_out) / (float)nbytes_in, + 100.0 * (1.0 - (float)nbytes_out / (float)nbytes_in), + nbytes_in, + nbytes_out ); + + return; + + errhandler: + bzWriteClose ( &bzerr_dummy, bzf, 1, &nbytes_in, &nbytes_out ); + switch (bzerr) { + case BZ_MEM_ERROR: + outOfMemory (); + case BZ_IO_ERROR: + errhandler_io: + ioError(); break; + default: + panic ( "compress:unexpected error" ); + } + + panic ( "compress:end" ); + /*notreached*/ } + /*---------------------------------------------*/ Bool uncompressStream ( FILE *zStream, FILE *stream ) { - UChar magic1, magic2, magic3, magic4; - UChar magic5, magic6; - UInt32 storedBlockCRC, storedCombinedCRC; - UInt32 computedBlockCRC, computedCombinedCRC; - Int32 currBlockNo; - IntNative retVal; + BZFILE* bzf = NULL; + Int32 bzerr, bzerr_dummy, ret, nread, streamNo, i; + UChar obuf[5000]; + UChar unused[BZ_MAX_UNUSED]; + Int32 nUnused; + UChar* unusedTmp; + + nUnused = 0; + streamNo = 0; SET_BINARY_MODE(stream); SET_BINARY_MODE(zStream); - ERROR_IF_NOT_ZERO ( ferror(stream) ); - ERROR_IF_NOT_ZERO ( ferror(zStream) ); - - bsSetStream ( zStream, False ); - - /*-- - A bad magic number is `recoverable from'; - return with False so the caller skips the file. - --*/ - magic1 = bsGetUChar (); - magic2 = bsGetUChar (); - magic3 = bsGetUChar (); - magic4 = bsGetUChar (); - if (magic1 != 'B' || - magic2 != 'Z' || - magic3 != 'h' || - magic4 < '1' || - magic4 > '9') { - bsFinishedWithStream(); - retVal = fclose ( stream ); - ERROR_IF_EOF ( retVal ); - return False; - } - - setDecompressStructureSizes ( magic4 - '0' ); - computedCombinedCRC = 0; - - if (verbosity >= 2) fprintf ( stderr, "\n " ); - currBlockNo = 0; + if (ferror(stream)) goto errhandler_io; + if (ferror(zStream)) goto errhandler_io; while (True) { - magic1 = bsGetUChar (); - magic2 = bsGetUChar (); - magic3 = bsGetUChar (); - magic4 = bsGetUChar (); - magic5 = bsGetUChar (); - magic6 = bsGetUChar (); - if (magic1 == 0x17 && magic2 == 0x72 && - magic3 == 0x45 && magic4 == 0x38 && - magic5 == 0x50 && magic6 == 0x90) break; - - if (magic1 != 0x31 || magic2 != 0x41 || - magic3 != 0x59 || magic4 != 0x26 || - magic5 != 0x53 || magic6 != 0x59) badBlockHeader(); - - storedBlockCRC = bsGetUInt32 (); - - if (bsR(1) == 1) - blockRandomised = True; else - blockRandomised = False; - - currBlockNo++; - if (verbosity >= 2) - fprintf ( stderr, "[%d: huff+mtf ", currBlockNo ); - getAndMoveToFrontDecode (); - ERROR_IF_NOT_ZERO ( ferror(zStream) ); - - initialiseCRC(); - if (verbosity >= 2) fprintf ( stderr, "rt+rld" ); - if (smallMode) - undoReversibleTransformation_small ( stream ); - else - undoReversibleTransformation_fast ( stream ); - - ERROR_IF_NOT_ZERO ( ferror(stream) ); - computedBlockCRC = getFinalCRC(); - if (verbosity >= 3) - fprintf ( stderr, " {0x%x, 0x%x}", storedBlockCRC, computedBlockCRC ); - if (verbosity >= 2) fprintf ( stderr, "] " ); + bzf = bzReadOpen ( + &bzerr, zStream, verbosity, + (int)smallMode, unused, nUnused + ); + if (bzf == NULL || bzerr != BZ_OK) goto errhandler; + streamNo++; + + while (bzerr == BZ_OK) { + nread = bzRead ( &bzerr, bzf, obuf, 5000 ); + if (bzerr == BZ_DATA_ERROR_MAGIC) goto errhandler; + if ((bzerr == BZ_OK || bzerr == BZ_STREAM_END) && nread > 0) + fwrite ( obuf, sizeof(UChar), nread, stream ); + if (ferror(stream)) goto errhandler_io; + } + if (bzerr != BZ_STREAM_END) goto errhandler; - /*-- A bad CRC is considered a fatal error. --*/ - if (storedBlockCRC != computedBlockCRC) - crcError ( storedBlockCRC, computedBlockCRC ); + bzReadGetUnused ( &bzerr, bzf, (void**)(&unusedTmp), &nUnused ); + if (bzerr != BZ_OK) panic ( "decompress:bzReadGetUnused" ); - computedCombinedCRC = (computedCombinedCRC << 1) | (computedCombinedCRC >> 31); - computedCombinedCRC ^= computedBlockCRC; - }; + for (i = 0; i < nUnused; i++) unused[i] = unusedTmp[i]; - if (verbosity >= 2) fprintf ( stderr, "\n " ); + bzReadClose ( &bzerr, bzf ); + if (bzerr != BZ_OK) panic ( "decompress:bzReadGetUnused" ); - storedCombinedCRC = bsGetUInt32 (); - if (verbosity >= 2) - fprintf ( stderr, - "combined CRCs: stored = 0x%x, computed = 0x%x\n ", - storedCombinedCRC, computedCombinedCRC ); - if (storedCombinedCRC != computedCombinedCRC) - crcError ( storedCombinedCRC, computedCombinedCRC ); + if (nUnused == 0 && myfeof(zStream)) break; + } - bsFinishedWithStream (); - ERROR_IF_NOT_ZERO ( ferror(zStream) ); - retVal = fclose ( zStream ); - ERROR_IF_EOF ( retVal ); + if (ferror(zStream)) goto errhandler_io; + ret = fclose ( zStream ); + if (ret == EOF) goto errhandler_io; - ERROR_IF_NOT_ZERO ( ferror(stream) ); - retVal = fflush ( stream ); - ERROR_IF_NOT_ZERO ( retVal ); + if (ferror(stream)) goto errhandler_io; + ret = fflush ( stream ); + if (ret != 0) goto errhandler_io; if (stream != stdout) { - retVal = fclose ( stream ); - ERROR_IF_EOF ( retVal ); + ret = fclose ( stream ); + if (ret == EOF) goto errhandler_io; } + if (verbosity >= 2) fprintf ( stderr, "\n " ); return True; + + errhandler: + bzReadClose ( &bzerr_dummy, bzf ); + switch (bzerr) { + case BZ_IO_ERROR: + errhandler_io: + ioError(); break; + case BZ_DATA_ERROR: + crcError(); + case BZ_MEM_ERROR: + outOfMemory(); + case BZ_UNEXPECTED_EOF: + compressedStreamEOF(); + case BZ_DATA_ERROR_MAGIC: + if (streamNo == 1) { + return False; + } else { + fprintf ( stderr, + "\n%s: %s: trailing garbage after EOF ignored\n", + progName, inName ); + return True; + } + default: + panic ( "decompress:unexpected error" ); + } + + panic ( "decompress:end" ); + return True; /*notreached*/ } /*---------------------------------------------*/ Bool testStream ( FILE *zStream ) { - UChar magic1, magic2, magic3, magic4; - UChar magic5, magic6; - UInt32 storedBlockCRC, storedCombinedCRC; - UInt32 computedBlockCRC, computedCombinedCRC; - Int32 currBlockNo; - IntNative retVal; + BZFILE* bzf = NULL; + Int32 bzerr, bzerr_dummy, ret, nread, streamNo, i; + UChar obuf[5000]; + UChar unused[BZ_MAX_UNUSED]; + Int32 nUnused; + UChar* unusedTmp; + + nUnused = 0; + streamNo = 0; SET_BINARY_MODE(zStream); - ERROR_IF_NOT_ZERO ( ferror(zStream) ); - - bsSetStream ( zStream, False ); - - magic1 = bsGetUChar (); - magic2 = bsGetUChar (); - magic3 = bsGetUChar (); - magic4 = bsGetUChar (); - if (magic1 != 'B' || - magic2 != 'Z' || - magic3 != 'h' || - magic4 < '1' || - magic4 > '9') { - bsFinishedWithStream(); - fclose ( zStream ); - fprintf ( stderr, "\n%s: bad magic number (ie, not created by bzip2)\n", - inName ); - return False; - } + if (ferror(zStream)) goto errhandler_io; - smallMode = True; - setDecompressStructureSizes ( magic4 - '0' ); - computedCombinedCRC = 0; + while (True) { - if (verbosity >= 2) fprintf ( stderr, "\n" ); - currBlockNo = 0; + bzf = bzReadOpen ( + &bzerr, zStream, verbosity, + (int)smallMode, unused, nUnused + ); + if (bzf == NULL || bzerr != BZ_OK) goto errhandler; + streamNo++; - while (True) { - magic1 = bsGetUChar (); - magic2 = bsGetUChar (); - magic3 = bsGetUChar (); - magic4 = bsGetUChar (); - magic5 = bsGetUChar (); - magic6 = bsGetUChar (); - if (magic1 == 0x17 && magic2 == 0x72 && - magic3 == 0x45 && magic4 == 0x38 && - magic5 == 0x50 && magic6 == 0x90) break; - - currBlockNo++; - if (magic1 != 0x31 || magic2 != 0x41 || - magic3 != 0x59 || magic4 != 0x26 || - magic5 != 0x53 || magic6 != 0x59) { - bsFinishedWithStream(); - fclose ( zStream ); - fprintf ( stderr, - "\n%s, block %d: bad header (not == 0x314159265359)\n", - inName, currBlockNo ); - return False; - } - storedBlockCRC = bsGetUInt32 (); - - if (bsR(1) == 1) - blockRandomised = True; else - blockRandomised = False; - - if (verbosity >= 2) - fprintf ( stderr, " block [%d: huff+mtf ", currBlockNo ); - getAndMoveToFrontDecode (); - ERROR_IF_NOT_ZERO ( ferror(zStream) ); - - initialiseCRC(); - if (verbosity >= 2) fprintf ( stderr, "rt+rld" ); - undoReversibleTransformation_small ( NULL ); - - computedBlockCRC = getFinalCRC(); - if (verbosity >= 3) - fprintf ( stderr, " {0x%x, 0x%x}", storedBlockCRC, computedBlockCRC ); - if (verbosity >= 2) fprintf ( stderr, "] " ); - - if (storedBlockCRC != computedBlockCRC) { - bsFinishedWithStream(); - fclose ( zStream ); - fprintf ( stderr, "\n%s, block %d: computed CRC does not match stored one\n", - inName, currBlockNo ); - return False; + while (bzerr == BZ_OK) { + nread = bzRead ( &bzerr, bzf, obuf, 5000 ); + if (bzerr == BZ_DATA_ERROR_MAGIC) goto errhandler; } + if (bzerr != BZ_STREAM_END) goto errhandler; - if (verbosity >= 2) fprintf ( stderr, "ok\n" ); - computedCombinedCRC = (computedCombinedCRC << 1) | (computedCombinedCRC >> 31); - computedCombinedCRC ^= computedBlockCRC; - }; + bzReadGetUnused ( &bzerr, bzf, (void**)(&unusedTmp), &nUnused ); + if (bzerr != BZ_OK) panic ( "test:bzReadGetUnused" ); + + for (i = 0; i < nUnused; i++) unused[i] = unusedTmp[i]; + + bzReadClose ( &bzerr, bzf ); + if (bzerr != BZ_OK) panic ( "test:bzReadGetUnused" ); + if (nUnused == 0 && myfeof(zStream)) break; - storedCombinedCRC = bsGetUInt32 (); - if (verbosity >= 2) - fprintf ( stderr, - " combined CRCs: stored = 0x%x, computed = 0x%x\n ", - storedCombinedCRC, computedCombinedCRC ); - if (storedCombinedCRC != computedCombinedCRC) { - bsFinishedWithStream(); - fclose ( zStream ); - fprintf ( stderr, "\n%s: computed CRC does not match stored one\n", - inName ); - return False; } - bsFinishedWithStream (); - ERROR_IF_NOT_ZERO ( ferror(zStream) ); - retVal = fclose ( zStream ); - ERROR_IF_EOF ( retVal ); + if (ferror(zStream)) goto errhandler_io; + ret = fclose ( zStream ); + if (ret == EOF) goto errhandler_io; + + if (verbosity >= 2) fprintf ( stderr, "\n " ); return True; -} + errhandler: + bzReadClose ( &bzerr_dummy, bzf ); + switch (bzerr) { + case BZ_IO_ERROR: + errhandler_io: + ioError(); break; + case BZ_DATA_ERROR: + fprintf ( stderr, + "\n%s: data integrity (CRC) error in data\n", + inName ); + return False; + case BZ_MEM_ERROR: + outOfMemory(); + case BZ_UNEXPECTED_EOF: + fprintf ( stderr, + "\n%s: file ends unexpectedly\n", + inName ); + return False; + case BZ_DATA_ERROR_MAGIC: + if (streamNo == 1) { + fprintf ( stderr, + "\n%s: bad magic number (ie, not created by bzip2)\n", + inName ); + return False; + } else { + fprintf ( stderr, + "\n%s: %s: trailing garbage after EOF ignored\n", + progName, inName ); + return True; + } + default: + panic ( "test:unexpected error" ); + } + + panic ( "test:end" ); + return True; /*notreached*/ +} /*---------------------------------------------------*/ @@ -3059,8 +592,7 @@ void showFileNames ( void ) fprintf ( stderr, "\tInput file = %s, output file = %s\n", - inName==NULL ? "(null)" : inName, - outName==NULL ? "(null)" : outName + inName, outName ); } @@ -3072,8 +604,7 @@ void cleanUpAndFail ( Int32 ec ) if ( srcMode == SM_F2F && opMode != OM_TEST ) { fprintf ( stderr, "%s: Deleting output file %s, if it exists.\n", - progName, - outName==NULL ? "(null)" : outName ); + progName, outName ); if (outputHandleJustInCase != NULL) fclose ( outputHandleJustInCase ); retVal = remove ( outName ); @@ -3108,11 +639,10 @@ void panic ( Char* s ) /*---------------------------------------------*/ -void badBGLengths ( void ) +void crcError () { fprintf ( stderr, - "\n%s: error when reading background model code lengths,\n" - "\twhich probably means the compressed file is corrupted.\n", + "\n%s: Data integrity error when decompressing.\n", progName ); showFileNames(); cadvise(); @@ -3120,19 +650,6 @@ void badBGLengths ( void ) } -/*---------------------------------------------*/ -void crcError ( UInt32 crcStored, UInt32 crcComputed ) -{ - fprintf ( stderr, - "\n%s: Data integrity error when decompressing.\n" - "\tStored CRC = 0x%x, computed CRC = 0x%x\n", - progName, crcStored, crcComputed ); - showFileNames(); - cadvise(); - cleanUpAndFail( 2 ); -} - - /*---------------------------------------------*/ void compressedStreamEOF ( void ) { @@ -3159,46 +676,6 @@ void ioError ( ) } -/*---------------------------------------------*/ -void blockOverrun () -{ - fprintf ( stderr, - "\n%s: block overrun during decompression,\n" - "\twhich probably means the compressed file\n" - "\tis corrupted.\n", - progName ); - showFileNames(); - cadvise(); - cleanUpAndFail( 2 ); -} - - -/*---------------------------------------------*/ -void badBlockHeader () -{ - fprintf ( stderr, - "\n%s: bad block header in the compressed file,\n" - "\twhich probably means it is corrupted.\n", - progName ); - showFileNames(); - cadvise(); - cleanUpAndFail( 2 ); -} - - -/*---------------------------------------------*/ -void bitStreamEOF () -{ - fprintf ( stderr, - "\n%s: read past the end of compressed data,\n" - "\twhich probably means it is corrupted.\n", - progName ); - showFileNames(); - cadvise(); - cleanUpAndFail( 2 ); -} - - /*---------------------------------------------*/ void mySignalCatcher ( IntNative n ) { @@ -3233,27 +710,11 @@ void mySIGSEGVorSIGBUScatcher ( IntNative n ) /*---------------------------------------------*/ -void uncompressOutOfMemory ( Int32 draw, Int32 blockSize ) -{ - fprintf ( stderr, - "\n%s: Can't allocate enough memory for decompression.\n" - "\tRequested %d bytes for a block size of %d.\n" - "\tTry selecting space-economic decompress (with flag -s)\n" - "\tand failing that, find a machine with more memory.\n", - progName, draw, blockSize ); - showFileNames(); - cleanUpAndFail(1); -} - - -/*---------------------------------------------*/ -void compressOutOfMemory ( Int32 draw, Int32 blockSize ) +void outOfMemory ( void ) { fprintf ( stderr, - "\n%s: Can't allocate enough memory for compression.\n" - "\tRequested %d bytes for a block size of %d.\n" - "\tTry selecting a small block size (with flag -s).\n", - progName, draw, blockSize ); + "\n%s: couldn't allocate enough memory\n", + progName ); showFileNames(); cleanUpAndFail(1); } @@ -3273,6 +734,24 @@ void pad ( Char *s ) } +/*---------------------------------------------*/ +void copyFileName ( Char* to, Char* from ) +{ + if ( strlen(from) > FILE_NAME_LEN-10 ) { + fprintf ( + stderr, + "bzip2: file name\n`%s'\nis suspiciously (> 1024 chars) long.\n" + "Try using a reasonable file name instead. Sorry! :)\n", + from + ); + exit(1); + } + + strncpy(to,from,FILE_NAME_LEN-10); + to[FILE_NAME_LEN-10]='\0'; +} + + /*---------------------------------------------*/ Bool fileExists ( Char* name ) { @@ -3287,7 +766,7 @@ Bool fileExists ( Char* name ) /*-- if in doubt, return True --*/ -Bool notABogStandardFile ( Char* name ) +Bool notAStandardFile ( Char* name ) { IntNative i; struct MY_STAT statBuf; @@ -3300,9 +779,9 @@ Bool notABogStandardFile ( Char* name ) /*---------------------------------------------*/ -void copyDateAndPermissions ( Char *srcName, Char *dstName ) +void copyDatePermissionsAndOwner ( Char *srcName, Char *dstName ) { - #if BZ_UNIX +#if BZ_UNIX IntNative retVal; struct MY_STAT statBuf; struct utimbuf uTimBuf; @@ -3314,12 +793,33 @@ void copyDateAndPermissions ( Char *srcName, Char *dstName ) retVal = chmod ( dstName, statBuf.st_mode ); ERROR_IF_NOT_ZERO ( retVal ); + /* Not sure if this is really portable or not. Causes + problems on my x86-Linux Redhat 5.0 box. Decided + to omit it from 0.9.0. JRS, 27 June 98. If you + understand Unix file semantics and portability issues + well enough to fix this properly, drop me a line + at jseward@acm.org. + retVal = chown ( dstName, statBuf.st_uid, statBuf.st_gid ); + ERROR_IF_NOT_ZERO ( retVal ); + */ retVal = utime ( dstName, &uTimBuf ); ERROR_IF_NOT_ZERO ( retVal ); - #endif +#endif +} + + +/*---------------------------------------------*/ +void setInterimPermissions ( Char *dstName ) +{ +#if BZ_UNIX + IntNative retVal; + retVal = chmod ( dstName, S_IRUSR | S_IWUSR ); + ERROR_IF_NOT_ZERO ( retVal ); +#endif } + /*---------------------------------------------*/ Bool endsInBz2 ( Char* name ) { @@ -3353,13 +853,13 @@ void compress ( Char *name ) panic ( "compress: bad modes\n" ); switch (srcMode) { - case SM_I2O: strcpy ( inName, "(stdin)" ); - strcpy ( outName, "(stdout)" ); break; - case SM_F2F: strcpy ( inName, name ); - strcpy ( outName, name ); + case SM_I2O: copyFileName ( inName, "(stdin)" ); + copyFileName ( outName, "(stdout)" ); break; + case SM_F2F: copyFileName ( inName, name ); + copyFileName ( outName, name ); strcat ( outName, ".bz2" ); break; - case SM_F2O: strcpy ( inName, name ); - strcpy ( outName, "(stdout)" ); break; + case SM_F2O: copyFileName ( inName, name ); + copyFileName ( outName, "(stdout)" ); break; } if ( srcMode != SM_I2O && containsDubiousChars ( inName ) ) { @@ -3377,12 +877,12 @@ void compress ( Char *name ) progName, inName ); return; } - if ( srcMode != SM_I2O && notABogStandardFile ( inName )) { + if ( srcMode != SM_I2O && notAStandardFile ( inName )) { fprintf ( stderr, "%s: Input file %s is not a normal file, skipping.\n", progName, inName ); return; } - if ( srcMode == SM_F2F && fileExists ( outName ) ) { + if ( srcMode == SM_F2F && !forceOverwrite && fileExists ( outName ) ) { fprintf ( stderr, "%s: Output file %s already exists, skipping.\n", progName, outName ); return; @@ -3434,6 +934,7 @@ void compress ( Char *name ) progName, inName ); return; }; + setInterimPermissions ( outName ); break; default: @@ -3454,7 +955,7 @@ void compress ( Char *name ) /*--- If there was an I/O error, we won't get here. ---*/ if ( srcMode == SM_F2F ) { - copyDateAndPermissions ( inName, outName ); + copyDatePermissionsAndOwner ( inName, outName ); if ( !keepInputFiles ) { IntNative retVal = remove ( inName ); ERROR_IF_NOT_ZERO ( retVal ); @@ -3474,15 +975,15 @@ void uncompress ( Char *name ) panic ( "uncompress: bad modes\n" ); switch (srcMode) { - case SM_I2O: strcpy ( inName, "(stdin)" ); - strcpy ( outName, "(stdout)" ); break; - case SM_F2F: strcpy ( inName, name ); - strcpy ( outName, name ); + case SM_I2O: copyFileName ( inName, "(stdin)" ); + copyFileName ( outName, "(stdout)" ); break; + case SM_F2F: copyFileName ( inName, name ); + copyFileName ( outName, name ); if (endsInBz2 ( outName )) outName [ strlen ( outName ) - 4 ] = '\0'; break; - case SM_F2O: strcpy ( inName, name ); - strcpy ( outName, "(stdout)" ); break; + case SM_F2O: copyFileName ( inName, name ); + copyFileName ( outName, "(stdout)" ); break; } if ( srcMode != SM_I2O && containsDubiousChars ( inName ) ) { @@ -3501,12 +1002,12 @@ void uncompress ( Char *name ) progName, inName ); return; } - if ( srcMode != SM_I2O && notABogStandardFile ( inName )) { + if ( srcMode != SM_I2O && notAStandardFile ( inName )) { fprintf ( stderr, "%s: Input file %s is not a normal file, skipping.\n", progName, inName ); return; } - if ( srcMode == SM_F2F && fileExists ( outName ) ) { + if ( srcMode == SM_F2F && !forceOverwrite && fileExists ( outName ) ) { fprintf ( stderr, "%s: Output file %s already exists, skipping.\n", progName, outName ); return; @@ -3550,6 +1051,7 @@ void uncompress ( Char *name ) progName, inName ); return; }; + setInterimPermissions ( outName ); break; default: @@ -3571,7 +1073,7 @@ void uncompress ( Char *name ) /*--- If there was an I/O error, we won't get here. ---*/ if ( magicNumberOK ) { if ( srcMode == SM_F2F ) { - copyDateAndPermissions ( inName, outName ); + copyDatePermissionsAndOwner ( inName, outName ); if ( !keepInputFiles ) { IntNative retVal = remove ( inName ); ERROR_IF_NOT_ZERO ( retVal ); @@ -3607,11 +1109,11 @@ void testf ( Char *name ) if (name == NULL && srcMode != SM_I2O) panic ( "testf: bad modes\n" ); - strcpy ( outName, "(none)" ); + copyFileName ( outName, "(none)" ); switch (srcMode) { - case SM_I2O: strcpy ( inName, "(stdin)" ); break; - case SM_F2F: strcpy ( inName, name ); break; - case SM_F2O: strcpy ( inName, name ); break; + case SM_I2O: copyFileName ( inName, "(stdin)" ); break; + case SM_F2F: copyFileName ( inName, name ); break; + case SM_F2O: copyFileName ( inName, name ); break; } if ( srcMode != SM_I2O && containsDubiousChars ( inName ) ) { @@ -3630,7 +1132,7 @@ void testf ( Char *name ) progName, inName ); return; } - if ( srcMode != SM_I2O && notABogStandardFile ( inName )) { + if ( srcMode != SM_I2O && notAStandardFile ( inName )) { fprintf ( stderr, "%s: Input file %s is not a normal file, skipping.\n", progName, inName ); return; @@ -3684,25 +1186,18 @@ void license ( void ) fprintf ( stderr, "bzip2, a block-sorting file compressor. " - "Version 0.1pl2, 29-Aug-97.\n" + "Version 0.9.0c, 18-Oct-98.\n" " \n" - " Copyright (C) 1996, 1997 by Julian Seward.\n" + " Copyright (C) 1996, 1997, 1998 by Julian Seward.\n" " \n" " This program is free software; you can redistribute it and/or modify\n" - " it under the terms of the GNU General Public License as published by\n" - " the Free Software Foundation; either version 2 of the License, or\n" - " (at your option) any later version.\n" + " it under the terms set out in the LICENSE file, which is included\n" + " in the bzip2-0.9.0c source distribution.\n" " \n" " This program is distributed in the hope that it will be useful,\n" " but WITHOUT ANY WARRANTY; without even the implied warranty of\n" " MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\n" - " GNU General Public License for more details.\n" - " \n" - " You should have received a copy of the GNU General Public License\n" - " along with this program; if not, write to the Free Software\n" - " Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.\n" - " \n" - " The GNU General Public License is contained in the file LICENSE.\n" + " LICENSE file for more details.\n" " \n" ); } @@ -3714,16 +1209,17 @@ void usage ( Char *fullProgName ) fprintf ( stderr, "bzip2, a block-sorting file compressor. " - "Version 0.1pl2, 29-Aug-97.\n" + "Version 0.9.0c, 18-Oct-98.\n" "\n usage: %s [flags and input files in any order]\n" "\n" " -h --help print this message\n" " -d --decompress force decompression\n" - " -f --compress force compression\n" + " -z --compress force compression\n" + " -k --keep keep (don't delete) input files\n" + " -f --force overwrite existing output filess\n" " -t --test test compressed file integrity\n" " -c --stdout output to standard out\n" " -v --verbose be verbose (a 2nd -v gives more)\n" - " -k --keep keep (don't delete) input files\n" " -L --license display software version & license\n" " -V --version display software version & license\n" " -s --small use less memory (at most 2500k)\n" @@ -3731,15 +1227,16 @@ void usage ( Char *fullProgName ) " --repetitive-fast compress repetitive blocks faster\n" " --repetitive-best compress repetitive blocks better\n" "\n" - " If invoked as `bzip2', the default action is to compress.\n" - " as `bunzip2', the default action is to decompress.\n" + " If invoked as `bzip2', default action is to compress.\n" + " as `bunzip2', default action is to decompress.\n" + " as `bz2cat', default action is to decompress to stdout.\n" "\n" " If no file names are given, bzip2 compresses or decompresses\n" " from standard input to standard output. You can combine\n" - " flags, so `-v -4' means the same as -v4 or -4v, &c.\n" - #if BZ_UNIX + " short flags, so `-v -4' means the same as -v4 or -4v, &c.\n" +#if BZ_UNIX "\n" - #endif +#endif , fullProgName @@ -3776,14 +1273,7 @@ void *myMalloc ( Int32 n ) void* p; p = malloc ( (size_t)n ); - if (p == NULL) { - fprintf ( - stderr, - "%s: `malloc' failed on request for %d bytes.\n", - progName, n - ); - exit ( 1 ); - } + if (p == NULL) outOfMemory (); return p; } @@ -3817,7 +1307,6 @@ Cell *snocString ( Cell *root, Char *name ) } - /*---------------------------------------------*/ #define ISFLAG(s) (strcmp(aa->name, (s))==0) @@ -3829,11 +1318,6 @@ IntNative main ( IntNative argc, Char *argv[] ) Cell *argList; Cell *aa; - - #if DEBUG - fprintf ( stderr, "bzip2: *** compiled with debugging ON ***\n" ); - #endif - /*-- Be really really really paranoid :-) --*/ if (sizeof(Int32) != 4 || sizeof(UInt32) != 4 || sizeof(Int16) != 2 || sizeof(UInt16) != 2 || @@ -3844,7 +1328,7 @@ IntNative main ( IntNative argc, Char *argv[] ) "\tof 4, 2 and 1 bytes to run properly, and they don't.\n" "\tProbably you can fix this by defining them correctly,\n" "\tand recompiling. Bye!\n" ); - exit(1); + exit(3); } @@ -3852,35 +1336,28 @@ IntNative main ( IntNative argc, Char *argv[] ) signal (SIGINT, mySignalCatcher); signal (SIGTERM, mySignalCatcher); signal (SIGSEGV, mySIGSEGVorSIGBUScatcher); - #if BZ_UNIX +#if BZ_UNIX signal (SIGHUP, mySignalCatcher); signal (SIGBUS, mySIGSEGVorSIGBUScatcher); - #endif +#endif /*-- Initialise --*/ outputHandleJustInCase = NULL; - ftab = NULL; - ll4 = NULL; - ll16 = NULL; - ll8 = NULL; - tt = NULL; - block = NULL; - zptr = NULL; smallMode = False; keepInputFiles = False; + forceOverwrite = False; verbosity = 0; blockSize100k = 9; testFailsExist = False; - bsStream = NULL; numFileNames = 0; numFilesProcessed = 0; workFactor = 30; - strcpy ( inName, "(none)" ); - strcpy ( outName, "(none)" ); + copyFileName ( inName, "(none)" ); + copyFileName ( outName, "(none)" ); - strcpy ( progNameReally, argv[0] ); + copyFileName ( progNameReally, argv[0] ); progName = &progNameReally[0]; for (tmp = &progNameReally[0]; *tmp != '\0'; tmp++) if (*tmp == PATH_SEP) progName = tmp + 1; @@ -3903,20 +1380,26 @@ IntNative main ( IntNative argc, Char *argv[] ) } - /*-- Determine what to do (compress/uncompress/test). --*/ + /*-- Determine source modes; flag handling may change this too. --*/ + if (numFileNames == 0) + srcMode = SM_I2O; else srcMode = SM_F2F; + + + /*-- Determine what to do (compress/uncompress/test/cat). --*/ /*-- Note that subsequent flag handling may change this. --*/ opMode = OM_Z; - if ( (strcmp ( "bunzip2", progName ) == 0) || - (strcmp ( "BUNZIP2", progName ) == 0) || - (strcmp ( "bunzip2.exe", progName ) == 0) || - (strcmp ( "BUNZIP2.EXE", progName ) == 0) ) + if ( (strstr ( progName, "unzip" ) != 0) || + (strstr ( progName, "UNZIP" ) != 0) ) opMode = OM_UNZ; - - /*-- Determine source modes; flag handling may change this too. --*/ - if (numFileNames == 0) - srcMode = SM_I2O; else srcMode = SM_F2F; + if ( (strstr ( progName, "z2cat" ) != 0) || + (strstr ( progName, "Z2CAT" ) != 0) || + (strstr ( progName, "zcat" ) != 0) || + (strstr ( progName, "ZCAT" ) != 0) ) { + opMode = OM_UNZ; + srcMode = (numFileNames == 0) ? SM_I2O : SM_F2O; + } /*-- Look at the flags. --*/ @@ -3926,7 +1409,8 @@ IntNative main ( IntNative argc, Char *argv[] ) switch (aa->name[j]) { case 'c': srcMode = SM_F2O; break; case 'd': opMode = OM_UNZ; break; - case 'f': opMode = OM_Z; break; + case 'z': opMode = OM_Z; break; + case 'f': forceOverwrite = True; break; case 't': opMode = OM_TEST; break; case 'k': keepInputFiles = True; break; case 's': smallMode = True; break; @@ -3957,6 +1441,7 @@ IntNative main ( IntNative argc, Char *argv[] ) if (ISFLAG("--stdout")) srcMode = SM_F2O; else if (ISFLAG("--decompress")) opMode = OM_UNZ; else if (ISFLAG("--compress")) opMode = OM_Z; else + if (ISFLAG("--force")) forceOverwrite = True; else if (ISFLAG("--test")) opMode = OM_TEST; else if (ISFLAG("--keep")) keepInputFiles = True; else if (ISFLAG("--small")) smallMode = True; else @@ -3974,14 +1459,9 @@ IntNative main ( IntNative argc, Char *argv[] ) } } + if (verbosity > 4) verbosity = 4; if (opMode == OM_Z && smallMode) blockSize100k = 2; - if (opMode == OM_Z && srcMode == SM_F2O && numFileNames > 1) { - fprintf ( stderr, "%s: I won't compress multiple files to stdout.\n", - progName ); - exit ( 1 ); - } - if (srcMode == SM_F2O && numFileNames == 0) { fprintf ( stderr, "%s: -c expects at least one filename.\n", progName ); @@ -3997,7 +1477,6 @@ IntNative main ( IntNative argc, Char *argv[] ) if (opMode != OM_Z) blockSize100k = 0; if (opMode == OM_Z) { - allocateCompressStructures(); if (srcMode == SM_I2O) compress ( NULL ); else diff --git a/bzip2.txt b/bzip2.txt index aee8e2b..898dfe8 100644 --- a/bzip2.txt +++ b/bzip2.txt @@ -1,22 +1,22 @@ - - bzip2(1) bzip2(1) NAME - bzip2, bunzip2 - a block-sorting file compressor, v0.1 + bzip2, bunzip2 - a block-sorting file compressor, v0.9.0 + bzcat - decompresses files to stdout bzip2recover - recovers data from damaged bzip2 files SYNOPSIS - bzip2 [ -cdfkstvVL123456789 ] [ filenames ... ] - bunzip2 [ -kvsVL ] [ filenames ... ] + bzip2 [ -cdfkstvzVL123456789 ] [ filenames ... ] + bunzip2 [ -fkvsVL ] [ filenames ... ] + bzcat [ -s ] [ filenames ... ] bzip2recover filename DESCRIPTION - Bzip2 compresses files using the Burrows-Wheeler block- + bzip2 compresses files using the Burrows-Wheeler block- sorting text compression algorithm, and Huffman coding. Compression is generally considerably better than that achieved by more conventional LZ77/LZ78-based compressors, @@ -26,7 +26,7 @@ DESCRIPTION The command-line options are deliberately very similar to those of GNU Gzip, but they are not identical. - Bzip2 expects a list of file names to accompany the com- + bzip2 expects a list of file names to accompany the com- mand-line flags. Each file is replaced by a compressed version of itself, with the name "original_name.bz2". Each compressed file has the same modification date and @@ -38,8 +38,8 @@ DESCRIPTION cepts, or have serious file name length restrictions, such as MS-DOS. - Bzip2 and bunzip2 will not overwrite existing files; if - you want this to happen, you should delete them first. + bzip2 and bunzip2 will by default not overwrite existing + files; if you want this to happen, specify the -f flag. If no file names are specified, bzip2 compresses from standard input to standard output. In this case, bzip2 @@ -47,28 +47,29 @@ DESCRIPTION this would be entirely incomprehensible and therefore pointless. - Bunzip2 (or bzip2 -d ) decompresses and restores all spec- + bunzip2 (or bzip2 -d ) decompresses and restores all spec- ified files whose names end in ".bz2". Files without this suffix are ignored. Again, supplying no filenames causes decompression from standard input to standard output. - You can also compress or decompress files to the standard - output by giving the -c flag. You can decompress multiple - files like this, but you may only compress a single file - this way, since it would otherwise be difficult to sepa- - rate out the compressed representations of the original - files. - - - - 1 - - - - - -bzip2(1) bzip2(1) + bunzip2 will correctly decompress a file which is the con- + catenation of two or more compressed files. The result is + the concatenation of the corresponding uncompressed files. + Integrity testing (-t) of concatenated compressed files is + also supported. + You can also compress or decompress files to the standard + output by giving the -c flag. Multiple files may be com- + pressed and decompressed like this. The resulting outputs + are fed sequentially to stdout. Compression of multiple + files in this manner generates a stream containing multi- + ple compressed file representations. Such a stream can be + decompressed correctly only by bzip2 version 0.9.0 or + later. Earlier versions of bzip2 will stop after decom- + pressing the first file in the stream. + + bzcat (or bzip2 -dc ) decompresses all specified files to + the standard output. Compression is always performed, even if the compressed file is slightly larger than the original. Files of less @@ -108,13 +109,14 @@ MEMORY MANAGEMENT file, and bunzip2 then allocates itself just enough memory to decompress the file. Since block sizes are stored in compressed files, it follows that the flags -1 to -9 are - irrelevant to and so ignored during decompression. Com- - pression and decompression requirements, in bytes, can be - estimated as: + irrelevant to and so ignored during decompression. + + Compression and decompression requirements, in bytes, can + be estimated as: Compression: 400k + ( 7 x block size ) - Decompression: 100k + ( 5 x block size ), or + Decompression: 100k + ( 4 x block size ), or 100k + ( 2.5 x block size ) Larger block sizes give rapidly diminishing marginal @@ -125,19 +127,8 @@ MEMORY MANAGEMENT requirement is set at compression-time by the choice of block size. - - - 2 - - - - - -bzip2(1) bzip2(1) - - For files compressed with the default 900k block size, - bunzip2 will require about 4600 kbytes to decompress. To + bunzip2 will require about 3700 kbytes to decompress. To support decompression of any file on a 4 megabyte machine, bunzip2 has an option to decompress using approximately half this amount of memory, about 2300 kbytes. Decompres- @@ -157,8 +148,8 @@ bzip2(1) bzip2(1) file 20,000 bytes long with the flag -9 will cause the compressor to allocate around 6700k of memory, but only touch 400k + 20000 * 7 = 540 kbytes of it. Similarly, the - decompressor will allocate 4600k but only touch 100k + - 20000 * 5 = 200 kbytes. + decompressor will allocate 3700k but only touch 100k + + 20000 * 4 = 180 kbytes. Here is a table which summarises the maximum memory usage for different block sizes. Also recorded is the total @@ -172,15 +163,15 @@ bzip2(1) bzip2(1) Compress Decompress Decompress Corpus Flag usage usage -s usage Size - -1 1100k 600k 350k 914704 - -2 1800k 1100k 600k 877703 - -3 2500k 1600k 850k 860338 - -4 3200k 2100k 1100k 846899 - -5 3900k 2600k 1350k 845160 - -6 4600k 3100k 1600k 838626 - -7 5400k 3600k 1850k 834096 - -8 6000k 4100k 2100k 828642 - -9 6700k 4600k 2350k 828642 + -1 1100k 500k 350k 914704 + -2 1800k 900k 600k 877703 + -3 2500k 1300k 850k 860338 + -4 3200k 1700k 1100k 846899 + -5 3900k 2100k 1350k 845160 + -6 4600k 2500k 1600k 838626 + -7 5400k 2900k 1850k 834096 + -8 6000k 3300k 2100k 828642 + -9 6700k 3700k 2350k 828642 OPTIONS @@ -189,47 +180,37 @@ OPTIONS decompress multiple files to stdout, but will only compress a single file to stdout. - - - - - 3 - - - - - -bzip2(1) bzip2(1) - - -d --decompress - Force decompression. Bzip2 and bunzip2 are really - the same program, and the decision about whether to - compress or decompress is done on the basis of - which name is used. This flag overrides that mech- - anism, and forces bzip2 to decompress. + Force decompression. bzip2, bunzip2 and bzcat are + really the same program, and the decision about + what actions to take is done on the basis of which + name is used. This flag overrides that mechanism, + and forces bzip2 to decompress. - -f --compress + -z --compress The complement to -d: forces compression, regard- less of the invokation name. -t --test Check integrity of the specified file(s), but don't decompress them. This really performs a trial - decompression and throws away the result, using the - low-memory decompression algorithm (see -s). + decompression and throws away the result. + + -f --force + Force overwrite of output files. Normally, bzip2 + will not overwrite existing output files. -k --keep Keep (don't delete) input files during compression or decompression. -s --small - Reduce memory usage, both for compression and - decompression. Files are decompressed using a mod- - ified algorithm which only requires 2.5 bytes per - block byte. This means any file can be decom- - pressed in 2300k of memory, albeit somewhat more - slowly than usual. + Reduce memory usage, for compression, decompression + and testing. Files are decompressed and tested + using a modified algorithm which only requires 2.5 + bytes per block byte. This means any file can be + decompressed in 2300k of memory, albeit at about + half the normal speed. During compression, -s selects a block size of 200k, which limits memory use to around the same @@ -238,36 +219,21 @@ bzip2(1) bzip2(1) megabytes or less), use -s for everything. See MEMORY MANAGEMENT above. - -v --verbose Verbose mode -- show the compression ratio for each file processed. Further -v's increase the ver- bosity level, spewing out lots of information which is primarily of interest for diagnostic purposes. - -L --license + -L --license -V --version Display the software version, license terms and conditions. - -V --version - Same as -L. - -1 to -9 Set the block size to 100 k, 200 k .. 900 k when compressing. Has no effect when decompressing. See MEMORY MANAGEMENT above. - - - 4 - - - - - -bzip2(1) bzip2(1) - - --repetitive-fast bzip2 injects some small pseudo-random variations into very repetitive blocks to limit worst-case @@ -278,7 +244,6 @@ bzip2(1) bzip2(1) would take before resorting to randomisation. This flag makes it give up much sooner. - --repetitive-best Opposite of --repetitive-fast; try a lot harder before resorting to randomisation. @@ -306,10 +271,10 @@ RECOVERING DATA FROM DAMAGED FILES bzip2recover takes a single argument, the name of the dam- aged file, and writes a number of files "rec0001file.bz2", "rec0002file.bz2", etc, containing the extracted blocks. - The output filenames are designed so that the use of wild- - cards in subsequent processing -- for example, "bzip2 -dc - rec*file.bz2 > recovered_data" -- lists the files in the - "right" order. + The output filenames are designed so that the use of + wildcards in subsequent processing -- for example, "bzip2 + -dc rec*file.bz2 > recovered_data" -- lists the files in + the "right" order. bzip2recover should be of most use dealing with large .bz2 files, as these will contain many blocks. It is clearly @@ -322,18 +287,6 @@ RECOVERING DATA FROM DAMAGED FILES PERFORMANCE NOTES The sorting phase of compression gathers together similar - - - - 5 - - - - - -bzip2(1) bzip2(1) - - strings in the file. Because of this, files containing very long runs of repeated symbols, like "aabaabaabaab ..." (repeated several hundred times) may compress @@ -348,10 +301,6 @@ bzip2(1) bzip2(1) severe slowness in compression, try making the block size as small as possible, with flag -1. - Incompressible or virtually-incompressible data may decom- - press rather more slowly than one would hope. This is due - to a naive implementation of the move-to-front coder. - bzip2 usually allocates several megabytes of memory to operate in, and then charges all over it in a fairly ran- dom fashion. This means that performance, both for com- @@ -362,12 +311,6 @@ bzip2(1) bzip2(1) large performance improvements. I imagine bzip2 will per- form best on machines with very large caches. - Test mode (-t) uses the low-memory decompression algorithm - (-s). This means test mode does not run as fast as it - could; it could run as fast as the normal decompression - machinery. This could easily be fixed at the cost of some - code bloat. - CAVEATS I/O error messages are not as helpful as they could be. @@ -375,91 +318,38 @@ CAVEATS but the details of what the problem is sometimes seem rather misleading. - This manual page pertains to version 0.1 of bzip2. It may - well happen that some future version will use a different - compressed file format. If you try to decompress, using - 0.1, a .bz2 file created with some future version which - uses a different compressed file format, 0.1 will complain - that your file "is not a bzip2 file". If that happens, - you should obtain a more recent version of bzip2 and use - that to decompress the file. + This manual page pertains to version 0.9.0 of bzip2. Com- + pressed data created by this version is entirely forwards + and backwards compatible with the previous public release, + version 0.1pl2, but with the following exception: 0.9.0 + can correctly decompress multiple concatenated compressed + files. 0.1pl2 cannot do this; it will stop after decom- + pressing just the first file in the stream. Wildcard expansion for Windows 95 and NT is flaky. - bzip2recover uses 32-bit integers to represent bit posi- - tions in compressed files, so it cannot handle compressed - - - - 6 - - - - - -bzip2(1) bzip2(1) - - - files more than 512 megabytes long. This could easily be + bzip2recover uses 32-bit integers to represent bit posi- + tions in compressed files, so it cannot handle compressed + files more than 512 megabytes long. This could easily be fixed. - bzip2recover sometimes reports a very small, incomplete - final block. This is spurious and can be safely ignored. - - -RELATIONSHIP TO bzip-0.21 - This program is a descendant of the bzip program, version - 0.21, which I released in August 1996. The primary dif- - ference of bzip2 is its avoidance of the possibly patented - algorithms which were used in 0.21. bzip2 also brings - various useful refinements (-s, -t), uses less memory, - decompresses significantly faster, and has support for - recovering data from damaged files. - - Because bzip2 uses Huffman coding to construct the com- - pressed bitstream, rather than the arithmetic coding used - in 0.21, the compressed representations generated by the - two programs are incompatible, and they will not interop- - erate. The change in suffix from .bz to .bz2 reflects - this. It would have been helpful to at least allow bzip2 - to decompress files created by 0.21, but this would defeat - the primary aim of having a patent-free compressor. - - For a more precise statement about patent issues in bzip2, - please see the README file in the distribution. - - Huffman coding necessarily involves some coding ineffi- - ciency compared to arithmetic coding. This means that - bzip2 compresses about 1% worse than 0.21, an unfortunate - but unavoidable fact-of-life. On the other hand, decom- - pression is approximately 50% faster for the same reason, - and the change in file format gave an opportunity to add - data-recovery features. So it is not all bad. - AUTHOR Julian Seward, jseward@acm.org. - - The ideas embodied in bzip and bzip2 are due to (at least) - the following people: Michael Burrows and David Wheeler - (for the block sorting transformation), David Wheeler - (again, for the Huffman coder), Peter Fenwick (for the - structured coding model in 0.21, and many refinements), - and Alistair Moffat, Radford Neal and Ian Witten (for the - arithmetic coder in 0.21). I am much indebted for their - help, support and advice. See the file ALGORITHMS in the - source distribution for pointers to sources of documenta- - tion. Christian von Roques encouraged me to look for - faster sorting algorithms, so as to speed up compression. - Bela Lubkin encouraged me to improve the worst-case com- - pression performance. Many people sent patches, helped + http://www.muraroa.demon.co.uk + + The ideas embodied in bzip2 are due to (at least) the fol- + lowing people: Michael Burrows and David Wheeler (for the + block sorting transformation), David Wheeler (again, for + the Huffman coder), Peter Fenwick (for the structured cod- + ing model in the original bzip, and many refinements), and + Alistair Moffat, Radford Neal and Ian Witten (for the + arithmetic coder in the original bzip). I am much + indebted for their help, support and advice. See the man- + ual in the source distribution for pointers to sources of + documentation. Christian von Roques encouraged me to look + for faster sorting algorithms, so as to speed up compres- + sion. Bela Lubkin encouraged me to improve the worst-case + compression performance. Many people sent patches, helped with portability problems, lent machines, gave advice and were generally helpful. - - - - - - 7 - - diff --git a/bzip2recover.c b/bzip2recover.c index 0eef0e6..0e2822b 100644 --- a/bzip2recover.c +++ b/bzip2recover.c @@ -7,43 +7,63 @@ /*-- This program is bzip2recover, a program to attempt data salvage from damaged files created by the accompanying - bzip2-0.1 program. - - Copyright (C) 1996, 1997 by Julian Seward. - Guildford, Surrey, UK - email: jseward@acm.org - - This program is free software; you can redistribute it and/or modify - it under the terms of the GNU General Public License as published by - the Free Software Foundation; either version 2 of the License, or - (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - - The GNU General Public License is contained in the file LICENSE. + bzip2-0.9.0c program. + + Copyright (C) 1996-1998 Julian R Seward. All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + 1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + 2. The origin of this software must not be misrepresented; you must + not claim that you wrote the original software. If you use this + software in a product, an acknowledgment in the product + documentation would be appreciated but is not required. + + 3. Altered source versions must be plainly marked as such, and must + not be misrepresented as being the original software. + + 4. The name of the author may not be used to endorse or promote + products derived from this software without specific prior written + permission. + + THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS + OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED + WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY + DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE + GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS + INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, + WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING + NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS + SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + Julian Seward, Guildford, Surrey, UK. + jseward@acm.org + bzip2/libbzip2 version 0.9.0c of 18 October 1998 --*/ +/*-- + This program is a complete hack and should be rewritten + properly. It isn't very complicated. +--*/ #include #include -#include #include -#include /*-- or try string.h --*/ +#include -#define UInt32 unsigned int -#define Int32 int -#define UChar unsigned char -#define Char char -#define Bool unsigned char -#define True 1 -#define False 0 +typedef unsigned int UInt32; +typedef int Int32; +typedef unsigned char UChar; +typedef char Char; +typedef unsigned char Bool; +#define True ((Bool)1) +#define False ((Bool)0) Char inFileName[2000]; @@ -191,8 +211,9 @@ void bsClose ( BitStream* bs ) if (retVal == EOF) writeError(); } retVal = fclose ( bs->handle ); - if (retVal == EOF) + if (retVal == EOF) { if (bs->mode == 'w') writeError(); else readError(); + } free ( bs ); } @@ -248,13 +269,19 @@ Int32 main ( Int32 argc, Char** argv ) UInt32 bitsRead; UInt32 bStart[20000]; UInt32 bEnd[20000]; + + UInt32 rbStart[20000]; + UInt32 rbEnd[20000]; + Int32 rbCtr; + + UInt32 buffHi, buffLo, blockCRC; Char* p; strcpy ( progName, argv[0] ); inFileName[0] = outFileName[0] = 0; - fprintf ( stderr, "bzip2recover: extracts blocks from damaged .bz2 files.\n" ); + fprintf ( stderr, "bzip2recover v0.9.0c: extracts blocks from damaged .bz2 files.\n" ); if (argc != 2) { fprintf ( stderr, "%s: usage is `%s damaged_file_name'.\n", @@ -278,6 +305,8 @@ Int32 main ( Int32 argc, Char** argv ) currBlock = 0; bStart[currBlock] = 0; + rbCtr = 0; + while (True) { b = bsGetBit ( bsIn ); bitsRead++; @@ -303,19 +332,25 @@ Int32 main ( Int32 argc, Char** argv ) if (bitsRead > 49) bEnd[currBlock] = bitsRead-49; else bEnd[currBlock] = 0; - if (currBlock > 0) + if (currBlock > 0 && + (bEnd[currBlock] - bStart[currBlock]) >= 130) { fprintf ( stderr, " block %d runs from %d to %d\n", - currBlock, bStart[currBlock], bEnd[currBlock] ); + rbCtr+1, bStart[currBlock], bEnd[currBlock] ); + rbStart[rbCtr] = bStart[currBlock]; + rbEnd[rbCtr] = bEnd[currBlock]; + rbCtr++; + } currBlock++; + bStart[currBlock] = bitsRead; } } bsClose ( bsIn ); - /*-- identified blocks run from 1 to currBlock inclusive. --*/ + /*-- identified blocks run from 1 to rbCtr inclusive. --*/ - if (currBlock < 1) { + if (rbCtr < 1) { fprintf ( stderr, "%s: sorry, I couldn't find any block boundaries.\n", progName ); @@ -336,23 +371,23 @@ Int32 main ( Int32 argc, Char** argv ) bitsRead = 0; outFile = NULL; - wrBlock = 1; + wrBlock = 0; while (True) { b = bsGetBit(bsIn); if (b == 2) break; buffHi = (buffHi << 1) | (buffLo >> 31); buffLo = (buffLo << 1) | (b & 1); - if (bitsRead == 47+bStart[wrBlock]) + if (bitsRead == 47+rbStart[wrBlock]) blockCRC = (buffHi << 16) | (buffLo >> 16); - if (outFile != NULL && bitsRead >= bStart[wrBlock] - && bitsRead <= bEnd[wrBlock]) { + if (outFile != NULL && bitsRead >= rbStart[wrBlock] + && bitsRead <= rbEnd[wrBlock]) { bsPutBit ( bsWr, b ); } bitsRead++; - if (bitsRead == bEnd[wrBlock]+1) { + if (bitsRead == rbEnd[wrBlock]+1) { if (outFile != NULL) { bsPutUChar ( bsWr, 0x17 ); bsPutUChar ( bsWr, 0x72 ); bsPutUChar ( bsWr, 0x45 ); bsPutUChar ( bsWr, 0x38 ); @@ -360,18 +395,18 @@ Int32 main ( Int32 argc, Char** argv ) bsPutUInt32 ( bsWr, blockCRC ); bsClose ( bsWr ); } - if (wrBlock >= currBlock) break; + if (wrBlock >= rbCtr) break; wrBlock++; } else - if (bitsRead == bStart[wrBlock]) { + if (bitsRead == rbStart[wrBlock]) { outFileName[0] = 0; - sprintf ( outFileName, "rec%4d", wrBlock ); + sprintf ( outFileName, "rec%4d", wrBlock+1 ); for (p = outFileName; *p != 0; p++) if (*p == ' ') *p = '0'; strcat ( outFileName, inFileName ); if ( !endsInBz2(outFileName)) strcat ( outFileName, ".bz2" ); fprintf ( stderr, " writing block %d to `%s' ...\n", - wrBlock, outFileName ); + wrBlock+1, outFileName ); outFile = fopen ( outFileName, "wb" ); if (outFile == NULL) { diff --git a/bzlib.c b/bzlib.c new file mode 100644 index 0000000..362e8ff --- /dev/null +++ b/bzlib.c @@ -0,0 +1,1512 @@ + +/*-------------------------------------------------------------*/ +/*--- Library top-level functions. ---*/ +/*--- bzlib.c ---*/ +/*-------------------------------------------------------------*/ + +/*-- + This file is a part of bzip2 and/or libbzip2, a program and + library for lossless, block-sorting data compression. + + Copyright (C) 1996-1998 Julian R Seward. All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + 1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + 2. The origin of this software must not be misrepresented; you must + not claim that you wrote the original software. If you use this + software in a product, an acknowledgment in the product + documentation would be appreciated but is not required. + + 3. Altered source versions must be plainly marked as such, and must + not be misrepresented as being the original software. + + 4. The name of the author may not be used to endorse or promote + products derived from this software without specific prior written + permission. + + THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS + OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED + WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY + DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE + GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS + INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, + WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING + NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS + SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + Julian Seward, Guildford, Surrey, UK. + jseward@acm.org + bzip2/libbzip2 version 0.9.0c of 18 October 1998 + + This program is based on (at least) the work of: + Mike Burrows + David Wheeler + Peter Fenwick + Alistair Moffat + Radford Neal + Ian H. Witten + Robert Sedgewick + Jon L. Bentley + + For more information on these sources, see the manual. +--*/ + +/*-- + CHANGES + ~~~~~~~ + 0.9.0 -- original version. + + 0.9.0a/b -- no changes in this file. + + 0.9.0c + * made zero-length BZ_FLUSH work correctly in bzCompress(). + * fixed bzWrite/bzRead to ignore zero-length requests. + * fixed bzread to correctly handle read requests after EOF. + * wrong parameter order in call to bzDecompressInit in + bzBuffToBuffDecompress. Fixed. +--*/ + +#include "bzlib_private.h" + + +/*---------------------------------------------------*/ +/*--- Compression stuff ---*/ +/*---------------------------------------------------*/ + + +/*---------------------------------------------------*/ +#ifndef BZ_NO_STDIO +void bz__AssertH__fail ( int errcode ) +{ + fprintf(stderr, + "\n\nbzip2/libbzip2, v0.9.0c: internal error number %d.\n" + "This is a bug in bzip2/libbzip2, v0.9.0c. Please report\n" + "it to me at: jseward@acm.org. If this happened when\n" + "you were using some program which uses libbzip2 as a\n" + "component, you should also report this bug to the author(s)\n" + "of that program. Please make an effort to report this bug;\n" + "timely and accurate bug reports eventually lead to higher\n" + "quality software. Thx. Julian Seward, 18 October 1998.\n\n", + errcode + ); + exit(3); +} +#endif + + +/*---------------------------------------------------*/ +static +void* default_bzalloc ( void* opaque, Int32 items, Int32 size ) +{ + void* v = malloc ( items * size ); + return v; +} + +static +void default_bzfree ( void* opaque, void* addr ) +{ + if (addr != NULL) free ( addr ); +} + + +/*---------------------------------------------------*/ +static +void prepare_new_block ( EState* s ) +{ + Int32 i; + s->nblock = 0; + s->numZ = 0; + s->state_out_pos = 0; + BZ_INITIALISE_CRC ( s->blockCRC ); + for (i = 0; i < 256; i++) s->inUse[i] = False; + s->blockNo++; +} + + +/*---------------------------------------------------*/ +static +void init_RL ( EState* s ) +{ + s->state_in_ch = 256; + s->state_in_len = 0; +} + + +static +Bool isempty_RL ( EState* s ) +{ + if (s->state_in_ch < 256 && s->state_in_len > 0) + return False; else + return True; +} + + +/*---------------------------------------------------*/ +int BZ_API(bzCompressInit) + ( bz_stream* strm, + int blockSize100k, + int verbosity, + int workFactor ) +{ + Int32 n; + EState* s; + + if (strm == NULL || + blockSize100k < 1 || blockSize100k > 9 || + workFactor < 0 || workFactor > 250) + return BZ_PARAM_ERROR; + + if (workFactor == 0) workFactor = 30; + if (strm->bzalloc == NULL) strm->bzalloc = default_bzalloc; + if (strm->bzfree == NULL) strm->bzfree = default_bzfree; + + s = BZALLOC( sizeof(EState) ); + if (s == NULL) return BZ_MEM_ERROR; + s->strm = strm; + + s->block = NULL; + s->quadrant = NULL; + s->zptr = NULL; + s->ftab = NULL; + + n = 100000 * blockSize100k; + s->block = BZALLOC( (n + BZ_NUM_OVERSHOOT_BYTES) * sizeof(UChar) ); + s->quadrant = BZALLOC( (n + BZ_NUM_OVERSHOOT_BYTES) * sizeof(Int16) ); + s->zptr = BZALLOC( n * sizeof(Int32) ); + s->ftab = BZALLOC( 65537 * sizeof(Int32) ); + + if (s->block == NULL || s->quadrant == NULL || + s->zptr == NULL || s->ftab == NULL) { + if (s->block != NULL) BZFREE(s->block); + if (s->quadrant != NULL) BZFREE(s->quadrant); + if (s->zptr != NULL) BZFREE(s->zptr); + if (s->ftab != NULL) BZFREE(s->ftab); + if (s != NULL) BZFREE(s); + return BZ_MEM_ERROR; + } + + s->szptr = (UInt16*)(s->zptr); + + s->blockNo = 0; + s->state = BZ_S_INPUT; + s->mode = BZ_M_RUNNING; + s->combinedCRC = 0; + s->blockSize100k = blockSize100k; + s->nblockMAX = 100000 * blockSize100k - 19; + s->verbosity = verbosity; + s->workFactor = workFactor; + s->nBlocksRandomised = 0; + strm->state = s; + strm->total_in = 0; + strm->total_out = 0; + init_RL ( s ); + prepare_new_block ( s ); + return BZ_OK; +} + + +/*---------------------------------------------------*/ +static +void add_pair_to_block ( EState* s ) +{ + Int32 i; + UChar ch = (UChar)(s->state_in_ch); + for (i = 0; i < s->state_in_len; i++) { + BZ_UPDATE_CRC( s->blockCRC, ch ); + } + s->inUse[s->state_in_ch] = True; + switch (s->state_in_len) { + case 1: + s->block[s->nblock] = (UChar)ch; s->nblock++; + break; + case 2: + s->block[s->nblock] = (UChar)ch; s->nblock++; + s->block[s->nblock] = (UChar)ch; s->nblock++; + break; + case 3: + s->block[s->nblock] = (UChar)ch; s->nblock++; + s->block[s->nblock] = (UChar)ch; s->nblock++; + s->block[s->nblock] = (UChar)ch; s->nblock++; + break; + default: + s->inUse[s->state_in_len-4] = True; + s->block[s->nblock] = (UChar)ch; s->nblock++; + s->block[s->nblock] = (UChar)ch; s->nblock++; + s->block[s->nblock] = (UChar)ch; s->nblock++; + s->block[s->nblock] = (UChar)ch; s->nblock++; + s->block[s->nblock] = (UChar)(s->state_in_len-4); + s->nblock++; + break; + } +} + + +/*---------------------------------------------------*/ +static +void flush_RL ( EState* s ) +{ + if (s->state_in_ch < 256) add_pair_to_block ( s ); + init_RL ( s ); +} + + +/*---------------------------------------------------*/ +#define ADD_CHAR_TO_BLOCK(zs,zchh0) \ +{ \ + UInt32 zchh = (UInt32)(zchh0); \ + /*-- fast track the common case --*/ \ + if (zchh != zs->state_in_ch && \ + zs->state_in_len == 1) { \ + UChar ch = (UChar)(zs->state_in_ch); \ + BZ_UPDATE_CRC( zs->blockCRC, ch ); \ + zs->inUse[zs->state_in_ch] = True; \ + zs->block[zs->nblock] = (UChar)ch; \ + zs->nblock++; \ + zs->state_in_ch = zchh; \ + } \ + else \ + /*-- general, uncommon cases --*/ \ + if (zchh != zs->state_in_ch || \ + zs->state_in_len == 255) { \ + if (zs->state_in_ch < 256) \ + add_pair_to_block ( zs ); \ + zs->state_in_ch = zchh; \ + zs->state_in_len = 1; \ + } else { \ + zs->state_in_len++; \ + } \ +} + + +/*---------------------------------------------------*/ +static +Bool copy_input_until_stop ( EState* s ) +{ + Bool progress_in = False; + + if (s->mode == BZ_M_RUNNING) { + + /*-- fast track the common case --*/ + while (True) { + /*-- block full? --*/ + if (s->nblock >= s->nblockMAX) break; + /*-- no input? --*/ + if (s->strm->avail_in == 0) break; + progress_in = True; + ADD_CHAR_TO_BLOCK ( s, (UInt32)(*((UChar*)(s->strm->next_in))) ); + s->strm->next_in++; + s->strm->avail_in--; + s->strm->total_in++; + } + + } else { + + /*-- general, uncommon case --*/ + while (True) { + /*-- block full? --*/ + if (s->nblock >= s->nblockMAX) break; + /*-- no input? --*/ + if (s->strm->avail_in == 0) break; + /*-- flush/finish end? --*/ + if (s->avail_in_expect == 0) break; + progress_in = True; + ADD_CHAR_TO_BLOCK ( s, (UInt32)(*((UChar*)(s->strm->next_in))) ); + s->strm->next_in++; + s->strm->avail_in--; + s->strm->total_in++; + s->avail_in_expect--; + } + } + return progress_in; +} + + +/*---------------------------------------------------*/ +static +Bool copy_output_until_stop ( EState* s ) +{ + Bool progress_out = False; + + while (True) { + + /*-- no output space? --*/ + if (s->strm->avail_out == 0) break; + + /*-- block done? --*/ + if (s->state_out_pos >= s->numZ) break; + + progress_out = True; + *(s->strm->next_out) = ((UChar*)(s->quadrant))[s->state_out_pos]; + s->state_out_pos++; + s->strm->avail_out--; + s->strm->next_out++; + s->strm->total_out++; + + } + + return progress_out; +} + + +/*---------------------------------------------------*/ +static +Bool handle_compress ( bz_stream* strm ) +{ + Bool progress_in = False; + Bool progress_out = False; + EState* s = strm->state; + + while (True) { + + if (s->state == BZ_S_OUTPUT) { + progress_out |= copy_output_until_stop ( s ); + if (s->state_out_pos < s->numZ) break; + if (s->mode == BZ_M_FINISHING && + s->avail_in_expect == 0 && + isempty_RL(s)) break; + prepare_new_block ( s ); + s->state = BZ_S_INPUT; + if (s->mode == BZ_M_FLUSHING && + s->avail_in_expect == 0 && + isempty_RL(s)) break; + } + + if (s->state == BZ_S_INPUT) { + progress_in |= copy_input_until_stop ( s ); + if (s->mode != BZ_M_RUNNING && s->avail_in_expect == 0) { + flush_RL ( s ); + compressBlock ( s, s->mode == BZ_M_FINISHING ); + s->state = BZ_S_OUTPUT; + } + else + if (s->nblock >= s->nblockMAX) { + compressBlock ( s, False ); + s->state = BZ_S_OUTPUT; + } + else + if (s->strm->avail_in == 0) { + break; + } + } + + } + + return progress_in || progress_out; +} + + +/*---------------------------------------------------*/ +int BZ_API(bzCompress) ( bz_stream *strm, int action ) +{ + Bool progress; + EState* s; + if (strm == NULL) return BZ_PARAM_ERROR; + s = strm->state; + if (s == NULL) return BZ_PARAM_ERROR; + if (s->strm != strm) return BZ_PARAM_ERROR; + + preswitch: + switch (s->mode) { + + case BZ_M_IDLE: + return BZ_SEQUENCE_ERROR; + + case BZ_M_RUNNING: + if (action == BZ_RUN) { + progress = handle_compress ( strm ); + return progress ? BZ_RUN_OK : BZ_PARAM_ERROR; + } + else + if (action == BZ_FLUSH) { + s->avail_in_expect = strm->avail_in; + s->mode = BZ_M_FLUSHING; + goto preswitch; + } + else + if (action == BZ_FINISH) { + s->avail_in_expect = strm->avail_in; + s->mode = BZ_M_FINISHING; + goto preswitch; + } + else + return BZ_PARAM_ERROR; + + case BZ_M_FLUSHING: + if (action != BZ_FLUSH) return BZ_SEQUENCE_ERROR; + if (s->avail_in_expect != s->strm->avail_in) return BZ_SEQUENCE_ERROR; + progress = handle_compress ( strm ); + if (s->avail_in_expect > 0 || !isempty_RL(s) || + s->state_out_pos < s->numZ) return BZ_FLUSH_OK; + s->mode = BZ_M_RUNNING; + return BZ_RUN_OK; + + case BZ_M_FINISHING: + if (action != BZ_FINISH) return BZ_SEQUENCE_ERROR; + if (s->avail_in_expect != s->strm->avail_in) return BZ_SEQUENCE_ERROR; + progress = handle_compress ( strm ); + if (!progress) return BZ_SEQUENCE_ERROR; + if (s->avail_in_expect > 0 || !isempty_RL(s) || + s->state_out_pos < s->numZ) return BZ_FINISH_OK; + s->mode = BZ_M_IDLE; + return BZ_STREAM_END; + } + return BZ_OK; /*--not reached--*/ +} + + +/*---------------------------------------------------*/ +int BZ_API(bzCompressEnd) ( bz_stream *strm ) +{ + EState* s; + if (strm == NULL) return BZ_PARAM_ERROR; + s = strm->state; + if (s == NULL) return BZ_PARAM_ERROR; + if (s->strm != strm) return BZ_PARAM_ERROR; + + if (s->block != NULL) BZFREE(s->block); + if (s->quadrant != NULL) BZFREE(s->quadrant); + if (s->zptr != NULL) BZFREE(s->zptr); + if (s->ftab != NULL) BZFREE(s->ftab); + BZFREE(strm->state); + + strm->state = NULL; + + return BZ_OK; +} + + +/*---------------------------------------------------*/ +/*--- Decompression stuff ---*/ +/*---------------------------------------------------*/ + +/*---------------------------------------------------*/ +int BZ_API(bzDecompressInit) + ( bz_stream* strm, + int verbosity, + int small ) +{ + DState* s; + + if (strm == NULL) return BZ_PARAM_ERROR; + if (small != 0 && small != 1) return BZ_PARAM_ERROR; + if (verbosity < 0 || verbosity > 4) return BZ_PARAM_ERROR; + + if (strm->bzalloc == NULL) strm->bzalloc = default_bzalloc; + if (strm->bzfree == NULL) strm->bzfree = default_bzfree; + + s = BZALLOC( sizeof(DState) ); + if (s == NULL) return BZ_MEM_ERROR; + s->strm = strm; + strm->state = s; + s->state = BZ_X_MAGIC_1; + s->bsLive = 0; + s->bsBuff = 0; + s->calculatedCombinedCRC = 0; + strm->total_in = 0; + strm->total_out = 0; + s->smallDecompress = (Bool)small; + s->ll4 = NULL; + s->ll16 = NULL; + s->tt = NULL; + s->currBlockNo = 0; + s->verbosity = verbosity; + + return BZ_OK; +} + + +/*---------------------------------------------------*/ +static +void unRLE_obuf_to_output_FAST ( DState* s ) +{ + UChar k1; + + if (s->blockRandomised) { + + while (True) { + /* try to finish existing run */ + while (True) { + if (s->strm->avail_out == 0) return; + if (s->state_out_len == 0) break; + *( (UChar*)(s->strm->next_out) ) = s->state_out_ch; + BZ_UPDATE_CRC ( s->calculatedBlockCRC, s->state_out_ch ); + s->state_out_len--; + s->strm->next_out++; + s->strm->avail_out--; + s->strm->total_out++; + } + + /* can a new run be started? */ + if (s->nblock_used == s->save_nblock+1) return; + + + s->state_out_len = 1; + s->state_out_ch = s->k0; + BZ_GET_FAST(k1); BZ_RAND_UPD_MASK; + k1 ^= BZ_RAND_MASK; s->nblock_used++; + if (s->nblock_used == s->save_nblock+1) continue; + if (k1 != s->k0) { s->k0 = k1; continue; }; + + s->state_out_len = 2; + BZ_GET_FAST(k1); BZ_RAND_UPD_MASK; + k1 ^= BZ_RAND_MASK; s->nblock_used++; + if (s->nblock_used == s->save_nblock+1) continue; + if (k1 != s->k0) { s->k0 = k1; continue; }; + + s->state_out_len = 3; + BZ_GET_FAST(k1); BZ_RAND_UPD_MASK; + k1 ^= BZ_RAND_MASK; s->nblock_used++; + if (s->nblock_used == s->save_nblock+1) continue; + if (k1 != s->k0) { s->k0 = k1; continue; }; + + BZ_GET_FAST(k1); BZ_RAND_UPD_MASK; + k1 ^= BZ_RAND_MASK; s->nblock_used++; + s->state_out_len = ((Int32)k1) + 4; + BZ_GET_FAST(s->k0); BZ_RAND_UPD_MASK; + s->k0 ^= BZ_RAND_MASK; s->nblock_used++; + } + + } else { + + /* restore */ + UInt32 c_calculatedBlockCRC = s->calculatedBlockCRC; + UChar c_state_out_ch = s->state_out_ch; + Int32 c_state_out_len = s->state_out_len; + Int32 c_nblock_used = s->nblock_used; + Int32 c_k0 = s->k0; + UInt32* c_tt = s->tt; + UInt32 c_tPos = s->tPos; + char* cs_next_out = s->strm->next_out; + unsigned int cs_avail_out = s->strm->avail_out; + /* end restore */ + + UInt32 avail_out_INIT = cs_avail_out; + Int32 s_save_nblockPP = s->save_nblock+1; + + while (True) { + + /* try to finish existing run */ + if (c_state_out_len > 0) { + while (True) { + if (cs_avail_out == 0) goto return_notr; + if (c_state_out_len == 1) break; + *( (UChar*)(cs_next_out) ) = c_state_out_ch; + BZ_UPDATE_CRC ( c_calculatedBlockCRC, c_state_out_ch ); + c_state_out_len--; + cs_next_out++; + cs_avail_out--; + } + s_state_out_len_eq_one: + { + if (cs_avail_out == 0) { + c_state_out_len = 1; goto return_notr; + }; + *( (UChar*)(cs_next_out) ) = c_state_out_ch; + BZ_UPDATE_CRC ( c_calculatedBlockCRC, c_state_out_ch ); + cs_next_out++; + cs_avail_out--; + } + } + /* can a new run be started? */ + if (c_nblock_used == s_save_nblockPP) { + c_state_out_len = 0; goto return_notr; + }; + c_state_out_ch = c_k0; + BZ_GET_FAST_C(k1); c_nblock_used++; + if (k1 != c_k0) { + c_k0 = k1; goto s_state_out_len_eq_one; + }; + if (c_nblock_used == s_save_nblockPP) + goto s_state_out_len_eq_one; + + c_state_out_len = 2; + BZ_GET_FAST_C(k1); c_nblock_used++; + if (c_nblock_used == s_save_nblockPP) continue; + if (k1 != c_k0) { c_k0 = k1; continue; }; + + c_state_out_len = 3; + BZ_GET_FAST_C(k1); c_nblock_used++; + if (c_nblock_used == s_save_nblockPP) continue; + if (k1 != c_k0) { c_k0 = k1; continue; }; + + BZ_GET_FAST_C(k1); c_nblock_used++; + c_state_out_len = ((Int32)k1) + 4; + BZ_GET_FAST_C(c_k0); c_nblock_used++; + } + + return_notr: + s->strm->total_out += (avail_out_INIT - cs_avail_out); + + /* save */ + s->calculatedBlockCRC = c_calculatedBlockCRC; + s->state_out_ch = c_state_out_ch; + s->state_out_len = c_state_out_len; + s->nblock_used = c_nblock_used; + s->k0 = c_k0; + s->tt = c_tt; + s->tPos = c_tPos; + s->strm->next_out = cs_next_out; + s->strm->avail_out = cs_avail_out; + /* end save */ + } +} + + + +/*---------------------------------------------------*/ +__inline__ Int32 indexIntoF ( Int32 indx, Int32 *cftab ) +{ + Int32 nb, na, mid; + nb = 0; + na = 256; + do { + mid = (nb + na) >> 1; + if (indx >= cftab[mid]) nb = mid; else na = mid; + } + while (na - nb != 1); + return nb; +} + + +/*---------------------------------------------------*/ +static +void unRLE_obuf_to_output_SMALL ( DState* s ) +{ + UChar k1; + + if (s->blockRandomised) { + + while (True) { + /* try to finish existing run */ + while (True) { + if (s->strm->avail_out == 0) return; + if (s->state_out_len == 0) break; + *( (UChar*)(s->strm->next_out) ) = s->state_out_ch; + BZ_UPDATE_CRC ( s->calculatedBlockCRC, s->state_out_ch ); + s->state_out_len--; + s->strm->next_out++; + s->strm->avail_out--; + s->strm->total_out++; + } + + /* can a new run be started? */ + if (s->nblock_used == s->save_nblock+1) return; + + + s->state_out_len = 1; + s->state_out_ch = s->k0; + BZ_GET_SMALL(k1); BZ_RAND_UPD_MASK; + k1 ^= BZ_RAND_MASK; s->nblock_used++; + if (s->nblock_used == s->save_nblock+1) continue; + if (k1 != s->k0) { s->k0 = k1; continue; }; + + s->state_out_len = 2; + BZ_GET_SMALL(k1); BZ_RAND_UPD_MASK; + k1 ^= BZ_RAND_MASK; s->nblock_used++; + if (s->nblock_used == s->save_nblock+1) continue; + if (k1 != s->k0) { s->k0 = k1; continue; }; + + s->state_out_len = 3; + BZ_GET_SMALL(k1); BZ_RAND_UPD_MASK; + k1 ^= BZ_RAND_MASK; s->nblock_used++; + if (s->nblock_used == s->save_nblock+1) continue; + if (k1 != s->k0) { s->k0 = k1; continue; }; + + BZ_GET_SMALL(k1); BZ_RAND_UPD_MASK; + k1 ^= BZ_RAND_MASK; s->nblock_used++; + s->state_out_len = ((Int32)k1) + 4; + BZ_GET_SMALL(s->k0); BZ_RAND_UPD_MASK; + s->k0 ^= BZ_RAND_MASK; s->nblock_used++; + } + + } else { + + while (True) { + /* try to finish existing run */ + while (True) { + if (s->strm->avail_out == 0) return; + if (s->state_out_len == 0) break; + *( (UChar*)(s->strm->next_out) ) = s->state_out_ch; + BZ_UPDATE_CRC ( s->calculatedBlockCRC, s->state_out_ch ); + s->state_out_len--; + s->strm->next_out++; + s->strm->avail_out--; + s->strm->total_out++; + } + + /* can a new run be started? */ + if (s->nblock_used == s->save_nblock+1) return; + + s->state_out_len = 1; + s->state_out_ch = s->k0; + BZ_GET_SMALL(k1); s->nblock_used++; + if (s->nblock_used == s->save_nblock+1) continue; + if (k1 != s->k0) { s->k0 = k1; continue; }; + + s->state_out_len = 2; + BZ_GET_SMALL(k1); s->nblock_used++; + if (s->nblock_used == s->save_nblock+1) continue; + if (k1 != s->k0) { s->k0 = k1; continue; }; + + s->state_out_len = 3; + BZ_GET_SMALL(k1); s->nblock_used++; + if (s->nblock_used == s->save_nblock+1) continue; + if (k1 != s->k0) { s->k0 = k1; continue; }; + + BZ_GET_SMALL(k1); s->nblock_used++; + s->state_out_len = ((Int32)k1) + 4; + BZ_GET_SMALL(s->k0); s->nblock_used++; + } + + } +} + + +/*---------------------------------------------------*/ +int BZ_API(bzDecompress) ( bz_stream *strm ) +{ + DState* s; + if (strm == NULL) return BZ_PARAM_ERROR; + s = strm->state; + if (s == NULL) return BZ_PARAM_ERROR; + if (s->strm != strm) return BZ_PARAM_ERROR; + + while (True) { + if (s->state == BZ_X_IDLE) return BZ_SEQUENCE_ERROR; + if (s->state == BZ_X_OUTPUT) { + if (s->smallDecompress) + unRLE_obuf_to_output_SMALL ( s ); else + unRLE_obuf_to_output_FAST ( s ); + if (s->nblock_used == s->save_nblock+1 && s->state_out_len == 0) { + BZ_FINALISE_CRC ( s->calculatedBlockCRC ); + if (s->verbosity >= 3) + VPrintf2 ( " {0x%x, 0x%x}", s->storedBlockCRC, + s->calculatedBlockCRC ); + if (s->verbosity >= 2) VPrintf0 ( "]" ); + if (s->calculatedBlockCRC != s->storedBlockCRC) + return BZ_DATA_ERROR; + s->calculatedCombinedCRC + = (s->calculatedCombinedCRC << 1) | + (s->calculatedCombinedCRC >> 31); + s->calculatedCombinedCRC ^= s->calculatedBlockCRC; + s->state = BZ_X_BLKHDR_1; + } else { + return BZ_OK; + } + } + if (s->state >= BZ_X_MAGIC_1) { + Int32 r = decompress ( s ); + if (r == BZ_STREAM_END) { + if (s->verbosity >= 3) + VPrintf2 ( "\n combined CRCs: stored = 0x%x, computed = 0x%x", + s->storedCombinedCRC, s->calculatedCombinedCRC ); + if (s->calculatedCombinedCRC != s->storedCombinedCRC) + return BZ_DATA_ERROR; + return r; + } + if (s->state != BZ_X_OUTPUT) return r; + } + } + + AssertH ( 0, 6001 ); + /*notreached*/ +} + + +/*---------------------------------------------------*/ +int BZ_API(bzDecompressEnd) ( bz_stream *strm ) +{ + DState* s; + if (strm == NULL) return BZ_PARAM_ERROR; + s = strm->state; + if (s == NULL) return BZ_PARAM_ERROR; + if (s->strm != strm) return BZ_PARAM_ERROR; + + if (s->tt != NULL) BZFREE(s->tt); + if (s->ll16 != NULL) BZFREE(s->ll16); + if (s->ll4 != NULL) BZFREE(s->ll4); + + BZFREE(strm->state); + strm->state = NULL; + + return BZ_OK; +} + + +#ifndef BZ_NO_STDIO +/*---------------------------------------------------*/ +/*--- File I/O stuff ---*/ +/*---------------------------------------------------*/ + +#define BZ_SETERR(eee) \ +{ \ + if (bzerror != NULL) *bzerror = eee; \ + if (bzf != NULL) bzf->lastErr = eee; \ +} + +typedef + struct { + FILE* handle; + Char buf[BZ_MAX_UNUSED]; + Int32 bufN; + Bool writing; + bz_stream strm; + Int32 lastErr; + Bool initialisedOk; + } + bzFile; + + +/*---------------------------------------------*/ +static Bool myfeof ( FILE* f ) +{ + Int32 c = fgetc ( f ); + if (c == EOF) return True; + ungetc ( c, f ); + return False; +} + + +/*---------------------------------------------------*/ +BZFILE* BZ_API(bzWriteOpen) + ( int* bzerror, + FILE* f, + int blockSize100k, + int verbosity, + int workFactor ) +{ + Int32 ret; + bzFile* bzf = NULL; + + BZ_SETERR(BZ_OK); + + if (f == NULL || + (blockSize100k < 1 || blockSize100k > 9) || + (workFactor < 0 || workFactor > 250) || + (verbosity < 0 || verbosity > 4)) + { BZ_SETERR(BZ_PARAM_ERROR); return NULL; }; + + if (ferror(f)) + { BZ_SETERR(BZ_IO_ERROR); return NULL; }; + + bzf = malloc ( sizeof(bzFile) ); + if (bzf == NULL) + { BZ_SETERR(BZ_MEM_ERROR); return NULL; }; + + BZ_SETERR(BZ_OK); + bzf->initialisedOk = False; + bzf->bufN = 0; + bzf->handle = f; + bzf->writing = True; + bzf->strm.bzalloc = NULL; + bzf->strm.bzfree = NULL; + bzf->strm.opaque = NULL; + + if (workFactor == 0) workFactor = 30; + ret = bzCompressInit ( &(bzf->strm), blockSize100k, + verbosity, workFactor ); + if (ret != BZ_OK) + { BZ_SETERR(ret); free(bzf); return NULL; }; + + bzf->strm.avail_in = 0; + bzf->initialisedOk = True; + return bzf; +} + + + +/*---------------------------------------------------*/ +void BZ_API(bzWrite) + ( int* bzerror, + BZFILE* b, + void* buf, + int len ) +{ + Int32 n, n2, ret; + bzFile* bzf = (bzFile*)b; + + BZ_SETERR(BZ_OK); + if (bzf == NULL || buf == NULL || len < 0) + { BZ_SETERR(BZ_PARAM_ERROR); return; }; + if (!(bzf->writing)) + { BZ_SETERR(BZ_SEQUENCE_ERROR); return; }; + if (ferror(bzf->handle)) + { BZ_SETERR(BZ_IO_ERROR); return; }; + + if (len == 0) + { BZ_SETERR(BZ_OK); return; }; + + bzf->strm.avail_in = len; + bzf->strm.next_in = buf; + + while (True) { + bzf->strm.avail_out = BZ_MAX_UNUSED; + bzf->strm.next_out = bzf->buf; + ret = bzCompress ( &(bzf->strm), BZ_RUN ); + if (ret != BZ_RUN_OK) + { BZ_SETERR(ret); return; }; + + if (bzf->strm.avail_out < BZ_MAX_UNUSED) { + n = BZ_MAX_UNUSED - bzf->strm.avail_out; + n2 = fwrite ( (void*)(bzf->buf), sizeof(UChar), + n, bzf->handle ); + if (n != n2 || ferror(bzf->handle)) + { BZ_SETERR(BZ_IO_ERROR); return; }; + } + + if (bzf->strm.avail_in == 0) + { BZ_SETERR(BZ_OK); return; }; + } +} + + +/*---------------------------------------------------*/ +void BZ_API(bzWriteClose) + ( int* bzerror, + BZFILE* b, + int abandon, + unsigned int* nbytes_in, + unsigned int* nbytes_out ) +{ + Int32 n, n2, ret; + bzFile* bzf = (bzFile*)b; + + if (bzf == NULL) + { BZ_SETERR(BZ_OK); return; }; + if (!(bzf->writing)) + { BZ_SETERR(BZ_SEQUENCE_ERROR); return; }; + if (ferror(bzf->handle)) + { BZ_SETERR(BZ_IO_ERROR); return; }; + + if (nbytes_in != NULL) *nbytes_in = 0; + if (nbytes_out != NULL) *nbytes_out = 0; + + if ((!abandon) && bzf->lastErr == BZ_OK) { + while (True) { + bzf->strm.avail_out = BZ_MAX_UNUSED; + bzf->strm.next_out = bzf->buf; + ret = bzCompress ( &(bzf->strm), BZ_FINISH ); + if (ret != BZ_FINISH_OK && ret != BZ_STREAM_END) + { BZ_SETERR(ret); return; }; + + if (bzf->strm.avail_out < BZ_MAX_UNUSED) { + n = BZ_MAX_UNUSED - bzf->strm.avail_out; + n2 = fwrite ( (void*)(bzf->buf), sizeof(UChar), + n, bzf->handle ); + if (n != n2 || ferror(bzf->handle)) + { BZ_SETERR(BZ_IO_ERROR); return; }; + } + + if (ret == BZ_STREAM_END) break; + } + } + + if ( !abandon && !ferror ( bzf->handle ) ) { + fflush ( bzf->handle ); + if (ferror(bzf->handle)) + { BZ_SETERR(BZ_IO_ERROR); return; }; + } + + if (nbytes_in != NULL) *nbytes_in = bzf->strm.total_in; + if (nbytes_out != NULL) *nbytes_out = bzf->strm.total_out; + + BZ_SETERR(BZ_OK); + bzCompressEnd ( &(bzf->strm) ); + free ( bzf ); +} + + +/*---------------------------------------------------*/ +BZFILE* BZ_API(bzReadOpen) + ( int* bzerror, + FILE* f, + int verbosity, + int small, + void* unused, + int nUnused ) +{ + bzFile* bzf = NULL; + int ret; + + BZ_SETERR(BZ_OK); + + if (f == NULL || + (small != 0 && small != 1) || + (verbosity < 0 || verbosity > 4) || + (unused == NULL && nUnused != 0) || + (unused != NULL && (nUnused < 0 || nUnused > BZ_MAX_UNUSED))) + { BZ_SETERR(BZ_PARAM_ERROR); return NULL; }; + + if (ferror(f)) + { BZ_SETERR(BZ_IO_ERROR); return NULL; }; + + bzf = malloc ( sizeof(bzFile) ); + if (bzf == NULL) + { BZ_SETERR(BZ_MEM_ERROR); return NULL; }; + + BZ_SETERR(BZ_OK); + + bzf->initialisedOk = False; + bzf->handle = f; + bzf->bufN = 0; + bzf->writing = False; + bzf->strm.bzalloc = NULL; + bzf->strm.bzfree = NULL; + bzf->strm.opaque = NULL; + + while (nUnused > 0) { + bzf->buf[bzf->bufN] = *((UChar*)(unused)); bzf->bufN++; + unused = ((void*)( 1 + ((UChar*)(unused)) )); + nUnused--; + } + + ret = bzDecompressInit ( &(bzf->strm), verbosity, small ); + if (ret != BZ_OK) + { BZ_SETERR(ret); free(bzf); return NULL; }; + + bzf->strm.avail_in = bzf->bufN; + bzf->strm.next_in = bzf->buf; + + bzf->initialisedOk = True; + return bzf; +} + + +/*---------------------------------------------------*/ +void BZ_API(bzReadClose) ( int *bzerror, BZFILE *b ) +{ + bzFile* bzf = (bzFile*)b; + + BZ_SETERR(BZ_OK); + if (bzf == NULL) + { BZ_SETERR(BZ_OK); return; }; + + if (bzf->writing) + { BZ_SETERR(BZ_SEQUENCE_ERROR); return; }; + + if (bzf->initialisedOk) + (void)bzDecompressEnd ( &(bzf->strm) ); + free ( bzf ); +} + + +/*---------------------------------------------------*/ +int BZ_API(bzRead) + ( int* bzerror, + BZFILE* b, + void* buf, + int len ) +{ + Int32 n, ret; + bzFile* bzf = (bzFile*)b; + + BZ_SETERR(BZ_OK); + + if (bzf == NULL || buf == NULL || len < 0) + { BZ_SETERR(BZ_PARAM_ERROR); return 0; }; + + if (bzf->writing) + { BZ_SETERR(BZ_SEQUENCE_ERROR); return 0; }; + + if (len == 0) + { BZ_SETERR(BZ_OK); return 0; }; + + bzf->strm.avail_out = len; + bzf->strm.next_out = buf; + + while (True) { + + if (ferror(bzf->handle)) + { BZ_SETERR(BZ_IO_ERROR); return 0; }; + + if (bzf->strm.avail_in == 0 && !myfeof(bzf->handle)) { + n = fread ( bzf->buf, sizeof(UChar), + BZ_MAX_UNUSED, bzf->handle ); + if (ferror(bzf->handle)) + { BZ_SETERR(BZ_IO_ERROR); return 0; }; + bzf->bufN = n; + bzf->strm.avail_in = bzf->bufN; + bzf->strm.next_in = bzf->buf; + } + + ret = bzDecompress ( &(bzf->strm) ); + + if (ret != BZ_OK && ret != BZ_STREAM_END) + { BZ_SETERR(ret); return 0; }; + + if (ret == BZ_OK && myfeof(bzf->handle) && + bzf->strm.avail_in == 0 && bzf->strm.avail_out > 0) + { BZ_SETERR(BZ_UNEXPECTED_EOF); return 0; }; + + if (ret == BZ_STREAM_END) + { BZ_SETERR(BZ_STREAM_END); + return len - bzf->strm.avail_out; }; + if (bzf->strm.avail_out == 0) + { BZ_SETERR(BZ_OK); return len; }; + + } + + return 0; /*not reached*/ +} + + +/*---------------------------------------------------*/ +void BZ_API(bzReadGetUnused) + ( int* bzerror, + BZFILE* b, + void** unused, + int* nUnused ) +{ + bzFile* bzf = (bzFile*)b; + if (bzf == NULL) + { BZ_SETERR(BZ_PARAM_ERROR); return; }; + if (bzf->lastErr != BZ_STREAM_END) + { BZ_SETERR(BZ_SEQUENCE_ERROR); return; }; + if (unused == NULL || nUnused == NULL) + { BZ_SETERR(BZ_PARAM_ERROR); return; }; + + BZ_SETERR(BZ_OK); + *nUnused = bzf->strm.avail_in; + *unused = bzf->strm.next_in; +} +#endif + + +/*---------------------------------------------------*/ +/*--- Misc convenience stuff ---*/ +/*---------------------------------------------------*/ + +/*---------------------------------------------------*/ +int BZ_API(bzBuffToBuffCompress) + ( char* dest, + unsigned int* destLen, + char* source, + unsigned int sourceLen, + int blockSize100k, + int verbosity, + int workFactor ) +{ + bz_stream strm; + int ret; + + if (dest == NULL || destLen == NULL || + source == NULL || + blockSize100k < 1 || blockSize100k > 9 || + verbosity < 0 || verbosity > 4 || + workFactor < 0 || workFactor > 250) + return BZ_PARAM_ERROR; + + if (workFactor == 0) workFactor = 30; + strm.bzalloc = NULL; + strm.bzfree = NULL; + strm.opaque = NULL; + ret = bzCompressInit ( &strm, blockSize100k, + verbosity, workFactor ); + if (ret != BZ_OK) return ret; + + strm.next_in = source; + strm.next_out = dest; + strm.avail_in = sourceLen; + strm.avail_out = *destLen; + + ret = bzCompress ( &strm, BZ_FINISH ); + if (ret == BZ_FINISH_OK) goto output_overflow; + if (ret != BZ_STREAM_END) goto errhandler; + + /* normal termination */ + *destLen -= strm.avail_out; + bzCompressEnd ( &strm ); + return BZ_OK; + + output_overflow: + bzCompressEnd ( &strm ); + return BZ_OUTBUFF_FULL; + + errhandler: + bzCompressEnd ( &strm ); + return ret; +} + + +/*---------------------------------------------------*/ +int BZ_API(bzBuffToBuffDecompress) + ( char* dest, + unsigned int* destLen, + char* source, + unsigned int sourceLen, + int small, + int verbosity ) +{ + bz_stream strm; + int ret; + + if (dest == NULL || destLen == NULL || + source == NULL || + (small != 0 && small != 1) || + verbosity < 0 || verbosity > 4) + return BZ_PARAM_ERROR; + + strm.bzalloc = NULL; + strm.bzfree = NULL; + strm.opaque = NULL; + ret = bzDecompressInit ( &strm, verbosity, small ); + if (ret != BZ_OK) return ret; + + strm.next_in = source; + strm.next_out = dest; + strm.avail_in = sourceLen; + strm.avail_out = *destLen; + + ret = bzDecompress ( &strm ); + if (ret == BZ_OK) goto output_overflow_or_eof; + if (ret != BZ_STREAM_END) goto errhandler; + + /* normal termination */ + *destLen -= strm.avail_out; + bzDecompressEnd ( &strm ); + return BZ_OK; + + output_overflow_or_eof: + if (strm.avail_out > 0) { + bzDecompressEnd ( &strm ); + return BZ_UNEXPECTED_EOF; + } else { + bzDecompressEnd ( &strm ); + return BZ_OUTBUFF_FULL; + }; + + errhandler: + bzDecompressEnd ( &strm ); + return BZ_SEQUENCE_ERROR; +} + + +/*---------------------------------------------------*/ +/*-- + Code contributed by Yoshioka Tsuneo + (QWF00133@niftyserve.or.jp/tsuneo-y@is.aist-nara.ac.jp), + to support better zlib compatibility. + This code is not _officially_ part of libbzip2 (yet); + I haven't tested it, documented it, or considered the + threading-safeness of it. + If this code breaks, please contact both Yoshioka and me. +--*/ +/*---------------------------------------------------*/ + +/*---------------------------------------------------*/ +/*-- + return version like "0.9.0c". +--*/ +const char * BZ_API(bzlibVersion)(void) +{ + return BZ_VERSION; +} + + +#ifndef BZ_NO_STDIO +/*---------------------------------------------------*/ + +#if defined(_WIN32) || defined(OS2) || defined(MSDOS) +# include +# include +# define SET_BINARY_MODE(file) setmode(fileno(file),O_BINARY) +#else +# define SET_BINARY_MODE(file) +#endif +static +BZFILE * bzopen_or_bzdopen + ( const char *path, /* no use when bzdopen */ + int fd, /* no use when bzdopen */ + const char *mode, + int open_mode) /* bzopen: 0, bzdopen:1 */ +{ + int bzerr; + char unused[BZ_MAX_UNUSED]; + int blockSize100k = 9; + int writing = 0; + char mode2[10] = ""; + FILE *fp = NULL; + BZFILE *bzfp = NULL; + int verbosity = 0; + int workFactor = 30; + int smallMode = 0; + int nUnused = 0; + + if(mode==NULL){return NULL;} + while(*mode){ + switch(*mode){ + case 'r': + writing = 0;break; + case 'w': + writing = 1;break; + case 's': + smallMode = 1;break; + default: + if(isdigit(*mode)){ + blockSize100k = 0; + while(isdigit(*mode)){ + blockSize100k = blockSize100k*10 + *mode-'0'; + mode++; + } + }else{ + /* ignore */ + } + } + mode++; + } + strcat(mode2, writing ? "w" : "r" ); + strcat(mode2,"b"); /* binary mode */ + + if(open_mode==0){ + if(path==NULL || strcmp(path,"")==0){ + fp = (writing ? stdout : stdin); + SET_BINARY_MODE(fp); + }else{ + fp = fopen(path,mode2); + } + }else{ +#ifdef BZ_STRICT_ANSI + fp = NULL; +#else + fp = fdopen(fd,mode2); +#endif + } + if(fp==NULL){return NULL;} + + if(writing){ + bzfp = bzWriteOpen(&bzerr,fp,blockSize100k,verbosity,workFactor); + }else{ + bzfp = bzReadOpen(&bzerr,fp,verbosity,smallMode,unused,nUnused); + } + if(bzfp==NULL){ + if(fp!=stdin && fp!=stdout) fclose(fp); + return NULL; + } + return bzfp; +} + + +/*---------------------------------------------------*/ +/*-- + open file for read or write. + ex) bzopen("file","w9") + case path="" or NULL => use stdin or stdout. +--*/ +BZFILE * BZ_API(bzopen) + ( const char *path, + const char *mode ) +{ + return bzopen_or_bzdopen(path,-1,mode,/*bzopen*/0); +} + + +/*---------------------------------------------------*/ +BZFILE * BZ_API(bzdopen) + ( int fd, + const char *mode ) +{ + return bzopen_or_bzdopen(NULL,fd,mode,/*bzdopen*/1); +} + + +/*---------------------------------------------------*/ +int BZ_API(bzread) (BZFILE* b, void* buf, int len ) +{ + int bzerr, nread; + if (((bzFile*)b)->lastErr == BZ_STREAM_END) return 0; + nread = bzRead(&bzerr,b,buf,len); + if (bzerr == BZ_OK || bzerr == BZ_STREAM_END) { + return nread; + } else { + return -1; + } +} + + +/*---------------------------------------------------*/ +int BZ_API(bzwrite) (BZFILE* b, void* buf, int len ) +{ + int bzerr; + + bzWrite(&bzerr,b,buf,len); + if(bzerr == BZ_OK){ + return len; + }else{ + return -1; + } +} + + +/*---------------------------------------------------*/ +int BZ_API(bzflush) (BZFILE *b) +{ + /* do nothing now... */ + return 0; +} + + +/*---------------------------------------------------*/ +void BZ_API(bzclose) (BZFILE* b) +{ + int bzerr; + FILE *fp = ((bzFile *)b)->handle; + + if(b==NULL){return;} + if(((bzFile*)b)->writing){ + bzWriteClose(&bzerr,b,0,NULL,NULL); + if(bzerr != BZ_OK){ + bzWriteClose(NULL,b,1,NULL,NULL); + } + }else{ + bzReadClose(&bzerr,b); + } + if(fp!=stdin && fp!=stdout){ + fclose(fp); + } +} + + +/*---------------------------------------------------*/ +/*-- + return last error code +--*/ +static char *bzerrorstrings[] = { + "OK" + ,"SEQUENCE_ERROR" + ,"PARAM_ERROR" + ,"MEM_ERROR" + ,"DATA_ERROR" + ,"DATA_ERROR_MAGIC" + ,"IO_ERROR" + ,"UNEXPECTED_EOF" + ,"OUTBUFF_FULL" + ,"???" /* for future */ + ,"???" /* for future */ + ,"???" /* for future */ + ,"???" /* for future */ + ,"???" /* for future */ + ,"???" /* for future */ +}; + + +const char * BZ_API(bzerror) (BZFILE *b, int *errnum) +{ + int err = ((bzFile *)b)->lastErr; + + if(err>0) err = 0; + *errnum = err; + return bzerrorstrings[err*-1]; +} +#endif + + +/*-------------------------------------------------------------*/ +/*--- end bzlib.c ---*/ +/*-------------------------------------------------------------*/ diff --git a/bzlib.h b/bzlib.h new file mode 100644 index 0000000..bb62273 --- /dev/null +++ b/bzlib.h @@ -0,0 +1,299 @@ + +/*-------------------------------------------------------------*/ +/*--- Public header file for the library. ---*/ +/*--- bzlib.h ---*/ +/*-------------------------------------------------------------*/ + +/*-- + This file is a part of bzip2 and/or libbzip2, a program and + library for lossless, block-sorting data compression. + + Copyright (C) 1996-1998 Julian R Seward. All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + 1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + 2. The origin of this software must not be misrepresented; you must + not claim that you wrote the original software. If you use this + software in a product, an acknowledgment in the product + documentation would be appreciated but is not required. + + 3. Altered source versions must be plainly marked as such, and must + not be misrepresented as being the original software. + + 4. The name of the author may not be used to endorse or promote + products derived from this software without specific prior written + permission. + + THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS + OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED + WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY + DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE + GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS + INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, + WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING + NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS + SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + Julian Seward, Guildford, Surrey, UK. + jseward@acm.org + bzip2/libbzip2 version 0.9.0c of 18 October 1998 + + This program is based on (at least) the work of: + Mike Burrows + David Wheeler + Peter Fenwick + Alistair Moffat + Radford Neal + Ian H. Witten + Robert Sedgewick + Jon L. Bentley + + For more information on these sources, see the manual. +--*/ + + +#ifndef _BZLIB_H +#define _BZLIB_H + +#define BZ_RUN 0 +#define BZ_FLUSH 1 +#define BZ_FINISH 2 + +#define BZ_OK 0 +#define BZ_RUN_OK 1 +#define BZ_FLUSH_OK 2 +#define BZ_FINISH_OK 3 +#define BZ_STREAM_END 4 +#define BZ_SEQUENCE_ERROR (-1) +#define BZ_PARAM_ERROR (-2) +#define BZ_MEM_ERROR (-3) +#define BZ_DATA_ERROR (-4) +#define BZ_DATA_ERROR_MAGIC (-5) +#define BZ_IO_ERROR (-6) +#define BZ_UNEXPECTED_EOF (-7) +#define BZ_OUTBUFF_FULL (-8) + +typedef + struct { + char *next_in; + unsigned int avail_in; + unsigned int total_in; + + char *next_out; + unsigned int avail_out; + unsigned int total_out; + + void *state; + + void *(*bzalloc)(void *,int,int); + void (*bzfree)(void *,void *); + void *opaque; + } + bz_stream; + + +#ifndef BZ_IMPORT +#define BZ_EXPORT +#endif + +#ifdef _WIN32 +# include +# include +# ifdef small + /* windows.h define small to char */ +# undef small +# endif +# ifdef BZ_EXPORT +# define BZ_API(func) WINAPI func +# define BZ_EXTERN extern +# else + /* import windows dll dynamically */ +# define BZ_API(func) (WINAPI * func) +# define BZ_EXTERN +# endif +#else +# define BZ_API(func) func +# define BZ_EXTERN extern +#endif + + +/*-- Core (low-level) library functions --*/ + +BZ_EXTERN int BZ_API(bzCompressInit) ( + bz_stream* strm, + int blockSize100k, + int verbosity, + int workFactor + ); + +BZ_EXTERN int BZ_API(bzCompress) ( + bz_stream* strm, + int action + ); + +BZ_EXTERN int BZ_API(bzCompressEnd) ( + bz_stream* strm + ); + +BZ_EXTERN int BZ_API(bzDecompressInit) ( + bz_stream *strm, + int verbosity, + int small + ); + +BZ_EXTERN int BZ_API(bzDecompress) ( + bz_stream* strm + ); + +BZ_EXTERN int BZ_API(bzDecompressEnd) ( + bz_stream *strm + ); + + + +/*-- High(er) level library functions --*/ + +#ifndef BZ_NO_STDIO +#define BZ_MAX_UNUSED 5000 + +typedef void BZFILE; + +BZ_EXTERN BZFILE* BZ_API(bzReadOpen) ( + int* bzerror, + FILE* f, + int verbosity, + int small, + void* unused, + int nUnused + ); + +BZ_EXTERN void BZ_API(bzReadClose) ( + int* bzerror, + BZFILE* b + ); + +BZ_EXTERN void BZ_API(bzReadGetUnused) ( + int* bzerror, + BZFILE* b, + void** unused, + int* nUnused + ); + +BZ_EXTERN int BZ_API(bzRead) ( + int* bzerror, + BZFILE* b, + void* buf, + int len + ); + +BZ_EXTERN BZFILE* BZ_API(bzWriteOpen) ( + int* bzerror, + FILE* f, + int blockSize100k, + int verbosity, + int workFactor + ); + +BZ_EXTERN void BZ_API(bzWrite) ( + int* bzerror, + BZFILE* b, + void* buf, + int len + ); + +BZ_EXTERN void BZ_API(bzWriteClose) ( + int* bzerror, + BZFILE* b, + int abandon, + unsigned int* nbytes_in, + unsigned int* nbytes_out + ); +#endif + + +/*-- Utility functions --*/ + +BZ_EXTERN int BZ_API(bzBuffToBuffCompress) ( + char* dest, + unsigned int* destLen, + char* source, + unsigned int sourceLen, + int blockSize100k, + int verbosity, + int workFactor + ); + +BZ_EXTERN int BZ_API(bzBuffToBuffDecompress) ( + char* dest, + unsigned int* destLen, + char* source, + unsigned int sourceLen, + int small, + int verbosity + ); + + +/*-- + Code contributed by Yoshioka Tsuneo + (QWF00133@niftyserve.or.jp/tsuneo-y@is.aist-nara.ac.jp), + to support better zlib compatibility. + This code is not _officially_ part of libbzip2 (yet); + I haven't tested it, documented it, or considered the + threading-safeness of it. + If this code breaks, please contact both Yoshioka and me. +--*/ + +BZ_EXTERN const char * BZ_API(bzlibVersion) ( + void + ); + +#ifndef BZ_NO_STDIO +BZ_EXTERN BZFILE * BZ_API(bzopen) ( + const char *path, + const char *mode + ); + +BZ_EXTERN BZFILE * BZ_API(bzdopen) ( + int fd, + const char *mode + ); + +BZ_EXTERN int BZ_API(bzread) ( + BZFILE* b, + void* buf, + int len + ); + +BZ_EXTERN int BZ_API(bzwrite) ( + BZFILE* b, + void* buf, + int len + ); + +BZ_EXTERN int BZ_API(bzflush) ( + BZFILE* b + ); + +BZ_EXTERN void BZ_API(bzclose) ( + BZFILE* b + ); + +BZ_EXTERN const char * BZ_API(bzerror) ( + BZFILE *b, + int *errnum + ); +#endif + + +#endif + +/*-------------------------------------------------------------*/ +/*--- end bzlib.h ---*/ +/*-------------------------------------------------------------*/ diff --git a/bzlib_private.h b/bzlib_private.h new file mode 100644 index 0000000..4044aef --- /dev/null +++ b/bzlib_private.h @@ -0,0 +1,523 @@ + +/*-------------------------------------------------------------*/ +/*--- Private header file for the library. ---*/ +/*--- bzlib_private.h ---*/ +/*-------------------------------------------------------------*/ + +/*-- + This file is a part of bzip2 and/or libbzip2, a program and + library for lossless, block-sorting data compression. + + Copyright (C) 1996-1998 Julian R Seward. All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + 1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + 2. The origin of this software must not be misrepresented; you must + not claim that you wrote the original software. If you use this + software in a product, an acknowledgment in the product + documentation would be appreciated but is not required. + + 3. Altered source versions must be plainly marked as such, and must + not be misrepresented as being the original software. + + 4. The name of the author may not be used to endorse or promote + products derived from this software without specific prior written + permission. + + THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS + OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED + WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY + DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE + GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS + INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, + WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING + NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS + SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + Julian Seward, Guildford, Surrey, UK. + jseward@acm.org + bzip2/libbzip2 version 0.9.0c of 18 October 1998 + + This program is based on (at least) the work of: + Mike Burrows + David Wheeler + Peter Fenwick + Alistair Moffat + Radford Neal + Ian H. Witten + Robert Sedgewick + Jon L. Bentley + + For more information on these sources, see the manual. +--*/ + + +#ifndef _BZLIB_PRIVATE_H +#define _BZLIB_PRIVATE_H + +#include + +#ifndef BZ_NO_STDIO +#include +#include +#include +#endif + +#include "bzlib.h" + + + +/*-- General stuff. --*/ + +#define BZ_VERSION "0.9.0c" + +typedef char Char; +typedef unsigned char Bool; +typedef unsigned char UChar; +typedef int Int32; +typedef unsigned int UInt32; +typedef short Int16; +typedef unsigned short UInt16; + +#define True ((Bool)1) +#define False ((Bool)0) + +#ifndef __GNUC__ +#define __inline__ /* */ +#endif + +#ifndef BZ_NO_STDIO +extern void bz__AssertH__fail ( int errcode ); +#define AssertH(cond,errcode) \ + { if (!(cond)) bz__AssertH__fail ( errcode ); } +#if BZ_DEBUG +#define AssertD(cond,msg) \ + { if (!(cond)) { \ + fprintf ( stderr, \ + "\n\nlibbzip2(debug build): internal error\n\t%s\n", msg );\ + exit(1); \ + }} +#else +#define AssertD(cond,msg) /* */ +#endif +#define VPrintf0(zf) \ + fprintf(stderr,zf) +#define VPrintf1(zf,za1) \ + fprintf(stderr,zf,za1) +#define VPrintf2(zf,za1,za2) \ + fprintf(stderr,zf,za1,za2) +#define VPrintf3(zf,za1,za2,za3) \ + fprintf(stderr,zf,za1,za2,za3) +#define VPrintf4(zf,za1,za2,za3,za4) \ + fprintf(stderr,zf,za1,za2,za3,za4) +#define VPrintf5(zf,za1,za2,za3,za4,za5) \ + fprintf(stderr,zf,za1,za2,za3,za4,za5) +#else +extern void bz_internal_error ( int errcode ); +#define AssertH(cond,errcode) \ + { if (!(cond)) bz_internal_error ( errcode ); } +#define AssertD(cond,msg) /* */ +#define VPrintf0(zf) /* */ +#define VPrintf1(zf,za1) /* */ +#define VPrintf2(zf,za1,za2) /* */ +#define VPrintf3(zf,za1,za2,za3) /* */ +#define VPrintf4(zf,za1,za2,za3,za4) /* */ +#define VPrintf5(zf,za1,za2,za3,za4,za5) /* */ +#endif + + +#define BZALLOC(nnn) (strm->bzalloc)(strm->opaque,(nnn),1) +#define BZFREE(ppp) (strm->bzfree)(strm->opaque,(ppp)) + + +/*-- Constants for the back end. --*/ + +#define BZ_MAX_ALPHA_SIZE 258 +#define BZ_MAX_CODE_LEN 23 + +#define BZ_RUNA 0 +#define BZ_RUNB 1 + +#define BZ_N_GROUPS 6 +#define BZ_G_SIZE 50 +#define BZ_N_ITERS 4 + +#define BZ_MAX_SELECTORS (2 + (900000 / BZ_G_SIZE)) + + + +/*-- Stuff for randomising repetitive blocks. --*/ + +extern Int32 rNums[512]; + +#define BZ_RAND_DECLS \ + Int32 rNToGo; \ + Int32 rTPos \ + +#define BZ_RAND_INIT_MASK \ + s->rNToGo = 0; \ + s->rTPos = 0 \ + +#define BZ_RAND_MASK ((s->rNToGo == 1) ? 1 : 0) + +#define BZ_RAND_UPD_MASK \ + if (s->rNToGo == 0) { \ + s->rNToGo = rNums[s->rTPos]; \ + s->rTPos++; \ + if (s->rTPos == 512) s->rTPos = 0; \ + } \ + s->rNToGo--; + + + +/*-- Stuff for doing CRCs. --*/ + +extern UInt32 crc32Table[256]; + +#define BZ_INITIALISE_CRC(crcVar) \ +{ \ + crcVar = 0xffffffffL; \ +} + +#define BZ_FINALISE_CRC(crcVar) \ +{ \ + crcVar = ~(crcVar); \ +} + +#define BZ_UPDATE_CRC(crcVar,cha) \ +{ \ + crcVar = (crcVar << 8) ^ \ + crc32Table[(crcVar >> 24) ^ \ + ((UChar)cha)]; \ +} + + + +/*-- States and modes for compression. --*/ + +#define BZ_M_IDLE 1 +#define BZ_M_RUNNING 2 +#define BZ_M_FLUSHING 3 +#define BZ_M_FINISHING 4 + +#define BZ_S_OUTPUT 1 +#define BZ_S_INPUT 2 + +#define BZ_NUM_OVERSHOOT_BYTES 20 + + + +/*-- Structure holding all the compression-side stuff. --*/ + +typedef + struct { + /* pointer back to the struct bz_stream */ + bz_stream* strm; + + /* mode this stream is in, and whether inputting */ + /* or outputting data */ + Int32 mode; + Int32 state; + + /* remembers avail_in when flush/finish requested */ + UInt32 avail_in_expect; + + /* for doing the block sorting */ + UChar* block; + UInt16* quadrant; + UInt32* zptr; + UInt16* szptr; + Int32* ftab; + Int32 workDone; + Int32 workLimit; + Int32 workFactor; + Bool firstAttempt; + Bool blockRandomised; + Int32 origPtr; + + /* run-length-encoding of the input */ + UInt32 state_in_ch; + Int32 state_in_len; + BZ_RAND_DECLS; + + /* input and output limits and current posns */ + Int32 nblock; + Int32 nblockMAX; + Int32 numZ; + Int32 state_out_pos; + + /* map of bytes used in block */ + Int32 nInUse; + Bool inUse[256]; + UChar unseqToSeq[256]; + + /* the buffer for bit stream creation */ + UInt32 bsBuff; + Int32 bsLive; + + /* block and combined CRCs */ + UInt32 blockCRC; + UInt32 combinedCRC; + + /* misc administratium */ + Int32 verbosity; + Int32 blockNo; + Int32 nBlocksRandomised; + Int32 blockSize100k; + + /* stuff for coding the MTF values */ + Int32 nMTF; + Int32 mtfFreq [BZ_MAX_ALPHA_SIZE]; + UChar selector [BZ_MAX_SELECTORS]; + UChar selectorMtf[BZ_MAX_SELECTORS]; + + UChar len [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE]; + Int32 code [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE]; + Int32 rfreq[BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE]; + + } + EState; + + + +/*-- externs for compression. --*/ + +extern void +blockSort ( EState* ); + +extern void +compressBlock ( EState*, Bool ); + +extern void +bsInitWrite ( EState* ); + +extern void +hbAssignCodes ( Int32*, UChar*, Int32, Int32, Int32 ); + +extern void +hbMakeCodeLengths ( UChar*, Int32*, Int32, Int32 ); + + + +/*-- states for decompression. --*/ + +#define BZ_X_IDLE 1 +#define BZ_X_OUTPUT 2 + +#define BZ_X_MAGIC_1 10 +#define BZ_X_MAGIC_2 11 +#define BZ_X_MAGIC_3 12 +#define BZ_X_MAGIC_4 13 +#define BZ_X_BLKHDR_1 14 +#define BZ_X_BLKHDR_2 15 +#define BZ_X_BLKHDR_3 16 +#define BZ_X_BLKHDR_4 17 +#define BZ_X_BLKHDR_5 18 +#define BZ_X_BLKHDR_6 19 +#define BZ_X_BCRC_1 20 +#define BZ_X_BCRC_2 21 +#define BZ_X_BCRC_3 22 +#define BZ_X_BCRC_4 23 +#define BZ_X_RANDBIT 24 +#define BZ_X_ORIGPTR_1 25 +#define BZ_X_ORIGPTR_2 26 +#define BZ_X_ORIGPTR_3 27 +#define BZ_X_MAPPING_1 28 +#define BZ_X_MAPPING_2 29 +#define BZ_X_SELECTOR_1 30 +#define BZ_X_SELECTOR_2 31 +#define BZ_X_SELECTOR_3 32 +#define BZ_X_CODING_1 33 +#define BZ_X_CODING_2 34 +#define BZ_X_CODING_3 35 +#define BZ_X_MTF_1 36 +#define BZ_X_MTF_2 37 +#define BZ_X_MTF_3 38 +#define BZ_X_MTF_4 39 +#define BZ_X_MTF_5 40 +#define BZ_X_MTF_6 41 +#define BZ_X_ENDHDR_2 42 +#define BZ_X_ENDHDR_3 43 +#define BZ_X_ENDHDR_4 44 +#define BZ_X_ENDHDR_5 45 +#define BZ_X_ENDHDR_6 46 +#define BZ_X_CCRC_1 47 +#define BZ_X_CCRC_2 48 +#define BZ_X_CCRC_3 49 +#define BZ_X_CCRC_4 50 + + + +/*-- Constants for the fast MTF decoder. --*/ + +#define MTFA_SIZE 4096 +#define MTFL_SIZE 16 + + + +/*-- Structure holding all the decompression-side stuff. --*/ + +typedef + struct { + /* pointer back to the struct bz_stream */ + bz_stream* strm; + + /* state indicator for this stream */ + Int32 state; + + /* for doing the final run-length decoding */ + UChar state_out_ch; + Int32 state_out_len; + Bool blockRandomised; + BZ_RAND_DECLS; + + /* the buffer for bit stream reading */ + UInt32 bsBuff; + Int32 bsLive; + + /* misc administratium */ + Int32 blockSize100k; + Bool smallDecompress; + Int32 currBlockNo; + Int32 verbosity; + + /* for undoing the Burrows-Wheeler transform */ + Int32 origPtr; + UInt32 tPos; + Int32 k0; + Int32 unzftab[256]; + Int32 nblock_used; + Int32 cftab[257]; + Int32 cftabCopy[257]; + + /* for undoing the Burrows-Wheeler transform (FAST) */ + UInt32 *tt; + + /* for undoing the Burrows-Wheeler transform (SMALL) */ + UInt16 *ll16; + UChar *ll4; + + /* stored and calculated CRCs */ + UInt32 storedBlockCRC; + UInt32 storedCombinedCRC; + UInt32 calculatedBlockCRC; + UInt32 calculatedCombinedCRC; + + /* map of bytes used in block */ + Int32 nInUse; + Bool inUse[256]; + Bool inUse16[16]; + UChar seqToUnseq[256]; + + /* for decoding the MTF values */ + UChar mtfa [MTFA_SIZE]; + Int32 mtfbase[256 / MTFL_SIZE]; + UChar selector [BZ_MAX_SELECTORS]; + UChar selectorMtf[BZ_MAX_SELECTORS]; + UChar len [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE]; + + Int32 limit [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE]; + Int32 base [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE]; + Int32 perm [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE]; + Int32 minLens[BZ_N_GROUPS]; + + /* save area for scalars in the main decompress code */ + Int32 save_i; + Int32 save_j; + Int32 save_t; + Int32 save_alphaSize; + Int32 save_nGroups; + Int32 save_nSelectors; + Int32 save_EOB; + Int32 save_groupNo; + Int32 save_groupPos; + Int32 save_nextSym; + Int32 save_nblockMAX; + Int32 save_nblock; + Int32 save_es; + Int32 save_N; + Int32 save_curr; + Int32 save_zt; + Int32 save_zn; + Int32 save_zvec; + Int32 save_zj; + Int32 save_gSel; + Int32 save_gMinlen; + Int32* save_gLimit; + Int32* save_gBase; + Int32* save_gPerm; + + } + DState; + + + +/*-- Macros for decompression. --*/ + +#define BZ_GET_FAST(cccc) \ + s->tPos = s->tt[s->tPos]; \ + cccc = (UChar)(s->tPos & 0xff); \ + s->tPos >>= 8; + +#define BZ_GET_FAST_C(cccc) \ + c_tPos = c_tt[c_tPos]; \ + cccc = (UChar)(c_tPos & 0xff); \ + c_tPos >>= 8; + +#define SET_LL4(i,n) \ + { if (((i) & 0x1) == 0) \ + s->ll4[(i) >> 1] = (s->ll4[(i) >> 1] & 0xf0) | (n); else \ + s->ll4[(i) >> 1] = (s->ll4[(i) >> 1] & 0x0f) | ((n) << 4); \ + } + +#define GET_LL4(i) \ + (((UInt32)(s->ll4[(i) >> 1])) >> (((i) << 2) & 0x4) & 0xF) + +#define SET_LL(i,n) \ + { s->ll16[i] = (UInt16)(n & 0x0000ffff); \ + SET_LL4(i, n >> 16); \ + } + +#define GET_LL(i) \ + (((UInt32)s->ll16[i]) | (GET_LL4(i) << 16)) + +#define BZ_GET_SMALL(cccc) \ + cccc = indexIntoF ( s->tPos, s->cftab ); \ + s->tPos = GET_LL(s->tPos); + + +/*-- externs for decompression. --*/ + +extern Int32 +indexIntoF ( Int32, Int32* ); + +extern Int32 +decompress ( DState* ); + +extern void +hbCreateDecodeTables ( Int32*, Int32*, Int32*, UChar*, + Int32, Int32, Int32 ); + + +#endif + + +/*-- BZ_NO_STDIO seems to make NULL disappear on some platforms. --*/ + +#ifdef BZ_NO_STDIO +#ifndef NULL +#define NULL 0 +#endif +#endif + + +/*-------------------------------------------------------------*/ +/*--- end bzlib_private.h ---*/ +/*-------------------------------------------------------------*/ diff --git a/compress.c b/compress.c new file mode 100644 index 0000000..23abd43 --- /dev/null +++ b/compress.c @@ -0,0 +1,588 @@ + +/*-------------------------------------------------------------*/ +/*--- Compression machinery (not incl block sorting) ---*/ +/*--- compress.c ---*/ +/*-------------------------------------------------------------*/ + +/*-- + This file is a part of bzip2 and/or libbzip2, a program and + library for lossless, block-sorting data compression. + + Copyright (C) 1996-1998 Julian R Seward. All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + 1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + 2. The origin of this software must not be misrepresented; you must + not claim that you wrote the original software. If you use this + software in a product, an acknowledgment in the product + documentation would be appreciated but is not required. + + 3. Altered source versions must be plainly marked as such, and must + not be misrepresented as being the original software. + + 4. The name of the author may not be used to endorse or promote + products derived from this software without specific prior written + permission. + + THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS + OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED + WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY + DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE + GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS + INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, + WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING + NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS + SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + Julian Seward, Guildford, Surrey, UK. + jseward@acm.org + bzip2/libbzip2 version 0.9.0 of 28 June 1998 + + This program is based on (at least) the work of: + Mike Burrows + David Wheeler + Peter Fenwick + Alistair Moffat + Radford Neal + Ian H. Witten + Robert Sedgewick + Jon L. Bentley + + For more information on these sources, see the manual. +--*/ + +/*-- + CHANGES + ~~~~~~~ + 0.9.0 -- original version. + + 0.9.0a/b -- no changes in this file. + + 0.9.0c + * changed setting of nGroups in sendMTFValues() so as to + do a bit better on small files +--*/ + +#include "bzlib_private.h" + + +/*---------------------------------------------------*/ +/*--- Bit stream I/O ---*/ +/*---------------------------------------------------*/ + +/*---------------------------------------------------*/ +void bsInitWrite ( EState* s ) +{ + s->bsLive = 0; + s->bsBuff = 0; +} + + +/*---------------------------------------------------*/ +static +void bsFinishWrite ( EState* s ) +{ + while (s->bsLive > 0) { + ((UChar*)(s->quadrant))[s->numZ] = (UChar)(s->bsBuff >> 24); + s->numZ++; + s->bsBuff <<= 8; + s->bsLive -= 8; + } +} + + +/*---------------------------------------------------*/ +#define bsNEEDW(nz) \ +{ \ + while (s->bsLive >= 8) { \ + ((UChar*)(s->quadrant))[s->numZ] \ + = (UChar)(s->bsBuff >> 24); \ + s->numZ++; \ + s->bsBuff <<= 8; \ + s->bsLive -= 8; \ + } \ +} + + +/*---------------------------------------------------*/ +static +void bsW ( EState* s, Int32 n, UInt32 v ) +{ + bsNEEDW ( n ); + s->bsBuff |= (v << (32 - s->bsLive - n)); + s->bsLive += n; +} + + +/*---------------------------------------------------*/ +static +void bsPutUInt32 ( EState* s, UInt32 u ) +{ + bsW ( s, 8, (u >> 24) & 0xffL ); + bsW ( s, 8, (u >> 16) & 0xffL ); + bsW ( s, 8, (u >> 8) & 0xffL ); + bsW ( s, 8, u & 0xffL ); +} + + +/*---------------------------------------------------*/ +static +void bsPutUChar ( EState* s, UChar c ) +{ + bsW( s, 8, (UInt32)c ); +} + + +/*---------------------------------------------------*/ +/*--- The back end proper ---*/ +/*---------------------------------------------------*/ + +/*---------------------------------------------------*/ +static +void makeMaps_e ( EState* s ) +{ + Int32 i; + s->nInUse = 0; + for (i = 0; i < 256; i++) + if (s->inUse[i]) { + s->unseqToSeq[i] = s->nInUse; + s->nInUse++; + } +} + + +/*---------------------------------------------------*/ +static +void generateMTFValues ( EState* s ) +{ + UChar yy[256]; + Int32 i, j; + UChar tmp; + UChar tmp2; + Int32 zPend; + Int32 wr; + Int32 EOB; + + makeMaps_e ( s ); + EOB = s->nInUse+1; + + for (i = 0; i <= EOB; i++) s->mtfFreq[i] = 0; + + wr = 0; + zPend = 0; + for (i = 0; i < s->nInUse; i++) yy[i] = (UChar) i; + + for (i = 0; i < s->nblock; i++) { + UChar ll_i; + + AssertD ( wr <= i, "generateMTFValues(1)" ); + j = s->zptr[i]-1; if (j < 0) j += s->nblock; + ll_i = s->unseqToSeq[s->block[j]]; + AssertD ( ll_i < s->nInUse, "generateMTFValues(2a)" ); + + j = 0; + tmp = yy[j]; + while ( ll_i != tmp ) { + j++; + tmp2 = tmp; + tmp = yy[j]; + yy[j] = tmp2; + }; + yy[0] = tmp; + + if (j == 0) { + zPend++; + } else { + if (zPend > 0) { + zPend--; + while (True) { + switch (zPend % 2) { + case 0: s->szptr[wr] = BZ_RUNA; wr++; s->mtfFreq[BZ_RUNA]++; break; + case 1: s->szptr[wr] = BZ_RUNB; wr++; s->mtfFreq[BZ_RUNB]++; break; + }; + if (zPend < 2) break; + zPend = (zPend - 2) / 2; + }; + zPend = 0; + } + s->szptr[wr] = j+1; wr++; s->mtfFreq[j+1]++; + } + } + + if (zPend > 0) { + zPend--; + while (True) { + switch (zPend % 2) { + case 0: s->szptr[wr] = BZ_RUNA; wr++; s->mtfFreq[BZ_RUNA]++; break; + case 1: s->szptr[wr] = BZ_RUNB; wr++; s->mtfFreq[BZ_RUNB]++; break; + }; + if (zPend < 2) break; + zPend = (zPend - 2) / 2; + }; + } + + s->szptr[wr] = EOB; wr++; s->mtfFreq[EOB]++; + + s->nMTF = wr; +} + + +/*---------------------------------------------------*/ +#define BZ_LESSER_ICOST 0 +#define BZ_GREATER_ICOST 15 + +static +void sendMTFValues ( EState* s ) +{ + Int32 v, t, i, j, gs, ge, totc, bt, bc, iter; + Int32 nSelectors, alphaSize, minLen, maxLen, selCtr; + Int32 nGroups, nBytes; + + /*-- + UChar len [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE]; + is a global since the decoder also needs it. + + Int32 code[BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE]; + Int32 rfreq[BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE]; + are also globals only used in this proc. + Made global to keep stack frame size small. + --*/ + + + UInt16 cost[BZ_N_GROUPS]; + Int32 fave[BZ_N_GROUPS]; + + if (s->verbosity >= 3) + VPrintf3( " %d in block, %d after MTF & 1-2 coding, " + "%d+2 syms in use\n", + s->nblock, s->nMTF, s->nInUse ); + + alphaSize = s->nInUse+2; + for (t = 0; t < BZ_N_GROUPS; t++) + for (v = 0; v < alphaSize; v++) + s->len[t][v] = BZ_GREATER_ICOST; + + /*--- Decide how many coding tables to use ---*/ + AssertH ( s->nMTF > 0, 3001 ); + if (s->nMTF < 200) nGroups = 2; else + if (s->nMTF < 600) nGroups = 3; else + if (s->nMTF < 1200) nGroups = 4; else + if (s->nMTF < 2400) nGroups = 5; else + nGroups = 6; + + /*--- Generate an initial set of coding tables ---*/ + { + Int32 nPart, remF, tFreq, aFreq; + + nPart = nGroups; + remF = s->nMTF; + gs = 0; + while (nPart > 0) { + tFreq = remF / nPart; + ge = gs-1; + aFreq = 0; + while (aFreq < tFreq && ge < alphaSize-1) { + ge++; + aFreq += s->mtfFreq[ge]; + } + + if (ge > gs + && nPart != nGroups && nPart != 1 + && ((nGroups-nPart) % 2 == 1)) { + aFreq -= s->mtfFreq[ge]; + ge--; + } + + if (s->verbosity >= 3) + VPrintf5( " initial group %d, [%d .. %d], " + "has %d syms (%4.1f%%)\n", + nPart, gs, ge, aFreq, + (100.0 * (float)aFreq) / (float)(s->nMTF) ); + + for (v = 0; v < alphaSize; v++) + if (v >= gs && v <= ge) + s->len[nPart-1][v] = BZ_LESSER_ICOST; else + s->len[nPart-1][v] = BZ_GREATER_ICOST; + + nPart--; + gs = ge+1; + remF -= aFreq; + } + } + + /*--- + Iterate up to BZ_N_ITERS times to improve the tables. + ---*/ + for (iter = 0; iter < BZ_N_ITERS; iter++) { + + for (t = 0; t < nGroups; t++) fave[t] = 0; + + for (t = 0; t < nGroups; t++) + for (v = 0; v < alphaSize; v++) + s->rfreq[t][v] = 0; + + nSelectors = 0; + totc = 0; + gs = 0; + while (True) { + + /*--- Set group start & end marks. --*/ + if (gs >= s->nMTF) break; + ge = gs + BZ_G_SIZE - 1; + if (ge >= s->nMTF) ge = s->nMTF-1; + + /*-- + Calculate the cost of this group as coded + by each of the coding tables. + --*/ + for (t = 0; t < nGroups; t++) cost[t] = 0; + + if (nGroups == 6) { + register UInt16 cost0, cost1, cost2, cost3, cost4, cost5; + cost0 = cost1 = cost2 = cost3 = cost4 = cost5 = 0; + for (i = gs; i <= ge; i++) { + UInt16 icv = s->szptr[i]; + cost0 += s->len[0][icv]; + cost1 += s->len[1][icv]; + cost2 += s->len[2][icv]; + cost3 += s->len[3][icv]; + cost4 += s->len[4][icv]; + cost5 += s->len[5][icv]; + } + cost[0] = cost0; cost[1] = cost1; cost[2] = cost2; + cost[3] = cost3; cost[4] = cost4; cost[5] = cost5; + } else { + for (i = gs; i <= ge; i++) { + UInt16 icv = s->szptr[i]; + for (t = 0; t < nGroups; t++) cost[t] += s->len[t][icv]; + } + } + + /*-- + Find the coding table which is best for this group, + and record its identity in the selector table. + --*/ + bc = 999999999; bt = -1; + for (t = 0; t < nGroups; t++) + if (cost[t] < bc) { bc = cost[t]; bt = t; }; + totc += bc; + fave[bt]++; + s->selector[nSelectors] = bt; + nSelectors++; + + /*-- + Increment the symbol frequencies for the selected table. + --*/ + for (i = gs; i <= ge; i++) + s->rfreq[bt][ s->szptr[i] ]++; + + gs = ge+1; + } + if (s->verbosity >= 3) { + VPrintf2 ( " pass %d: size is %d, grp uses are ", + iter+1, totc/8 ); + for (t = 0; t < nGroups; t++) + VPrintf1 ( "%d ", fave[t] ); + VPrintf0 ( "\n" ); + } + + /*-- + Recompute the tables based on the accumulated frequencies. + --*/ + for (t = 0; t < nGroups; t++) + hbMakeCodeLengths ( &(s->len[t][0]), &(s->rfreq[t][0]), + alphaSize, 20 ); + } + + + AssertH( nGroups < 8, 3002 ); + AssertH( nSelectors < 32768 && + nSelectors <= (2 + (900000 / BZ_G_SIZE)), + 3003 ); + + + /*--- Compute MTF values for the selectors. ---*/ + { + UChar pos[BZ_N_GROUPS], ll_i, tmp2, tmp; + for (i = 0; i < nGroups; i++) pos[i] = i; + for (i = 0; i < nSelectors; i++) { + ll_i = s->selector[i]; + j = 0; + tmp = pos[j]; + while ( ll_i != tmp ) { + j++; + tmp2 = tmp; + tmp = pos[j]; + pos[j] = tmp2; + }; + pos[0] = tmp; + s->selectorMtf[i] = j; + } + }; + + /*--- Assign actual codes for the tables. --*/ + for (t = 0; t < nGroups; t++) { + minLen = 32; + maxLen = 0; + for (i = 0; i < alphaSize; i++) { + if (s->len[t][i] > maxLen) maxLen = s->len[t][i]; + if (s->len[t][i] < minLen) minLen = s->len[t][i]; + } + AssertH ( !(maxLen > 20), 3004 ); + AssertH ( !(minLen < 1), 3005 ); + hbAssignCodes ( &(s->code[t][0]), &(s->len[t][0]), + minLen, maxLen, alphaSize ); + } + + /*--- Transmit the mapping table. ---*/ + { + Bool inUse16[16]; + for (i = 0; i < 16; i++) { + inUse16[i] = False; + for (j = 0; j < 16; j++) + if (s->inUse[i * 16 + j]) inUse16[i] = True; + } + + nBytes = s->numZ; + for (i = 0; i < 16; i++) + if (inUse16[i]) bsW(s,1,1); else bsW(s,1,0); + + for (i = 0; i < 16; i++) + if (inUse16[i]) + for (j = 0; j < 16; j++) { + if (s->inUse[i * 16 + j]) bsW(s,1,1); else bsW(s,1,0); + } + + if (s->verbosity >= 3) + VPrintf1( " bytes: mapping %d, ", s->numZ-nBytes ); + } + + /*--- Now the selectors. ---*/ + nBytes = s->numZ; + bsW ( s, 3, nGroups ); + bsW ( s, 15, nSelectors ); + for (i = 0; i < nSelectors; i++) { + for (j = 0; j < s->selectorMtf[i]; j++) bsW(s,1,1); + bsW(s,1,0); + } + if (s->verbosity >= 3) + VPrintf1( "selectors %d, ", s->numZ-nBytes ); + + /*--- Now the coding tables. ---*/ + nBytes = s->numZ; + + for (t = 0; t < nGroups; t++) { + Int32 curr = s->len[t][0]; + bsW ( s, 5, curr ); + for (i = 0; i < alphaSize; i++) { + while (curr < s->len[t][i]) { bsW(s,2,2); curr++; /* 10 */ }; + while (curr > s->len[t][i]) { bsW(s,2,3); curr--; /* 11 */ }; + bsW ( s, 1, 0 ); + } + } + + if (s->verbosity >= 3) + VPrintf1 ( "code lengths %d, ", s->numZ-nBytes ); + + /*--- And finally, the block data proper ---*/ + nBytes = s->numZ; + selCtr = 0; + gs = 0; + while (True) { + if (gs >= s->nMTF) break; + ge = gs + BZ_G_SIZE - 1; + if (ge >= s->nMTF) ge = s->nMTF-1; + for (i = gs; i <= ge; i++) { + AssertH ( s->selector[selCtr] < nGroups, 3006 ); + bsW ( s, + s->len [s->selector[selCtr]] [s->szptr[i]], + s->code [s->selector[selCtr]] [s->szptr[i]] ); + } + + gs = ge+1; + selCtr++; + } + AssertH( selCtr == nSelectors, 3007 ); + + if (s->verbosity >= 3) + VPrintf1( "codes %d\n", s->numZ-nBytes ); +} + + +/*---------------------------------------------------*/ +void compressBlock ( EState* s, Bool is_last_block ) +{ + if (s->nblock > 0) { + + BZ_FINALISE_CRC ( s->blockCRC ); + s->combinedCRC = (s->combinedCRC << 1) | (s->combinedCRC >> 31); + s->combinedCRC ^= s->blockCRC; + if (s->blockNo > 1) s->numZ = 0; + + if (s->verbosity >= 2) + VPrintf4( " block %d: crc = 0x%8x, " + "combined CRC = 0x%8x, size = %d\n", + s->blockNo, s->blockCRC, s->combinedCRC, s->nblock ); + + blockSort ( s ); + } + + /*-- If this is the first block, create the stream header. --*/ + if (s->blockNo == 1) { + bsInitWrite ( s ); + bsPutUChar ( s, 'B' ); + bsPutUChar ( s, 'Z' ); + bsPutUChar ( s, 'h' ); + bsPutUChar ( s, '0' + s->blockSize100k ); + } + + if (s->nblock > 0) { + + bsPutUChar ( s, 0x31 ); bsPutUChar ( s, 0x41 ); + bsPutUChar ( s, 0x59 ); bsPutUChar ( s, 0x26 ); + bsPutUChar ( s, 0x53 ); bsPutUChar ( s, 0x59 ); + + /*-- Now the block's CRC, so it is in a known place. --*/ + bsPutUInt32 ( s, s->blockCRC ); + + /*-- Now a single bit indicating randomisation. --*/ + if (s->blockRandomised) { + bsW(s,1,1); s->nBlocksRandomised++; + } else + bsW(s,1,0); + + bsW ( s, 24, s->origPtr ); + generateMTFValues ( s ); + sendMTFValues ( s ); + } + + + /*-- If this is the last block, add the stream trailer. --*/ + if (is_last_block) { + + if (s->verbosity >= 2 && s->nBlocksRandomised > 0) + VPrintf2 ( " %d block%s needed randomisation\n", + s->nBlocksRandomised, + s->nBlocksRandomised == 1 ? "" : "s" ); + + bsPutUChar ( s, 0x17 ); bsPutUChar ( s, 0x72 ); + bsPutUChar ( s, 0x45 ); bsPutUChar ( s, 0x38 ); + bsPutUChar ( s, 0x50 ); bsPutUChar ( s, 0x90 ); + bsPutUInt32 ( s, s->combinedCRC ); + if (s->verbosity >= 2) + VPrintf1( " final combined CRC = 0x%x\n ", s->combinedCRC ); + bsFinishWrite ( s ); + } +} + + +/*-------------------------------------------------------------*/ +/*--- end compress.c ---*/ +/*-------------------------------------------------------------*/ diff --git a/crctable.c b/crctable.c new file mode 100644 index 0000000..2f3eacb --- /dev/null +++ b/crctable.c @@ -0,0 +1,144 @@ + +/*-------------------------------------------------------------*/ +/*--- Table for doing CRCs ---*/ +/*--- crctable.c ---*/ +/*-------------------------------------------------------------*/ + +/*-- + This file is a part of bzip2 and/or libbzip2, a program and + library for lossless, block-sorting data compression. + + Copyright (C) 1996-1998 Julian R Seward. All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + 1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + 2. The origin of this software must not be misrepresented; you must + not claim that you wrote the original software. If you use this + software in a product, an acknowledgment in the product + documentation would be appreciated but is not required. + + 3. Altered source versions must be plainly marked as such, and must + not be misrepresented as being the original software. + + 4. The name of the author may not be used to endorse or promote + products derived from this software without specific prior written + permission. + + THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS + OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED + WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY + DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE + GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS + INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, + WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING + NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS + SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + Julian Seward, Guildford, Surrey, UK. + jseward@acm.org + bzip2/libbzip2 version 0.9.0c of 18 October 1998 + + This program is based on (at least) the work of: + Mike Burrows + David Wheeler + Peter Fenwick + Alistair Moffat + Radford Neal + Ian H. Witten + Robert Sedgewick + Jon L. Bentley + + For more information on these sources, see the manual. +--*/ + + +#include "bzlib_private.h" + +/*-- + I think this is an implementation of the AUTODIN-II, + Ethernet & FDDI 32-bit CRC standard. Vaguely derived + from code by Rob Warnock, in Section 51 of the + comp.compression FAQ. +--*/ + +UInt32 crc32Table[256] = { + + /*-- Ugly, innit? --*/ + + 0x00000000L, 0x04c11db7L, 0x09823b6eL, 0x0d4326d9L, + 0x130476dcL, 0x17c56b6bL, 0x1a864db2L, 0x1e475005L, + 0x2608edb8L, 0x22c9f00fL, 0x2f8ad6d6L, 0x2b4bcb61L, + 0x350c9b64L, 0x31cd86d3L, 0x3c8ea00aL, 0x384fbdbdL, + 0x4c11db70L, 0x48d0c6c7L, 0x4593e01eL, 0x4152fda9L, + 0x5f15adacL, 0x5bd4b01bL, 0x569796c2L, 0x52568b75L, + 0x6a1936c8L, 0x6ed82b7fL, 0x639b0da6L, 0x675a1011L, + 0x791d4014L, 0x7ddc5da3L, 0x709f7b7aL, 0x745e66cdL, + 0x9823b6e0L, 0x9ce2ab57L, 0x91a18d8eL, 0x95609039L, + 0x8b27c03cL, 0x8fe6dd8bL, 0x82a5fb52L, 0x8664e6e5L, + 0xbe2b5b58L, 0xbaea46efL, 0xb7a96036L, 0xb3687d81L, + 0xad2f2d84L, 0xa9ee3033L, 0xa4ad16eaL, 0xa06c0b5dL, + 0xd4326d90L, 0xd0f37027L, 0xddb056feL, 0xd9714b49L, + 0xc7361b4cL, 0xc3f706fbL, 0xceb42022L, 0xca753d95L, + 0xf23a8028L, 0xf6fb9d9fL, 0xfbb8bb46L, 0xff79a6f1L, + 0xe13ef6f4L, 0xe5ffeb43L, 0xe8bccd9aL, 0xec7dd02dL, + 0x34867077L, 0x30476dc0L, 0x3d044b19L, 0x39c556aeL, + 0x278206abL, 0x23431b1cL, 0x2e003dc5L, 0x2ac12072L, + 0x128e9dcfL, 0x164f8078L, 0x1b0ca6a1L, 0x1fcdbb16L, + 0x018aeb13L, 0x054bf6a4L, 0x0808d07dL, 0x0cc9cdcaL, + 0x7897ab07L, 0x7c56b6b0L, 0x71159069L, 0x75d48ddeL, + 0x6b93dddbL, 0x6f52c06cL, 0x6211e6b5L, 0x66d0fb02L, + 0x5e9f46bfL, 0x5a5e5b08L, 0x571d7dd1L, 0x53dc6066L, + 0x4d9b3063L, 0x495a2dd4L, 0x44190b0dL, 0x40d816baL, + 0xaca5c697L, 0xa864db20L, 0xa527fdf9L, 0xa1e6e04eL, + 0xbfa1b04bL, 0xbb60adfcL, 0xb6238b25L, 0xb2e29692L, + 0x8aad2b2fL, 0x8e6c3698L, 0x832f1041L, 0x87ee0df6L, + 0x99a95df3L, 0x9d684044L, 0x902b669dL, 0x94ea7b2aL, + 0xe0b41de7L, 0xe4750050L, 0xe9362689L, 0xedf73b3eL, + 0xf3b06b3bL, 0xf771768cL, 0xfa325055L, 0xfef34de2L, + 0xc6bcf05fL, 0xc27dede8L, 0xcf3ecb31L, 0xcbffd686L, + 0xd5b88683L, 0xd1799b34L, 0xdc3abdedL, 0xd8fba05aL, + 0x690ce0eeL, 0x6dcdfd59L, 0x608edb80L, 0x644fc637L, + 0x7a089632L, 0x7ec98b85L, 0x738aad5cL, 0x774bb0ebL, + 0x4f040d56L, 0x4bc510e1L, 0x46863638L, 0x42472b8fL, + 0x5c007b8aL, 0x58c1663dL, 0x558240e4L, 0x51435d53L, + 0x251d3b9eL, 0x21dc2629L, 0x2c9f00f0L, 0x285e1d47L, + 0x36194d42L, 0x32d850f5L, 0x3f9b762cL, 0x3b5a6b9bL, + 0x0315d626L, 0x07d4cb91L, 0x0a97ed48L, 0x0e56f0ffL, + 0x1011a0faL, 0x14d0bd4dL, 0x19939b94L, 0x1d528623L, + 0xf12f560eL, 0xf5ee4bb9L, 0xf8ad6d60L, 0xfc6c70d7L, + 0xe22b20d2L, 0xe6ea3d65L, 0xeba91bbcL, 0xef68060bL, + 0xd727bbb6L, 0xd3e6a601L, 0xdea580d8L, 0xda649d6fL, + 0xc423cd6aL, 0xc0e2d0ddL, 0xcda1f604L, 0xc960ebb3L, + 0xbd3e8d7eL, 0xb9ff90c9L, 0xb4bcb610L, 0xb07daba7L, + 0xae3afba2L, 0xaafbe615L, 0xa7b8c0ccL, 0xa379dd7bL, + 0x9b3660c6L, 0x9ff77d71L, 0x92b45ba8L, 0x9675461fL, + 0x8832161aL, 0x8cf30badL, 0x81b02d74L, 0x857130c3L, + 0x5d8a9099L, 0x594b8d2eL, 0x5408abf7L, 0x50c9b640L, + 0x4e8ee645L, 0x4a4ffbf2L, 0x470cdd2bL, 0x43cdc09cL, + 0x7b827d21L, 0x7f436096L, 0x7200464fL, 0x76c15bf8L, + 0x68860bfdL, 0x6c47164aL, 0x61043093L, 0x65c52d24L, + 0x119b4be9L, 0x155a565eL, 0x18197087L, 0x1cd86d30L, + 0x029f3d35L, 0x065e2082L, 0x0b1d065bL, 0x0fdc1becL, + 0x3793a651L, 0x3352bbe6L, 0x3e119d3fL, 0x3ad08088L, + 0x2497d08dL, 0x2056cd3aL, 0x2d15ebe3L, 0x29d4f654L, + 0xc5a92679L, 0xc1683bceL, 0xcc2b1d17L, 0xc8ea00a0L, + 0xd6ad50a5L, 0xd26c4d12L, 0xdf2f6bcbL, 0xdbee767cL, + 0xe3a1cbc1L, 0xe760d676L, 0xea23f0afL, 0xeee2ed18L, + 0xf0a5bd1dL, 0xf464a0aaL, 0xf9278673L, 0xfde69bc4L, + 0x89b8fd09L, 0x8d79e0beL, 0x803ac667L, 0x84fbdbd0L, + 0x9abc8bd5L, 0x9e7d9662L, 0x933eb0bbL, 0x97ffad0cL, + 0xafb010b1L, 0xab710d06L, 0xa6322bdfL, 0xa2f33668L, + 0xbcb4666dL, 0xb8757bdaL, 0xb5365d03L, 0xb1f740b4L +}; + + +/*-------------------------------------------------------------*/ +/*--- end crctable.c ---*/ +/*-------------------------------------------------------------*/ diff --git a/decompress.c b/decompress.c new file mode 100644 index 0000000..ac2b0a5 --- /dev/null +++ b/decompress.c @@ -0,0 +1,636 @@ + +/*-------------------------------------------------------------*/ +/*--- Decompression machinery ---*/ +/*--- decompress.c ---*/ +/*-------------------------------------------------------------*/ + +/*-- + This file is a part of bzip2 and/or libbzip2, a program and + library for lossless, block-sorting data compression. + + Copyright (C) 1996-1998 Julian R Seward. All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + 1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + 2. The origin of this software must not be misrepresented; you must + not claim that you wrote the original software. If you use this + software in a product, an acknowledgment in the product + documentation would be appreciated but is not required. + + 3. Altered source versions must be plainly marked as such, and must + not be misrepresented as being the original software. + + 4. The name of the author may not be used to endorse or promote + products derived from this software without specific prior written + permission. + + THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS + OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED + WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY + DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE + GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS + INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, + WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING + NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS + SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + Julian Seward, Guildford, Surrey, UK. + jseward@acm.org + bzip2/libbzip2 version 0.9.0c of 18 October 1998 + + This program is based on (at least) the work of: + Mike Burrows + David Wheeler + Peter Fenwick + Alistair Moffat + Radford Neal + Ian H. Witten + Robert Sedgewick + Jon L. Bentley + + For more information on these sources, see the manual. +--*/ + + +#include "bzlib_private.h" + + +/*---------------------------------------------------*/ +static +void makeMaps_d ( DState* s ) +{ + Int32 i; + s->nInUse = 0; + for (i = 0; i < 256; i++) + if (s->inUse[i]) { + s->seqToUnseq[s->nInUse] = i; + s->nInUse++; + } +} + + +/*---------------------------------------------------*/ +#define RETURN(rrr) \ + { retVal = rrr; goto save_state_and_return; }; + +#define GET_BITS(lll,vvv,nnn) \ + case lll: s->state = lll; \ + while (True) { \ + if (s->bsLive >= nnn) { \ + UInt32 v; \ + v = (s->bsBuff >> \ + (s->bsLive-nnn)) & ((1 << nnn)-1); \ + s->bsLive -= nnn; \ + vvv = v; \ + break; \ + } \ + if (s->strm->avail_in == 0) RETURN(BZ_OK); \ + s->bsBuff \ + = (s->bsBuff << 8) | \ + ((UInt32) \ + (*((UChar*)(s->strm->next_in)))); \ + s->bsLive += 8; \ + s->strm->next_in++; \ + s->strm->avail_in--; \ + s->strm->total_in++; \ + } + +#define GET_UCHAR(lll,uuu) \ + GET_BITS(lll,uuu,8) + +#define GET_BIT(lll,uuu) \ + GET_BITS(lll,uuu,1) + +/*---------------------------------------------------*/ +#define GET_MTF_VAL(label1,label2,lval) \ +{ \ + if (groupPos == 0) { \ + groupNo++; \ + groupPos = BZ_G_SIZE; \ + gSel = s->selector[groupNo]; \ + gMinlen = s->minLens[gSel]; \ + gLimit = &(s->limit[gSel][0]); \ + gPerm = &(s->perm[gSel][0]); \ + gBase = &(s->base[gSel][0]); \ + } \ + groupPos--; \ + zn = gMinlen; \ + GET_BITS(label1, zvec, zn); \ + while (zvec > gLimit[zn]) { \ + zn++; \ + GET_BIT(label2, zj); \ + zvec = (zvec << 1) | zj; \ + }; \ + lval = gPerm[zvec - gBase[zn]]; \ +} + + +/*---------------------------------------------------*/ +Int32 decompress ( DState* s ) +{ + UChar uc; + Int32 retVal; + Int32 minLen, maxLen; + bz_stream* strm = s->strm; + + /* stuff that needs to be saved/restored */ + Int32 i ; + Int32 j; + Int32 t; + Int32 alphaSize; + Int32 nGroups; + Int32 nSelectors; + Int32 EOB; + Int32 groupNo; + Int32 groupPos; + Int32 nextSym; + Int32 nblockMAX; + Int32 nblock; + Int32 es; + Int32 N; + Int32 curr; + Int32 zt; + Int32 zn; + Int32 zvec; + Int32 zj; + Int32 gSel; + Int32 gMinlen; + Int32* gLimit; + Int32* gBase; + Int32* gPerm; + + if (s->state == BZ_X_MAGIC_1) { + /*initialise the save area*/ + s->save_i = 0; + s->save_j = 0; + s->save_t = 0; + s->save_alphaSize = 0; + s->save_nGroups = 0; + s->save_nSelectors = 0; + s->save_EOB = 0; + s->save_groupNo = 0; + s->save_groupPos = 0; + s->save_nextSym = 0; + s->save_nblockMAX = 0; + s->save_nblock = 0; + s->save_es = 0; + s->save_N = 0; + s->save_curr = 0; + s->save_zt = 0; + s->save_zn = 0; + s->save_zvec = 0; + s->save_zj = 0; + s->save_gSel = 0; + s->save_gMinlen = 0; + s->save_gLimit = NULL; + s->save_gBase = NULL; + s->save_gPerm = NULL; + } + + /*restore from the save area*/ + i = s->save_i; + j = s->save_j; + t = s->save_t; + alphaSize = s->save_alphaSize; + nGroups = s->save_nGroups; + nSelectors = s->save_nSelectors; + EOB = s->save_EOB; + groupNo = s->save_groupNo; + groupPos = s->save_groupPos; + nextSym = s->save_nextSym; + nblockMAX = s->save_nblockMAX; + nblock = s->save_nblock; + es = s->save_es; + N = s->save_N; + curr = s->save_curr; + zt = s->save_zt; + zn = s->save_zn; + zvec = s->save_zvec; + zj = s->save_zj; + gSel = s->save_gSel; + gMinlen = s->save_gMinlen; + gLimit = s->save_gLimit; + gBase = s->save_gBase; + gPerm = s->save_gPerm; + + retVal = BZ_OK; + + switch (s->state) { + + GET_UCHAR(BZ_X_MAGIC_1, uc); + if (uc != 'B') RETURN(BZ_DATA_ERROR_MAGIC); + + GET_UCHAR(BZ_X_MAGIC_2, uc); + if (uc != 'Z') RETURN(BZ_DATA_ERROR_MAGIC); + + GET_UCHAR(BZ_X_MAGIC_3, uc) + if (uc != 'h') RETURN(BZ_DATA_ERROR_MAGIC); + + GET_BITS(BZ_X_MAGIC_4, s->blockSize100k, 8) + if (s->blockSize100k < '1' || + s->blockSize100k > '9') RETURN(BZ_DATA_ERROR_MAGIC); + s->blockSize100k -= '0'; + + if (s->smallDecompress) { + s->ll16 = BZALLOC( s->blockSize100k * 100000 * sizeof(UInt16) ); + s->ll4 = BZALLOC( + ((1 + s->blockSize100k * 100000) >> 1) * sizeof(UChar) + ); + if (s->ll16 == NULL || s->ll4 == NULL) RETURN(BZ_MEM_ERROR); + } else { + s->tt = BZALLOC( s->blockSize100k * 100000 * sizeof(Int32) ); + if (s->tt == NULL) RETURN(BZ_MEM_ERROR); + } + + GET_UCHAR(BZ_X_BLKHDR_1, uc); + + if (uc == 0x17) goto endhdr_2; + if (uc != 0x31) RETURN(BZ_DATA_ERROR); + GET_UCHAR(BZ_X_BLKHDR_2, uc); + if (uc != 0x41) RETURN(BZ_DATA_ERROR); + GET_UCHAR(BZ_X_BLKHDR_3, uc); + if (uc != 0x59) RETURN(BZ_DATA_ERROR); + GET_UCHAR(BZ_X_BLKHDR_4, uc); + if (uc != 0x26) RETURN(BZ_DATA_ERROR); + GET_UCHAR(BZ_X_BLKHDR_5, uc); + if (uc != 0x53) RETURN(BZ_DATA_ERROR); + GET_UCHAR(BZ_X_BLKHDR_6, uc); + if (uc != 0x59) RETURN(BZ_DATA_ERROR); + + s->currBlockNo++; + if (s->verbosity >= 2) + VPrintf1 ( "\n [%d: huff+mtf ", s->currBlockNo ); + + s->storedBlockCRC = 0; + GET_UCHAR(BZ_X_BCRC_1, uc); + s->storedBlockCRC = (s->storedBlockCRC << 8) | ((UInt32)uc); + GET_UCHAR(BZ_X_BCRC_2, uc); + s->storedBlockCRC = (s->storedBlockCRC << 8) | ((UInt32)uc); + GET_UCHAR(BZ_X_BCRC_3, uc); + s->storedBlockCRC = (s->storedBlockCRC << 8) | ((UInt32)uc); + GET_UCHAR(BZ_X_BCRC_4, uc); + s->storedBlockCRC = (s->storedBlockCRC << 8) | ((UInt32)uc); + + GET_BITS(BZ_X_RANDBIT, s->blockRandomised, 1); + + s->origPtr = 0; + GET_UCHAR(BZ_X_ORIGPTR_1, uc); + s->origPtr = (s->origPtr << 8) | ((Int32)uc); + GET_UCHAR(BZ_X_ORIGPTR_2, uc); + s->origPtr = (s->origPtr << 8) | ((Int32)uc); + GET_UCHAR(BZ_X_ORIGPTR_3, uc); + s->origPtr = (s->origPtr << 8) | ((Int32)uc); + + /*--- Receive the mapping table ---*/ + for (i = 0; i < 16; i++) { + GET_BIT(BZ_X_MAPPING_1, uc); + if (uc == 1) + s->inUse16[i] = True; else + s->inUse16[i] = False; + } + + for (i = 0; i < 256; i++) s->inUse[i] = False; + + for (i = 0; i < 16; i++) + if (s->inUse16[i]) + for (j = 0; j < 16; j++) { + GET_BIT(BZ_X_MAPPING_2, uc); + if (uc == 1) s->inUse[i * 16 + j] = True; + } + makeMaps_d ( s ); + alphaSize = s->nInUse+2; + + /*--- Now the selectors ---*/ + GET_BITS(BZ_X_SELECTOR_1, nGroups, 3); + GET_BITS(BZ_X_SELECTOR_2, nSelectors, 15); + for (i = 0; i < nSelectors; i++) { + j = 0; + while (True) { + GET_BIT(BZ_X_SELECTOR_3, uc); + if (uc == 0) break; + j++; + if (j > 5) RETURN(BZ_DATA_ERROR); + } + s->selectorMtf[i] = j; + } + + /*--- Undo the MTF values for the selectors. ---*/ + { + UChar pos[BZ_N_GROUPS], tmp, v; + for (v = 0; v < nGroups; v++) pos[v] = v; + + for (i = 0; i < nSelectors; i++) { + v = s->selectorMtf[i]; + tmp = pos[v]; + while (v > 0) { pos[v] = pos[v-1]; v--; } + pos[0] = tmp; + s->selector[i] = tmp; + } + } + + /*--- Now the coding tables ---*/ + for (t = 0; t < nGroups; t++) { + GET_BITS(BZ_X_CODING_1, curr, 5); + for (i = 0; i < alphaSize; i++) { + while (True) { + if (curr < 1 || curr > 20) RETURN(BZ_DATA_ERROR); + GET_BIT(BZ_X_CODING_2, uc); + if (uc == 0) break; + GET_BIT(BZ_X_CODING_3, uc); + if (uc == 0) curr++; else curr--; + } + s->len[t][i] = curr; + } + } + + /*--- Create the Huffman decoding tables ---*/ + for (t = 0; t < nGroups; t++) { + minLen = 32; + maxLen = 0; + for (i = 0; i < alphaSize; i++) { + if (s->len[t][i] > maxLen) maxLen = s->len[t][i]; + if (s->len[t][i] < minLen) minLen = s->len[t][i]; + } + hbCreateDecodeTables ( + &(s->limit[t][0]), + &(s->base[t][0]), + &(s->perm[t][0]), + &(s->len[t][0]), + minLen, maxLen, alphaSize + ); + s->minLens[t] = minLen; + } + + /*--- Now the MTF values ---*/ + + EOB = s->nInUse+1; + nblockMAX = 100000 * s->blockSize100k; + groupNo = -1; + groupPos = 0; + + for (i = 0; i <= 255; i++) s->unzftab[i] = 0; + + /*-- MTF init --*/ + { + Int32 ii, jj, kk; + kk = MTFA_SIZE-1; + for (ii = 256 / MTFL_SIZE - 1; ii >= 0; ii--) { + for (jj = MTFL_SIZE-1; jj >= 0; jj--) { + s->mtfa[kk] = (UChar)(ii * MTFL_SIZE + jj); + kk--; + } + s->mtfbase[ii] = kk + 1; + } + } + /*-- end MTF init --*/ + + nblock = 0; + + GET_MTF_VAL(BZ_X_MTF_1, BZ_X_MTF_2, nextSym); + + while (True) { + + if (nextSym == EOB) break; + + if (nextSym == BZ_RUNA || nextSym == BZ_RUNB) { + + es = -1; + N = 1; + do { + if (nextSym == BZ_RUNA) es = es + (0+1) * N; else + if (nextSym == BZ_RUNB) es = es + (1+1) * N; + N = N * 2; + GET_MTF_VAL(BZ_X_MTF_3, BZ_X_MTF_4, nextSym); + } + while (nextSym == BZ_RUNA || nextSym == BZ_RUNB); + + es++; + uc = s->seqToUnseq[ s->mtfa[s->mtfbase[0]] ]; + s->unzftab[uc] += es; + + if (s->smallDecompress) + while (es > 0) { + s->ll16[nblock] = (UInt16)uc; + nblock++; + es--; + } + else + while (es > 0) { + s->tt[nblock] = (UInt32)uc; + nblock++; + es--; + }; + + if (nblock > nblockMAX) RETURN(BZ_DATA_ERROR); + continue; + + } else { + + if (nblock > nblockMAX) RETURN(BZ_DATA_ERROR); + + /*-- uc = MTF ( nextSym-1 ) --*/ + { + Int32 ii, jj, kk, pp, lno, off; + UInt32 nn; + nn = (UInt32)(nextSym - 1); + + if (nn < MTFL_SIZE) { + /* avoid general-case expense */ + pp = s->mtfbase[0]; + uc = s->mtfa[pp+nn]; + while (nn > 3) { + Int32 z = pp+nn; + s->mtfa[(z) ] = s->mtfa[(z)-1]; + s->mtfa[(z)-1] = s->mtfa[(z)-2]; + s->mtfa[(z)-2] = s->mtfa[(z)-3]; + s->mtfa[(z)-3] = s->mtfa[(z)-4]; + nn -= 4; + } + while (nn > 0) { + s->mtfa[(pp+nn)] = s->mtfa[(pp+nn)-1]; nn--; + }; + s->mtfa[pp] = uc; + } else { + /* general case */ + lno = nn / MTFL_SIZE; + off = nn % MTFL_SIZE; + pp = s->mtfbase[lno] + off; + uc = s->mtfa[pp]; + while (pp > s->mtfbase[lno]) { + s->mtfa[pp] = s->mtfa[pp-1]; pp--; + }; + s->mtfbase[lno]++; + while (lno > 0) { + s->mtfbase[lno]--; + s->mtfa[s->mtfbase[lno]] + = s->mtfa[s->mtfbase[lno-1] + MTFL_SIZE - 1]; + lno--; + } + s->mtfbase[0]--; + s->mtfa[s->mtfbase[0]] = uc; + if (s->mtfbase[0] == 0) { + kk = MTFA_SIZE-1; + for (ii = 256 / MTFL_SIZE-1; ii >= 0; ii--) { + for (jj = MTFL_SIZE-1; jj >= 0; jj--) { + s->mtfa[kk] = s->mtfa[s->mtfbase[ii] + jj]; + kk--; + } + s->mtfbase[ii] = kk + 1; + } + } + } + } + /*-- end uc = MTF ( nextSym-1 ) --*/ + + s->unzftab[s->seqToUnseq[uc]]++; + if (s->smallDecompress) + s->ll16[nblock] = (UInt16)(s->seqToUnseq[uc]); else + s->tt[nblock] = (UInt32)(s->seqToUnseq[uc]); + nblock++; + + GET_MTF_VAL(BZ_X_MTF_5, BZ_X_MTF_6, nextSym); + continue; + } + } + + s->state_out_len = 0; + s->state_out_ch = 0; + BZ_INITIALISE_CRC ( s->calculatedBlockCRC ); + s->state = BZ_X_OUTPUT; + if (s->verbosity >= 2) VPrintf0 ( "rt+rld" ); + + /*-- Set up cftab to facilitate generation of T^(-1) --*/ + s->cftab[0] = 0; + for (i = 1; i <= 256; i++) s->cftab[i] = s->unzftab[i-1]; + for (i = 1; i <= 256; i++) s->cftab[i] += s->cftab[i-1]; + + if (s->smallDecompress) { + + /*-- Make a copy of cftab, used in generation of T --*/ + for (i = 0; i <= 256; i++) s->cftabCopy[i] = s->cftab[i]; + + /*-- compute the T vector --*/ + for (i = 0; i < nblock; i++) { + uc = (UChar)(s->ll16[i]); + SET_LL(i, s->cftabCopy[uc]); + s->cftabCopy[uc]++; + } + + /*-- Compute T^(-1) by pointer reversal on T --*/ + i = s->origPtr; + j = GET_LL(i); + do { + Int32 tmp = GET_LL(j); + SET_LL(j, i); + i = j; + j = tmp; + } + while (i != s->origPtr); + + s->tPos = s->origPtr; + s->nblock_used = 0; + if (s->blockRandomised) { + BZ_RAND_INIT_MASK; + BZ_GET_SMALL(s->k0); s->nblock_used++; + BZ_RAND_UPD_MASK; s->k0 ^= BZ_RAND_MASK; + } else { + BZ_GET_SMALL(s->k0); s->nblock_used++; + } + + } else { + + /*-- compute the T^(-1) vector --*/ + for (i = 0; i < nblock; i++) { + uc = (UChar)(s->tt[i] & 0xff); + s->tt[s->cftab[uc]] |= (i << 8); + s->cftab[uc]++; + } + + s->tPos = s->tt[s->origPtr] >> 8; + s->nblock_used = 0; + if (s->blockRandomised) { + BZ_RAND_INIT_MASK; + BZ_GET_FAST(s->k0); s->nblock_used++; + BZ_RAND_UPD_MASK; s->k0 ^= BZ_RAND_MASK; + } else { + BZ_GET_FAST(s->k0); s->nblock_used++; + } + + } + + RETURN(BZ_OK); + + + + endhdr_2: + + GET_UCHAR(BZ_X_ENDHDR_2, uc); + if (uc != 0x72) RETURN(BZ_DATA_ERROR); + GET_UCHAR(BZ_X_ENDHDR_3, uc); + if (uc != 0x45) RETURN(BZ_DATA_ERROR); + GET_UCHAR(BZ_X_ENDHDR_4, uc); + if (uc != 0x38) RETURN(BZ_DATA_ERROR); + GET_UCHAR(BZ_X_ENDHDR_5, uc); + if (uc != 0x50) RETURN(BZ_DATA_ERROR); + GET_UCHAR(BZ_X_ENDHDR_6, uc); + if (uc != 0x90) RETURN(BZ_DATA_ERROR); + + s->storedCombinedCRC = 0; + GET_UCHAR(BZ_X_CCRC_1, uc); + s->storedCombinedCRC = (s->storedCombinedCRC << 8) | ((UInt32)uc); + GET_UCHAR(BZ_X_CCRC_2, uc); + s->storedCombinedCRC = (s->storedCombinedCRC << 8) | ((UInt32)uc); + GET_UCHAR(BZ_X_CCRC_3, uc); + s->storedCombinedCRC = (s->storedCombinedCRC << 8) | ((UInt32)uc); + GET_UCHAR(BZ_X_CCRC_4, uc); + s->storedCombinedCRC = (s->storedCombinedCRC << 8) | ((UInt32)uc); + + s->state = BZ_X_IDLE; + RETURN(BZ_STREAM_END); + + default: AssertH ( False, 4001 ); + } + + AssertH ( False, 4002 ); + + save_state_and_return: + + s->save_i = i; + s->save_j = j; + s->save_t = t; + s->save_alphaSize = alphaSize; + s->save_nGroups = nGroups; + s->save_nSelectors = nSelectors; + s->save_EOB = EOB; + s->save_groupNo = groupNo; + s->save_groupPos = groupPos; + s->save_nextSym = nextSym; + s->save_nblockMAX = nblockMAX; + s->save_nblock = nblock; + s->save_es = es; + s->save_N = N; + s->save_curr = curr; + s->save_zt = zt; + s->save_zn = zn; + s->save_zvec = zvec; + s->save_zj = zj; + s->save_gSel = gSel; + s->save_gMinlen = gMinlen; + s->save_gLimit = gLimit; + s->save_gBase = gBase; + s->save_gPerm = gPerm; + + return retVal; +} + + +/*-------------------------------------------------------------*/ +/*--- end decompress.c ---*/ +/*-------------------------------------------------------------*/ diff --git a/dlltest.c b/dlltest.c new file mode 100644 index 0000000..ee81bcd --- /dev/null +++ b/dlltest.c @@ -0,0 +1,163 @@ +/* + minibz2 + libbz2.dll test program. + by Yoshioka Tsuneo(QWF00133@nifty.ne.jp/tsuneo-y@is.aist-nara.ac.jp) + This file is Public Domain. + welcome any email to me. + + usage: minibz2 [-d] [-{1,2,..9}] [[srcfilename] destfilename] +*/ + +#define BZ_IMPORT +#include "bzlib.h" +#include +#include +#ifdef _WIN32 +#include +#endif + + +#ifdef _WIN32 + +#include +static int BZ2DLLLoaded = 0; +static HINSTANCE BZ2DLLhLib; +int BZ2DLLLoadLibrary(void) +{ + HINSTANCE hLib; + + if(BZ2DLLLoaded==1){return 0;} + hLib=LoadLibrary("libbz2.dll"); + if(hLib == NULL){ + puts("Can't load libbz2.dll"); + return -1; + } + BZ2DLLLoaded=1; + BZ2DLLhLib=hLib; + bzlibVersion=GetProcAddress(hLib,"bzlibVersion"); + bzopen=GetProcAddress(hLib,"bzopen"); + bzdopen=GetProcAddress(hLib,"bzdopen"); + bzread=GetProcAddress(hLib,"bzread"); + bzwrite=GetProcAddress(hLib,"bzwrite"); + bzflush=GetProcAddress(hLib,"bzflush"); + bzclose=GetProcAddress(hLib,"bzclose"); + bzerror=GetProcAddress(hLib,"bzerror"); + return 0; + +} +int BZ2DLLFreeLibrary(void) +{ + if(BZ2DLLLoaded==0){return 0;} + FreeLibrary(BZ2DLLhLib); + BZ2DLLLoaded=0; +} +#endif /* WIN32 */ + +void usage(void) +{ + puts("usage: minibz2 [-d] [-{1,2,..9}] [[srcfilename] destfilename]"); +} + +void main(int argc,char *argv[]) +{ + int decompress = 0; + int level = 9; + char *fn_r,*fn_w; + +#ifdef _WIN32 + if(BZ2DLLLoadLibrary()<0){ + puts("can't load dll"); + exit(1); + } +#endif + while(++argv,--argc){ + if(**argv =='-' || **argv=='/'){ + char *p; + + for(p=*argv+1;*p;p++){ + if(*p=='d'){ + decompress = 1; + }else if('1'<=*p && *p<='9'){ + level = *p - '0'; + }else{ + usage(); + exit(1); + } + } + }else{ + break; + } + } + if(argc>=1){ + fn_r = *argv; + argc--;argv++; + }else{ + fn_r = NULL; + } + if(argc>=1){ + fn_w = *argv; + argc--;argv++; + }else{ + fn_w = NULL; + } + { + int len; + char buff[0x1000]; + char mode[10]; + + if(decompress){ + BZFILE *BZ2fp_r; + FILE *fp_w; + + if(fn_w){ + if((fp_w = fopen(fn_w,"wb"))==NULL){ + printf("can't open [%s]\n",fn_w); + perror("reason:"); + exit(1); + } + }else{ + fp_w = stdout; + } + if((BZ2fp_r == NULL && (BZ2fp_r = bzdopen(fileno(stdin),"rb"))==NULL) + || (BZ2fp_r != NULL && (BZ2fp_r = bzopen(fn_r,"rb"))==NULL)){ + printf("can't bz2openstream\n"); + exit(1); + } + while((len=bzread(BZ2fp_r,buff,0x1000))>0){ + fwrite(buff,1,len,fp_w); + } + bzclose(BZ2fp_r); + if(fp_w != stdout) fclose(fp_w); + }else{ + BZFILE *BZ2fp_w; + FILE *fp_r; + + if(fn_r){ + if((fp_r = fopen(fn_r,"rb"))==NULL){ + printf("can't open [%s]\n",fn_r); + perror("reason:"); + exit(1); + } + }else{ + fp_r = stdin; + } + mode[0]='w'; + mode[1] = '0' + level; + mode[2] = '\0'; + + if((fn_w == NULL && (BZ2fp_w = bzdopen(fileno(stdout),mode))==NULL) + || (fn_w !=NULL && (BZ2fp_w = bzopen(fn_w,mode))==NULL)){ + printf("can't bz2openstream\n"); + exit(1); + } + while((len=fread(buff,1,0x1000,fp_r))>0){ + bzwrite(BZ2fp_w,buff,len); + } + bzclose(BZ2fp_w); + if(fp_r!=stdin)fclose(fp_r); + } + } +#ifdef _WIN32 + BZ2DLLFreeLibrary(); +#endif +} diff --git a/dlltest.dsp b/dlltest.dsp new file mode 100644 index 0000000..4b1615e --- /dev/null +++ b/dlltest.dsp @@ -0,0 +1,93 @@ +# Microsoft Developer Studio Project File - Name="dlltest" - Package Owner=<4> +# Microsoft Developer Studio Generated Build File, Format Version 5.00 +# ** ҏWȂł ** + +# TARGTYPE "Win32 (x86) Console Application" 0x0103 + +CFG=dlltest - Win32 Debug +!MESSAGE ͗LҲ̧قł͂܂B ۼުĂނ邽߂ɂ NMAKE gpĂB +!MESSAGE [Ҳ̧ق̴߰] ނgpĎsĂ +!MESSAGE +!MESSAGE NMAKE /f "dlltest.mak". +!MESSAGE +!MESSAGE NMAKE ̎sɍ\wł܂ +!MESSAGE ײݏϸۂ̐ݒ`܂B: +!MESSAGE +!MESSAGE NMAKE /f "dlltest.mak" CFG="dlltest - Win32 Debug" +!MESSAGE +!MESSAGE I”\ Ӱ: +!MESSAGE +!MESSAGE "dlltest - Win32 Release" ("Win32 (x86) Console Application" p) +!MESSAGE "dlltest - Win32 Debug" ("Win32 (x86) Console Application" p) +!MESSAGE + +# Begin Project +# PROP Scc_ProjName "" +# PROP Scc_LocalPath "" +CPP=cl.exe +RSC=rc.exe + +!IF "$(CFG)" == "dlltest - Win32 Release" + +# PROP BASE Use_MFC 0 +# PROP BASE Use_Debug_Libraries 0 +# PROP BASE Output_Dir "Release" +# PROP BASE Intermediate_Dir "Release" +# PROP BASE Target_Dir "" +# PROP Use_MFC 0 +# PROP Use_Debug_Libraries 0 +# PROP Output_Dir "Release" +# PROP Intermediate_Dir "Release" +# PROP Ignore_Export_Lib 0 +# PROP Target_Dir "" +# ADD BASE CPP /nologo /W3 /GX /O2 /D "WIN32" /D "NDEBUG" /D "_CONSOLE" /D "_MBCS" /YX /FD /c +# ADD CPP /nologo /W3 /GX /O2 /D "WIN32" /D "NDEBUG" /D "_CONSOLE" /D "_MBCS" /YX /FD /c +# ADD BASE RSC /l 0x411 /d "NDEBUG" +# ADD RSC /l 0x411 /d "NDEBUG" +BSC32=bscmake.exe +# ADD BASE BSC32 /nologo +# ADD BSC32 /nologo +LINK32=link.exe +# ADD BASE LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:console /machine:I386 +# ADD LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:console /machine:I386 /out:"minibz2.exe" + +!ELSEIF "$(CFG)" == "dlltest - Win32 Debug" + +# PROP BASE Use_MFC 0 +# PROP BASE Use_Debug_Libraries 1 +# PROP BASE Output_Dir "dlltest_" +# PROP BASE Intermediate_Dir "dlltest_" +# PROP BASE Target_Dir "" +# PROP Use_MFC 0 +# PROP Use_Debug_Libraries 1 +# PROP Output_Dir "dlltest_" +# PROP Intermediate_Dir "dlltest_" +# PROP Ignore_Export_Lib 0 +# PROP Target_Dir "" +# ADD BASE CPP /nologo /W3 /Gm /GX /Zi /Od /D "WIN32" /D "_DEBUG" /D "_CONSOLE" /D "_MBCS" /YX /FD /c +# ADD CPP /nologo /W3 /Gm /GX /Zi /Od /D "WIN32" /D "_DEBUG" /D "_CONSOLE" /D "_MBCS" /YX /FD /c +# ADD BASE RSC /l 0x411 /d "_DEBUG" +# ADD RSC /l 0x411 /d "_DEBUG" +BSC32=bscmake.exe +# ADD BASE BSC32 /nologo +# ADD BSC32 /nologo +LINK32=link.exe +# ADD BASE LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:console /debug /machine:I386 /pdbtype:sept +# ADD LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:console /debug /machine:I386 /out:"minibz2.exe" /pdbtype:sept + +!ENDIF + +# Begin Target + +# Name "dlltest - Win32 Release" +# Name "dlltest - Win32 Debug" +# Begin Source File + +SOURCE=.\bzlib.h +# End Source File +# Begin Source File + +SOURCE=.\dlltest.c +# End Source File +# End Target +# End Project diff --git a/howbig.c b/howbig.c new file mode 100644 index 0000000..9f2ad7c --- /dev/null +++ b/howbig.c @@ -0,0 +1,37 @@ + +#include +#include +#include "bzlib.h" + +unsigned char ibuff[1000000]; +unsigned char obuff[1000000]; + +void doone ( int n ) +{ + int i, j, k, q, nobuff; + q = 0; + + for (k = 0; k < 1; k++) { + for (i = 0; i < n; i++) + ibuff[i] = ((unsigned long)(random())) & 0xff; + nobuff = 1000000; + j = bzBuffToBuffCompress ( obuff, &nobuff, ibuff, n, 9,0,0 ); + assert (j == BZ_OK); + if (nobuff > q) q = nobuff; + } + printf ( "%d %d(%d)\n", n, q, (int)((float)n * 1.01 - (float)q) ); +} + +int main ( int argc, char** argv ) +{ + int i; + i = 0; + while (1) { + if (i >= 900000) break; + doone(i); + if ( (int)(1.10 * i) > i ) + i = (int)(1.10 * i); else i++; + } + + return 0; +} \ No newline at end of file diff --git a/huffman.c b/huffman.c new file mode 100644 index 0000000..8254990 --- /dev/null +++ b/huffman.c @@ -0,0 +1,228 @@ + +/*-------------------------------------------------------------*/ +/*--- Huffman coding low-level stuff ---*/ +/*--- huffman.c ---*/ +/*-------------------------------------------------------------*/ + +/*-- + This file is a part of bzip2 and/or libbzip2, a program and + library for lossless, block-sorting data compression. + + Copyright (C) 1996-1998 Julian R Seward. All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + 1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + 2. The origin of this software must not be misrepresented; you must + not claim that you wrote the original software. If you use this + software in a product, an acknowledgment in the product + documentation would be appreciated but is not required. + + 3. Altered source versions must be plainly marked as such, and must + not be misrepresented as being the original software. + + 4. The name of the author may not be used to endorse or promote + products derived from this software without specific prior written + permission. + + THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS + OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED + WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY + DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE + GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS + INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, + WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING + NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS + SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + Julian Seward, Guildford, Surrey, UK. + jseward@acm.org + bzip2/libbzip2 version 0.9.0c of 18 October 1998 + + This program is based on (at least) the work of: + Mike Burrows + David Wheeler + Peter Fenwick + Alistair Moffat + Radford Neal + Ian H. Witten + Robert Sedgewick + Jon L. Bentley + + For more information on these sources, see the manual. +--*/ + + +#include "bzlib_private.h" + +/*---------------------------------------------------*/ +#define WEIGHTOF(zz0) ((zz0) & 0xffffff00) +#define DEPTHOF(zz1) ((zz1) & 0x000000ff) +#define MYMAX(zz2,zz3) ((zz2) > (zz3) ? (zz2) : (zz3)) + +#define ADDWEIGHTS(zw1,zw2) \ + (WEIGHTOF(zw1)+WEIGHTOF(zw2)) | \ + (1 + MYMAX(DEPTHOF(zw1),DEPTHOF(zw2))) + +#define UPHEAP(z) \ +{ \ + Int32 zz, tmp; \ + zz = z; tmp = heap[zz]; \ + while (weight[tmp] < weight[heap[zz >> 1]]) { \ + heap[zz] = heap[zz >> 1]; \ + zz >>= 1; \ + } \ + heap[zz] = tmp; \ +} + +#define DOWNHEAP(z) \ +{ \ + Int32 zz, yy, tmp; \ + zz = z; tmp = heap[zz]; \ + while (True) { \ + yy = zz << 1; \ + if (yy > nHeap) break; \ + if (yy < nHeap && \ + weight[heap[yy+1]] < weight[heap[yy]]) \ + yy++; \ + if (weight[tmp] < weight[heap[yy]]) break; \ + heap[zz] = heap[yy]; \ + zz = yy; \ + } \ + heap[zz] = tmp; \ +} + + +/*---------------------------------------------------*/ +void hbMakeCodeLengths ( UChar *len, + Int32 *freq, + Int32 alphaSize, + Int32 maxLen ) +{ + /*-- + Nodes and heap entries run from 1. Entry 0 + for both the heap and nodes is a sentinel. + --*/ + Int32 nNodes, nHeap, n1, n2, i, j, k; + Bool tooLong; + + Int32 heap [ BZ_MAX_ALPHA_SIZE + 2 ]; + Int32 weight [ BZ_MAX_ALPHA_SIZE * 2 ]; + Int32 parent [ BZ_MAX_ALPHA_SIZE * 2 ]; + + for (i = 0; i < alphaSize; i++) + weight[i+1] = (freq[i] == 0 ? 1 : freq[i]) << 8; + + while (True) { + + nNodes = alphaSize; + nHeap = 0; + + heap[0] = 0; + weight[0] = 0; + parent[0] = -2; + + for (i = 1; i <= alphaSize; i++) { + parent[i] = -1; + nHeap++; + heap[nHeap] = i; + UPHEAP(nHeap); + } + + AssertH( nHeap < (BZ_MAX_ALPHA_SIZE+2), 2001 ); + + while (nHeap > 1) { + n1 = heap[1]; heap[1] = heap[nHeap]; nHeap--; DOWNHEAP(1); + n2 = heap[1]; heap[1] = heap[nHeap]; nHeap--; DOWNHEAP(1); + nNodes++; + parent[n1] = parent[n2] = nNodes; + weight[nNodes] = ADDWEIGHTS(weight[n1], weight[n2]); + parent[nNodes] = -1; + nHeap++; + heap[nHeap] = nNodes; + UPHEAP(nHeap); + } + + AssertH( nNodes < (BZ_MAX_ALPHA_SIZE * 2), 2002 ); + + tooLong = False; + for (i = 1; i <= alphaSize; i++) { + j = 0; + k = i; + while (parent[k] >= 0) { k = parent[k]; j++; } + len[i-1] = j; + if (j > maxLen) tooLong = True; + } + + if (! tooLong) break; + + for (i = 1; i < alphaSize; i++) { + j = weight[i] >> 8; + j = 1 + (j / 2); + weight[i] = j << 8; + } + } +} + + +/*---------------------------------------------------*/ +void hbAssignCodes ( Int32 *code, + UChar *length, + Int32 minLen, + Int32 maxLen, + Int32 alphaSize ) +{ + Int32 n, vec, i; + + vec = 0; + for (n = minLen; n <= maxLen; n++) { + for (i = 0; i < alphaSize; i++) + if (length[i] == n) { code[i] = vec; vec++; }; + vec <<= 1; + } +} + + +/*---------------------------------------------------*/ +void hbCreateDecodeTables ( Int32 *limit, + Int32 *base, + Int32 *perm, + UChar *length, + Int32 minLen, + Int32 maxLen, + Int32 alphaSize ) +{ + Int32 pp, i, j, vec; + + pp = 0; + for (i = minLen; i <= maxLen; i++) + for (j = 0; j < alphaSize; j++) + if (length[j] == i) { perm[pp] = j; pp++; }; + + for (i = 0; i < BZ_MAX_CODE_LEN; i++) base[i] = 0; + for (i = 0; i < alphaSize; i++) base[length[i]+1]++; + + for (i = 1; i < BZ_MAX_CODE_LEN; i++) base[i] += base[i-1]; + + for (i = 0; i < BZ_MAX_CODE_LEN; i++) limit[i] = 0; + vec = 0; + + for (i = minLen; i <= maxLen; i++) { + vec += (base[i+1] - base[i]); + limit[i] = vec-1; + vec <<= 1; + } + for (i = minLen + 1; i <= maxLen; i++) + base[i] = ((limit[i-1] + 1) << 1) - base[i]; +} + + +/*-------------------------------------------------------------*/ +/*--- end huffman.c ---*/ +/*-------------------------------------------------------------*/ diff --git a/libbz2.def b/libbz2.def new file mode 100644 index 0000000..ba0f54e --- /dev/null +++ b/libbz2.def @@ -0,0 +1,25 @@ +LIBRARY LIBBZ2 +DESCRIPTION "libbzip2: library for data compression" +EXPORTS + bzCompressInit + bzCompress + bzCompressEnd + bzDecompressInit + bzDecompress + bzDecompressEnd + bzReadOpen + bzReadClose + bzReadGetUnused + bzRead + bzWriteOpen + bzWrite + bzWriteClose + bzBuffToBuffCompress + bzBuffToBuffDecompress + bzlibVersion + bzopen + bzdopen + bzread + bzwrite + bzflush + bzclose diff --git a/libbz2.dsp b/libbz2.dsp new file mode 100644 index 0000000..a21a20f --- /dev/null +++ b/libbz2.dsp @@ -0,0 +1,130 @@ +# Microsoft Developer Studio Project File - Name="libbz2" - Package Owner=<4> +# Microsoft Developer Studio Generated Build File, Format Version 5.00 +# ** ҏWȂł ** + +# TARGTYPE "Win32 (x86) Dynamic-Link Library" 0x0102 + +CFG=libbz2 - Win32 Debug +!MESSAGE ͗LҲ̧قł͂܂B ۼުĂނ邽߂ɂ NMAKE gpĂB +!MESSAGE [Ҳ̧ق̴߰] ނgpĎsĂ +!MESSAGE +!MESSAGE NMAKE /f "libbz2.mak". +!MESSAGE +!MESSAGE NMAKE ̎sɍ\wł܂ +!MESSAGE ײݏϸۂ̐ݒ`܂B: +!MESSAGE +!MESSAGE NMAKE /f "libbz2.mak" CFG="libbz2 - Win32 Debug" +!MESSAGE +!MESSAGE I”\ Ӱ: +!MESSAGE +!MESSAGE "libbz2 - Win32 Release" ("Win32 (x86) Dynamic-Link Library" p) +!MESSAGE "libbz2 - Win32 Debug" ("Win32 (x86) Dynamic-Link Library" p) +!MESSAGE + +# Begin Project +# PROP Scc_ProjName "" +# PROP Scc_LocalPath "" +CPP=cl.exe +MTL=midl.exe +RSC=rc.exe + +!IF "$(CFG)" == "libbz2 - Win32 Release" + +# PROP BASE Use_MFC 0 +# PROP BASE Use_Debug_Libraries 0 +# PROP BASE Output_Dir "Release" +# PROP BASE Intermediate_Dir "Release" +# PROP BASE Target_Dir "" +# PROP Use_MFC 0 +# PROP Use_Debug_Libraries 0 +# PROP Output_Dir "Release" +# PROP Intermediate_Dir "Release" +# PROP Ignore_Export_Lib 0 +# PROP Target_Dir "" +# ADD BASE CPP /nologo /MT /W3 /GX /O2 /D "WIN32" /D "NDEBUG" /D "_WINDOWS" /YX /FD /c +# ADD CPP /nologo /MT /W3 /GX /O2 /D "WIN32" /D "NDEBUG" /D "_WINDOWS" /YX /FD /c +# ADD BASE MTL /nologo /D "NDEBUG" /mktyplib203 /o NUL /win32 +# ADD MTL /nologo /D "NDEBUG" /mktyplib203 /o NUL /win32 +# ADD BASE RSC /l 0x411 /d "NDEBUG" +# ADD RSC /l 0x411 /d "NDEBUG" +BSC32=bscmake.exe +# ADD BASE BSC32 /nologo +# ADD BSC32 /nologo +LINK32=link.exe +# ADD BASE LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:windows /dll /machine:I386 +# ADD LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:windows /dll /machine:I386 /out:"libbz2.dll" + +!ELSEIF "$(CFG)" == "libbz2 - Win32 Debug" + +# PROP BASE Use_MFC 0 +# PROP BASE Use_Debug_Libraries 1 +# PROP BASE Output_Dir "Debug" +# PROP BASE Intermediate_Dir "Debug" +# PROP BASE Target_Dir "" +# PROP Use_MFC 0 +# PROP Use_Debug_Libraries 1 +# PROP Output_Dir "Debug" +# PROP Intermediate_Dir "Debug" +# PROP Ignore_Export_Lib 0 +# PROP Target_Dir "" +# ADD BASE CPP /nologo /MTd /W3 /Gm /GX /Zi /Od /D "WIN32" /D "_DEBUG" /D "_WINDOWS" /YX /FD /c +# ADD CPP /nologo /MTd /W3 /Gm /GX /Zi /Od /D "WIN32" /D "_DEBUG" /D "_WINDOWS" /YX /FD /c +# ADD BASE MTL /nologo /D "_DEBUG" /mktyplib203 /o NUL /win32 +# ADD MTL /nologo /D "_DEBUG" /mktyplib203 /o NUL /win32 +# ADD BASE RSC /l 0x411 /d "_DEBUG" +# ADD RSC /l 0x411 /d "_DEBUG" +BSC32=bscmake.exe +# ADD BASE BSC32 /nologo +# ADD BSC32 /nologo +LINK32=link.exe +# ADD BASE LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:windows /dll /debug /machine:I386 /pdbtype:sept +# ADD LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:windows /dll /debug /machine:I386 /out:"libbz2.dll" /pdbtype:sept + +!ENDIF + +# Begin Target + +# Name "libbz2 - Win32 Release" +# Name "libbz2 - Win32 Debug" +# Begin Source File + +SOURCE=.\blocksort.c +# End Source File +# Begin Source File + +SOURCE=.\bzlib.c +# End Source File +# Begin Source File + +SOURCE=.\bzlib.h +# End Source File +# Begin Source File + +SOURCE=.\bzlib_private.h +# End Source File +# Begin Source File + +SOURCE=.\compress.c +# End Source File +# Begin Source File + +SOURCE=.\crctable.c +# End Source File +# Begin Source File + +SOURCE=.\decompress.c +# End Source File +# Begin Source File + +SOURCE=.\huffman.c +# End Source File +# Begin Source File + +SOURCE=.\libbz2.def +# End Source File +# Begin Source File + +SOURCE=.\randtable.c +# End Source File +# End Target +# End Project diff --git a/manual.texi b/manual.texi new file mode 100644 index 0000000..99ce661 --- /dev/null +++ b/manual.texi @@ -0,0 +1,2100 @@ +\input texinfo @c -*- Texinfo -*- +@setfilename bzip2.info + +@ignore +This file documents bzip2 version 0.9.0c, and associated library +libbzip2, written by Julian Seward (jseward@acm.org). + +Copyright (C) 1996-1998 Julian R Seward + +Permission is granted to make and distribute verbatim copies of +this manual provided the copyright notice and this permission notice +are preserved on all copies. + +Permission is granted to copy and distribute translations of this manual +into another language, under the above conditions for verbatim copies. +@end ignore + +@ifinfo +@format +START-INFO-DIR-ENTRY +* Bzip2: (bzip2). A program and library for data compression. +END-INFO-DIR-ENTRY +@end format + +@end ifinfo + +@iftex +@c @finalout +@settitle bzip2 and libbzip2 +@titlepage +@title bzip2 and libbzip2 +@subtitle a program and library for data compression +@subtitle copyright (C) 1996-1998 Julian Seward +@subtitle version 0.9.0c of 18 October 1998 +@author Julian Seward + +@end titlepage +@end iftex + + +@parindent 0mm +@parskip 2mm + + +This program, @code{bzip2}, +and associated library @code{libbzip2}, are +Copyright (C) 1996-1998 Julian R Seward. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +@itemize @bullet +@item + Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. +@item + The origin of this software must not be misrepresented; you must + not claim that you wrote the original software. If you use this + software in a product, an acknowledgment in the product + documentation would be appreciated but is not required. +@item + Altered source versions must be plainly marked as such, and must + not be misrepresented as being the original software. +@item + The name of the author may not be used to endorse or promote + products derived from this software without specific prior written + permission. +@end itemize +THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS +OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY +DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE +GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING +NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +Julian Seward, Guildford, Surrey, UK. + +@code{jseward@@acm.org} + +@code{http://www.muraroa.demon.co.uk} + +@code{bzip2}/@code{libbzip2} version 0.9.0c of 18 October 1998. + +PATENTS: To the best of my knowledge, @code{bzip2} does not use any patented +algorithms. However, I do not have the resources available to carry out +a full patent search. Therefore I cannot give any guarantee of the +above statement. + + + + + + + +@node Overview, Implementation, Top, Top +@chapter Introduction + +@code{bzip2} compresses files using the Burrows-Wheeler +block-sorting text compression algorithm, and Huffman coding. +Compression is generally considerably better than that +achieved by more conventional LZ77/LZ78-based compressors, +and approaches the performance of the PPM family of statistical compressors. + +@code{bzip2} is built on top of @code{libbzip2}, a flexible library +for handling compressed data in the @code{bzip2} format. This manual +describes both how to use the program and +how to work with the library interface. Most of the +manual is devoted to this library, not the program, +which is good news if your interest is only in the program. + +Chapter 2 describes how to use @code{bzip2}; this is the only part +you need to read if you just want to know how to operate the program. +Chapter 3 describes the programming interfaces in detail, and +Chapter 4 records some miscellaneous notes which I thought +ought to be recorded somewhere. + + +@chapter How to use @code{bzip2} + +This chapter contains a copy of the @code{bzip2} man page, +and nothing else. +@example +NAME + bzip2, bunzip2 - a block-sorting file compressor, v0.9.0 + bzcat - decompresses files to stdout + bzip2recover - recovers data from damaged bzip2 files + + +SYNOPSIS + bzip2 [ -cdfkstvzVL123456789 ] [ filenames ... ] + bunzip2 [ -fkvsVL ] [ filenames ... ] + bzcat [ -s ] [ filenames ... ] + bzip2recover filename + + +DESCRIPTION + bzip2 compresses files using the Burrows-Wheeler block- + sorting text compression algorithm, and Huffman coding. + Compression is generally considerably better than that + achieved by more conventional LZ77/LZ78-based compressors, + and approaches the performance of the PPM family of sta- + tistical compressors. + + The command-line options are deliberately very similar to + those of GNU Gzip, but they are not identical. + + bzip2 expects a list of file names to accompany the com- + mand-line flags. Each file is replaced by a compressed + version of itself, with the name "original_name.bz2". + Each compressed file has the same modification date and + permissions as the corresponding original, so that these + properties can be correctly restored at decompression + time. File name handling is naive in the sense that there + is no mechanism for preserving original file names, per- + missions and dates in filesystems which lack these con- + cepts, or have serious file name length restrictions, such + as MS-DOS. + + bzip2 and bunzip2 will by default not overwrite existing + files; if you want this to happen, specify the -f flag. + + If no file names are specified, bzip2 compresses from + standard input to standard output. In this case, bzip2 + will decline to write compressed output to a terminal, as + this would be entirely incomprehensible and therefore + pointless. + + bunzip2 (or bzip2 -d ) decompresses and restores all spec- + ified files whose names end in ".bz2". Files without this + suffix are ignored. Again, supplying no filenames causes + decompression from standard input to standard output. + + bunzip2 will correctly decompress a file which is the con- + catenation of two or more compressed files. The result is + the concatenation of the corresponding uncompressed files. + Integrity testing (-t) of concatenated compressed files is + also supported. + + You can also compress or decompress files to the standard + output by giving the -c flag. Multiple files may be com- + pressed and decompressed like this. The resulting outputs + are fed sequentially to stdout. Compression of multiple + files in this manner generates a stream containing multi- + ple compressed file representations. Such a stream can be + decompressed correctly only by bzip2 version 0.9.0 or + later. Earlier versions of bzip2 will stop after decom- + pressing the first file in the stream. + + bzcat (or bzip2 -dc ) decompresses all specified files to + the standard output. + + Compression is always performed, even if the compressed + file is slightly larger than the original. Files of less + than about one hundred bytes tend to get larger, since the + compression mechanism has a constant overhead in the + region of 50 bytes. Random data (including the output of + most file compressors) is coded at about 8.05 bits per + byte, giving an expansion of around 0.5%. + + As a self-check for your protection, bzip2 uses 32-bit + CRCs to make sure that the decompressed version of a file + is identical to the original. This guards against corrup- + tion of the compressed data, and against undetected bugs + in bzip2 (hopefully very unlikely). The chances of data + corruption going undetected is microscopic, about one + chance in four billion for each file processed. Be aware, + though, that the check occurs upon decompression, so it + can only tell you that that something is wrong. It can't + help you recover the original uncompressed data. You can + use bzip2recover to try to recover data from damaged + files. + + Return values: 0 for a normal exit, 1 for environmental + problems (file not found, invalid flags, I/O errors, &c), + 2 to indicate a corrupt compressed file, 3 for an internal + consistency error (eg, bug) which caused bzip2 to panic. + + +MEMORY MANAGEMENT + Bzip2 compresses large files in blocks. The block size + affects both the compression ratio achieved, and the + amount of memory needed both for compression and decom- + pression. The flags -1 through -9 specify the block size + to be 100,000 bytes through 900,000 bytes (the default) + respectively. At decompression-time, the block size used + for compression is read from the header of the compressed + file, and bunzip2 then allocates itself just enough memory + to decompress the file. Since block sizes are stored in + compressed files, it follows that the flags -1 to -9 are + irrelevant to and so ignored during decompression. + + Compression and decompression requirements, in bytes, can + be estimated as: + + Compression: 400k + ( 7 x block size ) + + Decompression: 100k + ( 4 x block size ), or + 100k + ( 2.5 x block size ) + + Larger block sizes give rapidly diminishing marginal + returns; most of the compression comes from the first two + or three hundred k of block size, a fact worth bearing in + mind when using bzip2 on small machines. It is also + important to appreciate that the decompression memory + requirement is set at compression-time by the choice of + block size. + + For files compressed with the default 900k block size, + bunzip2 will require about 3700 kbytes to decompress. To + support decompression of any file on a 4 megabyte machine, + bunzip2 has an option to decompress using approximately + half this amount of memory, about 2300 kbytes. Decompres- + sion speed is also halved, so you should use this option + only where necessary. The relevant flag is -s. + + In general, try and use the largest block size memory con- + straints allow, since that maximises the compression + achieved. Compression and decompression speed are virtu- + ally unaffected by block size. + + Another significant point applies to files which fit in a + single block -- that means most files you'd encounter + using a large block size. The amount of real memory + touched is proportional to the size of the file, since the + file is smaller than a block. For example, compressing a + file 20,000 bytes long with the flag -9 will cause the + compressor to allocate around 6700k of memory, but only + touch 400k + 20000 * 7 = 540 kbytes of it. Similarly, the + decompressor will allocate 3700k but only touch 100k + + 20000 * 4 = 180 kbytes. + + Here is a table which summarises the maximum memory usage + for different block sizes. Also recorded is the total + compressed size for 14 files of the Calgary Text Compres- + sion Corpus totalling 3,141,622 bytes. This column gives + some feel for how compression varies with block size. + These figures tend to understate the advantage of larger + block sizes for larger files, since the Corpus is domi- + nated by smaller files. + + Compress Decompress Decompress Corpus + Flag usage usage -s usage Size + + -1 1100k 500k 350k 914704 + -2 1800k 900k 600k 877703 + -3 2500k 1300k 850k 860338 + -4 3200k 1700k 1100k 846899 + -5 3900k 2100k 1350k 845160 + -6 4600k 2500k 1600k 838626 + -7 5400k 2900k 1850k 834096 + -8 6000k 3300k 2100k 828642 + -9 6700k 3700k 2350k 828642 + + +OPTIONS + -c --stdout + Compress or decompress to standard output. -c will + decompress multiple files to stdout, but will only + compress a single file to stdout. + + -d --decompress + Force decompression. bzip2, bunzip2 and bzcat are + really the same program, and the decision about + what actions to take is done on the basis of which + name is used. This flag overrides that mechanism, + and forces bzip2 to decompress. + + -z --compress + The complement to -d: forces compression, regard- + less of the invokation name. + + -t --test + Check integrity of the specified file(s), but don't + decompress them. This really performs a trial + decompression and throws away the result. + + -f --force + Force overwrite of output files. Normally, bzip2 + will not overwrite existing output files. + + -k --keep + Keep (don't delete) input files during compression + or decompression. + + -s --small + Reduce memory usage, for compression, decompression + and testing. Files are decompressed and tested + using a modified algorithm which only requires 2.5 + bytes per block byte. This means any file can be + decompressed in 2300k of memory, albeit at about + half the normal speed. + + During compression, -s selects a block size of + 200k, which limits memory use to around the same + figure, at the expense of your compression ratio. + In short, if your machine is low on memory (8 + megabytes or less), use -s for everything. See + MEMORY MANAGEMENT above. + + -v --verbose + Verbose mode -- show the compression ratio for each + file processed. Further -v's increase the ver- + bosity level, spewing out lots of information which + is primarily of interest for diagnostic purposes. + + -L --license -V --version + Display the software version, license terms and + conditions. + + -1 to -9 + Set the block size to 100 k, 200 k .. 900 k when + compressing. Has no effect when decompressing. + See MEMORY MANAGEMENT above. + + --repetitive-fast + bzip2 injects some small pseudo-random variations + into very repetitive blocks to limit worst-case + performance during compression. If sorting runs + into difficulties, the block is randomised, and + sorting is restarted. Very roughly, bzip2 persists + for three times as long as a well-behaved input + would take before resorting to randomisation. This + flag makes it give up much sooner. + + --repetitive-best + Opposite of --repetitive-fast; try a lot harder + before resorting to randomisation. + + +RECOVERING DATA FROM DAMAGED FILES + bzip2 compresses files in blocks, usually 900kbytes long. + Each block is handled independently. If a media or trans- + mission error causes a multi-block .bz2 file to become + damaged, it may be possible to recover data from the + undamaged blocks in the file. + + The compressed representation of each block is delimited + by a 48-bit pattern, which makes it possible to find the + block boundaries with reasonable certainty. Each block + also carries its own 32-bit CRC, so damaged blocks can be + distinguished from undamaged ones. + + bzip2recover is a simple program whose purpose is to + search for blocks in .bz2 files, and write each block out + into its own .bz2 file. You can then use bzip2 -t to test + the integrity of the resulting files, and decompress those + which are undamaged. + + bzip2recover takes a single argument, the name of the dam- + aged file, and writes a number of files "rec0001file.bz2", + "rec0002file.bz2", etc, containing the extracted blocks. + The output filenames are designed so that the use of + wildcards in subsequent processing -- for example, "bzip2 + -dc rec*file.bz2 > recovered_data" -- lists the files in + the "right" order. + + bzip2recover should be of most use dealing with large .bz2 + files, as these will contain many blocks. It is clearly + futile to use it on damaged single-block files, since a + damaged block cannot be recovered. If you wish to min- + imise any potential data loss through media or transmis- + sion errors, you might consider compressing with a smaller + block size. + + +PERFORMANCE NOTES + The sorting phase of compression gathers together similar + strings in the file. Because of this, files containing + very long runs of repeated symbols, like "aabaabaabaab + ..." (repeated several hundred times) may compress + extraordinarily slowly. You can use the -vvvvv option to + monitor progress in great detail, if you want. Decompres- + sion speed is unaffected. + + Such pathological cases seem rare in practice, appearing + mostly in artificially-constructed test files, and in low- + level disk images. It may be inadvisable to use bzip2 to + compress the latter. If you do get a file which causes + severe slowness in compression, try making the block size + as small as possible, with flag -1. + + bzip2 usually allocates several megabytes of memory to + operate in, and then charges all over it in a fairly ran- + dom fashion. This means that performance, both for com- + pressing and decompressing, is largely determined by the + speed at which your machine can service cache misses. + Because of this, small changes to the code to reduce the + miss rate have been observed to give disproportionately + large performance improvements. I imagine bzip2 will per- + form best on machines with very large caches. + + +CAVEATS + I/O error messages are not as helpful as they could be. + Bzip2 tries hard to detect I/O errors and exit cleanly, + but the details of what the problem is sometimes seem + rather misleading. + + This manual page pertains to version 0.9.0 of bzip2. Com- + pressed data created by this version is entirely forwards + and backwards compatible with the previous public release, + version 0.1pl2, but with the following exception: 0.9.0 + can correctly decompress multiple concatenated compressed + files. 0.1pl2 cannot do this; it will stop after decom- + pressing just the first file in the stream. + + Wildcard expansion for Windows 95 and NT is flaky. + + bzip2recover uses 32-bit integers to represent bit posi- + tions in compressed files, so it cannot handle compressed + files more than 512 megabytes long. This could easily be + fixed. + + +AUTHOR + Julian Seward, jseward@@acm.org. + + The ideas embodied in bzip2 are due to (at least) the fol- + lowing people: Michael Burrows and David Wheeler (for the + block sorting transformation), David Wheeler (again, for + the Huffman coder), Peter Fenwick (for the structured cod- + ing model in the original bzip, and many refinements), and + Alistair Moffat, Radford Neal and Ian Witten (for the + arithmetic coder in the original bzip). I am much + indebted for their help, support and advice. See the man- + ual in the source distribution for pointers to sources of + documentation. Christian von Roques encouraged me to look + for faster sorting algorithms, so as to speed up compres- + sion. Bela Lubkin encouraged me to improve the worst-case + compression performance. Many people sent patches, helped + with portability problems, lent machines, gave advice and + were generally helpful. +@end example + + + + + +@chapter Programming with @code{libbzip2} + +This chapter describes the programming interface to @code{libbzip2}. + +For general background information, particularly about memory +use and performance aspects, you'd be well advised to read Chapter 2 +as well. + +@section Top-level structure + +@code{libbzip2} is a flexible library for compressing and decompressing +data in the @code{bzip2} data format. Although packaged as a single +entity, it helps to regard the library as three separate parts: the low +level interface, and the high level interface, and some utility +functions. + +The structure of @code{libbzip2}'s interfaces is similar to +that of Jean-loup Gailly's and Mark Adler's excellent @code{zlib} +library. + +@subsection Low-level summary + +This interface provides services for compressing and decompressing +data in memory. There's no provision for dealing with files, streams +or any other I/O mechanisms, just straight memory-to-memory work. +In fact, this part of the library can be compiled without inclusion +of @code{stdio.h}, which may be helpful for embedded applications. + +The low-level part of the library has no global variables and +is therefore thread-safe. + +Six routines make up the low level interface: +@code{bzCompressInit}, @code{bzCompress}, and @* @code{bzCompressEnd} +for compression, +and a corresponding trio @code{bzDecompressInit}, @* @code{bzDecompress} +and @code{bzDecompressEnd} for decompression. +The @code{*Init} functions allocate +memory for compression/decompression and do other +initialisations, whilst the @code{*End} functions close down operations +and release memory. + +The real work is done by @code{bzCompress} and @code{bzDecompress}. +These compress/decompress data from a user-supplied input buffer +to a user-supplied output buffer. These buffers can be any size; +arbitrary quantities of data are handled by making repeated calls +to these functions. This is a flexible mechanism allowing a +consumer-pull style of activity, or producer-push, or a mixture of +both. + + + +@subsection High-level summary + +This interface provides some handy wrappers around the low-level +interface to facilitate reading and writing @code{bzip2} format +files (@code{.bz2} files). The routines provide hooks to facilitate +reading files in which the @code{bzip2} data stream is embedded +within some larger-scale file structure, or where there are +multiple @code{bzip2} data streams concatenated end-to-end. + +For reading files, @code{bzReadOpen}, @code{bzRead}, @code{bzReadClose} +and @code{bzReadGetUnused} are supplied. For writing files, +@code{bzWriteOpen}, @code{bzWrite} and @code{bzWriteFinish} are +available. + +As with the low-level library, no global variables are used +so the library is per se thread-safe. However, if I/O errors +occur whilst reading or writing the underlying compressed files, +you may have to consult @code{errno} to determine the cause of +the error. In that case, you'd need a C library which correctly +supports @code{errno} in a multithreaded environment. + +To make the library a little simpler and more portable, +@code{bzReadOpen} and @code{bzWriteOpen} require you to pass them file +handles (@code{FILE*}s) which have previously been opened for reading or +writing respectively. That avoids portability problems associated with +file operations and file attributes, whilst not being much of an +imposition on the programmer. + + + +@subsection Utility functions summary +For very simple needs, @code{bzBuffToBuffCompress} and +@code{bzBuffToBuffDecompress} are provided. These compress +data in memory from one buffer to another buffer in a single +function call. You should assess whether these functions +fulfill your memory-to-memory compression/decompression +requirements before investing effort in understanding the more +general but more complex low-level interface. + +Yoshioka Tsuneo (@code{QWF00133@@niftyserve.or.jp} / +@code{tsuneo-y@@is.aist-nara.ac.jp}) has contributed some functions to +give better @code{zlib} compatibility. These functions are +@code{bzopen}, @code{bzread}, @code{bzwrite}, @code{bzflush}, +@code{bzclose}, +@code{bzerror} and @code{bzlibVersion}. You may find these functions +more convenient for simple file reading and writing, than those in the +high-level interface. These functions are not (yet) officially part of +the library, and are not further documented here. If they break, you +get to keep all the pieces. I hope to document them properly when time +permits. + +Yoshioka also contributed modifications to allow the library to be +built as a Windows DLL. + + +@section Error handling + +The library is designed to recover cleanly in all situations, including +the worst-case situation of decompressing random data. I'm not +100% sure that it can always do this, so you might want to add +a signal handler to catch segmentation violations during decompression +if you are feeling especially paranoid. I would be interested in +hearing more about the robustness of the library to corrupted +compressed data. + +The file @code{bzlib.h} contains all definitions needed to use +the library. In particular, you should definitely not include +@code{bzlib_private.h}. + +In @code{bzlib.h}, the various return values are defined. The following +list is not intended as an exhaustive description of the circumstances +in which a given value may be returned -- those descriptions are given +later. Rather, it is intended to convey the rough meaning of each +return value. The first five actions are normal and not intended to +denote an error situation. +@table @code +@item BZ_OK +The requested action was completed successfully. +@item BZ_RUN_OK +@itemx BZ_FLUSH_OK +@itemx BZ_FINISH_OK +In @code{bzCompress}, the requested flush/finish/nothing-special action +was completed successfully. +@item BZ_STREAM_END +Compression of data was completed, or the logical stream end was +detected during decompression. +@end table + +The following return values indicate an error of some kind. +@table @code +@item BZ_SEQUENCE_ERROR +When using the library, it is important to call the functions in the +correct sequence and with data structures (buffers etc) in the correct +states. @code{libbzip2} checks as much as it can to ensure this is +happening, and returns @code{BZ_SEQUENCE_ERROR} if not. Code which +complies precisely with the function semantics, as detailed below, +should never receive this value; such an event denotes buggy code +which you should investigate. +@item BZ_PARAM_ERROR +Returned when a parameter to a function call is out of range +or otherwise manifestly incorrect. As with @code{BZ_SEQUENCE_ERROR}, +this denotes a bug in the client code. The distinction between +@code{BZ_PARAM_ERROR} and @code{BZ_SEQUENCE_ERROR} is a bit hazy, but still worth +making. +@item BZ_MEM_ERROR +Returned when a request to allocate memory failed. Note that the +quantity of memory needed to decompress a stream cannot be determined +until the stream's header has been read. So @code{bzDecompress} and +@code{bzRead} may return @code{BZ_MEM_ERROR} even though some of +the compressed data has been read. The same is not true for +compression; once @code{bzCompressInit} or @code{bzWriteOpen} have +successfully completed, @code{BZ_MEM_ERROR} cannot occur. +@item BZ_DATA_ERROR +Returned when a data integrity error is detected during decompression. +Most importantly, this means when stored and computed CRCs for the +data do not match. This value is also returned upon detection of any +other anomaly in the compressed data. +@item BZ_DATA_ERROR_MAGIC +As a special case of @code{BZ_DATA_ERROR}, it is sometimes useful to +know when the compressed stream does not start with the correct +magic bytes (@code{'B' 'Z' 'h'}). +@item BZ_IO_ERROR +Returned by @code{bzRead} and @code{bzRead} when there is an error +reading or writing in the compressed file, and by @code{bzReadOpen} +and @code{bzWriteOpen} for attempts to use a file for which the +error indicator (viz, @code{ferror(f)}) is set. +On receipt of @code{BZ_IO_ERROR}, the caller should consult +@code{errno} and/or @code{perror} to acquire operating-system +specific information about the problem. +@item BZ_UNEXPECTED_EOF +Returned by @code{bzRead} when the compressed file finishes +before the logical end of stream is detected. +@item BZ_OUTBUFF_FULL +Returned by @code{bzBuffToBuffCompress} and +@code{bzBuffToBuffDecompress} to indicate that the output data +will not fit into the output buffer provided. +@end table + + + +@section Low-level interface + +@subsection @code{bzCompressInit} +@example +typedef + struct @{ + char *next_in; + unsigned int avail_in; + unsigned int total_in; + + char *next_out; + unsigned int avail_out; + unsigned int total_out; + + void *state; + + void *(*bzalloc)(void *,int,int); + void (*bzfree)(void *,void *); + void *opaque; + @} + bz_stream; + +int bzCompressInit ( bz_stream *strm, + int blockSize100k, + int verbosity, + int workFactor ); + +@end example + +Prepares for compression. The @code{bz_stream} structure +holds all data pertaining to the compression activity. +A @code{bz_stream} structure should be allocated and initialised +prior to the call. +The fields of @code{bz_stream} +comprise the entirety of the user-visible data. @code{state} +is a pointer to the private data structures required for compression. + +Custom memory allocators are supported, via fields @code{bzalloc}, +@code{bzfree}, +and @code{opaque}. The value +@code{opaque} is passed to as the first argument to +all calls to @code{bzalloc} and @code{bzfree}, but is +otherwise ignored by the library. +The call @code{bzalloc ( opaque, n, m )} is expected to return a +pointer @code{p} to +@code{n * m} bytes of memory, and @code{bzfree ( opaque, p )} +should free +that memory. + +If you don't want to use a custom memory allocator, set @code{bzalloc}, +@code{bzfree} and +@code{opaque} to @code{NULL}, +and the library will then use the standard @code{malloc}/@code{free} +routines. + +Before calling @code{bzCompressInit}, fields @code{bzalloc}, +@code{bzfree} and @code{opaque} should +be filled appropriately, as just described. Upon return, the internal +state will have been allocated and initialised, and @code{total_in} and +@code{total_out} will have been set to zero. +These last two fields are used by the library +to inform the caller of the total amount of data passed into and out of +the library, respectively. You should not try to change them. + +Parameter @code{blockSize100k} specifies the block size to be used for +compression. It should be a value between 1 and 9 inclusive, and the +actual block size used is 100000 x this figure. 9 gives the best +compression but takes most memory. + +Parameter @code{verbosity} should be set to a number between 0 and 4 +inclusive. 0 is silent, and greater numbers give increasingly verbose +monitoring/debugging output. If the library has been compiled with +@code{-DBZ_NO_STDIO}, no such output will appear for any verbosity +setting. + +Parameter @code{workFactor} controls how the compression phase behaves +when presented with worst case, highly repetitive, input data. +If compression runs into difficulties caused by repetitive data, +some pseudo-random variations are inserted into the block, and +compression is restarted. Lower values of @code{workFactor} +reduce the tolerance of compression to repetitive data. +You should set this parameter carefully; too low, and +compression ratio suffers, too high, and your average-to-worst +case compression times can become very large. +The default value of 30 +gives reasonable behaviour over a wide range of circumstances. + +Allowable values range from 0 to 250 inclusive. 0 is a special +case, equivalent to using the default value of 30. + +Note that the randomisation process is entirely transparent. +If the library decides to randomise and restart compression on a +block, it does so without comment. Randomised blocks are +automatically de-randomised during decompression, so data +integrity is never compromised. + +Possible return values: +@display + @code{BZ_PARAM_ERROR} + if @code{strm} is @code{NULL} + or @code{blockSize} < 1 or @code{blockSize} > 9 + or @code{verbosity} < 0 or @code{verbosity} > 4 + or @code{workFactor} < 0 or @code{workFactor} > 250 + @code{BZ_MEM_ERROR} + if not enough memory is available + @code{BZ_OK} + otherwise +@end display +Allowable next actions: +@display + @code{bzCompress} + if @code{BZ_OK} is returned + no specific action needed in case of error +@end display + +@subsection @code{bzCompress} +@example + int bzCompress ( bz_stream *strm, int action ); +@end example +Provides more input and/or output buffer space for the library. The +caller maintains input and output buffers, and calls @code{bzCompress} to +transfer data between them. + +Before each call to @code{bzCompress}, @code{next_in} should point at +the data to be compressed, and @code{avail_in} should indicate how many +bytes the library may read. @code{bzCompress} updates @code{next_in}, +@code{avail_in} and @code{total_in} to reflect the number of bytes it +has read. + +Similarly, @code{next_out} should point to a buffer in which the +compressed data is to be placed, with @code{avail_out} indicating how +much output space is available. @code{bzCompress} updates +@code{next_out}, @code{avail_out} and @code{total_out} to reflect the +number of bytes output. + +You may provide and remove as little or as much data as you like on each +call of @code{bzCompress}. In the limit, it is acceptable to supply and +remove data one byte at a time, although this would be terribly +inefficient. You should always ensure that at least one byte of output +space is available at each call. + +A second purpose of @code{bzCompress} is to request a change of mode of the +compressed stream. + +Conceptually, a compressed stream can be in one of four states: IDLE, +RUNNING, FLUSHING and FINISHING. Before initialisation +(@code{bzCompressInit}) and after termination (@code{bzCompressEnd}), a +stream is regarded as IDLE. + +Upon initialisation (@code{bzCompressInit}), the stream is placed in the +RUNNING state. Subsequent calls to @code{bzCompress} should pass +@code{BZ_RUN} as the requested action; other actions are illegal and +will result in @code{BZ_SEQUENCE_ERROR}. + +At some point, the calling program will have provided all the input data +it wants to. It will then want to finish up -- in effect, asking the +library to process any data it might have buffered internally. In this +state, @code{bzCompress} will no longer attempt to read data from +@code{next_in}, but it will want to write data to @code{next_out}. +Because the output buffer supplied by the user can be arbitrarily small, +the finishing-up operation cannot necessarily be done with a single call +of @code{bzCompress}. + +Instead, the calling program passes @code{BZ_FINISH} as an action to +@code{bzCompress}. This changes the stream's state to FINISHING. Any +remaining input (ie, @code{next_in[0 .. avail_in-1]}) is compressed and +transferred to the output buffer. To do this, @code{bzCompress} must be +called repeatedly until all the output has been consumed. At that +point, @code{bzCompress} returns @code{BZ_STREAM_END}, and the stream's +state is set back to IDLE. @code{bzCompressEnd} should then be +called. + +Just to make sure the calling program does not cheat, the library makes +a note of @code{avail_in} at the time of the first call to +@code{bzCompress} which has @code{BZ_FINISH} as an action (ie, at the +time the program has announced its intention to not supply any more +input). By comparing this value with that of @code{avail_in} over +subsequent calls to @code{bzCompress}, the library can detect any +attempts to slip in more data to compress. Any calls for which this is +detected will return @code{BZ_SEQUENCE_ERROR}. This indicates a +programming mistake which should be corrected. + +Instead of asking to finish, the calling program may ask +@code{bzCompress} to take all the remaining input, compress it and +terminate the current (Burrows-Wheeler) compression block. This could +be useful for error control purposes. The mechanism is analogous to +that for finishing: call @code{bzCompress} with an action of +@code{BZ_FLUSH}, remove output data, and persist with the +@code{BZ_FLUSH} action until the value @code{BZ_RUN} is returned. As +with finishing, @code{bzCompress} detects any attempt to provide more +input data once the flush has begun. + +Once the flush is complete, the stream returns to the normal RUNNING +state. + +This all sounds pretty complex, but isn't really. Here's a table +which shows which actions are allowable in each state, what action +will be taken, what the next state is, and what the non-error return +values are. Note that you can't explicitly ask what state the +stream is in, but nor do you need to -- it can be inferred from the +values returned by @code{bzCompress}. +@display +IDLE/@code{any} + Illegal. IDLE state only exists after @code{bzCompressEnd} or + before @code{bzCompressInit}. + Return value = @code{BZ_SEQUENCE_ERROR} + +RUNNING/@code{BZ_RUN} + Compress from @code{next_in} to @code{next_out} as much as possible. + Next state = RUNNING + Return value = @code{BZ_RUN_OK} + +RUNNING/@code{BZ_FLUSH} + Remember current value of @code{next_in}. Compress from @code{next_in} + to @code{next_out} as much as possible, but do not accept any more input. + Next state = FLUSHING + Return value = @code{BZ_FLUSH_OK} + +RUNNING/@code{BZ_FINISH} + Remember current value of @code{next_in}. Compress from @code{next_in} + to @code{next_out} as much as possible, but do not accept any more input. + Next state = FINISHING + Return value = @code{BZ_FINISH_OK} + +FLUSHING/@code{BZ_FLUSH} + Compress from @code{next_in} to @code{next_out} as much as possible, + but do not accept any more input. + If all the existing input has been used up and all compressed + output has been removed + Next state = RUNNING; Return value = @code{BZ_RUN_OK} + else + Next state = FLUSHING; Return value = @code{BZ_FLUSH_OK} + +FLUSHING/other + Illegal. + Return value = @code{BZ_SEQUENCE_ERROR} + +FINISHING/@code{BZ_FINISH} + Compress from @code{next_in} to @code{next_out} as much as possible, + but to not accept any more input. + If all the existing input has been used up and all compressed + output has been removed + Next state = IDLE; Return value = @code{BZ_STREAM_END} + else + Next state = FINISHING; Return value = @code{BZ_FINISHING} + +FINISHING/other + Illegal. + Return value = @code{BZ_SEQUENCE_ERROR} +@end display + +That still looks complicated? Well, fair enough. The usual sequence +of calls for compressing a load of data is: +@itemize @bullet +@item Get started with @code{bzCompressInit}. +@item Shovel data in and shlurp out its compressed form using zero or more +calls of @code{bzCompress} with action = @code{BZ_RUN}. +@item Finish up. +Repeatedly call @code{bzCompress} with action = @code{BZ_FINISH}, +copying out the compressed output, until @code{BZ_STREAM_END} is returned. +@item Close up and go home. Call @code{bzCompressEnd}. +@end itemize +If the data you want to compress fits into your input buffer all +at once, you can skip the calls of @code{bzCompress ( ..., BZ_RUN )} and +just do the @code{bzCompress ( ..., BZ_FINISH )} calls. + +All required memory is allocated by @code{bzCompressInit}. The +compression library can accept any data at all (obviously). So you +shouldn't get any error return values from the @code{bzCompress} calls. +If you do, they will be @code{BZ_SEQUENCE_ERROR}, and indicate a bug in +your programming. + +Trivial other possible return values: +@display + @code{BZ_PARAM_ERROR} + if @code{strm} is @code{NULL}, or @code{strm->s} is @code{NULL} +@end display + +@subsection @code{bzCompressEnd} +@example +int bzCompressEnd ( bz_stream *strm ); +@end example +Releases all memory associated with a compression stream. + +Possible return values: +@display + @code{BZ_PARAM_ERROR} if @code{strm} is @code{NULL} or @code{strm->s} is @code{NULL} + @code{BZ_OK} otherwise +@end display + + +@subsection @code{bzDecompressInit} +@example +int bzDecompressInit ( bz_stream *strm, int verbosity, int small ); +@end example +Prepares for decompression. As with @code{bzCompressInit}, a +@code{bz_stream} record should be allocated and initialised before the +call. Fields @code{bzalloc}, @code{bzfree} and @code{opaque} should be +set if a custom memory allocator is required, or made @code{NULL} for +the normal @code{malloc}/@code{free} routines. Upon return, the internal +state will have been initialised, and @code{total_in} and +@code{total_out} will be zero. + +For the meaning of parameter @code{verbosity}, see @code{bzCompressInit}. + +If @code{small} is nonzero, the library will use an alternative +decompression algorithm which uses less memory but at the cost of +decompressing more slowly (roughly speaking, half the speed, but the +maximum memory requirement drops to around 2300k). See Chapter 2 for +more information on memory management. + +Note that the amount of memory needed to decompress +a stream cannot be determined until the stream's header has been read, +so even if @code{bzDecompressInit} succeeds, a subsequent +@code{bzDecompress} could fail with @code{BZ_MEM_ERROR}. + +Possible return values: +@display + @code{BZ_PARAM_ERROR} + if @code{(small != 0 && small != 1)} + or @code{(verbosity < 0 || verbosity > 4)} + @code{BZ_MEM_ERROR} + if insufficient memory is available +@end display + +Allowable next actions: +@display + @code{bzDecompress} + if @code{BZ_OK} was returned + no specific action required in case of error +@end display + + + +@subsection @code{bzDecompress} +@example +int bzDecompress ( bz_stream *strm ); +@end example +Provides more input and/out output buffer space for the library. The +caller maintains input and output buffers, and uses @code{bzDecompress} +to transfer data between them. + +Before each call to @code{bzDecompress}, @code{next_in} +should point at the compressed data, +and @code{avail_in} should indicate how many bytes the library +may read. @code{bzDecompress} updates @code{next_in}, @code{avail_in} +and @code{total_in} +to reflect the number of bytes it has read. + +Similarly, @code{next_out} should point to a buffer in which the uncompressed +output is to be placed, with @code{avail_out} indicating how much output space +is available. @code{bzCompress} updates @code{next_out}, +@code{avail_out} and @code{total_out} to reflect +the number of bytes output. + +You may provide and remove as little or as much data as you like on +each call of @code{bzDecompress}. +In the limit, it is acceptable to +supply and remove data one byte at a time, although this would be +terribly inefficient. You should always ensure that at least one +byte of output space is available at each call. + +Use of @code{bzDecompress} is simpler than @code{bzCompress}. + +You should provide input and remove output as described above, and +repeatedly call @code{bzDecompress} until @code{BZ_STREAM_END} is +returned. Appearance of @code{BZ_STREAM_END} denotes that +@code{bzDecompress} has detected the logical end of the compressed +stream. @code{bzDecompress} will not produce @code{BZ_STREAM_END} until +all output data has been placed into the output buffer, so once +@code{BZ_STREAM_END} appears, you are guaranteed to have available all +the decompressed output, and @code{bzDecompressEnd} can safely be +called. + +If case of an error return value, you should call @code{bzDecompressEnd} +to clean up and release memory. + +Possible return values: +@display + @code{BZ_PARAM_ERROR} + if @code{strm} is @code{NULL} or @code{strm->s} is @code{NULL} + or @code{strm->avail_out < 1} + @code{BZ_DATA_ERROR} + if a data integrity error is detected in the compressed stream + @code{BZ_DATA_ERROR_MAGIC} + if the compressed stream doesn't begin with the right magic bytes + @code{BZ_MEM_ERROR} + if there wasn't enough memory available + @code{BZ_STREAM_END} + if the logical end of the data stream was detected and all + output in has been consumed, eg @code{s->avail_out > 0} + @code{BZ_OK} + otherwise +@end display +Allowable next actions: +@display + @code{bzDecompress} + if @code{BZ_OK} was returned + @code{bzDecompressEnd} + otherwise +@end display + + +@subsection @code{bzDecompressEnd} +@example +int bzDecompressEnd ( bz_stream *strm ); +@end example +Releases all memory associated with a decompression stream. + +Possible return values: +@display + @code{BZ_PARAM_ERROR} + if @code{strm} is @code{NULL} or @code{strm->s} is @code{NULL} + @code{BZ_OK} + otherwise +@end display + +Allowable next actions: +@display + None. +@end display + + +@section High-level interface + +This interface provides functions for reading and writing +@code{bzip2} format files. First, some general points. + +@itemize @bullet +@item All of the functions take an @code{int*} first argument, + @code{bzerror}. + After each call, @code{bzerror} should be consulted first to determine + the outcome of the call. If @code{bzerror} is @code{BZ_OK}, + the call completed + successfully, and only then should the return value of the function + (if any) be consulted. If @code{bzerror} is @code{BZ_IO_ERROR}, + there was an error + reading/writing the underlying compressed file, and you should + then consult @code{errno}/@code{perror} to determine the + cause of the difficulty. + @code{bzerror} may also be set to various other values; precise details are + given on a per-function basis below. +@item If @code{bzerror} indicates an error + (ie, anything except @code{BZ_OK} and @code{BZ_STREAM_END}), + you should immediately call @code{bzReadClose} (or @code{bzWriteClose}, + depending on whether you are attempting to read or to write) + to free up all resources associated + with the stream. Once an error has been indicated, behaviour of all calls + except @code{bzReadClose} (@code{bzWriteClose}) is undefined. + The implication is that (1) @code{bzerror} should + be checked after each call, and (2) if @code{bzerror} indicates an error, + @code{bzReadClose} (@code{bzWriteClose}) should then be called to clean up. +@item The @code{FILE*} arguments passed to + @code{bzReadOpen}/@code{bzWriteOpen} + should be set to binary mode. + Most Unix systems will do this by default, but other platforms, + including Windows and Mac, will not. If you omit this, you may + encounter problems when moving code to new platforms. +@item Memory allocation requests are handled by + @code{malloc}/@code{free}. + At present + there is no facility for user-defined memory allocators in the file I/O + functions (could easily be added, though). +@end itemize + + + +@subsection @code{bzReadOpen} +@example + typedef void BZFILE; + + BZFILE *bzReadOpen ( int *bzerror, FILE *f, + int small, int verbosity, + void *unused, int nUnused ); +@end example +Prepare to read compressed data from file handle @code{f}. @code{f} +should refer to a file which has been opened for reading, and for which +the error indicator (@code{ferror(f)})is not set. If @code{small} is 1, +the library will try to decompress using less memory, at the expense of +speed. + +For reasons explained below, @code{bzRead} will decompress the +@code{nUnused} bytes starting at @code{unused}, before starting to read +from the file @code{f}. At most @code{BZ_MAX_UNUSED} bytes may be +supplied like this. If this facility is not required, you should pass +@code{NULL} and @code{0} for @code{unused} and n@code{Unused} +respectively. + +For the meaning of parameters @code{small} and @code{verbosity}, +see @code{bzDecompressInit}. + +The amount of memory needed to decompress a file cannot be determined +until the file's header has been read. So it is possible that +@code{bzReadOpen} returns @code{BZ_OK} but a subsequent call of +@code{bzRead} will return @code{BZ_MEM_ERROR}. + +Possible assignments to @code{bzerror}: +@display + @code{BZ_PARAM_ERROR} + if @code{f} is @code{NULL} + or @code{small} is neither @code{0} nor @code{1} + or @code{(unused == NULL && nUnused != 0)} + or @code{(unused != NULL && !(0 <= nUnused <= BZ_MAX_UNUSED))} + @code{BZ_IO_ERROR} + if @code{ferror(f)} is nonzero + @code{BZ_MEM_ERROR} + if insufficient memory is available + @code{BZ_OK} + otherwise. +@end display + +Possible return values: +@display + Pointer to an abstract @code{BZFILE} + if @code{bzerror} is @code{BZ_OK} + @code{NULL} + otherwise +@end display + +Allowable next actions: +@display + @code{bzRead} + if @code{bzerror} is @code{BZ_OK} + @code{bzClose} + otherwise +@end display + + +@subsection @code{bzRead} +@example + int bzRead ( int *bzerror, BZFILE *b, void *buf, int len ); +@end example +Reads up to @code{len} (uncompressed) bytes from the compressed file +@code{b} into +the buffer @code{buf}. If the read was successful, +@code{bzerror} is set to @code{BZ_OK} +and the number of bytes read is returned. If the logical end-of-stream +was detected, @code{bzerror} will be set to @code{BZ_STREAM_END}, +and the number +of bytes read is returned. All other @code{bzerror} values denote an error. + +@code{bzRead} will supply @code{len} bytes, +unless the logical stream end is detected +or an error occurs. Because of this, it is possible to detect the +stream end by observing when the number of bytes returned is +less than the number +requested. Nevertheless, this is regarded as inadvisable; you should +instead check @code{bzerror} after every call and watch out for +@code{BZ_STREAM_END}. + +Internally, @code{bzRead} copies data from the compressed file in chunks +of size @code{BZ_MAX_UNUSED} bytes +before decompressing it. If the file contains more bytes than strictly +needed to reach the logical end-of-stream, @code{bzRead} will almost certainly +read some of the trailing data before signalling @code{BZ_SEQUENCE_END}. +To collect the read but unused data once @code{BZ_SEQUENCE_END} has +appeared, call @code{bzReadGetUnused} immediately before @code{bzReadClose}. + +Possible assignments to @code{bzerror}: +@display + @code{BZ_PARAM_ERROR} + if @code{b} is @code{NULL} or @code{buf} is @code{NULL} or @code{len < 0} + @code{BZ_SEQUENCE_ERROR} + if @code{b} was opened with @code{bzWriteOpen} + @code{BZ_IO_ERROR} + if there is an error reading from the compressed file + @code{BZ_UNEXPECTED_EOF} + if the compressed file ended before the logical end-of-stream was detected + @code{BZ_DATA_ERROR} + if a data integrity error was detected in the compressed stream + @code{BZ_DATA_ERROR_MAGIC} + if the stream does not begin with the requisite header bytes (ie, is not + a @code{bzip2} data file). This is really a special case of @code{BZ_DATA_ERROR}. + @code{BZ_MEM_ERROR} + if insufficient memory was available + @code{BZ_STREAM_END} + if the logical end of stream was detected. + @code{BZ_OK} + otherwise. +@end display + +Possible return values: +@display + number of bytes read + if @code{bzerror} is @code{BZ_OK} or @code{BZ_STREAM_END} + undefined + otherwise +@end display + +Allowable next actions: +@display + collect data from @code{buf}, then @code{bzRead} or @code{bzReadClose} + if @code{bzerror} is @code{BZ_OK} + collect data from @code{buf}, then @code{bzReadClose} or @code{bzReadGetUnused} + if @code{bzerror} is @code{BZ_SEQUENCE_END} + @code{bzReadClose} + otherwise +@end display + + + +@subsection @code{bzReadGetUnused} +@example + void bzReadGetUnused ( int* bzerror, BZFILE *b, + void** unused, int* nUnused ); +@end example +Returns data which was read from the compressed file but was not needed +to get to the logical end-of-stream. @code{*unused} is set to the address +of the data, and @code{*nUnused} to the number of bytes. @code{*nUnused} will +be set to a value between @code{0} and @code{BZ_MAX_UNUSED} inclusive. + +This function may only be called once @code{bzRead} has signalled +@code{BZ_STREAM_END} but before @code{bzReadClose}. + +Possible assignments to @code{bzerror}: +@display + @code{BZ_PARAM_ERROR} + if @code{b} is @code{NULL} + or @code{unused} is @code{NULL} or @code{nUnused} is @code{NULL} + @code{BZ_SEQUENCE_ERROR} + if @code{BZ_STREAM_END} has not been signalled + or if @code{b} was opened with @code{bzWriteOpen} + @code{BZ_OK} + otherwise +@end display + +Allowable next actions: +@display + @code{bzReadClose} +@end display + + +@subsection @code{bzReadClose} +@example + void bzReadClose ( int *bzerror, BZFILE *b ); +@end example +Releases all memory pertaining to the compressed file @code{b}. +@code{bzReadClose} does not call @code{fclose} on the underlying file +handle, so you should do that yourself if appropriate. +@code{bzReadClose} should be called to clean up after all error +situations. + +Possible assignments to @code{bzerror}: +@display + @code{BZ_SEQUENCE_ERROR} + if @code{b} was opened with @code{bzOpenWrite} + @code{BZ_OK} + otherwise +@end display + +Allowable next actions: +@display + none +@end display + + + +@subsection @code{bzWriteOpen} +@example + BZFILE *bzWriteOpen ( int *bzerror, FILE *f, + int blockSize100k, int verbosity, + int workFactor ); +@end example +Prepare to write compressed data to file handle @code{f}. +@code{f} should refer to +a file which has been opened for writing, and for which the error +indicator (@code{ferror(f)})is not set. + +For the meaning of parameters @code{blockSize100k}, +@code{verbosity} and @code{workFactor}, see +@* @code{bzCompressInit}. + +All required memory is allocated at this stage, so if the call +completes successfully, @code{BZ_MEM_ERROR} cannot be signalled by a +subsequent call to @code{bzWrite}. + +Possible assignments to @code{bzerror}: +@display + @code{BZ_PARAM_ERROR} + if @code{f} is @code{NULL} + or @code{blockSize100k < 1} or @code{blockSize100k > 9} + @code{BZ_IO_ERROR} + if @code{ferror(f)} is nonzero + @code{BZ_MEM_ERROR} + if insufficient memory is available + @code{BZ_OK} + otherwise +@end display + +Possible return values: +@display + Pointer to an abstract @code{BZFILE} + if @code{bzerror} is @code{BZ_OK} + @code{NULL} + otherwise +@end display + +Allowable next actions: +@display + @code{bzWrite} + if @code{bzerror} is @code{BZ_OK} + (you could go directly to @code{bzWriteClose}, but this would be pretty pointless) + @code{bzWriteClose} + otherwise +@end display + + + +@subsection @code{bzWrite} +@example + void bzWrite ( int *bzerror, BZFILE *b, void *buf, int len ); +@end example +Absorbs @code{len} bytes from the buffer @code{buf}, eventually to be +compressed and written to the file. + +Possible assignments to @code{bzerror}: +@display + @code{BZ_PARAM_ERROR} + if @code{b} is @code{NULL} or @code{buf} is @code{NULL} or @code{len < 0} + @code{BZ_SEQUENCE_ERROR} + if b was opened with @code{bzReadOpen} + @code{BZ_IO_ERROR} + if there is an error writing the compressed file. + @code{BZ_OK} + otherwise +@end display + + + + +@subsection @code{bzWriteClose} +@example + int bzWriteClose ( int *bzerror, BZFILE* f, + int abandon, + unsigned int* nbytes_in, + unsigned int* nbytes_out ); +@end example + +Compresses and flushes to the compressed file all data so far supplied +by @code{bzWrite}. The logical end-of-stream markers are also written, so +subsequent calls to @code{bzWrite} are illegal. All memory associated +with the compressed file @code{b} is released. +@code{fflush} is called on the +compressed file, but it is not @code{fclose}'d. + +If @code{bzWriteClose} is called to clean up after an error, the only +action is to release the memory. The library records the error codes +issued by previous calls, so this situation will be detected +automatically. There is no attempt to complete the compression +operation, nor to @code{fflush} the compressed file. You can force this +behaviour to happen even in the case of no error, by passing a nonzero +value to @code{abandon}. + +If @code{nbytes_in} is non-null, @code{*nbytes_in} will be set to be the +total volume of uncompressed data handled. Similarly, @code{nbytes_out} +will be set to the total volume of compressed data written. + +Possible assignments to @code{bzerror}: +@display + @code{BZ_SEQUENCE_ERROR} + if @code{b} was opened with @code{bzReadOpen} + @code{BZ_IO_ERROR} + if there is an error writing the compressed file + @code{BZ_OK} + otherwise +@end display + +@subsection Handling embedded compressed data streams + +The high-level library facilitates use of +@code{bzip2} data streams which form some part of a surrounding, larger +data stream. +@itemize @bullet +@item For writing, the library takes an open file handle, writes +compressed data to it, @code{fflush}es it but does not @code{fclose} it. +The calling application can write its own data before and after the +compressed data stream, using that same file handle. +@item Reading is more complex, and the facilities are not as general +as they could be since generality is hard to reconcile with efficiency. +@code{bzRead} reads from the compressed file in blocks of size +@code{BZ_MAX_UNUSED} bytes, and in doing so probably will overshoot +the logical end of compressed stream. +To recover this data once decompression has +ended, call @code{bzReadGetUnused} after the last call of @code{bzRead} +(the one returning @code{BZ_STREAM_END}) but before calling +@code{bzReadClose}. +@end itemize + +This mechanism makes it easy to decompress multiple @code{bzip2} +streams placed end-to-end. As the end of one stream, when @code{bzRead} +returns @code{BZ_STREAM_END}, call @code{bzReadGetUnused} to collect the +unused data (copy it into your own buffer somewhere). +That data forms the start of the next compressed stream. +To start uncompressing that next stream, call @code{bzReadOpen} again, +feeding in the unused data via the @code{unused}/@code{nUnused} +parameters. +Keep doing this until @code{BZ_STREAM_END} return coincides with the +physical end of file (@code{feof(f)}). In this situation +@code{bzReadGetUnused} +will of course return no data. + +This should give some feel for how the high-level interface can be used. +If you require extra flexibility, you'll have to bite the bullet and get +to grips with the low-level interface. + +@subsection Standard file-reading/writing code +Here's how you'd write data to a compressed file: +@example @code +FILE* f; +BZFILE* b; +int nBuf; +char buf[ /* whatever size you like */ ]; +int bzerror; +int nWritten; + +f = fopen ( "myfile.bz2", "w" ); +if (!f) @{ + /* handle error */ +@} +b = bzWriteOpen ( &bzerror, f, 9 ); +if (bzerror != BZ_OK) @{ + bzWriteClose ( b ); + /* handle error */ +@} + +while ( /* condition */ ) @{ + /* get data to write into buf, and set nBuf appropriately */ + nWritten = bzWrite ( &bzerror, b, buf, nBuf ); + if (bzerror == BZ_IO_ERROR) @{ + bzWriteClose ( &bzerror, b ); + /* handle error */ + @} +@} + +bzWriteClose ( &bzerror, b ); +if (bzerror == BZ_IO_ERROR) @{ + /* handle error */ +@} +@end example +And to read from a compressed file: +@example +FILE* f; +BZFILE* b; +int nBuf; +char buf[ /* whatever size you like */ ]; +int bzerror; +int nWritten; + +f = fopen ( "myfile.bz2", "r" ); +if (!f) @{ + /* handle error */ +@} +b = bzReadOpen ( &bzerror, f, 0, NULL, 0 ); +if (bzerror != BZ_OK) @{ + bzReadClose ( &bzerror, b ); + /* handle error */ +@} + +bzerror = BZ_OK; +while (bzerror == BZ_OK && /* arbitrary other conditions */) @{ + nBuf = bzRead ( &bzerror, b, buf, /* size of buf */ ); + if (bzerror == BZ_OK) @{ + /* do something with buf[0 .. nBuf-1] */ + @} +@} +if (bzerror != BZ_STREAM_END) @{ + bzReadClose ( &bzerror, b ); + /* handle error */ +@} else @{ + bzReadClose ( &bzerror ); +@} +@end example + + + +@section Utility functions +@subsection @code{bzBuffToBuffCompress} +@example + int bzBuffToBuffCompress( char* dest, + unsigned int* destLen, + char* source, + unsigned int sourceLen, + int blockSize100k, + int verbosity, + int workFactor ); +@end example +Attempts to compress the data in @code{source[0 .. sourceLen-1]} +into the destination buffer, @code{dest[0 .. *destLen-1]}. +If the destination buffer is big enough, @code{*destLen} is +set to the size of the compressed data, and @code{BZ_OK} is +returned. If the compressed data won't fit, @code{*destLen} +is unchanged, and @code{BZ_OUTBUFF_FULL} is returned. + +Compression in this manner is a one-shot event, done with a single call +to this function. The resulting compressed data is a complete +@code{bzip2} format data stream. There is no mechanism for making +additional calls to provide extra input data. If you want that kind of +mechanism, use the low-level interface. + +For the meaning of parameters @code{blockSize100k}, @code{verbosity} +and @code{workFactor}, @* see @code{bzCompressInit}. + +To guarantee that the compressed data will fit in its buffer, allocate +an output buffer of size 1% larger than the uncompressed data, plus +six hundred extra bytes. + +@code{bzBuffToBuffDecompress} will not write data at or +beyond @code{dest[*destLen]}, even in case of buffer overflow. + +Possible return values: +@display + @code{BZ_PARAM_ERROR} + if @code{dest} is @code{NULL} or @code{destLen} is @code{NULL} + or @code{blockSize100k < 1} or @code{blockSize100k > 9} + or @code{verbosity < 0} or @code{verbosity > 4} + or @code{workFactor < 0} or @code{workFactor > 250} + @code{BZ_MEM_ERROR} + if insufficient memory is available + @code{BZ_OUTBUFF_FULL} + if the size of the compressed data exceeds @code{*destLen} + @code{BZ_OK} + otherwise +@end display + + + +@subsection @code{bzBuffToBuffDecompress} +@example + int bzBuffToBuffDecompress ( char* dest, + unsigned int* destLen, + char* source, + unsigned int sourceLen, + int small, + int verbosity ); +@end example +Attempts to decompress the data in @code{source[0 .. sourceLen-1]} +into the destination buffer, @code{dest[0 .. *destLen-1]}. +If the destination buffer is big enough, @code{*destLen} is +set to the size of the uncompressed data, and @code{BZ_OK} is +returned. If the compressed data won't fit, @code{*destLen} +is unchanged, and @code{BZ_OUTBUFF_FULL} is returned. + +@code{source} is assumed to hold a complete @code{bzip2} format +data stream. @code{bzBuffToBuffDecompress} tries to decompress +the entirety of the stream into the output buffer. + +For the meaning of parameters @code{small} and @code{verbosity}, +see @code{bzDecompressInit}. + +Because the compression ratio of the compressed data cannot be known in +advance, there is no easy way to guarantee that the output buffer will +be big enough. You may of course make arrangements in your code to +record the size of the uncompressed data, but such a mechanism is beyond +the scope of this library. + +@code{bzBuffToBuffDecompress} will not write data at or +beyond @code{dest[*destLen]}, even in case of buffer overflow. + +Possible return values: +@display + @code{BZ_PARAM_ERROR} + if @code{dest} is @code{NULL} or @code{destLen} is @code{NULL} + or @code{small != 0 && small != 1} + or @code{verbosity < 0} or @code{verbosity > 4} + @code{BZ_MEM_ERROR} + if insufficient memory is available + @code{BZ_OUTBUFF_FULL} + if the size of the compressed data exceeds @code{*destLen} + @code{BZ_DATA_ERROR} + if a data integrity error was detected in the compressed data + @code{BZ_DATA_ERROR_MAGIC} + if the compressed data doesn't begin with the right magic bytes + @code{BZ_UNEXPECTED_EOF} + if the compressed data ends unexpectedly + @code{BZ_OK} + otherwise +@end display + + + +@section Using the library in a @code{stdio}-free environment + +@subsection Getting rid of @code{stdio} + +In a deeply embedded application, you might want to use just +the memory-to-memory functions. You can do this conveniently +by compiling the library with preprocessor symbol @code{BZ_NO_STDIO} +defined. Doing this gives you a library containing only the following +eight functions: + +@code{bzCompressInit}, @code{bzCompress}, @code{bzCompressEnd} @* +@code{bzDecompressInit}, @code{bzDecompress}, @code{bzDecompressEnd} @* +@code{bzBuffToBuffCompress}, @code{bzBuffToBuffDecompress} + +When compiled like this, all functions will ignore @code{verbosity} +settings. + +@subsection Critical error handling +@code{libbzip2} contains a number of internal assertion checks which +should, needless to say, never be activated. Nevertheless, if an +assertion should fail, behaviour depends on whether or not the library +was compiled with @code{BZ_NO_STDIO} set. + +For a normal compile, an assertion failure yields the message +@example + bzip2/libbzip2, v0.9.0: internal error number N. + This is a bug in bzip2/libbzip2, v0.9.0. Please report + it to me at: jseward@@acm.org. If this happened when + you were using some program which uses libbzip2 as a + component, you should also report this bug to the author(s) + of that program. Please make an effort to report this bug; + timely and accurate bug reports eventually lead to higher + quality software. Thx. Julian Seward, 27 June 1998. +@end example +where @code{N} is some error code number. @code{exit(3)} +is then called. + +For a @code{stdio}-free library, assertion failures result +in a call to a function declared as: +@example + extern void bz_internal_error ( int errcode ); +@end example +The relevant code is passed as a parameter. You should supply +such a function. + +In either case, once an assertion failure has occurred, any +@code{bz_stream} records involved can be regarded as invalid. +You should not attempt to resume normal operation with them. + +You may, of course, change critical error handling to suit +your needs. As I said above, critical errors indicate bugs +in the library and should not occur. All "normal" error +situations are indicated via error return codes from functions, +and can be recovered from. + + +@section Making a Windows DLL +Everything related to Windows has been contributed by Yoshioka Tsuneo +@* (@code{QWF00133@@niftyserve.or.jp} / +@code{tsuneo-y@@is.aist-nara.ac.jp}), so you should send your queries to +him (but perhaps Cc: me, @code{jseward@@acm.org}). + +My vague understanding of what to do is: using Visual C++ 5.0, +open the project file @code{libbz2.dsp}, and build. That's all. + +If you can't +open the project file for some reason, make a new one, naming these files: +@code{blocksort.c}, @code{bzlib.c}, @code{compress.c}, +@code{crctable.c}, @code{decompress.c}, @code{huffman.c}, @* +@code{randtable.c} and @code{libbz2.def}. You might also need +to name the header files @code{bzlib.h} and @code{bzlib_private.h}. + +If you don't use VC++, you may need to define the proprocessor symbol +@code{_WIN32}. + +Finally, @code{dlltest.c} is a sample program using the DLL. It has a +project file, @code{dlltest.dsp}. + +I haven't tried any of this stuff myself, but it all looks plausible. + + + +@chapter Miscellanea + +These are just some random thoughts of mine. Your mileage may +vary. + +@section Limitations of the compressed file format +@code{bzip2-0.9.0} uses exactly the same file format as the previous +version, @code{bzip2-0.1}. This decision was made in the interests of +stability. Creating yet another incompatible compressed file format +would create further confusion and disruption for users. + +Nevertheless, this is not a painless decision. Development +work since the release of @code{bzip2-0.1} in August 1997 +has shown complexities in the file format which slow down +decompression and, in retrospect, are unnecessary. These are: +@itemize @bullet +@item The run-length encoder, which is the first of the + compression transformations, is entirely irrelevant. + The original purpose was to protect the sorting algorithm + from the very worst case input: a string of repeated + symbols. But algorithm steps Q6a and Q6b in the original + Burrows-Wheeler technical report (SRC-124) show how + repeats can be handled without difficulty in block + sorting. +@item The randomisation mechanism doesn't really need to be + there. Udi Manber and Gene Myers published a suffix + array construction algorithm a few years back, which + can be employed to sort any block, no matter how + repetitive, in O(N log N) time. Subsequent work by + Kunihiko Sadakane has produced a derivative O(N (log N)^2) + algorithm which usually outperforms the Manber-Myers + algorithm. + + I could have changed to Sadakane's algorithm, but I find + it to be slower than @code{bzip2}'s existing algorithm for + most inputs, and the randomisation mechanism protects + adequately against bad cases. I didn't think it was + a good tradeoff to make. Partly this is due to the fact + that I was not flooded with email complaints about + @code{bzip2-0.1}'s performance on repetitive data, so + perhaps it isn't a problem for real inputs. + + Probably the best long-term solution + is to use the existing sorting + algorithm initially, and fall back to a O(N (log N)^2) + algorithm if the standard algorithm gets into difficulties. + This can be done without much difficulty; I made + a prototype implementation of it some months now. +@item The compressed file format was never designed to be + handled by a library, and I have had to jump though + some hoops to produce an efficient implementation of + decompression. It's a bit hairy. Try passing + @code{decompress.c} through the C preprocessor + and you'll see what I mean. Much of this complexity + could have been avoided if the compressed size of + each block of data was recorded in the data stream. +@item An Adler-32 checksum, rather than a CRC32 checksum, + would be faster to compute. +@end itemize +It would be fair to say that the @code{bzip2} format was frozen +before I properly and fully understood the performance +consequences of doing so. + +Improvements which I have been able to incorporate into +0.9.0, despite using the same file format, are: +@itemize @bullet +@item Single array implementation of the inverse BWT. This + significantly speeds up decompression, presumably + because it reduces the number of cache misses. +@item Faster inverse MTF transform for large MTF values. The + new implementation is based on the notion of sliding blocks + of values. +@item @code{bzip2-0.9.0} now reads and writes files with @code{fread} + and @code{fwrite}; version 0.1 used @code{putc} and @code{getc}. + Duh! I'm embarrassed at my own moronicness (moronicity?) on this + one. + +@end itemize +Further ahead, it would be nice +to be able to do random access into files. This will +require some careful design of compressed file formats. + + + +@section Portability issues +After some consideration, I have decided not to use +GNU @code{autoconf} to configure 0.9.0. + +@code{autoconf}, admirable and wonderful though it is, +mainly assists with portability problems between Unix-like +platforms. But @code{bzip2} doesn't have much in the way +of portability problems on Unix; most of the difficulties appear +when porting to the Mac, or to Microsoft's operating systems. +@code{autoconf} doesn't help in those cases, and brings in a +whole load of new complexity. + +Most people should be able to compile the library and program +under Unix straight out-of-the-box, so to speak, especially +if you have a version of GNU C available. + +There are a couple of @code{__inline__} directives in the code. GNU C +(@code{gcc}) should be able to handle them. If your compiler doesn't +like them, just @code{#define} @code{__inline__} to be null. One +easy way to do this is to compile with the flag @code{-D__inline__=}, +which should be understood by most Unix compilers. + +If you still have difficulties, try compiling with the macro +@code{BZ_STRICT_ANSI} defined. This should enable you to build the +library in a strictly ANSI compliant environment. Building the program +itself like this is dangerous and not supported, since you remove +@code{bzip2}'s checks against compressing directories, symbolic links, +devices, and other not-really-a-file entities. This could cause +filesystem corruption! + +One other thing: if you create a @code{bzip2} binary for public +distribution, please try and link it statically (@code{gcc -s}). This +avoids all sorts of library-version issues that others may encounter +later on. + + +@section Reporting bugs +I tried pretty hard to make sure @code{bzip2} is +bug free, both by design and by testing. Hopefully +you'll never need to read this section for real. + +Nevertheless, if @code{bzip2} dies with a segmentation +fault, a bus error or an internal assertion failure, it +will ask you to email me a bug report. Experience with +version 0.1 shows that almost all these problems can +be traced to either compiler bugs or hardware problems. +@itemize @bullet +@item +Recompile the program with no optimisation, and see if it +works. And/or try a different compiler. +I heard all sorts of stories about various flavours +of GNU C (and other compilers) generating bad code for +@code{bzip2}, and I've run across two such examples myself. + +2.7.X versions of GNU C are known to generate bad code from +time to time, at high optimisation levels. +If you get problems, try using the flags +@code{-O2} @code{-fomit-frame-pointer} @code{-fno-strength-reduce}. +You should specifically @emph{not} use @code{-funroll-loops}. + +You may notice that the Makefile runs four tests as part of +the build process. If the program passes all of these, it's +a pretty good (but not 100%) indication that the compiler has +done its job correctly. +@item +If @code{bzip2} crashes randomly, and the crashes are not +repeatable, you may have a flaky memory subsystem. @code{bzip2} +really hammers your memory hierarchy, and if it's a bit marginal, +you may get these problems. Ditto if your disk or I/O subsystem +is slowly failing. Yup, this really does happen. + +Try using a different machine of the same type, and see if +you can repeat the problem. +@item This isn't really a bug, but ... If @code{bzip2} tells +you your file is corrupted on decompression, and you +obtained the file via FTP, there is a possibility that you +forgot to tell FTP to do a binary mode transfer. That absolutely +will cause the file to be non-decompressible. You'll have to transfer +it again. +@end itemize + +If you've incorporated @code{libbzip2} into your own program +and are getting problems, please, please, please, check that the +parameters you are passing in calls to the library, are +correct, and in accordance with what the documentation says +is allowable. I have tried to make the library robust against +such problems, but I'm sure I haven't succeeded. + +Finally, if the above comments don't help, you'll have to send +me a bug report. Now, it's just amazing how many people will +send me a bug report saying something like +@display + bzip2 crashed with segmentation fault on my machine +@end display +and absolutely nothing else. Needless to say, a such a report +is @emph{totally, utterly, completely and comprehensively 100% useless; +a waste of your time, my time, and net bandwidth}. +With no details at all, there's no way I can possibly begin +to figure out what the problem is. + +The rules of the game are: facts, facts, facts. Don't omit +them because "oh, they won't be relevant". At the bare +minimum: +@display + Machine type. Operating system version. + Exact version of @code{bzip2} (do @code{bzip2 -V}). + Exact version of the compiler used. + Flags passed to the compiler. +@end display +However, the most important single thing that will help me is +the file that you were trying to compress or decompress at the +time the problem happened. Without that, my ability to do anything +more than speculate about the cause, is limited. + +Please remember that I connect to the Internet with a modem, so +you should contact me before mailing me huge files. + + +@section Did you get the right package? + +@code{bzip2} is a resource hog. It soaks up large amounts of CPU cycles +and memory. Also, it gives very large latencies. In the worst case, you +can feed many megabytes of uncompressed data into the library before +getting any compressed output, so this probably rules out applications +requiring interactive behaviour. + +These aren't faults of my implementation, I hope, but more +an intrinsic property of the Burrows-Wheeler transform (unfortunately). +Maybe this isn't what you want. + +If you want a compressor and/or library which is faster, uses less +memory but gets pretty good compression, and has minimal latency, +consider Jean-loup +Gailly's and Mark Adler's work, @code{zlib-1.1.2} and +@code{gzip-1.2.4}. Look for them at +@code{http://www.cdrom.com/pub/infozip/zlib} and +@code{http://www.gzip.org} respectively. + +For something faster and lighter still, you might try Markus F X J +Oberhumer's @code{LZO} real-time compression/decompression library, at +@* @code{http://wildsau.idv.uni-linz.ac.at/mfx/lzo.html}. + +If you want to use the @code{bzip2} algorithms to compress small blocks +of data, 64k bytes or smaller, for example on an on-the-fly disk +compressor, you'd be well advised not to use this library. Instead, +I've made a special library tuned for that kind of use. It's part of +@code{e2compr-0.40}, an on-the-fly disk compressor for the Linux +@code{ext2} filesystem. Look at +@code{http://www.netspace.net.au/~reiter/e2compr}. + + + +@section Testing + +A record of the tests I've done. + +First, some data sets: +@itemize @bullet +@item B: a directory containing a 6001 files, one for every length in the + range 0 to 6000 bytes. The files contain random lowercase + letters. 18.7 megabytes. +@item H: my home directory tree. Documents, source code, mail files, + compressed data. H contains B, and also a directory of + files designed as boundary cases for the sorting; mostly very + repetitive, nasty files. 445 megabytes. +@item A: directory tree holding various applications built from source: + @code{egcs-1.0.2}, @code{gcc-2.8.1}, KDE Beta 4, GTK, Octave, etc. + 827 megabytes. +@item P: directory tree holding large amounts of source code (@code{.tar} + files) of the entire GNU distribution, plus a couple of + Linux distributions. 2400 megabytes. +@end itemize +The tests conducted are as follows. Each test means compressing +(a copy of) each file in the data set, decompressing it and +comparing it against the original. + +First, a bunch of tests with block sizes, internal buffer +sizes and randomisation lengths set very small, +to detect any problems with the +blocking, buffering and randomisation mechanisms. +This required modifying the source code so as to try to +break it. +@enumerate +@item Data set H, with + buffer size of 1 byte, and block size of 23 bytes. +@item Data set B, buffer sizes 1 byte, block size 1 byte. +@item As (2) but small-mode decompression (first 1700 files). +@item As (2) with block size 2 bytes. +@item As (2) with block size 3 bytes. +@item As (2) with block size 4 bytes. +@item As (2) with block size 5 bytes. +@item As (2) with block size 6 bytes and small-mode decompression. +@item H with normal buffer sizes (5000 bytes), normal block + size (up to 900000 bytes), but with randomisation + mechanism running intensely (randomising approximately every + third byte). +@item As (9) with small-mode decompression. +@end enumerate +Then some tests with unmodified source code. +@enumerate +@item H, all settings normal. +@item As (1), with small-mode decompress. +@item H, compress with flag @code{-1}. +@item H, compress with flag @code{-s}, decompress with flag @code{-s}. +@item Forwards compatibility: H, @code{bzip2-0.1pl2} compressing, + @code{bzip2-0.9.0} decompressing, all settings normal. +@item Backwards compatibility: H, @code{bzip2-0.9.0} compressing, + @code{bzip2-0.1pl2} decompressing, all settings normal. +@item Bigger tests: A, all settings normal. +@item P, all settings normal. +@item Misc test: about 100 megabytes of @code{.tar} files with + @code{bzip2} compiled with Purify. +@item Misc tests to make sure it builds and runs ok on non-Linux/x86 + platforms. +@end enumerate +These tests were conducted on a 205 MHz Cyrix 6x86MX machine, running +Linux 2.0.32. They represent nearly a week of continuous computation. +All tests completed successfully. + + +@section Further reading +@code{bzip2} is not research work, in the sense that it doesn't present +any new ideas. Rather, it's an engineering exercise based on existing +ideas. + +Four documents describe essentially all the ideas behind @code{bzip2}: +@example +Michael Burrows and D. J. Wheeler: + "A block-sorting lossless data compression algorithm" + 10th May 1994. + Digital SRC Research Report 124. + ftp://ftp.digital.com/pub/DEC/SRC/research-reports/SRC-124.ps.gz + If you have trouble finding it, try searching at the + New Zealand Digital Library, http://www.nzdl.org. + +Daniel S. Hirschberg and Debra A. LeLewer + "Efficient Decoding of Prefix Codes" + Communications of the ACM, April 1990, Vol 33, Number 4. + You might be able to get an electronic copy of this + from the ACM Digital Library. + +David J. Wheeler + Program bred3.c and accompanying document bred3.ps. + This contains the idea behind the multi-table Huffman + coding scheme. + ftp://ftp.cl.cam.ac.uk/pub/user/djw3/ + +Jon L. Bentley and Robert Sedgewick + "Fast Algorithms for Sorting and Searching Strings" + Available from Sedgewick's web page, + www.cs.princeton.edu/~rs +@end example +The following paper gives valuable additional insights into the +algorithm, but is not immediately the basis of any code +used in bzip2. +@example +Peter Fenwick: + Block Sorting Text Compression + Proceedings of the 19th Australasian Computer Science Conference, + Melbourne, Australia. Jan 31 - Feb 2, 1996. + ftp://ftp.cs.auckland.ac.nz/pub/peter-f/ACSC96paper.ps +@end example +Kunihiko Sadakane's sorting algorithm, mentioned above, +is available from: +@example +http://naomi.is.s.u-tokyo.ac.jp/~sada/papers/Sada98b.ps.gz +@end example +The Manber-Myers suffix array construction +algorithm is described in a paper +available from: +@example +http://www.cs.arizona.edu/people/gene/PAPERS/suffix.ps +@end example + + + +@contents + +@bye + diff --git a/randtable.c b/randtable.c new file mode 100644 index 0000000..27b34af --- /dev/null +++ b/randtable.c @@ -0,0 +1,124 @@ + +/*-------------------------------------------------------------*/ +/*--- Table for randomising repetitive blocks ---*/ +/*--- randtable.c ---*/ +/*-------------------------------------------------------------*/ + +/*-- + This file is a part of bzip2 and/or libbzip2, a program and + library for lossless, block-sorting data compression. + + Copyright (C) 1996-1998 Julian R Seward. All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + 1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + 2. The origin of this software must not be misrepresented; you must + not claim that you wrote the original software. If you use this + software in a product, an acknowledgment in the product + documentation would be appreciated but is not required. + + 3. Altered source versions must be plainly marked as such, and must + not be misrepresented as being the original software. + + 4. The name of the author may not be used to endorse or promote + products derived from this software without specific prior written + permission. + + THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS + OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED + WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY + DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE + GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS + INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, + WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING + NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS + SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + Julian Seward, Guildford, Surrey, UK. + jseward@acm.org + bzip2/libbzip2 version 0.9.0c of 18 October 1998 + + This program is based on (at least) the work of: + Mike Burrows + David Wheeler + Peter Fenwick + Alistair Moffat + Radford Neal + Ian H. Witten + Robert Sedgewick + Jon L. Bentley + + For more information on these sources, see the manual. +--*/ + + +#include "bzlib_private.h" + + +/*---------------------------------------------*/ +Int32 rNums[512] = { + 619, 720, 127, 481, 931, 816, 813, 233, 566, 247, + 985, 724, 205, 454, 863, 491, 741, 242, 949, 214, + 733, 859, 335, 708, 621, 574, 73, 654, 730, 472, + 419, 436, 278, 496, 867, 210, 399, 680, 480, 51, + 878, 465, 811, 169, 869, 675, 611, 697, 867, 561, + 862, 687, 507, 283, 482, 129, 807, 591, 733, 623, + 150, 238, 59, 379, 684, 877, 625, 169, 643, 105, + 170, 607, 520, 932, 727, 476, 693, 425, 174, 647, + 73, 122, 335, 530, 442, 853, 695, 249, 445, 515, + 909, 545, 703, 919, 874, 474, 882, 500, 594, 612, + 641, 801, 220, 162, 819, 984, 589, 513, 495, 799, + 161, 604, 958, 533, 221, 400, 386, 867, 600, 782, + 382, 596, 414, 171, 516, 375, 682, 485, 911, 276, + 98, 553, 163, 354, 666, 933, 424, 341, 533, 870, + 227, 730, 475, 186, 263, 647, 537, 686, 600, 224, + 469, 68, 770, 919, 190, 373, 294, 822, 808, 206, + 184, 943, 795, 384, 383, 461, 404, 758, 839, 887, + 715, 67, 618, 276, 204, 918, 873, 777, 604, 560, + 951, 160, 578, 722, 79, 804, 96, 409, 713, 940, + 652, 934, 970, 447, 318, 353, 859, 672, 112, 785, + 645, 863, 803, 350, 139, 93, 354, 99, 820, 908, + 609, 772, 154, 274, 580, 184, 79, 626, 630, 742, + 653, 282, 762, 623, 680, 81, 927, 626, 789, 125, + 411, 521, 938, 300, 821, 78, 343, 175, 128, 250, + 170, 774, 972, 275, 999, 639, 495, 78, 352, 126, + 857, 956, 358, 619, 580, 124, 737, 594, 701, 612, + 669, 112, 134, 694, 363, 992, 809, 743, 168, 974, + 944, 375, 748, 52, 600, 747, 642, 182, 862, 81, + 344, 805, 988, 739, 511, 655, 814, 334, 249, 515, + 897, 955, 664, 981, 649, 113, 974, 459, 893, 228, + 433, 837, 553, 268, 926, 240, 102, 654, 459, 51, + 686, 754, 806, 760, 493, 403, 415, 394, 687, 700, + 946, 670, 656, 610, 738, 392, 760, 799, 887, 653, + 978, 321, 576, 617, 626, 502, 894, 679, 243, 440, + 680, 879, 194, 572, 640, 724, 926, 56, 204, 700, + 707, 151, 457, 449, 797, 195, 791, 558, 945, 679, + 297, 59, 87, 824, 713, 663, 412, 693, 342, 606, + 134, 108, 571, 364, 631, 212, 174, 643, 304, 329, + 343, 97, 430, 751, 497, 314, 983, 374, 822, 928, + 140, 206, 73, 263, 980, 736, 876, 478, 430, 305, + 170, 514, 364, 692, 829, 82, 855, 953, 676, 246, + 369, 970, 294, 750, 807, 827, 150, 790, 288, 923, + 804, 378, 215, 828, 592, 281, 565, 555, 710, 82, + 896, 831, 547, 261, 524, 462, 293, 465, 502, 56, + 661, 821, 976, 991, 658, 869, 905, 758, 745, 193, + 768, 550, 608, 933, 378, 286, 215, 979, 792, 961, + 61, 688, 793, 644, 986, 403, 106, 366, 905, 644, + 372, 567, 466, 434, 645, 210, 389, 550, 919, 135, + 780, 773, 635, 389, 707, 100, 626, 958, 165, 504, + 920, 176, 193, 713, 857, 265, 203, 50, 668, 108, + 645, 990, 626, 197, 510, 357, 358, 850, 858, 364, + 936, 638 +}; + + +/*-------------------------------------------------------------*/ +/*--- end randtable.c ---*/ +/*-------------------------------------------------------------*/ diff --git a/test.bat b/test.bat deleted file mode 100644 index 30b747d..0000000 --- a/test.bat +++ /dev/null @@ -1,9 +0,0 @@ -@rem -@rem MSDOS test driver for bzip2 -@rem -type words1 -.\bzip2 -1 < sample1.ref > sample1.rbz -.\bzip2 -2 < sample2.ref > sample2.rbz -.\bzip2 -dvv < sample1.bz2 > sample1.tst -.\bzip2 -dvv < sample2.bz2 > sample2.tst -type words3sh \ No newline at end of file diff --git a/test.cmd b/test.cmd deleted file mode 100644 index f7bc866..0000000 --- a/test.cmd +++ /dev/null @@ -1,9 +0,0 @@ -@rem -@rem OS/2 test driver for bzip2 -@rem -type words1 -.\bzip2 -1 < sample1.ref > sample1.rbz -.\bzip2 -2 < sample2.ref > sample2.rbz -.\bzip2 -dvv < sample1.bz2 > sample1.tst -.\bzip2 -dvv < sample2.bz2 > sample2.tst -type words3sh \ No newline at end of file diff --git a/words0 b/words0 deleted file mode 100644 index 527fb43..0000000 --- a/words0 +++ /dev/null @@ -1,7 +0,0 @@ -***-------------------------------------------------*** -***--------- IMPORTANT: READ WHAT FOLLOWS! ---------*** -***--------- viz: pay attention :-) ---------*** -***-------------------------------------------------*** - -Compiling bzip2 ... - diff --git a/words1 b/words1 index c75293b..a891431 100644 --- a/words1 +++ b/words1 @@ -1,5 +1,4 @@ - Doing 4 tests (2 compress, 2 uncompress) ... If there's a problem, things might stop at this point. diff --git a/words2 b/words2 index d3cafb9..203ee39 100644 --- a/words2 +++ b/words2 @@ -1,5 +1,4 @@ - Checking test results. If any of the four "cmp"s which follow report any differences, something is wrong. If you can't easily figure out what, please let me know (jseward@acm.org). diff --git a/words3 b/words3 index 5739d18..10bb2e9 100644 --- a/words3 +++ b/words3 @@ -1,23 +1,20 @@ - If you got this far and the "cmp"s didn't find anything amiss, looks -like you're in business. You should install bzip2 and bunzip2: +like you're in business. You should install bzip2, bunzip2 and bzcat: - copy bzip2 to a public place, maybe /usr/bin. - In that public place, make bunzip2 a symbolic link - to the bzip2 you just copied there. + Copy bzip2 and bzip2recover to a public place, maybe /usr/bin. + In that public place, make bunzip2 and bzcat be + symbolic links to the bzip2 you just copied there. Put the manual page, bzip2.1, somewhere appropriate; perhaps in /usr/man/man1. -Complete instructions for use are in the preformatted -manual page, in the file bzip2.1.preformatted. +Instructions for use are in the preformatted manual page, in the file +bzip2.txt. For more detailed documentation, read the full manual. +It is available in Postscript form (manual.ps) and HTML form +(manual_toc.html). You can also do "bzip2 --help" to see some helpful information. - "bzip2 -L" displays the software license. -Please read the README file carefully. -Finally, note that bzip2 comes with ABSOLUTELY NO WARRANTY. - -Happy compressing! +Happy compressing. -- JRS, 30 August 1998. diff --git a/words3sh b/words3sh deleted file mode 100644 index 1139177..0000000 --- a/words3sh +++ /dev/null @@ -1,12 +0,0 @@ -If you got this far and the "bzip2 -dvv"s give identical -stored vs computed CRCs, you're probably in business. -Complete instructions for use are in the preformatted manual page, -in the file bzip2.txt. - -You can also do "bzip2 --help" to see some helpful information. -"bzip2 -L" displays the software license. - -Please read the README file carefully. -Finally, note that bzip2 comes with ABSOLUTELY NO WARRANTY. - -Happy compressing! \ No newline at end of file -- cgit v1.2.3 From f93cd82a9a7094ad90fd19bbc6ccf6f4627f8060 Mon Sep 17 00:00:00 2001 From: Julian Seward Date: Sat, 4 Sep 1999 22:13:13 +0200 Subject: bzip2-0.9.5d --- CHANGES | 55 + LICENSE | 6 +- Makefile | 89 +- README | 78 +- Y2K_INFO | 34 + blocksort.c | 1242 +- bzip2.1 | 610 +- bzip2.1.preformatted | 440 +- bzip2.c | 545 +- bzip2.txt | 336 +- bzip2recover.c | 21 +- bzlib.c | 138 +- bzlib.h | 13 +- bzlib_private.h | 47 +- compress.c | 141 +- crctable.c | 6 +- decompress.c | 8 +- dlltest.c | 242 +- howbig.c | 37 - huffman.c | 6 +- makefile.msc | 65 + manual.texi | 869 +- randtable.c | 6 +- sample3.bz2 | Bin 0 -> 235 bytes sample3.ref | 30007 +++++++++++++++++++++++++++++++++++++++++++++++++ words1 | 2 +- words3 | 21 +- 27 files changed, 32912 insertions(+), 2152 deletions(-) create mode 100644 Y2K_INFO delete mode 100644 howbig.c create mode 100644 makefile.msc create mode 100644 sample3.bz2 create mode 100644 sample3.ref diff --git a/CHANGES b/CHANGES index ac00f3a..0acb1c2 100644 --- a/CHANGES +++ b/CHANGES @@ -43,3 +43,58 @@ In compress.c: do a bit better on small files. This _does_ effect bzip2.c. + +0.9.5a +~~~~~~ +Major change: add a fallback sorting algorithm (blocksort.c) +to give reasonable behaviour even for very repetitive inputs. +Nuked --repetitive-best and --repetitive-fast since they are +no longer useful. + +Minor changes: mostly a whole bunch of small changes/ +bugfixes in the driver (bzip2.c). Changes pertaining to the +user interface are: + + allow decompression of symlink'd files to stdout + decompress/test files even without .bz2 extension + give more accurate error messages for I/O errors + when compressing/decompressing to stdout, don't catch control-C + read flags from BZIP2 and BZIP environment variables + decline to break hard links to a file unless forced with -f + allow -c flag even with no filenames + preserve file ownerships as far as possible + make -s -1 give the expected block size (100k) + add a flag -q --quiet to suppress nonessential warnings + stop decoding flags after --, so files beginning in - can be handled + resolved inconsistent naming: bzcat or bz2cat ? + bzip2 --help now returns 0 + +Programming-level changes are: + + fixed syntax error in GET_LL4 for Borland C++ 5.02 + let bzBuffToBuffDecompress return BZ_DATA_ERROR{_MAGIC} + fix overshoot of mode-string end in bzopen_or_bzdopen + wrapped bzlib.h in #ifdef __cplusplus ... extern "C" { ... } + close file handles under all error conditions + added minor mods so it compiles with DJGPP out of the box + fixed Makefile so it doesn't give problems with BSD make + fix uninitialised memory reads in dlltest.c + +0.9.5b +~~~~~~ +Open stdin/stdout in binary mode for DJGPP. + +0.9.5c +~~~~~~ +Changed BZ_N_OVERSHOOT to be ... + 2 instead of ... + 1. The + 1 +version could cause the sorted order to be wrong in some extremely +obscure cases. Also changed setting of quadrant in blocksort.c. + +0.9.5d +~~~~~~ +The only functional change is to make bzlibVersion() in the library +return the correct string. This has no effect whatsoever on the +functioning of the bzip2 program or library. Added a couple of casts +so the library compiles without warnings at level 3 in MS Visual +Studio 6.0. Included a Y2K statement in the file Y2K_INFO. All other +changes are minor documentation changes. diff --git a/LICENSE b/LICENSE index 3de0301..bc0069a 100644 --- a/LICENSE +++ b/LICENSE @@ -1,6 +1,6 @@ This program, "bzip2" and associated library "libbzip2", are -copyright (C) 1996-1998 Julian R Seward. All rights reserved. +copyright (C) 1996-1999 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -33,7 +33,7 @@ WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -Julian Seward, Guildford, Surrey, UK. +Julian Seward, Cambridge, UK. jseward@acm.org -bzip2/libbzip2 version 0.9.0 of 28 June 1998 +bzip2/libbzip2 version 0.9.5 of 24 May 1999 diff --git a/Makefile b/Makefile index 8ebea66..8a1235d 100644 --- a/Makefile +++ b/Makefile @@ -1,6 +1,7 @@ +SHELL=/bin/sh CC=gcc -CFLAGS=-Wall -O2 -fomit-frame-pointer -fno-strength-reduce +CFLAGS=-Wall -Winline -O2 -fomit-frame-pointer -fno-strength-reduce OBJS= blocksort.o \ huffman.o \ @@ -10,37 +11,93 @@ OBJS= blocksort.o \ decompress.o \ bzlib.o -all: lib bzip2 test +all: libbz2.a bzip2 bzip2recover test -bzip2: lib - $(CC) $(CFLAGS) -c bzip2.c +bzip2: libbz2.a bzip2.o $(CC) $(CFLAGS) -o bzip2 bzip2.o -L. -lbz2 - $(CC) $(CFLAGS) -o bzip2recover bzip2recover.c -lib: $(OBJS) +bzip2recover: bzip2recover.o + $(CC) $(CFLAGS) -o bzip2recover bzip2recover.o + +libbz2.a: $(OBJS) rm -f libbz2.a - ar clq libbz2.a $(OBJS) + ar cq libbz2.a $(OBJS) + @if ( test -f /usr/bin/ranlib -o -f /bin/ranlib -o \ + -f /usr/ccs/bin/ranlib ) ; then \ + echo ranlib libbz2.a ; \ + ranlib libbz2.a ; \ + fi test: bzip2 @cat words1 - ./bzip2 -1 < sample1.ref > sample1.rb2 - ./bzip2 -2 < sample2.ref > sample2.rb2 - ./bzip2 -d < sample1.bz2 > sample1.tst - ./bzip2 -d < sample2.bz2 > sample2.tst - @cat words2 + ./bzip2 -1 < sample1.ref > sample1.rb2 + ./bzip2 -2 < sample2.ref > sample2.rb2 + ./bzip2 -3 < sample3.ref > sample3.rb2 + ./bzip2 -d < sample1.bz2 > sample1.tst + ./bzip2 -d < sample2.bz2 > sample2.tst + ./bzip2 -ds < sample3.bz2 > sample3.tst cmp sample1.bz2 sample1.rb2 cmp sample2.bz2 sample2.rb2 + cmp sample3.bz2 sample3.rb2 cmp sample1.tst sample1.ref cmp sample2.tst sample2.ref + cmp sample3.tst sample3.ref @cat words3 +PREFIX=/usr + +install: bzip2 bzip2recover + if ( test ! -d $(PREFIX)/bin ) ; then mkdir $(PREFIX)/bin ; fi + if ( test ! -d $(PREFIX)/lib ) ; then mkdir $(PREFIX)/lib ; fi + if ( test ! -d $(PREFIX)/man ) ; then mkdir $(PREFIX)/man ; fi + if ( test ! -d $(PREFIX)/man/man1 ) ; then mkdir $(PREFIX)/man/man1 ; fi + if ( test ! -d $(PREFIX)/include ) ; then mkdir $(PREFIX)/include ; fi + cp -f bzip2 $(PREFIX)/bin/bzip2 + cp -f bzip2 $(PREFIX)/bin/bunzip2 + cp -f bzip2 $(PREFIX)/bin/bzcat + cp -f bzip2recover $(PREFIX)/bin/bzip2recover + chmod a+x $(PREFIX)/bin/bzip2 + chmod a+x $(PREFIX)/bin/bunzip2 + chmod a+x $(PREFIX)/bin/bzcat + chmod a+x $(PREFIX)/bin/bzip2recover + cp -f bzip2.1 $(PREFIX)/man/man1 + chmod a+r $(PREFIX)/man/man1/bzip2.1 + cp -f bzlib.h $(PREFIX)/include + chmod a+r $(PREFIX)/include/bzlib.h + cp -f libbz2.a $(PREFIX)/lib + chmod a+r $(PREFIX)/lib/libbz2.a clean: - rm -f *.o libbz2.a bzip2 bzip2recover sample1.rb2 sample2.rb2 sample1.tst sample2.tst + rm -f *.o libbz2.a bzip2 bzip2recover \ + sample1.rb2 sample2.rb2 sample3.rb2 \ + sample1.tst sample2.tst sample3.tst -.c.o: $*.o bzlib.h bzlib_private.h - $(CC) $(CFLAGS) -c $*.c -o $*.o +blocksort.o: blocksort.c + $(CC) $(CFLAGS) -c blocksort.c +huffman.o: huffman.c + $(CC) $(CFLAGS) -c huffman.c +crctable.o: crctable.c + $(CC) $(CFLAGS) -c crctable.c +randtable.o: randtable.c + $(CC) $(CFLAGS) -c randtable.c +compress.o: compress.c + $(CC) $(CFLAGS) -c compress.c +decompress.o: decompress.c + $(CC) $(CFLAGS) -c decompress.c +bzlib.o: bzlib.c + $(CC) $(CFLAGS) -c bzlib.c +bzip2.o: bzip2.c + $(CC) $(CFLAGS) -c bzip2.c +bzip2recover.o: bzip2recover.c + $(CC) $(CFLAGS) -c bzip2recover.c tarfile: - tar cvf interim.tar *.c *.h Makefile manual.texi manual.ps LICENSE bzip2.1 bzip2.1.preformatted bzip2.txt words1 words2 words3 sample1.ref sample2.ref sample1.bz2 sample2.bz2 *.html README CHANGES libbz2.def libbz2.dsp dlltest.dsp + tar cvf interim.tar blocksort.c huffman.c crctable.c \ + randtable.c compress.c decompress.c bzlib.c bzip2.c \ + bzip2recover.c bzlib.h bzlib_private.h Makefile manual.texi \ + manual.ps LICENSE bzip2.1 bzip2.1.preformatted bzip2.txt \ + words1 words2 words3 sample1.ref sample2.ref sample3.ref \ + sample1.bz2 sample2.bz2 sample3.bz2 dlltest.c \ + *.html README CHANGES libbz2.def libbz2.dsp \ + dlltest.dsp makefile.msc Y2K_INFO diff --git a/README b/README index 2f59ef7..ee70649 100644 --- a/README +++ b/README @@ -1,48 +1,44 @@ - This is the README for bzip2, a block-sorting file compressor, version -0.9.0. This version is fully compatible with the previous public -release, bzip2-0.1pl2. +0.9.5d. This version is fully compatible with the previous public +releases, bzip2-0.1pl2 and bzip2-0.9.0. -bzip2-0.9.0 is distributed under a BSD-style license. For details, +bzip2-0.9.5 is distributed under a BSD-style license. For details, see the file LICENSE. -Complete documentation is available in Postscript form (manual.ps) -or html (manual_toc.html). A plain-text version of the manual page is -available as bzip2.txt. +Complete documentation is available in Postscript form (manual.ps) or +html (manual_toc.html). A plain-text version of the manual page is +available as bzip2.txt. A statement about Y2K issues is now included +in the file Y2K_INFO. HOW TO BUILD -- UNIX -Type `make'. - -This creates binaries "bzip2" and "bzip2recover". - -It also runs four compress-decompress tests to make sure things are -working properly. If all goes well, you should be up & running. -Please be sure to read the output from `make' just to be sure that the -tests went ok. - -To install bzip2 properly: - -* Copy the binaries "bzip2" and "bzip2recover" to a publically visible - place, possibly /usr/bin or /usr/local/bin. +Type `make'. This builds the library libbz2.a and then the +programs bzip2 and bzip2recover. Six self-tests are run. +If the self-tests complete ok, carry on to installation: -* In that directory, make "bunzip2" and "bzcat" be symbolic links - to "bzip2". +To install in /usr/bin, /usr/lib, /usr/man and /usr/include, type + make install +To install somewhere else, eg, /xxx/yyy/{bin,lib,man,include}, type + make install PREFIX=/xxx/yyy +If you are (justifiably) paranoid and want to see what 'make install' +is going to do, you can first do + make -n install or + make -n install PREFIX=/xxx/yyy respectively. +The -n instructs make to show the commands it would execute, but +not actually execute them. -* Copy the manual page, bzip2.1, to the relevant place. - Probably the right place is /usr/man/man1/. - -If you want to program with the library, you'll need to copy libbz2.a -and bzlib.h to /usr/lib and /usr/include respectively. - HOW TO BUILD -- Windows 95, NT, DOS, Mac, etc. It's difficult for me to support compilation on all these platforms. My approach is to collect binaries for these platforms, and put them -on my web page (http://www.muraroa.demon.co.uk). Look there. +on my web page (http://www.muraroa.demon.co.uk). Look there. However +(FWIW), bzip2-0.9.5 is very standard ANSI C and should compile +unmodified with MS Visual C. For Win32, there is one important +caveat: in bzip2.c, you must set BZ_UNIX to 0 and BZ_LCCWIN32 to 1 +before building. VALIDATION @@ -112,26 +108,32 @@ WHAT'S NEW IN 0.9.0 (as compared to 0.1pl2) ? * Much more documentation, i.e., a proper user manual * Hopefully, improved portability (at least of the library) +WHAT'S NEW IN 0.9.5 ? + + * Compression speed is much less sensitive to the input + data than in previous versions. Specifically, the very + slow performance caused by repetitive data is fixed. + * Many small improvements in file and flag handling. + * A Y2K statement. I hope you find bzip2 useful. Feel free to contact me at jseward@acm.org if you have any suggestions or queries. Many people mailed me with comments, suggestions and patches after the releases of bzip-0.15, -bzip-0.21 and bzip2-0.1pl2, and the changes in bzip2 are largely a -result of this feedback. I thank you for your comments. +bzip-0.21, bzip2-0.1pl2 and bzip2-0.9.0, and the changes in bzip2 are +largely a result of this feedback. I thank you for your comments. -At least for the time being, bzip2's "home" is +At least for the time being, bzip2's "home" is (or can be reached via) http://www.muraroa.demon.co.uk. Julian Seward jseward@acm.org -Manchester, UK -18 July 1996 (version 0.15) +Cambridge, UK +18 July 1996 (version 0.15) 25 August 1996 (version 0.21) - -Guildford, Surrey, UK -7 August 1997 (bzip2, version 0.1) + 7 August 1997 (bzip2, version 0.1) 29 August 1997 (bzip2, version 0.1pl2) 23 August 1998 (bzip2, version 0.9.0) - + 8 June 1999 (bzip2, version 0.9.5) + 4 Sept 1999 (bzip2, version 0.9.5d) diff --git a/Y2K_INFO b/Y2K_INFO new file mode 100644 index 0000000..55fd56a --- /dev/null +++ b/Y2K_INFO @@ -0,0 +1,34 @@ + +Y2K status of bzip2 and libbzip2, versions 0.1, 0.9.0 and 0.9.5 +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Informally speaking: + bzip2 is a compression program built on top of libbzip2, + a library which does the real work of compression and + decompression. As far as I am aware, libbzip2 does not have + any date-related code at all. + + bzip2 itself copies dates from source to destination files + when compressing or decompressing, using the 'stat' and 'utime' + UNIX system calls. It doesn't examine, manipulate or store the + dates in any way. So as far as I can see, there shouldn't be any + problem with bzip2 providing 'stat' and 'utime' work correctly + on your system. + + On non-unix platforms (those for which BZ_UNIX in bzip2.c is + not set to 1), bzip2 doesn't even do the date copying. + + Overall, informally speaking, I don't think bzip2 or libbzip2 + have a Y2K problem. + +Formally speaking: + I am not prepared to offer you any assurance whatsoever + regarding Y2K issues in my software. You alone assume the + entire risk of using the software. The disclaimer of liability + in the LICENSE file in the bzip2 source distribution continues + to apply on this issue as with every other issue pertaining + to the software. + +Julian Seward +Cambridge, UK +25 August 1999 diff --git a/blocksort.c b/blocksort.c index d8bb26a..85a02de 100644 --- a/blocksort.c +++ b/blocksort.c @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-1998 Julian R Seward. All rights reserved. + Copyright (C) 1996-1999 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -41,9 +41,9 @@ NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - Julian Seward, Guildford, Surrey, UK. + Julian Seward, Cambridge, UK. jseward@acm.org - bzip2/libbzip2 version 0.9.0c of 18 October 1998 + bzip2/libbzip2 version 0.9.5 of 24 May 1999 This program is based on (at least) the work of: Mike Burrows @@ -62,106 +62,404 @@ #include "bzlib_private.h" /*---------------------------------------------*/ -/*-- - Compare two strings in block. We assume (see - discussion above) that i1 and i2 have a max - offset of 10 on entry, and that the first - bytes of both block and quadrant have been - copied into the "overshoot area", ie - into the subscript range - [nblock .. nblock+NUM_OVERSHOOT_BYTES-1]. ---*/ -static __inline__ Bool fullGtU ( UChar* block, - UInt16* quadrant, - UInt32 nblock, - Int32* workDone, - Int32 i1, - Int32 i2 - ) +/*--- Fallback O(N log(N)^2) sorting ---*/ +/*--- algorithm, for repetitive blocks ---*/ +/*---------------------------------------------*/ + +/*---------------------------------------------*/ +static +__inline__ +void fallbackSimpleSort ( UInt32* fmap, + UInt32* eclass, + Int32 lo, + Int32 hi ) +{ + Int32 i, j, tmp; + UInt32 ec_tmp; + + if (lo == hi) return; + + if (hi - lo > 3) { + for ( i = hi-4; i >= lo; i-- ) { + tmp = fmap[i]; + ec_tmp = eclass[tmp]; + for ( j = i+4; j <= hi && ec_tmp > eclass[fmap[j]]; j += 4 ) + fmap[j-4] = fmap[j]; + fmap[j-4] = tmp; + } + } + + for ( i = hi-1; i >= lo; i-- ) { + tmp = fmap[i]; + ec_tmp = eclass[tmp]; + for ( j = i+1; j <= hi && ec_tmp > eclass[fmap[j]]; j++ ) + fmap[j-1] = fmap[j]; + fmap[j-1] = tmp; + } +} + + +/*---------------------------------------------*/ +#define fswap(zz1, zz2) \ + { Int32 zztmp = zz1; zz1 = zz2; zz2 = zztmp; } + +#define fvswap(zzp1, zzp2, zzn) \ +{ \ + Int32 yyp1 = (zzp1); \ + Int32 yyp2 = (zzp2); \ + Int32 yyn = (zzn); \ + while (yyn > 0) { \ + fswap(fmap[yyp1], fmap[yyp2]); \ + yyp1++; yyp2++; yyn--; \ + } \ +} + + +#define fmin(a,b) ((a) < (b)) ? (a) : (b) + +#define fpush(lz,hz) { stackLo[sp] = lz; \ + stackHi[sp] = hz; \ + sp++; } + +#define fpop(lz,hz) { sp--; \ + lz = stackLo[sp]; \ + hz = stackHi[sp]; } + +#define FALLBACK_QSORT_SMALL_THRESH 10 +#define FALLBACK_QSORT_STACK_SIZE 100 + + +static +void fallbackQSort3 ( UInt32* fmap, + UInt32* eclass, + Int32 loSt, + Int32 hiSt ) +{ + Int32 unLo, unHi, ltLo, gtHi, n, m; + Int32 sp, lo, hi; + UInt32 med, r, r3; + Int32 stackLo[FALLBACK_QSORT_STACK_SIZE]; + Int32 stackHi[FALLBACK_QSORT_STACK_SIZE]; + + r = 0; + + sp = 0; + fpush ( loSt, hiSt ); + + while (sp > 0) { + + AssertH ( sp < FALLBACK_QSORT_STACK_SIZE, 1004 ); + + fpop ( lo, hi ); + if (hi - lo < FALLBACK_QSORT_SMALL_THRESH) { + fallbackSimpleSort ( fmap, eclass, lo, hi ); + continue; + } + + /* Random partitioning. Median of 3 sometimes fails to + avoid bad cases. Median of 9 seems to help but + looks rather expensive. This too seems to work but + is cheaper. Guidance for the magic constants + 7621 and 32768 is taken from Sedgewick's algorithms + book, chapter 35. + */ + r = ((r * 7621) + 1) % 32768; + r3 = r % 3; + if (r3 == 0) med = eclass[fmap[lo]]; else + if (r3 == 1) med = eclass[fmap[(lo+hi)>>1]]; else + med = eclass[fmap[hi]]; + + unLo = ltLo = lo; + unHi = gtHi = hi; + + while (1) { + while (1) { + if (unLo > unHi) break; + n = (Int32)eclass[fmap[unLo]] - (Int32)med; + if (n == 0) { + fswap(fmap[unLo], fmap[ltLo]); + ltLo++; unLo++; + continue; + }; + if (n > 0) break; + unLo++; + } + while (1) { + if (unLo > unHi) break; + n = (Int32)eclass[fmap[unHi]] - (Int32)med; + if (n == 0) { + fswap(fmap[unHi], fmap[gtHi]); + gtHi--; unHi--; + continue; + }; + if (n < 0) break; + unHi--; + } + if (unLo > unHi) break; + fswap(fmap[unLo], fmap[unHi]); unLo++; unHi--; + } + + AssertD ( unHi == unLo-1, "fallbackQSort3(2)" ); + + if (gtHi < ltLo) continue; + + n = fmin(ltLo-lo, unLo-ltLo); fvswap(lo, unLo-n, n); + m = fmin(hi-gtHi, gtHi-unHi); fvswap(unLo, hi-m+1, m); + + n = lo + unLo - ltLo - 1; + m = hi - (gtHi - unHi) + 1; + + if (n - lo > hi - m) { + fpush ( lo, n ); + fpush ( m, hi ); + } else { + fpush ( m, hi ); + fpush ( lo, n ); + } + } +} + +#undef fmin +#undef fpush +#undef fpop +#undef fswap +#undef fvswap +#undef FALLBACK_QSORT_SMALL_THRESH +#undef FALLBACK_QSORT_STACK_SIZE + + +/*---------------------------------------------*/ +/* Pre: + nblock > 0 + eclass exists for [0 .. nblock-1] + ((UInt16*)eclass) [0 .. nblock-1] [15:8] holds block + ptr exists for [0 .. nblock-1] + + Post: + ((UInt16*)eclass) [0 .. nblock-1] [15:8] holds block + All other areas of eclass destroyed + fmap [0 .. nblock-1] holds sorted order + bhtab [ 0 .. 2+(nblock/32) ] destroyed +*/ + +#define SET_BH(zz) bhtab[(zz) >> 5] |= (1 << ((zz) & 31)) +#define CLEAR_BH(zz) bhtab[(zz) >> 5] &= ~(1 << ((zz) & 31)) +#define ISSET_BH(zz) (bhtab[(zz) >> 5] & (1 << ((zz) & 31))) +#define WORD_BH(zz) bhtab[(zz) >> 5] +#define UNALIGNED_BH(zz) ((zz) & 0x01f) + +static +void fallbackSort ( UInt32* fmap, + UInt32* eclass, + UInt32* bhtab, + Int32 nblock, + Int32 verb ) +{ + Int32 ftab[257]; + Int32 ftabCopy[256]; + Int32 H, i, j, k, l, r, cc, cc1; + Int32 nNotDone; + Int32 nBhtab; + UInt16* eclass16 = (UInt16*)eclass; + + /*-- + Initial 1-char radix sort to generate + initial fmap and initial BH bits. + --*/ + if (verb >= 4) + VPrintf0 ( " bucket sorting ...\n" ); + for (i = 0; i < 257; i++) ftab[i] = 0; + for (i = 0; i < nblock; i++) ftab[eclass16[i] >> 8]++; + for (i = 0; i < 256; i++) ftabCopy[i] = ftab[i]; + for (i = 1; i < 257; i++) ftab[i] += ftab[i-1]; + + for (i = 0; i < nblock; i++) { + j = eclass16[i] >> 8; + k = ftab[j] - 1; + ftab[j] = k; + fmap[k] = i; + } + + nBhtab = 2 + (nblock / 32); + for (i = 0; i < nBhtab; i++) bhtab[i] = 0; + for (i = 0; i < 256; i++) SET_BH(ftab[i]); + + /*-- + Inductively refine the buckets. Kind-of an + "exponential radix sort" (!), inspired by the + Manber-Myers suffix array construction algorithm. + --*/ + + /*-- set sentinel bits for block-end detection --*/ + for (i = 0; i < 32; i++) { + SET_BH(nblock + 2*i); + CLEAR_BH(nblock + 2*i + 1); + } + + /*-- the log(N) loop --*/ + H = 1; + while (1) { + + if (verb >= 4) + VPrintf1 ( " depth %6d has ", H ); + + j = 0; + for (i = 0; i < nblock; i++) { + if (ISSET_BH(i)) j = i; + k = fmap[i] - H; if (k < 0) k += nblock; + eclass[k] = j; + } + + nNotDone = 0; + r = -1; + while (1) { + + /*-- find the next non-singleton bucket --*/ + k = r + 1; + while (ISSET_BH(k) && UNALIGNED_BH(k)) k++; + if (ISSET_BH(k)) { + while (WORD_BH(k) == 0xffffffff) k += 32; + while (ISSET_BH(k)) k++; + } + l = k - 1; + if (l >= nblock) break; + while (!ISSET_BH(k) && UNALIGNED_BH(k)) k++; + if (!ISSET_BH(k)) { + while (WORD_BH(k) == 0x00000000) k += 32; + while (!ISSET_BH(k)) k++; + } + r = k - 1; + if (r >= nblock) break; + + /*-- now [l, r] bracket current bucket --*/ + if (r > l) { + nNotDone += (r - l + 1); + fallbackQSort3 ( fmap, eclass, l, r ); + + /*-- scan bucket and generate header bits-- */ + cc = -1; + for (i = l; i <= r; i++) { + cc1 = eclass[fmap[i]]; + if (cc != cc1) { SET_BH(i); cc = cc1; }; + } + } + } + + if (verb >= 4) + VPrintf1 ( "%6d unresolved strings\n", nNotDone ); + + H *= 2; + if (H > nblock || nNotDone == 0) break; + } + + /*-- + Reconstruct the original block in + eclass16 [0 .. nblock-1] [15:8], since the + previous phase destroyed it. + --*/ + if (verb >= 4) + VPrintf0 ( " reconstructing block ...\n" ); + j = 0; + for (i = 0; i < nblock; i++) { + while (ftabCopy[j] == 0) j++; + ftabCopy[j]--; + eclass16[fmap[i]] = j << 8; + } + AssertH ( j < 256, 1005 ); +} + +#undef SET_BH +#undef CLEAR_BH +#undef ISSET_BH +#undef WORD_BH +#undef UNALIGNED_BH + + +/*---------------------------------------------*/ +/*--- The main, O(N^2 log(N)) sorting ---*/ +/*--- algorithm. Faster for "normal" ---*/ +/*--- non-repetitive blocks. ---*/ +/*---------------------------------------------*/ + +/*---------------------------------------------*/ +static +__inline__ +Bool mainGtU ( UInt32 i1, + UInt32 i2, + UInt16* block, + UInt16* quadrant, + UInt32 nblock, + Int32* budget ) { Int32 k; - UChar c1, c2; UInt16 s1, s2; - AssertD ( i1 != i2, "fullGtU(1)" ); + AssertD ( i1 != i2, "mainGtU" ); - c1 = block[i1]; - c2 = block[i2]; - if (c1 != c2) return (c1 > c2); - i1++; i2++; + s1 = block[i1]; s2 = block[i2]; + if (s1 != s2) return (s1 > s2); + i1 += 2; i2 += 2; - c1 = block[i1]; - c2 = block[i2]; - if (c1 != c2) return (c1 > c2); - i1++; i2++; + s1 = block[i1]; s2 = block[i2]; + if (s1 != s2) return (s1 > s2); + i1 += 2; i2 += 2; - c1 = block[i1]; - c2 = block[i2]; - if (c1 != c2) return (c1 > c2); - i1++; i2++; + s1 = block[i1]; s2 = block[i2]; + if (s1 != s2) return (s1 > s2); + i1 += 2; i2 += 2; - c1 = block[i1]; - c2 = block[i2]; - if (c1 != c2) return (c1 > c2); - i1++; i2++; + s1 = block[i1]; s2 = block[i2]; + if (s1 != s2) return (s1 > s2); + i1 += 2; i2 += 2; - c1 = block[i1]; - c2 = block[i2]; - if (c1 != c2) return (c1 > c2); - i1++; i2++; + s1 = block[i1]; s2 = block[i2]; + if (s1 != s2) return (s1 > s2); + i1 += 2; i2 += 2; - c1 = block[i1]; - c2 = block[i2]; - if (c1 != c2) return (c1 > c2); - i1++; i2++; + s1 = block[i1]; s2 = block[i2]; + if (s1 != s2) return (s1 > s2); + i1 += 2; i2 += 2; - k = nblock; + k = nblock + 8; do { - c1 = block[i1]; - c2 = block[i2]; - if (c1 != c2) return (c1 > c2); - s1 = quadrant[i1]; - s2 = quadrant[i2]; + s1 = block[i1]; s2 = block[i2]; if (s1 != s2) return (s1 > s2); - i1++; i2++; + s1 = quadrant[i1]; s2 = quadrant[i2]; + if (s1 != s2) return (s1 > s2); + i1 += 2; i2 += 2; - c1 = block[i1]; - c2 = block[i2]; - if (c1 != c2) return (c1 > c2); - s1 = quadrant[i1]; - s2 = quadrant[i2]; + s1 = block[i1]; s2 = block[i2]; + if (s1 != s2) return (s1 > s2); + s1 = quadrant[i1]; s2 = quadrant[i2]; if (s1 != s2) return (s1 > s2); - i1++; i2++; + i1 += 2; i2 += 2; - c1 = block[i1]; - c2 = block[i2]; - if (c1 != c2) return (c1 > c2); - s1 = quadrant[i1]; - s2 = quadrant[i2]; + s1 = block[i1]; s2 = block[i2]; if (s1 != s2) return (s1 > s2); - i1++; i2++; + s1 = quadrant[i1]; s2 = quadrant[i2]; + if (s1 != s2) return (s1 > s2); + i1 += 2; i2 += 2; - c1 = block[i1]; - c2 = block[i2]; - if (c1 != c2) return (c1 > c2); - s1 = quadrant[i1]; - s2 = quadrant[i2]; + s1 = block[i1]; s2 = block[i2]; + if (s1 != s2) return (s1 > s2); + s1 = quadrant[i1]; s2 = quadrant[i2]; if (s1 != s2) return (s1 > s2); - i1++; i2++; + i1 += 2; i2 += 2; if (i1 >= nblock) i1 -= nblock; if (i2 >= nblock) i2 -= nblock; - k -= 4; - (*workDone)++; + k -= 8; + (*budget)--; } while (k >= 0); return False; } + /*---------------------------------------------*/ /*-- Knuth's increments seem to work better @@ -169,22 +467,22 @@ static __inline__ Bool fullGtU ( UChar* block, because the number of elems to sort is usually small, typically <= 20. --*/ -static Int32 incs[14] = { 1, 4, 13, 40, 121, 364, 1093, 3280, - 9841, 29524, 88573, 265720, - 797161, 2391484 }; - -static void simpleSort ( EState* s, Int32 lo, Int32 hi, Int32 d ) +Int32 incs[14] = { 1, 4, 13, 40, 121, 364, 1093, 3280, + 9841, 29524, 88573, 265720, + 797161, 2391484 }; + +static +void mainSimpleSort ( UInt32* ptr, + UInt16* block, + UInt16* quadrant, + Int32 nblock, + Int32 lo, + Int32 hi, + Int32 d, + Int32* budget ) { Int32 i, j, h, bigN, hp; - Int32 v; - - UChar* block = s->block; - UInt32* zptr = s->zptr; - UInt16* quadrant = s->quadrant; - Int32* workDone = &(s->workDone); - Int32 nblock = s->nblock; - Int32 workLimit = s->workLimit; - Bool firstAttempt = s->firstAttempt; + UInt32 v; bigN = hi - lo + 1; if (bigN < 2) return; @@ -195,49 +493,53 @@ static void simpleSort ( EState* s, Int32 lo, Int32 hi, Int32 d ) for (; hp >= 0; hp--) { h = incs[hp]; + i = lo + h; while (True) { /*-- copy 1 --*/ if (i > hi) break; - v = zptr[i]; + v = ptr[i]; j = i; - while ( fullGtU ( block, quadrant, nblock, workDone, - zptr[j-h]+d, v+d ) ) { - zptr[j] = zptr[j-h]; + while ( mainGtU ( + ptr[j-h]+d, v+d, block, quadrant, nblock, budget + ) ) { + ptr[j] = ptr[j-h]; j = j - h; if (j <= (lo + h - 1)) break; } - zptr[j] = v; + ptr[j] = v; i++; /*-- copy 2 --*/ if (i > hi) break; - v = zptr[i]; + v = ptr[i]; j = i; - while ( fullGtU ( block, quadrant, nblock, workDone, - zptr[j-h]+d, v+d ) ) { - zptr[j] = zptr[j-h]; + while ( mainGtU ( + ptr[j-h]+d, v+d, block, quadrant, nblock, budget + ) ) { + ptr[j] = ptr[j-h]; j = j - h; if (j <= (lo + h - 1)) break; } - zptr[j] = v; + ptr[j] = v; i++; /*-- copy 3 --*/ if (i > hi) break; - v = zptr[i]; + v = ptr[i]; j = i; - while ( fullGtU ( block, quadrant, nblock, workDone, - zptr[j-h]+d, v+d ) ) { - zptr[j] = zptr[j-h]; + while ( mainGtU ( + ptr[j-h]+d, v+d, block, quadrant, nblock, budget + ) ) { + ptr[j] = ptr[j-h]; j = j - h; if (j <= (lo + h - 1)) break; } - zptr[j] = v; + ptr[j] = v; i++; - if (*workDone > workLimit && firstAttempt) return; + if (*budget < 0) return; } } } @@ -252,20 +554,26 @@ static void simpleSort ( EState* s, Int32 lo, Int32 hi, Int32 d ) Sedgewick and Jon L. Bentley. --*/ -#define swap(lv1, lv2) \ - { Int32 tmp = lv1; lv1 = lv2; lv2 = tmp; } - -static void vswap ( UInt32* zptr, Int32 p1, Int32 p2, Int32 n ) -{ - while (n > 0) { - swap(zptr[p1], zptr[p2]); - p1++; p2++; n--; - } +#define mswap(zz1, zz2) \ + { Int32 zztmp = zz1; zz1 = zz2; zz2 = zztmp; } + +#define mvswap(zzp1, zzp2, zzn) \ +{ \ + Int32 yyp1 = (zzp1); \ + Int32 yyp2 = (zzp2); \ + Int32 yyn = (zzn); \ + while (yyn > 0) { \ + mswap(ptr[yyp1], ptr[yyp2]); \ + yyp1++; yyp2++; yyn--; \ + } \ } -static UChar med3 ( UChar a, UChar b, UChar c ) + +static +__inline__ +UInt16 mmed3 ( UInt16 a, UInt16 b, UInt16 c ) { - UChar t; + UInt16 t; if (a > b) { t = a; a = b; b = t; }; if (b > c) { t = b; b = c; c = t; }; if (a > b) b = a; @@ -273,66 +581,72 @@ static UChar med3 ( UChar a, UChar b, UChar c ) } -#define min(a,b) ((a) < (b)) ? (a) : (b) +#define mmin(a,b) ((a) < (b)) ? (a) : (b) -typedef - struct { Int32 ll; Int32 hh; Int32 dd; } - StackElem; +#define mpush(lz,hz,dz) { stackLo[sp] = lz; \ + stackHi[sp] = hz; \ + stackD [sp] = dz; \ + sp++; } -#define push(lz,hz,dz) { stack[sp].ll = lz; \ - stack[sp].hh = hz; \ - stack[sp].dd = dz; \ - sp++; } +#define mpop(lz,hz,dz) { sp--; \ + lz = stackLo[sp]; \ + hz = stackHi[sp]; \ + dz = stackD [sp]; } -#define pop(lz,hz,dz) { sp--; \ - lz = stack[sp].ll; \ - hz = stack[sp].hh; \ - dz = stack[sp].dd; } -#define SMALL_THRESH 20 -#define DEPTH_THRESH 10 +#define mnextsize(az) (nextHi[az]-nextLo[az]) + +#define mnextswap(az,bz) \ + { Int32 tz; \ + tz = nextLo[az]; nextLo[az] = nextLo[bz]; nextLo[bz] = tz; \ + tz = nextHi[az]; nextHi[az] = nextHi[bz]; nextHi[bz] = tz; \ + tz = nextD [az]; nextD [az] = nextD [bz]; nextD [bz] = tz; } -/*-- - If you are ever unlucky/improbable enough - to get a stack overflow whilst sorting, - increase the following constant and try - again. In practice I have never seen the - stack go above 27 elems, so the following - limit seems very generous. ---*/ -#define QSORT_STACK_SIZE 1000 +#define MAIN_QSORT_SMALL_THRESH 20 +#define MAIN_QSORT_DEPTH_THRESH (BZ_N_RADIX + BZ_N_QSORT) +#define MAIN_QSORT_STACK_SIZE 100 -static void qSort3 ( EState* s, Int32 loSt, Int32 hiSt, Int32 dSt ) +static +void mainQSort3 ( UInt32* ptr, + UInt16* block, + UInt16* quadrant, + Int32 nblock, + Int32 loSt, + Int32 hiSt, + Int32 dSt, + Int32* budget ) { - Int32 unLo, unHi, ltLo, gtHi, med, n, m; + Int32 unLo, unHi, ltLo, gtHi, n, m, med; Int32 sp, lo, hi, d; - StackElem stack[QSORT_STACK_SIZE]; - UChar* block = s->block; - UInt32* zptr = s->zptr; - Int32* workDone = &(s->workDone); - Int32 workLimit = s->workLimit; - Bool firstAttempt = s->firstAttempt; + Int32 stackLo[MAIN_QSORT_STACK_SIZE]; + Int32 stackHi[MAIN_QSORT_STACK_SIZE]; + Int32 stackD [MAIN_QSORT_STACK_SIZE]; + + Int32 nextLo[3]; + Int32 nextHi[3]; + Int32 nextD [3]; sp = 0; - push ( loSt, hiSt, dSt ); + mpush ( loSt, hiSt, dSt ); while (sp > 0) { - AssertH ( sp < QSORT_STACK_SIZE, 1001 ); + AssertH ( sp < MAIN_QSORT_STACK_SIZE, 1001 ); - pop ( lo, hi, d ); - - if (hi - lo < SMALL_THRESH || d > DEPTH_THRESH) { - simpleSort ( s, lo, hi, d ); - if (*workDone > workLimit && firstAttempt) return; + mpop ( lo, hi, d ); + if (hi - lo < MAIN_QSORT_SMALL_THRESH || + d > MAIN_QSORT_DEPTH_THRESH) { + mainSimpleSort ( ptr, block, quadrant, nblock, lo, hi, d, budget ); + if (*budget < 0) return; continue; } - med = med3 ( block[zptr[ lo ]+d], - block[zptr[ hi ]+d], - block[zptr[ (lo+hi)>>1 ]+d] ); + med = (Int32) + mmed3 ( block[ptr[ lo ]+d], + block[ptr[ hi ]+d], + block[ptr[ (lo+hi)>>1 ]+d] ); unLo = ltLo = lo; unHi = gtHi = hi; @@ -340,370 +654,412 @@ static void qSort3 ( EState* s, Int32 loSt, Int32 hiSt, Int32 dSt ) while (True) { while (True) { if (unLo > unHi) break; - n = ((Int32)block[zptr[unLo]+d]) - med; - if (n == 0) { swap(zptr[unLo], zptr[ltLo]); ltLo++; unLo++; continue; }; + n = ((Int32)block[ptr[unLo]+d]) - med; + if (n == 0) { + mswap(ptr[unLo], ptr[ltLo]); + ltLo++; unLo++; continue; + }; if (n > 0) break; unLo++; } while (True) { if (unLo > unHi) break; - n = ((Int32)block[zptr[unHi]+d]) - med; - if (n == 0) { swap(zptr[unHi], zptr[gtHi]); gtHi--; unHi--; continue; }; + n = ((Int32)block[ptr[unHi]+d]) - med; + if (n == 0) { + mswap(ptr[unHi], ptr[gtHi]); + gtHi--; unHi--; continue; + }; if (n < 0) break; unHi--; } if (unLo > unHi) break; - swap(zptr[unLo], zptr[unHi]); unLo++; unHi--; + mswap(ptr[unLo], ptr[unHi]); unLo++; unHi--; } - AssertD ( unHi == unLo-1, "bad termination in qSort3" ); + AssertD ( unHi == unLo-1, "mainQSort3(2)" ); if (gtHi < ltLo) { - push(lo, hi, d+1 ); + mpush(lo, hi, d+2 ); continue; } - n = min(ltLo-lo, unLo-ltLo); vswap(zptr, lo, unLo-n, n); - m = min(hi-gtHi, gtHi-unHi); vswap(zptr, unLo, hi-m+1, m); + n = mmin(ltLo-lo, unLo-ltLo); mvswap(lo, unLo-n, n); + m = mmin(hi-gtHi, gtHi-unHi); mvswap(unLo, hi-m+1, m); n = lo + unLo - ltLo - 1; m = hi - (gtHi - unHi) + 1; - push ( lo, n, d ); - push ( n+1, m-1, d+1 ); - push ( m, hi, d ); + nextLo[0] = lo; nextHi[0] = n; nextD[0] = d; + nextLo[1] = m; nextHi[1] = hi; nextD[1] = d; + nextLo[2] = n+1; nextHi[2] = m-1; nextD[2] = d+2; + + if (mnextsize(0) < mnextsize(1)) mnextswap(0,1); + if (mnextsize(1) < mnextsize(2)) mnextswap(1,2); + if (mnextsize(0) < mnextsize(1)) mnextswap(0,1); + + AssertD (mnextsize(0) >= mnextsize(1), "mainQSort3(8)" ); + AssertD (mnextsize(1) >= mnextsize(2), "mainQSort3(9)" ); + + mpush (nextLo[0], nextHi[0], nextD[0]); + mpush (nextLo[1], nextHi[1], nextD[1]); + mpush (nextLo[2], nextHi[2], nextD[2]); } } +#undef mswap +#undef mvswap +#undef mpush +#undef mpop +#undef mmin +#undef mnextsize +#undef mnextswap +#undef MAIN_QSORT_SMALL_THRESH +#undef MAIN_QSORT_DEPTH_THRESH +#undef MAIN_QSORT_STACK_SIZE + /*---------------------------------------------*/ +/* Pre: + nblock > N_OVERSHOOT + block32 exists for [0 .. nblock-1 +N_OVERSHOOT] + ((UInt16*)block32) [0 .. nblock-1] [15:8] holds block + ptr exists for [0 .. nblock-1] + + Post: + ((UInt16*)block32) [0 .. nblock-1] [15:8] holds block + All other areas of block32 destroyed + ftab [0 .. 65536 ] destroyed + ptr [0 .. nblock-1] holds sorted order + if (*budget < 0), sorting was abandoned +*/ #define BIGFREQ(b) (ftab[((b)+1) << 8] - ftab[(b) << 8]) - #define SETMASK (1 << 21) #define CLEARMASK (~(SETMASK)) -static void sortMain ( EState* s ) +static +void mainSort ( UInt32* ptr, + UInt16* block, + UInt16* quadrant, + UInt32* ftab, + Int32 nblock, + Int32 verb, + Int32* budget ) { - Int32 i, j, k, ss, sb; - Int32 runningOrder[256]; - Int32 copy[256]; - Bool bigDone[256]; - UChar c1, c2; - Int32 numQSorted; - - UChar* block = s->block; - UInt32* zptr = s->zptr; - UInt16* quadrant = s->quadrant; - Int32* ftab = s->ftab; - Int32* workDone = &(s->workDone); - Int32 nblock = s->nblock; - Int32 workLimit = s->workLimit; - Bool firstAttempt = s->firstAttempt; - - /*-- - In the various block-sized structures, live data runs - from 0 to last+NUM_OVERSHOOT_BYTES inclusive. First, - set up the overshoot area for block. + Int32 i, j, k, m, ss, sb; + Int32 runningOrder[256]; + Int32 copy[256]; + Bool bigDone[256]; + UChar c1; + Int32 numQSorted; + Int32 biggestSoFar; + UInt16 s; + + if (verb >= 4) VPrintf0 ( " main sort initialise ...\n" ); + + /*-- Stripe the block data into 16 bits, and at the + same time set up the 2-byte frequency table --*/ + for (i = 65536; i >= 0; i--) ftab[i] = 0; + + s = block[0]; + for (i = 1; i < nblock; i++) { + quadrant[i] = 0; + s = (s << 8) | block[i]; + block[i-1] = s; + ftab[s]++; + } + quadrant[0] = 0; + s = (s << 8) | (block[0] >> 8); + block[nblock-1] = s; + ftab[s]++; + + /*-- (emphasises close relationship of block & quadrant) --*/ + for (i = 0; i < BZ_N_OVERSHOOT; i++) { + block [nblock+i] = block[i]; + quadrant[nblock+i] = 0; + } - if (s->verbosity >= 4) - VPrintf0( " sort initialise ...\n" ); - - for (i = 0; i < BZ_NUM_OVERSHOOT_BYTES; i++) - block[nblock+i] = block[i % nblock]; - for (i = 0; i < nblock+BZ_NUM_OVERSHOOT_BYTES; i++) - quadrant[i] = 0; - - - if (nblock <= 4000) { - - /*-- - Use simpleSort(), since the full sorting mechanism - has quite a large constant overhead. - --*/ - if (s->verbosity >= 4) VPrintf0( " simpleSort ...\n" ); - for (i = 0; i < nblock; i++) zptr[i] = i; - firstAttempt = False; - *workDone = workLimit = 0; - simpleSort ( s, 0, nblock-1, 0 ); - if (s->verbosity >= 4) VPrintf0( " simpleSort done.\n" ); + if (verb >= 4) VPrintf0 ( " bucket sorting ...\n" ); - } else { + /*-- Complete the initial radix sort --*/ + for (i = 1; i <= 65536; i++) ftab[i] += ftab[i-1]; - numQSorted = 0; - for (i = 0; i <= 255; i++) bigDone[i] = False; + for (i = 0; i < nblock; i++) { + s = block[i]; + j = ftab[s] - 1; + ftab[s] = j; + ptr[j] = i; + } - if (s->verbosity >= 4) VPrintf0( " bucket sorting ...\n" ); + /*-- + Now ftab contains the first loc of every small bucket. + Calculate the running order, from smallest to largest + big bucket. + --*/ + for (i = 0; i <= 255; i++) { + bigDone [i] = False; + runningOrder[i] = i; + } - for (i = 0; i <= 65536; i++) ftab[i] = 0; + { + Int32 vv; + Int32 h = 1; + do h = 3 * h + 1; while (h <= 256); + do { + h = h / 3; + for (i = h; i <= 255; i++) { + vv = runningOrder[i]; + j = i; + while ( BIGFREQ(runningOrder[j-h]) > BIGFREQ(vv) ) { + runningOrder[j] = runningOrder[j-h]; + j = j - h; + if (j <= (h - 1)) goto zero; + } + zero: + runningOrder[j] = vv; + } + } while (h != 1); + } - c1 = block[nblock-1]; - for (i = 0; i < nblock; i++) { - c2 = block[i]; - ftab[(c1 << 8) + c2]++; - c1 = c2; - } + /*-- + The main sorting loop. + --*/ - for (i = 1; i <= 65536; i++) ftab[i] += ftab[i-1]; + biggestSoFar = numQSorted = 0; - c1 = block[0]; - for (i = 0; i < nblock-1; i++) { - c2 = block[i+1]; - j = (c1 << 8) + c2; - c1 = c2; - ftab[j]--; - zptr[ftab[j]] = i; - } - j = (block[nblock-1] << 8) + block[0]; - ftab[j]--; - zptr[ftab[j]] = nblock-1; + for (i = 0; i <= 255; i++) { /*-- - Now ftab contains the first loc of every small bucket. - Calculate the running order, from smallest to largest - big bucket. + Process big buckets, starting with the least full. + Basically this is a 4-step process in which we call + mainQSort3 to sort the small buckets [ss, j], but + also make a big effort to avoid the calls if we can. --*/ + ss = runningOrder[i]; - for (i = 0; i <= 255; i++) runningOrder[i] = i; - - { - Int32 vv; - Int32 h = 1; - do h = 3 * h + 1; while (h <= 256); - do { - h = h / 3; - for (i = h; i <= 255; i++) { - vv = runningOrder[i]; - j = i; - while ( BIGFREQ(runningOrder[j-h]) > BIGFREQ(vv) ) { - runningOrder[j] = runningOrder[j-h]; - j = j - h; - if (j <= (h - 1)) goto zero; + /*-- + Step 1: + Complete the big bucket [ss] by quicksorting + any unsorted small buckets [ss, j], for j != ss. + Hopefully previous pointer-scanning phases have already + completed many of the small buckets [ss, j], so + we don't have to sort them at all. + --*/ + for (j = 0; j <= 255; j++) { + if (j != ss) { + sb = (ss << 8) + j; + if ( ! (ftab[sb] & SETMASK) ) { + Int32 lo = ftab[sb] & CLEARMASK; + Int32 hi = (ftab[sb+1] & CLEARMASK) - 1; + if (hi > lo) { + if (verb >= 4) + VPrintf4 ( " qsort [0x%x, 0x%x] " + "done %d this %d\n", + ss, j, numQSorted, hi - lo + 1 ); + mainQSort3 ( + ptr, block, quadrant, nblock, + lo, hi, BZ_N_RADIX, budget + ); + numQSorted += (hi - lo + 1); + if (*budget < 0) return; } - zero: - runningOrder[j] = vv; } - } while (h != 1); + ftab[sb] |= SETMASK; + } } /*-- - The main sorting loop. + Step 2: + Deal specially with case [ss, ss]. This establishes the + sorted order for [ss, ss] without any comparisons. + A clever trick, cryptically described as steps Q6b and Q6c + in SRC-124 (aka BW94). Compared to bzip2, this makes it + practical not to use a preliminary run-length coder. --*/ + { + Int32 put0, get0, put1, get1; + Int32 sbn = (ss << 8) + ss; + Int32 lo = ftab[sbn] & CLEARMASK; + Int32 hi = (ftab[sbn+1] & CLEARMASK) - 1; + UChar ssc = (UChar)ss; + put0 = lo; + get0 = ftab[ss << 8] & CLEARMASK; + put1 = hi; + get1 = (ftab[(ss+1) << 8] & CLEARMASK) - 1; + while (get0 < put0) { + j = ptr[get0]-1; if (j < 0) j += nblock; + c1 = (UChar)(block[j] >> 8); + if (c1 == ssc) { ptr[put0] = j; put0++; }; + get0++; + } + while (get1 > put1) { + j = ptr[get1]-1; if (j < 0) j += nblock; + c1 = (UChar)(block[j] >> 8); + if (c1 == ssc) { ptr[put1] = j; put1--; }; + get1--; + } + ftab[sbn] |= SETMASK; + } - for (i = 0; i <= 255; i++) { - - /*-- - Process big buckets, starting with the least full. - Basically this is a 4-step process in which we call - qSort3 to sort the small buckets [ss, j], but - also make a big effort to avoid the calls if we can. - --*/ - ss = runningOrder[i]; - - /*-- - Step 1: - Complete the big bucket [ss] by quicksorting - any unsorted small buckets [ss, j], for j != ss. - Hopefully previous pointer-scanning phases have already - completed many of the small buckets [ss, j], so - we don't have to sort them at all. - --*/ - for (j = 0; j <= 255; j++) { - if (j != ss) { - sb = (ss << 8) + j; - if ( ! (ftab[sb] & SETMASK) ) { - Int32 lo = ftab[sb] & CLEARMASK; - Int32 hi = (ftab[sb+1] & CLEARMASK) - 1; - if (hi > lo) { - if (s->verbosity >= 4) - VPrintf4( " qsort [0x%x, 0x%x] done %d this %d\n", - ss, j, numQSorted, hi - lo + 1 ); - qSort3 ( s, lo, hi, 2 ); - numQSorted += ( hi - lo + 1 ); - if (*workDone > workLimit && firstAttempt) return; - } + /*-- + Step 3: + The [ss] big bucket is now done. Record this fact, + and update the quadrant descriptors. Remember to + update quadrants in the overshoot area too, if + necessary. The "if (i < 255)" test merely skips + this updating for the last bucket processed, since + updating for the last bucket is pointless. + + The quadrant array provides a way to incrementally + cache sort orderings, as they appear, so as to + make subsequent comparisons in fullGtU() complete + faster. For repetitive blocks this makes a big + difference (but not big enough to be able to avoid + the fallback sorting mechanism, exponential radix sort). + + The precise meaning is: at all times: + + for 0 <= i < nblock and 0 <= j <= nblock + + if block[i] != block[j], + + then the relative values of quadrant[i] and + quadrant[j] are meaningless. + + else { + if quadrant[i] < quadrant[j] + then the string starting at i lexicographically + precedes the string starting at j + + else if quadrant[i] > quadrant[j] + then the string starting at j lexicographically + precedes the string starting at i + + else + the relative ordering of the strings starting + at i and j has not yet been determined. } - ftab[sb] |= SETMASK; - } - } + --*/ + bigDone[ss] = True; - /*-- - Step 2: - Deal specially with case [ss, ss]. This establishes the - sorted order for [ss, ss] without any comparisons. - A clever trick, cryptically described as steps Q6b and Q6c - in SRC-124 (aka BW94). This makes it entirely practical to - not use a preliminary run-length coder, but unfortunately - we are now stuck with the .bz2 file format. - --*/ - { - Int32 put0, get0, put1, get1; - Int32 sbn = (ss << 8) + ss; - Int32 lo = ftab[sbn] & CLEARMASK; - Int32 hi = (ftab[sbn+1] & CLEARMASK) - 1; - UChar ssc = (UChar)ss; - put0 = lo; - get0 = ftab[ss << 8] & CLEARMASK; - put1 = hi; - get1 = (ftab[(ss+1) << 8] & CLEARMASK) - 1; - while (get0 < put0) { - j = zptr[get0]-1; if (j < 0) j += nblock; - c1 = block[j]; - if (c1 == ssc) { zptr[put0] = j; put0++; }; - get0++; - } - while (get1 > put1) { - j = zptr[get1]-1; if (j < 0) j += nblock; - c1 = block[j]; - if (c1 == ssc) { zptr[put1] = j; put1--; }; - get1--; - } - ftab[sbn] |= SETMASK; - } + if (i < 255) { + Int32 bbStart = ftab[ss << 8] & CLEARMASK; + Int32 bbSize = (ftab[(ss+1) << 8] & CLEARMASK) - bbStart; + Int32 shifts = 0; - /*-- - Step 3: - The [ss] big bucket is now done. Record this fact, - and update the quadrant descriptors. Remember to - update quadrants in the overshoot area too, if - necessary. The "if (i < 255)" test merely skips - this updating for the last bucket processed, since - updating for the last bucket is pointless. - - The quadrant array provides a way to incrementally - cache sort orderings, as they appear, so as to - make subsequent comparisons in fullGtU() complete - faster. For repetitive blocks this makes a big - difference (but not big enough to be able to avoid - randomisation for very repetitive data.) - - The precise meaning is: at all times: - - for 0 <= i < nblock and 0 <= j <= nblock - - if block[i] != block[j], - - then the relative values of quadrant[i] and - quadrant[j] are meaningless. - - else { - if quadrant[i] < quadrant[j] - then the string starting at i lexicographically - precedes the string starting at j - - else if quadrant[i] > quadrant[j] - then the string starting at j lexicographically - precedes the string starting at i - - else - the relative ordering of the strings starting - at i and j has not yet been determined. - } - --*/ - bigDone[ss] = True; - - if (i < 255) { - Int32 bbStart = ftab[ss << 8] & CLEARMASK; - Int32 bbSize = (ftab[(ss+1) << 8] & CLEARMASK) - bbStart; - Int32 shifts = 0; - - while ((bbSize >> shifts) > 65534) shifts++; - - for (j = 0; j < bbSize; j++) { - Int32 a2update = zptr[bbStart + j]; - UInt16 qVal = (UInt16)(j >> shifts); - quadrant[a2update] = qVal; - if (a2update < BZ_NUM_OVERSHOOT_BYTES) - quadrant[a2update + nblock] = qVal; - } + while ((bbSize >> shifts) > 65534) shifts++; - AssertH ( ( ((bbSize-1) >> shifts) <= 65535 ), 1002 ); + for (j = 0; j < bbSize; j++) { + Int32 a2update = ptr[bbStart + j]; + UInt16 qVal = (UInt16)(j >> shifts); + quadrant[a2update] = qVal; + if (a2update < BZ_N_OVERSHOOT) + quadrant[a2update + nblock] = qVal; } + AssertH ( ((bbSize-1) >> shifts) <= 65535, 1002 ); + } - /*-- - Step 4: - Now scan this big bucket [ss] so as to synthesise the - sorted order for small buckets [t, ss] for all t != ss. - This will avoid doing Real Work in subsequent Step 1's. - --*/ - for (j = 0; j <= 255; j++) - copy[j] = ftab[(j << 8) + ss] & CLEARMASK; - - for (j = ftab[ss << 8] & CLEARMASK; - j < (ftab[(ss+1) << 8] & CLEARMASK); - j++) { - k = zptr[j]-1; if (k < 0) k += nblock; - c1 = block[k]; - if ( ! bigDone[c1] ) { - zptr[copy[c1]] = k; - copy[c1] ++; - } + /*-- + Step 4: + Now scan this big bucket [ss] so as to synthesise the + sorted order for small buckets [t, ss] for all t != ss. + This will avoid doing Real Work in subsequent Step 1's. + --*/ + for (j = 0; j <= 255; j++) + copy[j] = ftab[(j << 8) + ss] & CLEARMASK; + + m = ftab[(ss+1) << 8] & CLEARMASK; + for (j = ftab[ss << 8] & CLEARMASK; j < m; j++) { + k = ptr[j] - 1; if (k < 0) k += nblock; + c1 = (UChar)(block[k] >> 8); + if ( ! bigDone[c1] ) { + ptr[copy[c1]] = k; + copy[c1] ++; } - - for (j = 0; j <= 255; j++) ftab[(j << 8) + ss] |= SETMASK; } - if (s->verbosity >= 4) - VPrintf3( " %d pointers, %d sorted, %d scanned\n", - nblock, numQSorted, nblock - numQSorted ); - } -} - -/*---------------------------------------------*/ -static void randomiseBlock ( EState* s ) -{ - Int32 i; - BZ_RAND_INIT_MASK; - for (i = 0; i < 256; i++) s->inUse[i] = False; - - for (i = 0; i < s->nblock; i++) { - BZ_RAND_UPD_MASK; - s->block[i] ^= BZ_RAND_MASK; - s->inUse[s->block[i]] = True; + for (j = 0; j <= 255; j++) ftab[(j << 8) + ss] |= SETMASK; } + + if (verb >= 4) + VPrintf3 ( " %d pointers, %d sorted, %d scanned\n", + nblock, numQSorted, nblock - numQSorted ); } +#undef BIGFREQ +#undef SETMASK +#undef CLEARMASK + /*---------------------------------------------*/ +/* Pre: + nblock > 0 + arr2 exists for [0 .. nblock-1 +N_OVERSHOOT] + ((UInt16*)arr2) [0 .. nblock-1] [15:8] holds block + arr1 exists for [0 .. nblock-1] + + Post: + ((UInt16*)arr2) [0 .. nblock-1] [15:8] holds block + All other areas of block destroyed + ftab [ 0 .. 65536 ] destroyed + arr1 [0 .. nblock-1] holds sorted order +*/ void blockSort ( EState* s ) { - Int32 i; - - s->workLimit = s->workFactor * (s->nblock - 1); - s->workDone = 0; - s->blockRandomised = False; - s->firstAttempt = True; - - sortMain ( s ); - - if (s->verbosity >= 3) - VPrintf3( " %d work, %d block, ratio %5.2f\n", - s->workDone, s->nblock-1, - (float)(s->workDone) / (float)(s->nblock-1) ); - - if (s->workDone > s->workLimit && s->firstAttempt) { - if (s->verbosity >= 2) - VPrintf0( " sorting aborted; randomising block\n" ); - randomiseBlock ( s ); - s->workLimit = s->workDone = 0; - s->blockRandomised = True; - s->firstAttempt = False; - sortMain ( s ); - if (s->verbosity >= 3) - VPrintf3( " %d work, %d block, ratio %f\n", - s->workDone, s->nblock-1, - (float)(s->workDone) / (float)(s->nblock-1) ); + UInt32* ptr = s->ptr; + UInt16* block = s->block; + UInt32* ftab = s->ftab; + Int32 nblock = s->nblock; + Int32 verb = s->verbosity; + Int32 wfact = s->workFactor; + UInt16* quadrant; + Int32 budget; + Int32 budgetInit; + Int32 i; + + if (nblock < 10000) { + for (i = 0; i < nblock; i++) block[i] <<= 8; + fallbackSort ( s->arr1, s->arr2, ftab, nblock, verb ); + } else { + quadrant = &(block[nblock+BZ_N_OVERSHOOT]); + + /* (wfact-1) / 3 puts the default-factor-30 + transition point at very roughly the same place as + with v0.1 and v0.9.0. + Not that it particularly matters any more, since the + resulting compressed stream is now the same regardless + of whether or not we use the main sort or fallback sort. + */ + if (wfact < 1 ) wfact = 1; + if (wfact > 100) wfact = 100; + budgetInit = nblock * ((wfact-1) / 3); + budget = budgetInit; + + mainSort ( ptr, block, quadrant, ftab, nblock, verb, &budget ); + if (verb >= 3) + VPrintf3 ( " %d work, %d block, ratio %5.2f\n", + budgetInit - budget, + nblock, + (float)(budgetInit - budget) / + (float)(nblock==0 ? 1 : nblock) ); + if (budget < 0) { + if (verb >= 2) + VPrintf0 ( " too repetitive; using fallback" + " sorting algorithm\n" ); + fallbackSort ( s->arr1, s->arr2, ftab, nblock, verb ); + } } s->origPtr = -1; for (i = 0; i < s->nblock; i++) - if (s->zptr[i] == 0) - { s->origPtr = i; break; }; + if (ptr[i] == 0) + { s->origPtr = i; break; }; AssertH( s->origPtr != -1, 1003 ); } + /*-------------------------------------------------------------*/ /*--- end blocksort.c ---*/ /*-------------------------------------------------------------*/ diff --git a/bzip2.1 b/bzip2.1 index a6789a4..99eda9b 100644 --- a/bzip2.1 +++ b/bzip2.1 @@ -1,7 +1,7 @@ .PU .TH bzip2 1 .SH NAME -bzip2, bunzip2 \- a block-sorting file compressor, v0.9.0 +bzip2, bunzip2 \- a block-sorting file compressor, v0.9.5 .br bzcat \- decompresses files to stdout .br @@ -10,7 +10,7 @@ bzip2recover \- recovers data from damaged bzip2 files .SH SYNOPSIS .ll +8 .B bzip2 -.RB [ " \-cdfkstvzVL123456789 " ] +.RB [ " \-cdfkqstvzVL123456789 " ] [ .I "filenames \&..." ] @@ -18,13 +18,13 @@ bzip2recover \- recovers data from damaged bzip2 files .br .B bunzip2 .RB [ " \-fkvsVL " ] -[ +[ .I "filenames \&..." ] -.br +.br .B bzcat .RB [ " \-s " ] -[ +[ .I "filenames \&..." ] .br @@ -33,211 +33,171 @@ bzip2recover \- recovers data from damaged bzip2 files .SH DESCRIPTION .I bzip2 -compresses files using the Burrows-Wheeler block-sorting -text compression algorithm, and Huffman coding. -Compression is generally considerably -better than that -achieved by more conventional LZ77/LZ78-based compressors, -and approaches the performance of the PPM family of statistical -compressors. +compresses files using the Burrows-Wheeler block sorting +text compression algorithm, and Huffman coding. Compression is +generally considerably better than that achieved by more conventional +LZ77/LZ78-based compressors, and approaches the performance of the PPM +family of statistical compressors. The command-line options are deliberately very similar to those of -.I GNU Gzip, +.I GNU gzip, but they are not identical. -.I bzip2 -expects a list of file names to accompany the command-line flags. -Each file is replaced by a compressed version of itself, -with the name "original_name.bz2". -Each compressed file has the same modification date and permissions -as the corresponding original, so that these properties can be -correctly restored at decompression time. File name handling is -naive in the sense that there is no mechanism for preserving -original file names, permissions and dates in filesystems -which lack these concepts, or have serious file name length -restrictions, such as MS-DOS. +.I bzip2 +expects a list of file names to accompany the +command-line flags. Each file is replaced by a compressed version of +itself, with the name "original_name.bz2". +Each compressed file +has the same modification date, permissions, and, when possible, +ownership as the corresponding original, so that these properties can +be correctly restored at decompression time. File name handling is +naive in the sense that there is no mechanism for preserving original +file names, permissions, ownerships or dates in filesystems which lack +these concepts, or have serious file name length restrictions, such as +MS-DOS. .I bzip2 and .I bunzip2 -will by default not overwrite existing files; -if you want this to happen, specify the \-f flag. +will by default not overwrite existing +files. If you want this to happen, specify the \-f flag. If no file names are specified, .I bzip2 -compresses from standard input to standard output. -In this case, +compresses from standard +input to standard output. In this case, .I bzip2 -will decline to write compressed output to a terminal, as -this would be entirely incomprehensible and therefore pointless. +will decline to +write compressed output to a terminal, as this would be entirely +incomprehensible and therefore pointless. .I bunzip2 (or -.I bzip2 \-d -) decompresses and restores all specified files whose names -end in ".bz2". -Files without this suffix are ignored. -Again, supplying no filenames -causes decompression from standard input to standard output. - -.I bunzip2 -will correctly decompress a file which is the concatenation -of two or more compressed files. The result is the concatenation -of the corresponding uncompressed files. Integrity testing -(\-t) of concatenated compressed files is also supported. - -You can also compress or decompress files to -the standard output by giving the \-c flag. -Multiple files may be compressed and decompressed like this. -The resulting outputs are fed sequentially to stdout. -Compression of multiple files in this manner generates -a stream containing multiple compressed file representations. -Such a stream can be decompressed correctly only by +.I bzip2 \-d) +decompresses all +specified files. Files which were not created by .I bzip2 -version 0.9.0 or later. Earlier versions of +will be detected and ignored, and a warning issued. .I bzip2 -will stop after decompressing the first file in the stream. +attempts to guess the filename for the decompressed file +from that of the compressed file as follows: + + filename.bz2 becomes filename + filename.bz becomes filename + filename.tbz2 becomes filename.tar + filename.tbz becomes filename.tar + anyothername becomes anyothername.out + +If the file does not end in one of the recognised endings, +.I .bz2, +.I .bz, +.I .tbz2 +or +.I .tbz, +.I bzip2 +complains that it cannot +guess the name of the original file, and uses the original name +with +.I .out +appended. + +As with compression, supplying no +filenames causes decompression from +standard input to standard output. + +.I bunzip2 +will correctly decompress a file which is the +concatenation of two or more compressed files. The result is the +concatenation of the corresponding uncompressed files. Integrity +testing (\-t) +of concatenated +compressed files is also supported. + +You can also compress or decompress files to the standard output by +giving the \-c flag. Multiple files may be compressed and +decompressed like this. The resulting outputs are fed sequentially to +stdout. Compression of multiple files +in this manner generates a stream +containing multiple compressed file representations. Such a stream +can be decompressed correctly only by +.I bzip2 +version 0.9.0 or +later. Earlier versions of +.I bzip2 +will stop after decompressing +the first file in the stream. .I bzcat (or -.I bzip2 \-dc -) decompresses all specified files to the standard output. - -Compression is always performed, even if the compressed file is -slightly larger than the original. Files of less than about -one hundred bytes tend to get larger, since the compression -mechanism has a constant overhead in the region of 50 bytes. -Random data (including the output of most file compressors) -is coded at about 8.05 bits per byte, giving an expansion of -around 0.5%. - -As a self-check for your protection, +.I bzip2 -dc) +decompresses all specified files to +the standard output. + .I bzip2 -uses 32-bit CRCs to make sure that the decompressed -version of a file is identical to the original. -This guards against corruption of the compressed data, -and against undetected bugs in +will read arguments from the environment variables +.I BZIP2 +and +.I BZIP, +in that order, and will process them +before any arguments read from the command line. This gives a +convenient way to supply default arguments. + +Compression is always performed, even if the compressed +file is slightly +larger than the original. Files of less than about one hundred bytes +tend to get larger, since the compression mechanism has a constant +overhead in the region of 50 bytes. Random data (including the output +of most file compressors) is coded at about 8.05 bits per byte, giving +an expansion of around 0.5%. + +As a self-check for your protection, +.I +bzip2 +uses 32-bit CRCs to +make sure that the decompressed version of a file is identical to the +original. This guards against corruption of the compressed data, and +against undetected bugs in .I bzip2 -(hopefully very unlikely). -The chances of data corruption going undetected is -microscopic, about one chance in four billion -for each file processed. Be aware, though, that the check -occurs upon decompression, so it can only tell you that -that something is wrong. It can't help you recover the -original uncompressed data. -You can use +(hopefully very unlikely). The +chances of data corruption going undetected is microscopic, about one +chance in four billion for each file processed. Be aware, though, that +the check occurs upon decompression, so it can only tell you that +something is wrong. It can't help you +recover the original uncompressed +data. You can use .I bzip2recover -to try to recover data from damaged files. - -Return values: -0 for a normal exit, -1 for environmental -problems (file not found, invalid flags, I/O errors, &c), -2 to indicate a corrupt compressed file, -3 for an internal consistency error (eg, bug) which caused -.I bzip2 -to panic. +to try to recover data from +damaged files. -.SH MEMORY MANAGEMENT -.I Bzip2 -compresses large files in blocks. The block size affects both the -compression ratio achieved, and the amount of memory needed both for -compression and decompression. The flags \-1 through \-9 -specify the block size to be 100,000 bytes through 900,000 bytes -(the default) respectively. At decompression-time, the block size used for -compression is read from the header of the compressed file, and -.I bunzip2 -then allocates itself just enough memory to decompress the file. -Since block sizes are stored in compressed files, it follows that the flags -\-1 to \-9 -are irrelevant to and so ignored during decompression. -Compression and decompression requirements, in bytes, can be estimated as: - - Compression: 400k + ( 7 x block size ) - - Decompression: 100k + ( 4 x block size ), or -.br - 100k + ( 2.5 x block size ) - -Larger block sizes give rapidly diminishing marginal returns; most -of the -compression comes from the first two or three hundred k of block size, -a fact worth bearing in mind when using +Return values: 0 for a normal exit, 1 for environmental problems (file +not found, invalid flags, I/O errors, &c), 2 to indicate a corrupt +compressed file, 3 for an internal consistency error (eg, bug) which +caused .I bzip2 -on small machines. It is also important to appreciate that the -decompression memory requirement is set at compression-time by the -choice of block size. - -For files compressed with the default 900k block size, -.I bunzip2 -will require about 3700 kbytes to decompress. -To support decompression of any file on a 4 megabyte machine, -.I bunzip2 -has an option to decompress using approximately half this -amount of memory, about 2300 kbytes. Decompression speed is -also halved, so you should use this option only where necessary. -The relevant flag is \-s. - -In general, try and use the largest block size -memory constraints allow, since that maximises the compression -achieved. Compression and decompression -speed are virtually unaffected by block size. - -Another significant point applies to files which fit in a single -block -- that means most files you'd encounter using a large -block size. The amount of real memory touched is proportional -to the size of the file, since the file is smaller than a block. -For example, compressing a file 20,000 bytes long with the flag -\-9 -will cause the compressor to allocate around -6700k of memory, but only touch 400k + 20000 * 7 = 540 -kbytes of it. Similarly, the decompressor will allocate 3700k but -only touch 100k + 20000 * 4 = 180 kbytes. - -Here is a table which summarises the maximum memory usage for -different block sizes. Also recorded is the total compressed -size for 14 files of the Calgary Text Compression Corpus -totalling 3,141,622 bytes. This column gives some feel for how -compression varies with block size. These figures tend to understate -the advantage of larger block sizes for larger files, since the -Corpus is dominated by smaller files. - - Compress Decompress Decompress Corpus - Flag usage usage -s usage Size - - -1 1100k 500k 350k 914704 - -2 1800k 900k 600k 877703 - -3 2500k 1300k 850k 860338 - -4 3200k 1700k 1100k 846899 - -5 3900k 2100k 1350k 845160 - -6 4600k 2500k 1600k 838626 - -7 5400k 2900k 1850k 834096 - -8 6000k 3300k 2100k 828642 - -9 6700k 3700k 2350k 828642 +to panic. .SH OPTIONS .TP .B \-c --stdout -Compress or decompress to standard output. \-c will decompress -multiple files to stdout, but will only compress a single file to -stdout. +Compress or decompress to standard output. .TP .B \-d --decompress -Force decompression. -.I bzip2, -.I bunzip2 +Force decompression. +.I bzip2, +.I bunzip2 and -.I bzcat -are really the same program, and the decision about what actions -to take is done on the basis of which name is -used. This flag overrides that mechanism, and forces +.I bzcat +are +really the same program, and the decision about what actions to take is +done on the basis of which name is used. This flag overrides that +mechanism, and forces .I bzip2 to decompress. -.TP +.TP .B \-z --compress -The complement to \-d: forces compression, regardless of the invokation -name. +The complement to \-d: forces compression, regardless of the +invokation name. .TP .B \-t --test Check integrity of the specified file(s), but don't decompress them. @@ -245,25 +205,31 @@ This really performs a trial decompression and throws away the result. .TP .B \-f --force Force overwrite of output files. Normally, -.I bzip2 -will not overwrite existing output files. +.I bzip2 +will not overwrite +existing output files. Also forces +.I bzip2 +to break hard links +to files, which it otherwise wouldn't do. .TP .B \-k --keep -Keep (don't delete) input files during compression or decompression. +Keep (don't delete) input files during compression +or decompression. .TP .B \-s --small -Reduce memory usage, for compression, decompression and -testing. -Files are decompressed and tested using a modified algorithm which only +Reduce memory usage, for compression, decompression and testing. Files +are decompressed and tested using a modified algorithm which only requires 2.5 bytes per block byte. This means any file can be -decompressed in 2300k of memory, albeit at about half the normal -speed. - -During compression, -s selects a block size of 200k, which limits -memory use to around the same figure, at the expense of your -compression ratio. In short, if your machine is low on memory -(8 megabytes or less), use -s for everything. See -MEMORY MANAGEMENT above. +decompressed in 2300k of memory, albeit at about half the normal speed. + +During compression, \-s selects a block size of 200k, which limits +memory use to around the same figure, at the expense of your compression +ratio. In short, if your machine is low on memory (8 megabytes or +less), use \-s for everything. See MEMORY MANAGEMENT below. +.TP +.B \-q --quiet +Suppress non-essential warning messages. Messages pertaining to +I/O errors and other critical events will not be suppressed. .TP .B \-v --verbose Verbose mode -- show the compression ratio for each file processed. @@ -273,147 +239,199 @@ information which is primarily of interest for diagnostic purposes. .B \-L --license -V --version Display the software version, license terms and conditions. .TP -.B \-1 to \-9 -Set the block size to 100 k, 200 k .. 900 k when -compressing. Has no effect when decompressing. -See MEMORY MANAGEMENT above. +.B \-1 to \-9 +Set the block size to 100 k, 200 k .. 900 k when compressing. Has no +effect when decompressing. See MEMORY MANAGEMENT below. .TP -.B \--repetitive-fast -.I bzip2 -injects some small pseudo-random variations -into very repetitive blocks to limit -worst-case performance during compression. -If sorting runs into difficulties, the block -is randomised, and sorting is restarted. -Very roughly, +.B \-- +Treats all subsequent arguments as file names, even if they start +with a dash. This is so you can handle files with names beginning +with a dash, for example: bzip2 \-- \-myfilename. +.TP +.B \--repetitive-fast --repetitive-best +These flags are redundant in versions 0.9.5 and above. They provided +some coarse control over the behaviour of the sorting algorithm in +earlier versions, which was sometimes useful. 0.9.5 and above have an +improved algorithm which renders these flags irrelevant. + +.SH MEMORY MANAGEMENT +.I bzip2 +compresses large files in blocks. The block size affects +both the compression ratio achieved, and the amount of memory needed for +compression and decompression. The flags \-1 through \-9 +specify the block size to be 100,000 bytes through 900,000 bytes (the +default) respectively. At decompression time, the block size used for +compression is read from the header of the compressed file, and +.I bunzip2 +then allocates itself just enough memory to decompress +the file. Since block sizes are stored in compressed files, it follows +that the flags \-1 to \-9 are irrelevant to and so ignored +during decompression. + +Compression and decompression requirements, +in bytes, can be estimated as: + + Compression: 400k + ( 8 x block size ) + + Decompression: 100k + ( 4 x block size ), or + 100k + ( 2.5 x block size ) + +Larger block sizes give rapidly diminishing marginal returns. Most of +the compression comes from the first two or three hundred k of block +size, a fact worth bearing in mind when using .I bzip2 -persists for three times as long as a well-behaved input -would take before resorting to randomisation. -This flag makes it give up much sooner. +on small machines. +It is also important to appreciate that the decompression memory +requirement is set at compression time by the choice of block size. -.TP -.B \--repetitive-best -Opposite of \--repetitive-fast; try a lot harder before -resorting to randomisation. +For files compressed with the default 900k block size, +.I bunzip2 +will require about 3700 kbytes to decompress. To support decompression +of any file on a 4 megabyte machine, +.I bunzip2 +has an option to +decompress using approximately half this amount of memory, about 2300 +kbytes. Decompression speed is also halved, so you should use this +option only where necessary. The relevant flag is -s. + +In general, try and use the largest block size memory constraints allow, +since that maximises the compression achieved. Compression and +decompression speed are virtually unaffected by block size. + +Another significant point applies to files which fit in a single block +-- that means most files you'd encounter using a large block size. The +amount of real memory touched is proportional to the size of the file, +since the file is smaller than a block. For example, compressing a file +20,000 bytes long with the flag -9 will cause the compressor to +allocate around 7600k of memory, but only touch 400k + 20000 * 8 = 560 +kbytes of it. Similarly, the decompressor will allocate 3700k but only +touch 100k + 20000 * 4 = 180 kbytes. + +Here is a table which summarises the maximum memory usage for different +block sizes. Also recorded is the total compressed size for 14 files of +the Calgary Text Compression Corpus totalling 3,141,622 bytes. This +column gives some feel for how compression varies with block size. +These figures tend to understate the advantage of larger block sizes for +larger files, since the Corpus is dominated by smaller files. + + Compress Decompress Decompress Corpus + Flag usage usage -s usage Size + + -1 1200k 500k 350k 914704 + -2 2000k 900k 600k 877703 + -3 2800k 1300k 850k 860338 + -4 3600k 1700k 1100k 846899 + -5 4400k 2100k 1350k 845160 + -6 5200k 2500k 1600k 838626 + -7 6100k 2900k 1850k 834096 + -8 6800k 3300k 2100k 828642 + -9 7600k 3700k 2350k 828642 .SH RECOVERING DATA FROM DAMAGED FILES .I bzip2 -compresses files in blocks, usually 900kbytes long. -Each block is handled independently. If a media or -transmission error causes a multi-block .bz2 -file to become damaged, -it may be possible to recover data from the undamaged blocks -in the file. - -The compressed representation of each block is delimited by -a 48-bit pattern, which makes it possible to find the block -boundaries with reasonable certainty. Each block also carries -its own 32-bit CRC, so damaged blocks can be -distinguished from undamaged ones. +compresses files in blocks, usually 900kbytes long. Each +block is handled independently. If a media or transmission error causes +a multi-block .bz2 +file to become damaged, it may be possible to +recover data from the undamaged blocks in the file. + +The compressed representation of each block is delimited by a 48-bit +pattern, which makes it possible to find the block boundaries with +reasonable certainty. Each block also carries its own 32-bit CRC, so +damaged blocks can be distinguished from undamaged ones. .I bzip2recover -is a simple program whose purpose is to search for -blocks in .bz2 files, and write each block out into -its own .bz2 file. You can then use -.I bzip2 -t -to test the integrity of the resulting files, -and decompress those which are undamaged. +is a simple program whose purpose is to search for +blocks in .bz2 files, and write each block out into its own .bz2 +file. You can then use +.I bzip2 +\-t +to test the +integrity of the resulting files, and decompress those which are +undamaged. .I bzip2recover -takes a single argument, the name of the damaged file, -and writes a number of files "rec0001file.bz2", "rec0002file.bz2", -etc, containing the extracted blocks. The output filenames -are designed so that the use of wildcards in subsequent processing --- for example, "bzip2 -dc rec*file.bz2 > recovered_data" -- -lists the files in the "right" order. +takes a single argument, the name of the damaged file, +and writes a number of files "rec0001file.bz2", +"rec0002file.bz2", etc, containing the extracted blocks. +The output filenames are designed so that the use of +wildcards in subsequent processing -- for example, +"bzip2 -dc rec*file.bz2 > recovered_data" -- lists the files in +the correct order. .I bzip2recover -should be of most use dealing with large .bz2 files, as -these will contain many blocks. It is clearly futile to -use it on damaged single-block files, since a damaged -block cannot be recovered. If you wish to minimise -any potential data loss through media or transmission -errors, you might consider compressing with a smaller +should be of most use dealing with large .bz2 +files, as these will contain many blocks. It is clearly +futile to use it on damaged single-block files, since a +damaged block cannot be recovered. If you wish to minimise +any potential data loss through media or transmission errors, +you might consider compressing with a smaller block size. .SH PERFORMANCE NOTES -The sorting phase of compression gathers together similar strings -in the file. Because of this, files containing very long -runs of repeated symbols, like "aabaabaabaab ..." (repeated -several hundred times) may compress extraordinarily slowly. -You can use the -\-vvvvv -option to monitor progress in great detail, if you want. -Decompression speed is unaffected. - -Such pathological cases -seem rare in practice, appearing mostly in artificially-constructed -test files, and in low-level disk images. It may be inadvisable to -use -.I bzip2 -to compress the latter. -If you do get a file which causes severe slowness in compression, -try making the block size as small as possible, with flag \-1. +The sorting phase of compression gathers together similar strings in the +file. Because of this, files containing very long runs of repeated +symbols, like "aabaabaabaab ..." (repeated several hundred times) may +compress more slowly than normal. Versions 0.9.5 and above fare much +better than previous versions in this respect. The ratio between +worst-case and average-case compression time is in the region of 10:1. +For previous versions, this figure was more like 100:1. You can use the +\-vvvv option to monitor progress in great detail, if you want. + +Decompression speed is unaffected by these phenomena. .I bzip2 -usually allocates several megabytes of memory to operate in, -and then charges all over it in a fairly random fashion. This -means that performance, both for compressing and decompressing, -is largely determined by the speed -at which your machine can service cache misses. -Because of this, small changes -to the code to reduce the miss rate have been observed to give -disproportionately large performance improvements. +usually allocates several megabytes of memory to operate +in, and then charges all over it in a fairly random fashion. This means +that performance, both for compressing and decompressing, is largely +determined by the speed at which your machine can service cache misses. +Because of this, small changes to the code to reduce the miss rate have +been observed to give disproportionately large performance improvements. I imagine .I bzip2 will perform best on machines with very large caches. .SH CAVEATS I/O error messages are not as helpful as they could be. -.I Bzip2 -tries hard to detect I/O errors and exit cleanly, but the -details of what the problem is sometimes seem rather misleading. +.I bzip2 +tries hard to detect I/O errors and exit cleanly, but the details of +what the problem is sometimes seem rather misleading. -This manual page pertains to version 0.9.0 of +This manual page pertains to version 0.9.5 of .I bzip2. -Compressed data created by this version is entirely forwards and -backwards compatible with the previous public release, version 0.1pl2, -but with the following exception: 0.9.0 can correctly decompress -multiple concatenated compressed files. 0.1pl2 cannot do this; it -will stop after decompressing just the first file in the stream. - -Wildcard expansion for Windows 95 and NT -is flaky. +Compressed +data created by this version is entirely forwards and backwards +compatible with the previous public releases, versions 0.1pl2 and 0.9.0, +but with the following exception: 0.9.0 and above can correctly +decompress multiple concatenated compressed files. 0.1pl2 cannot do +this; it will stop after decompressing just the first file in the +stream. .I bzip2recover uses 32-bit integers to represent bit positions in -compressed files, so it cannot handle compressed files -more than 512 megabytes long. This could easily be fixed. +compressed files, so it cannot handle compressed files more than 512 +megabytes long. This could easily be fixed. .SH AUTHOR Julian Seward, jseward@acm.org. http://www.muraroa.demon.co.uk -The ideas embodied in +The ideas embodied in .I bzip2 -are due to (at least) the following people: -Michael Burrows and David Wheeler (for the block sorting -transformation), David Wheeler (again, for the Huffman coder), -Peter Fenwick (for the structured coding model in the original -.I bzip, -and many refinements), -and -Alistair Moffat, Radford Neal and Ian Witten (for the arithmetic -coder in the original +are due to (at least) the following +people: Michael Burrows and David Wheeler (for the block sorting +transformation), David Wheeler (again, for the Huffman coder), Peter +Fenwick (for the structured coding model in the original +.I bzip, +and many refinements), and Alistair Moffat, Radford Neal and Ian Witten +(for the arithmetic coder in the original .I bzip). -I am much indebted for their help, support and advice. -See the manual in the source distribution for pointers to -sources of documentation. -Christian von Roques encouraged me to look for faster -sorting algorithms, so as to speed up compression. -Bela Lubkin encouraged me to improve the worst-case -compression performance. -Many people sent patches, helped with portability problems, -lent machines, gave advice and were generally helpful. - +I am much +indebted for their help, support and advice. See the manual in the +source distribution for pointers to sources of documentation. Christian +von Roques encouraged me to look for faster sorting algorithms, so as to +speed up compression. Bela Lubkin encouraged me to improve the +worst-case compression performance. Many people sent patches, helped +with portability problems, lent machines, gave advice and were generally +helpful. diff --git a/bzip2.1.preformatted b/bzip2.1.preformatted index 8c4fab1..96b44be 100644 --- a/bzip2.1.preformatted +++ b/bzip2.1.preformatted @@ -1,24 +1,20 @@ - -bzip2(1) bzip2(1) - - NNAAMMEE - bzip2, bunzip2 - a block-sorting file compressor, v0.9.0 + bzip2, bunzip2 - a block-sorting file compressor, v0.9.5 bzcat - decompresses files to stdout bzip2recover - recovers data from damaged bzip2 files SSYYNNOOPPSSIISS - bbzziipp22 [ --ccddffkkssttvvzzVVLL112233445566778899 ] [ _f_i_l_e_n_a_m_e_s _._._. ] + bbzziipp22 [ --ccddffkkqqssttvvzzVVLL112233445566778899 ] [ _f_i_l_e_n_a_m_e_s _._._. ] bbuunnzziipp22 [ --ffkkvvssVVLL ] [ _f_i_l_e_n_a_m_e_s _._._. ] bbzzccaatt [ --ss ] [ _f_i_l_e_n_a_m_e_s _._._. ] bbzziipp22rreeccoovveerr _f_i_l_e_n_a_m_e DDEESSCCRRIIPPTTIIOONN - _b_z_i_p_2 compresses files using the Burrows-Wheeler block- + _b_z_i_p_2 compresses files using the Burrows-Wheeler block sorting text compression algorithm, and Huffman coding. Compression is generally considerably better than that achieved by more conventional LZ77/LZ78-based compressors, @@ -26,22 +22,22 @@ DDEESSCCRRIIPPTTIIOONN tistical compressors. The command-line options are deliberately very similar to - those of _G_N_U _G_z_i_p_, but they are not identical. + those of _G_N_U _g_z_i_p_, but they are not identical. _b_z_i_p_2 expects a list of file names to accompany the com- mand-line flags. Each file is replaced by a compressed version of itself, with the name "original_name.bz2". - Each compressed file has the same modification date and - permissions as the corresponding original, so that these - properties can be correctly restored at decompression - time. File name handling is naive in the sense that there - is no mechanism for preserving original file names, per- - missions and dates in filesystems which lack these con- - cepts, or have serious file name length restrictions, such - as MS-DOS. + Each compressed file has the same modification date, per- + missions, and, when possible, ownership as the correspond- + ing original, so that these properties can be correctly + restored at decompression time. File name handling is + naive in the sense that there is no mechanism for preserv- + ing original file names, permissions, ownerships or dates + in filesystems which lack these concepts, or have serious + file name length restrictions, such as MS-DOS. _b_z_i_p_2 and _b_u_n_z_i_p_2 will by default not overwrite existing - files; if you want this to happen, specify the -f flag. + files. If you want this to happen, specify the -f flag. If no file names are specified, _b_z_i_p_2 compresses from standard input to standard output. In this case, _b_z_i_p_2 @@ -49,42 +45,50 @@ DDEESSCCRRIIPPTTIIOONN this would be entirely incomprehensible and therefore pointless. - _b_u_n_z_i_p_2 (or _b_z_i_p_2 _-_d ) decompresses and restores all spec- - ified files whose names end in ".bz2". Files without this - suffix are ignored. Again, supplying no filenames causes - decompression from standard input to standard output. + _b_u_n_z_i_p_2 (or _b_z_i_p_2 _-_d_) decompresses all specified files. + Files which were not created by _b_z_i_p_2 will be detected and + ignored, and a warning issued. _b_z_i_p_2 attempts to guess + the filename for the decompressed file from that of the + compressed file as follows: + + filename.bz2 becomes filename + filename.bz becomes filename + filename.tbz2 becomes filename.tar + filename.tbz becomes filename.tar + anyothername becomes anyothername.out + + If the file does not end in one of the recognised endings, + _._b_z_2_, _._b_z_, _._t_b_z_2 or _._t_b_z_, _b_z_i_p_2 complains that it cannot + guess the name of the original file, and uses the original + name with _._o_u_t appended. + + As with compression, supplying no filenames causes decom- + pression from standard input to standard output. _b_u_n_z_i_p_2 will correctly decompress a file which is the con- catenation of two or more compressed files. The result is the concatenation of the corresponding uncompressed files. Integrity testing (-t) of concatenated compressed files is - - - - 1 - - - - - -bzip2(1) bzip2(1) - - also supported. - You can also compress or decompress files to the standard - output by giving the -c flag. Multiple files may be com- + You can also compress or decompress files to the standard + output by giving the -c flag. Multiple files may be com- pressed and decompressed like this. The resulting outputs - are fed sequentially to stdout. Compression of multiple - files in this manner generates a stream containing multi- + are fed sequentially to stdout. Compression of multiple + files in this manner generates a stream containing multi- ple compressed file representations. Such a stream can be - decompressed correctly only by _b_z_i_p_2 version 0.9.0 or - later. Earlier versions of _b_z_i_p_2 will stop after decom- + decompressed correctly only by _b_z_i_p_2 version 0.9.0 or + later. Earlier versions of _b_z_i_p_2 will stop after decom- pressing the first file in the stream. - _b_z_c_a_t (or _b_z_i_p_2 _-_d_c ) decompresses all specified files to + _b_z_c_a_t (or _b_z_i_p_2 _-_d_c_) decompresses all specified files to the standard output. + _b_z_i_p_2 will read arguments from the environment variables + _B_Z_I_P_2 and _B_Z_I_P_, in that order, and will process them + before any arguments read from the command line. This + gives a convenient way to supply default arguments. + Compression is always performed, even if the compressed file is slightly larger than the original. Files of less than about one hundred bytes tend to get larger, since the @@ -101,121 +105,19 @@ bzip2(1) bzip2(1) corruption going undetected is microscopic, about one chance in four billion for each file processed. Be aware, though, that the check occurs upon decompression, so it - can only tell you that that something is wrong. It can't - help you recover the original uncompressed data. You can - use _b_z_i_p_2_r_e_c_o_v_e_r to try to recover data from damaged - files. + can only tell you that something is wrong. It can't help + you recover the original uncompressed data. You can use + _b_z_i_p_2_r_e_c_o_v_e_r to try to recover data from damaged files. - Return values: 0 for a normal exit, 1 for environmental - problems (file not found, invalid flags, I/O errors, &c), + Return values: 0 for a normal exit, 1 for environmental + problems (file not found, invalid flags, I/O errors, &c), 2 to indicate a corrupt compressed file, 3 for an internal consistency error (eg, bug) which caused _b_z_i_p_2 to panic. -MMEEMMOORRYY MMAANNAAGGEEMMEENNTT - _B_z_i_p_2 compresses large files in blocks. The block size - affects both the compression ratio achieved, and the - amount of memory needed both for compression and decom- - pression. The flags -1 through -9 specify the block size - to be 100,000 bytes through 900,000 bytes (the default) - respectively. At decompression-time, the block size used - for compression is read from the header of the compressed - file, and _b_u_n_z_i_p_2 then allocates itself just enough memory - to decompress the file. Since block sizes are stored in - compressed files, it follows that the flags -1 to -9 are - irrelevant to and so ignored during decompression. - - - - 2 - - - - - -bzip2(1) bzip2(1) - - - Compression and decompression requirements, in bytes, can - be estimated as: - - Compression: 400k + ( 7 x block size ) - - Decompression: 100k + ( 4 x block size ), or - 100k + ( 2.5 x block size ) - - Larger block sizes give rapidly diminishing marginal - returns; most of the compression comes from the first two - or three hundred k of block size, a fact worth bearing in - mind when using _b_z_i_p_2 on small machines. It is also - important to appreciate that the decompression memory - requirement is set at compression-time by the choice of - block size. - - For files compressed with the default 900k block size, - _b_u_n_z_i_p_2 will require about 3700 kbytes to decompress. To - support decompression of any file on a 4 megabyte machine, - _b_u_n_z_i_p_2 has an option to decompress using approximately - half this amount of memory, about 2300 kbytes. Decompres- - sion speed is also halved, so you should use this option - only where necessary. The relevant flag is -s. - - In general, try and use the largest block size memory con- - straints allow, since that maximises the compression - achieved. Compression and decompression speed are virtu- - ally unaffected by block size. - - Another significant point applies to files which fit in a - single block -- that means most files you'd encounter - using a large block size. The amount of real memory - touched is proportional to the size of the file, since the - file is smaller than a block. For example, compressing a - file 20,000 bytes long with the flag -9 will cause the - compressor to allocate around 6700k of memory, but only - touch 400k + 20000 * 7 = 540 kbytes of it. Similarly, the - decompressor will allocate 3700k but only touch 100k + - 20000 * 4 = 180 kbytes. - - Here is a table which summarises the maximum memory usage - for different block sizes. Also recorded is the total - compressed size for 14 files of the Calgary Text Compres- - sion Corpus totalling 3,141,622 bytes. This column gives - some feel for how compression varies with block size. - These figures tend to understate the advantage of larger - block sizes for larger files, since the Corpus is domi- - nated by smaller files. - - Compress Decompress Decompress Corpus - Flag usage usage -s usage Size - - -1 1100k 500k 350k 914704 - -2 1800k 900k 600k 877703 - - - - 3 - - - - - -bzip2(1) bzip2(1) - - - -3 2500k 1300k 850k 860338 - -4 3200k 1700k 1100k 846899 - -5 3900k 2100k 1350k 845160 - -6 4600k 2500k 1600k 838626 - -7 5400k 2900k 1850k 834096 - -8 6000k 3300k 2100k 828642 - -9 6700k 3700k 2350k 828642 - - OOPPTTIIOONNSS --cc ----ssttddoouutt - Compress or decompress to standard output. -c will - decompress multiple files to stdout, but will only - compress a single file to stdout. + Compress or decompress to standard output. --dd ----ddeeccoommpprreessss Force decompression. _b_z_i_p_2_, _b_u_n_z_i_p_2 and _b_z_c_a_t are @@ -235,7 +137,9 @@ OOPPTTIIOONNSS --ff ----ffoorrccee Force overwrite of output files. Normally, _b_z_i_p_2 - will not overwrite existing output files. + will not overwrite existing output files. Also + forces _b_z_i_p_2 to break hard links to files, which it + otherwise wouldn't do. --kk ----kkeeeepp Keep (don't delete) input files during compression @@ -254,19 +158,12 @@ OOPPTTIIOONNSS figure, at the expense of your compression ratio. In short, if your machine is low on memory (8 megabytes or less), use -s for everything. See - MEMORY MANAGEMENT above. - - - - - 4 - - - - - -bzip2(1) bzip2(1) + MEMORY MANAGEMENT below. + --qq ----qquuiieett + Suppress non-essential warning messages. Messages + pertaining to I/O errors and other critical events + will not be suppressed. --vv ----vveerrbboossee Verbose mode -- show the compression ratio for each @@ -281,22 +178,96 @@ bzip2(1) bzip2(1) --11 ttoo --99 Set the block size to 100 k, 200 k .. 900 k when compressing. Has no effect when decompressing. - See MEMORY MANAGEMENT above. + See MEMORY MANAGEMENT below. + + ---- Treats all subsequent arguments as file names, even + if they start with a dash. This is so you can han- + dle files with names beginning with a dash, for + example: bzip2 -- -myfilename. + + ----rreeppeettiittiivvee--ffaasstt ----rreeppeettiittiivvee--bbeesstt + These flags are redundant in versions 0.9.5 and + above. They provided some coarse control over the + behaviour of the sorting algorithm in earlier ver- + sions, which was sometimes useful. 0.9.5 and above + have an improved algorithm which renders these + flags irrelevant. + + +MMEEMMOORRYY MMAANNAAGGEEMMEENNTT + _b_z_i_p_2 compresses large files in blocks. The block size + affects both the compression ratio achieved, and the + amount of memory needed for compression and decompression. + The flags -1 through -9 specify the block size to be + 100,000 bytes through 900,000 bytes (the default) respec- + tively. At decompression time, the block size used for + compression is read from the header of the compressed + file, and _b_u_n_z_i_p_2 then allocates itself just enough memory + to decompress the file. Since block sizes are stored in + compressed files, it follows that the flags -1 to -9 are + irrelevant to and so ignored during decompression. - ----rreeppeettiittiivvee--ffaasstt - _b_z_i_p_2 injects some small pseudo-random variations - into very repetitive blocks to limit worst-case - performance during compression. If sorting runs - into difficulties, the block is randomised, and - sorting is restarted. Very roughly, _b_z_i_p_2 persists - for three times as long as a well-behaved input - would take before resorting to randomisation. This - flag makes it give up much sooner. + Compression and decompression requirements, in bytes, can + be estimated as: + Compression: 400k + ( 8 x block size ) - ----rreeppeettiittiivvee--bbeesstt - Opposite of --repetitive-fast; try a lot harder - before resorting to randomisation. + Decompression: 100k + ( 4 x block size ), or + 100k + ( 2.5 x block size ) + + Larger block sizes give rapidly diminishing marginal + returns. Most of the compression comes from the first two + or three hundred k of block size, a fact worth bearing in + mind when using _b_z_i_p_2 on small machines. It is also + important to appreciate that the decompression memory + requirement is set at compression time by the choice of + block size. + + For files compressed with the default 900k block size, + _b_u_n_z_i_p_2 will require about 3700 kbytes to decompress. To + support decompression of any file on a 4 megabyte machine, + _b_u_n_z_i_p_2 has an option to decompress using approximately + half this amount of memory, about 2300 kbytes. Decompres- + sion speed is also halved, so you should use this option + only where necessary. The relevant flag is -s. + + In general, try and use the largest block size memory con- + straints allow, since that maximises the compression + achieved. Compression and decompression speed are virtu- + ally unaffected by block size. + + Another significant point applies to files which fit in a + single block -- that means most files you'd encounter + using a large block size. The amount of real memory + touched is proportional to the size of the file, since the + file is smaller than a block. For example, compressing a + file 20,000 bytes long with the flag -9 will cause the + compressor to allocate around 7600k of memory, but only + touch 400k + 20000 * 8 = 560 kbytes of it. Similarly, the + decompressor will allocate 3700k but only touch 100k + + 20000 * 4 = 180 kbytes. + + Here is a table which summarises the maximum memory usage + for different block sizes. Also recorded is the total + compressed size for 14 files of the Calgary Text Compres- + sion Corpus totalling 3,141,622 bytes. This column gives + some feel for how compression varies with block size. + These figures tend to understate the advantage of larger + block sizes for larger files, since the Corpus is domi- + nated by smaller files. + + Compress Decompress Decompress Corpus + Flag usage usage -s usage Size + + -1 1200k 500k 350k 914704 + -2 2000k 900k 600k 877703 + -3 2800k 1300k 850k 860338 + -4 3600k 1700k 1100k 846899 + -5 4400k 2100k 1350k 845160 + -6 5200k 2500k 1600k 838626 + -7 6100k 2900k 1850k 834096 + -8 6800k 3300k 2100k 828642 + -9 7600k 3700k 2350k 828642 RREECCOOVVEERRIINNGG DDAATTAA FFRROOMM DDAAMMAAGGEEDD FFIILLEESS @@ -314,7 +285,7 @@ RREECCOOVVEERRIINNGG DDAATTAA FFRROOMM DDAAMMAAGGEEDD F _b_z_i_p_2_r_e_c_o_v_e_r is a simple program whose purpose is to search for blocks in .bz2 files, and write each block out - into its own .bz2 file. You can then use _b_z_i_p_2 _-_t to test + into its own .bz2 file. You can then use _b_z_i_p_2 -t to test the integrity of the resulting files, and decompress those which are undamaged. @@ -322,21 +293,9 @@ RREECCOOVVEERRIINNGG DDAATTAA FFRROOMM DDAAMMAAGGEEDD F aged file, and writes a number of files "rec0001file.bz2", "rec0002file.bz2", etc, containing the extracted blocks. The output filenames are designed so that the use of - - - - 5 - - - - - -bzip2(1) bzip2(1) - - wildcards in subsequent processing -- for example, "bzip2 - -dc rec*file.bz2 > recovered_data" -- lists the files in - the "right" order. + -dc rec*file.bz2 > recovered_data" -- lists the files in + the correct order. _b_z_i_p_2_r_e_c_o_v_e_r should be of most use dealing with large .bz2 files, as these will contain many blocks. It is clearly @@ -351,17 +310,15 @@ PPEERRFFOORRMMAANNCCEE NNOOTTEESS The sorting phase of compression gathers together similar strings in the file. Because of this, files containing very long runs of repeated symbols, like "aabaabaabaab - ..." (repeated several hundred times) may compress - extraordinarily slowly. You can use the -vvvvv option to - monitor progress in great detail, if you want. Decompres- - sion speed is unaffected. - - Such pathological cases seem rare in practice, appearing - mostly in artificially-constructed test files, and in low- - level disk images. It may be inadvisable to use _b_z_i_p_2 to - compress the latter. If you do get a file which causes - severe slowness in compression, try making the block size - as small as possible, with flag -1. + ..." (repeated several hundred times) may compress more + slowly than normal. Versions 0.9.5 and above fare much + better than previous versions in this respect. The ratio + between worst-case and average-case compression time is in + the region of 10:1. For previous versions, this figure + was more like 100:1. You can use the -vvvv option to mon- + itor progress in great detail, if you want. + + Decompression speed is unaffected by these phenomena. _b_z_i_p_2 usually allocates several megabytes of memory to operate in, and then charges all over it in a fairly ran- @@ -376,88 +333,43 @@ PPEERRFFOORRMMAANNCCEE NNOOTTEESS CCAAVVEEAATTSS I/O error messages are not as helpful as they could be. - _B_z_i_p_2 tries hard to detect I/O errors and exit cleanly, + _b_z_i_p_2 tries hard to detect I/O errors and exit cleanly, but the details of what the problem is sometimes seem rather misleading. - This manual page pertains to version 0.9.0 of _b_z_i_p_2_. Com- + This manual page pertains to version 0.9.5 of _b_z_i_p_2_. Com- pressed data created by this version is entirely forwards - and backwards compatible with the previous public release, - version 0.1pl2, but with the following exception: 0.9.0 - can correctly decompress multiple concatenated compressed - files. 0.1pl2 cannot do this; it will stop after decom- - pressing just the first file in the stream. - - - - - 6 - - - - - -bzip2(1) bzip2(1) - - - Wildcard expansion for Windows 95 and NT is flaky. - - _b_z_i_p_2_r_e_c_o_v_e_r uses 32-bit integers to represent bit posi- - tions in compressed files, so it cannot handle compressed - files more than 512 megabytes long. This could easily be + and backwards compatible with the previous public + releases, versions 0.1pl2 and 0.9.0, but with the follow- + ing exception: 0.9.0 and above can correctly decompress + multiple concatenated compressed files. 0.1pl2 cannot do + this; it will stop after decompressing just the first file + in the stream. + + _b_z_i_p_2_r_e_c_o_v_e_r uses 32-bit integers to represent bit posi- + tions in compressed files, so it cannot handle compressed + files more than 512 megabytes long. This could easily be fixed. AAUUTTHHOORR Julian Seward, jseward@acm.org. + http://www.muraroa.demon.co.uk The ideas embodied in _b_z_i_p_2 are due to (at least) the fol- - lowing people: Michael Burrows and David Wheeler (for the - block sorting transformation), David Wheeler (again, for + lowing people: Michael Burrows and David Wheeler (for the + block sorting transformation), David Wheeler (again, for the Huffman coder), Peter Fenwick (for the structured cod- ing model in the original _b_z_i_p_, and many refinements), and - Alistair Moffat, Radford Neal and Ian Witten (for the + Alistair Moffat, Radford Neal and Ian Witten (for the arithmetic coder in the original _b_z_i_p_)_. I am much indebted for their help, support and advice. See the man- - ual in the source distribution for pointers to sources of + ual in the source distribution for pointers to sources of documentation. Christian von Roques encouraged me to look - for faster sorting algorithms, so as to speed up compres- + for faster sorting algorithms, so as to speed up compres- sion. Bela Lubkin encouraged me to improve the worst-case compression performance. Many people sent patches, helped - with portability problems, lent machines, gave advice and + with portability problems, lent machines, gave advice and were generally helpful. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 7 - - diff --git a/bzip2.c b/bzip2.c index 6a3ab95..abb9530 100644 --- a/bzip2.c +++ b/bzip2.c @@ -7,7 +7,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-1998 Julian R Seward. All rights reserved. + Copyright (C) 1996-1999 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -40,9 +40,9 @@ NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - Julian Seward, Guildford, Surrey, UK. + Julian Seward, Cambridge, UK. jseward@acm.org - bzip2/libbzip2 version 0.9.0c of 18 October 1998 + bzip2/libbzip2 version 0.9.5 of 24 May 1999 This program is based on (at least) the work of: Mike Burrows @@ -123,8 +123,10 @@ --*/ #define BZ_LCCWIN32 0 -#ifdef _WIN32 +#if defined(_WIN32) && !defined(__CYGWIN32__) +#undef BZ_LCCWIN32 #define BZ_LCCWIN32 1 +#undef BZ_UNIX #define BZ_UNIX 0 #endif @@ -139,6 +141,8 @@ #include #include #include +#include +#include #include "bzlib.h" #define ERROR_IF_EOF(i) { if ((i) == EOF) ioError(); } @@ -166,6 +170,9 @@ # define APPEND_FILESPEC(root, name) \ root=snocString((root), (name)) +# define APPEND_FLAG(root, name) \ + root=snocString((root), (name)) + # define SET_BINARY_MODE(fd) /**/ # ifdef __GNUC__ @@ -173,6 +180,19 @@ # else # define NORETURN /**/ # endif +# ifdef __DJGPP__ +# include +# include +# undef MY_LSTAT +# define MY_LSTAT stat +# undef SET_BINARY_MODE +# define SET_BINARY_MODE(fd) \ + do { \ + int retVal = setmode ( fileno ( fd ), \ + O_BINARY ); \ + ERROR_IF_MINUS_ONE ( retVal ); \ + } while ( 0 ) +# endif #endif @@ -188,6 +208,9 @@ # define MY_STAT _stat # define MY_S_IFREG(x) ((x) & _S_IFREG) +# define APPEND_FLAG(root, name) \ + root=snocString((root), (name)) + # if 0 /*-- lcc-win32 seems to expand wildcards itself --*/ # define APPEND_FILESPEC(root, spec) \ @@ -254,7 +277,7 @@ typedef int IntNative; Int32 verbosity; Bool keepInputFiles, smallMode; -Bool forceOverwrite, testFailsExist; +Bool forceOverwrite, testFailsExist, noisy; Int32 numFileNames, numFilesProcessed, blockSize100k; @@ -274,8 +297,9 @@ Int32 srcMode; #define FILE_NAME_LEN 1034 Int32 longestFileName; -Char inName[FILE_NAME_LEN]; +Char inName [FILE_NAME_LEN]; Char outName[FILE_NAME_LEN]; +Char tmpName[FILE_NAME_LEN]; Char *progName; Char progNameReally[FILE_NAME_LEN]; FILE *outputHandleJustInCase; @@ -467,6 +491,7 @@ Bool uncompressStream ( FILE *zStream, FILE *stream ) if (streamNo == 1) { return False; } else { + if (noisy) fprintf ( stderr, "\n%s: %s: trailing garbage after EOF ignored\n", progName, inName ); @@ -532,32 +557,31 @@ Bool testStream ( FILE *zStream ) errhandler: bzReadClose ( &bzerr_dummy, bzf ); + if (verbosity == 0) + fprintf ( stderr, "%s: %s: ", progName, inName ); switch (bzerr) { case BZ_IO_ERROR: errhandler_io: ioError(); break; case BZ_DATA_ERROR: fprintf ( stderr, - "\n%s: data integrity (CRC) error in data\n", - inName ); + "data integrity (CRC) error in data\n" ); return False; case BZ_MEM_ERROR: outOfMemory(); case BZ_UNEXPECTED_EOF: fprintf ( stderr, - "\n%s: file ends unexpectedly\n", - inName ); + "file ends unexpectedly\n" ); return False; case BZ_DATA_ERROR_MAGIC: if (streamNo == 1) { fprintf ( stderr, - "\n%s: bad magic number (ie, not created by bzip2)\n", - inName ); + "bad magic number (file not created by bzip2)\n" ); return False; } else { + if (noisy) fprintf ( stderr, - "\n%s: %s: trailing garbage after EOF ignored\n", - progName, inName ); + "trailing garbage after EOF ignored\n" ); return True; } default: @@ -576,6 +600,7 @@ Bool testStream ( FILE *zStream ) /*---------------------------------------------*/ void cadvise ( void ) { + if (noisy) fprintf ( stderr, "\nIt is possible that the compressed file(s) have become corrupted.\n" @@ -589,6 +614,7 @@ void cadvise ( void ) /*---------------------------------------------*/ void showFileNames ( void ) { + if (noisy) fprintf ( stderr, "\tInput file = %s, output file = %s\n", @@ -603,6 +629,7 @@ void cleanUpAndFail ( Int32 ec ) IntNative retVal; if ( srcMode == SM_F2F && opMode != OM_TEST ) { + if (noisy) fprintf ( stderr, "%s: Deleting output file %s, if it exists.\n", progName, outName ); if (outputHandleJustInCase != NULL) @@ -613,7 +640,7 @@ void cleanUpAndFail ( Int32 ec ) "%s: WARNING: deletion of output file (apparently) failed.\n", progName ); } - if (numFileNames > 0 && numFilesProcessed < numFileNames) { + if (noisy && numFileNames > 0 && numFilesProcessed < numFileNames) { fprintf ( stderr, "%s: WARNING: some files have not been processed:\n" "\t%d specified on command line, %d not processed yet.\n\n", @@ -639,7 +666,7 @@ void panic ( Char* s ) /*---------------------------------------------*/ -void crcError () +void crcError ( void ) { fprintf ( stderr, "\n%s: Data integrity error when decompressing.\n", @@ -665,7 +692,7 @@ void compressedStreamEOF ( void ) /*---------------------------------------------*/ -void ioError ( ) +void ioError ( void ) { fprintf ( stderr, "\n%s: I/O or other error, bailing out. Possible reason follows.\n", @@ -680,7 +707,7 @@ void ioError ( ) void mySignalCatcher ( IntNative n ) { fprintf ( stderr, - "\n%s: Control-C (or similar) caught, quitting.\n", + "\n%s: Control-C or similar caught, quitting.\n", progName ); cleanUpAndFail(1); } @@ -740,9 +767,10 @@ void copyFileName ( Char* to, Char* from ) if ( strlen(from) > FILE_NAME_LEN-10 ) { fprintf ( stderr, - "bzip2: file name\n`%s'\nis suspiciously (> 1024 chars) long.\n" - "Try using a reasonable file name instead. Sorry! :)\n", - from + "bzip2: file name\n`%s'\n" + "is suspiciously (more than %d chars) long.\n" + "Try using a reasonable file name instead. Sorry! :-)\n", + from, FILE_NAME_LEN-10 ); exit(1); } @@ -778,6 +806,21 @@ Bool notAStandardFile ( Char* name ) } +/*---------------------------------------------*/ +/*-- + rac 11/21/98 see if file has hard links to it +--*/ +Int32 countHardLinks ( Char* name ) +{ + IntNative i; + struct MY_STAT statBuf; + + i = MY_LSTAT ( name, &statBuf ); + if (i != 0) return 0; + return (statBuf.st_nlink - 1); +} + + /*---------------------------------------------*/ void copyDatePermissionsAndOwner ( Char *srcName, Char *dstName ) { @@ -793,17 +836,14 @@ void copyDatePermissionsAndOwner ( Char *srcName, Char *dstName ) retVal = chmod ( dstName, statBuf.st_mode ); ERROR_IF_NOT_ZERO ( retVal ); - /* Not sure if this is really portable or not. Causes - problems on my x86-Linux Redhat 5.0 box. Decided - to omit it from 0.9.0. JRS, 27 June 98. If you - understand Unix file semantics and portability issues - well enough to fix this properly, drop me a line - at jseward@acm.org. - retVal = chown ( dstName, statBuf.st_uid, statBuf.st_gid ); - ERROR_IF_NOT_ZERO ( retVal ); - */ + retVal = utime ( dstName, &uTimBuf ); ERROR_IF_NOT_ZERO ( retVal ); + + retVal = chown ( dstName, statBuf.st_uid, statBuf.st_gid ); + /* chown() will in many cases return with EPERM, which can + be safely ignored. + */ #endif } @@ -819,20 +859,6 @@ void setInterimPermissions ( Char *dstName ) } - -/*---------------------------------------------*/ -Bool endsInBz2 ( Char* name ) -{ - Int32 n = strlen ( name ); - if (n <= 4) return False; - return - (name[n-4] == '.' && - name[n-3] == 'b' && - name[n-2] == 'z' && - name[n-1] == '2'); -} - - /*---------------------------------------------*/ Bool containsDubiousChars ( Char* name ) { @@ -844,49 +870,94 @@ Bool containsDubiousChars ( Char* name ) /*---------------------------------------------*/ -void compress ( Char *name ) +#define BZ_N_SUFFIX_PAIRS 4 + +Char* zSuffix[BZ_N_SUFFIX_PAIRS] + = { ".bz2", ".bz", ".tbz2", ".tbz" }; +Char* unzSuffix[BZ_N_SUFFIX_PAIRS] + = { "", "", ".tar", ".tar" }; + +Bool hasSuffix ( Char* s, Char* suffix ) { - FILE *inStr; - FILE *outStr; + Int32 ns = strlen(s); + Int32 nx = strlen(suffix); + if (ns < nx) return False; + if (strcmp(s + ns - nx, suffix) == 0) return True; + return False; +} +Bool mapSuffix ( Char* name, + Char* oldSuffix, Char* newSuffix ) +{ + if (!hasSuffix(name,oldSuffix)) return False; + name[strlen(name)-strlen(oldSuffix)] = 0; + strcat ( name, newSuffix ); + return True; +} + + +/*---------------------------------------------*/ +void compress ( Char *name ) +{ + FILE *inStr; + FILE *outStr; + Int32 n, i; if (name == NULL && srcMode != SM_I2O) panic ( "compress: bad modes\n" ); switch (srcMode) { - case SM_I2O: copyFileName ( inName, "(stdin)" ); - copyFileName ( outName, "(stdout)" ); break; - case SM_F2F: copyFileName ( inName, name ); - copyFileName ( outName, name ); - strcat ( outName, ".bz2" ); break; - case SM_F2O: copyFileName ( inName, name ); - copyFileName ( outName, "(stdout)" ); break; + case SM_I2O: + copyFileName ( inName, "(stdin)" ); + copyFileName ( outName, "(stdout)" ); + break; + case SM_F2F: + copyFileName ( inName, name ); + copyFileName ( outName, name ); + strcat ( outName, ".bz2" ); + break; + case SM_F2O: + copyFileName ( inName, name ); + copyFileName ( outName, "(stdout)" ); + break; } if ( srcMode != SM_I2O && containsDubiousChars ( inName ) ) { + if (noisy) fprintf ( stderr, "%s: There are no files matching `%s'.\n", progName, inName ); return; } if ( srcMode != SM_I2O && !fileExists ( inName ) ) { - fprintf ( stderr, "%s: Input file %s doesn't exist, skipping.\n", - progName, inName ); + fprintf ( stderr, "%s: Can't open input file %s: %s.\n", + progName, inName, strerror(errno) ); return; } - if ( srcMode != SM_I2O && endsInBz2 ( inName )) { - fprintf ( stderr, "%s: Input file name %s ends in `.bz2', skipping.\n", - progName, inName ); - return; + for (i = 0; i < BZ_N_SUFFIX_PAIRS; i++) { + if (hasSuffix(inName, zSuffix[i])) { + if (noisy) + fprintf ( stderr, + "%s: Input file %s already has %s suffix.\n", + progName, inName, zSuffix[i] ); + return; + } } - if ( srcMode != SM_I2O && notAStandardFile ( inName )) { - fprintf ( stderr, "%s: Input file %s is not a normal file, skipping.\n", + if ( srcMode == SM_F2F && !forceOverwrite && notAStandardFile ( inName )) { + if (noisy) + fprintf ( stderr, "%s: Input file %s is not a normal file.\n", progName, inName ); return; } if ( srcMode == SM_F2F && !forceOverwrite && fileExists ( outName ) ) { - fprintf ( stderr, "%s: Output file %s already exists, skipping.\n", + fprintf ( stderr, "%s: Output file %s already exists.\n", progName, outName ); return; } + if ( srcMode == SM_F2F && !forceOverwrite && + (n=countHardLinks ( inName )) > 0) { + fprintf ( stderr, "%s: Input file %s has %d other link%s.\n", + progName, inName, n, n > 1 ? "s" : "" ); + return; + } switch ( srcMode ) { @@ -912,11 +983,12 @@ void compress ( Char *name ) progName ); fprintf ( stderr, "%s: For help, type: `%s --help'.\n", progName, progName ); + if ( inStr != NULL ) fclose ( inStr ); return; }; if ( inStr == NULL ) { - fprintf ( stderr, "%s: Can't open input file %s, skipping.\n", - progName, inName ); + fprintf ( stderr, "%s: Can't open input file %s: %s.\n", + progName, inName, strerror(errno) ); return; }; break; @@ -925,13 +997,15 @@ void compress ( Char *name ) inStr = fopen ( inName, "rb" ); outStr = fopen ( outName, "wb" ); if ( outStr == NULL) { - fprintf ( stderr, "%s: Can't create output file %s, skipping.\n", - progName, outName ); + fprintf ( stderr, "%s: Can't create output file %s: %s.\n", + progName, outName, strerror(errno) ); + if ( inStr != NULL ) fclose ( inStr ); return; } if ( inStr == NULL ) { - fprintf ( stderr, "%s: Can't open input file %s, skipping.\n", - progName, inName ); + fprintf ( stderr, "%s: Can't open input file %s: %s.\n", + progName, inName, strerror(errno) ); + if ( outStr != NULL ) fclose ( outStr ); return; }; setInterimPermissions ( outName ); @@ -967,51 +1041,72 @@ void compress ( Char *name ) /*---------------------------------------------*/ void uncompress ( Char *name ) { - FILE *inStr; - FILE *outStr; - Bool magicNumberOK; + FILE *inStr; + FILE *outStr; + Int32 n, i; + Bool magicNumberOK; + Bool cantGuess; if (name == NULL && srcMode != SM_I2O) panic ( "uncompress: bad modes\n" ); + cantGuess = False; switch (srcMode) { - case SM_I2O: copyFileName ( inName, "(stdin)" ); - copyFileName ( outName, "(stdout)" ); break; - case SM_F2F: copyFileName ( inName, name ); - copyFileName ( outName, name ); - if (endsInBz2 ( outName )) - outName [ strlen ( outName ) - 4 ] = '\0'; - break; - case SM_F2O: copyFileName ( inName, name ); - copyFileName ( outName, "(stdout)" ); break; + case SM_I2O: + copyFileName ( inName, "(stdin)" ); + copyFileName ( outName, "(stdout)" ); + break; + case SM_F2F: + copyFileName ( inName, name ); + copyFileName ( outName, name ); + for (i = 0; i < BZ_N_SUFFIX_PAIRS; i++) + if (mapSuffix(outName,zSuffix[i],unzSuffix[i])) + goto zzz; + cantGuess = True; + strcat ( outName, ".out" ); + break; + case SM_F2O: + copyFileName ( inName, name ); + copyFileName ( outName, "(stdout)" ); + break; } + zzz: if ( srcMode != SM_I2O && containsDubiousChars ( inName ) ) { + if (noisy) fprintf ( stderr, "%s: There are no files matching `%s'.\n", progName, inName ); return; } if ( srcMode != SM_I2O && !fileExists ( inName ) ) { - fprintf ( stderr, "%s: Input file %s doesn't exist, skipping.\n", - progName, inName ); + fprintf ( stderr, "%s: Can't open input file %s: %s.\n", + progName, inName, strerror(errno) ); return; } - if ( srcMode != SM_I2O && !endsInBz2 ( inName )) { - fprintf ( stderr, - "%s: Input file name %s doesn't end in `.bz2', skipping.\n", - progName, inName ); - return; - } - if ( srcMode != SM_I2O && notAStandardFile ( inName )) { - fprintf ( stderr, "%s: Input file %s is not a normal file, skipping.\n", + if ( srcMode == SM_F2F && !forceOverwrite && notAStandardFile ( inName )) { + if (noisy) + fprintf ( stderr, "%s: Input file %s is not a normal file.\n", progName, inName ); return; } + if ( /* srcMode == SM_F2F implied && */ cantGuess ) { + if (noisy) + fprintf ( stderr, + "%s: Can't guess original name for %s -- using %s\n", + progName, inName, outName ); + /* just a warning, no return */ + } if ( srcMode == SM_F2F && !forceOverwrite && fileExists ( outName ) ) { - fprintf ( stderr, "%s: Output file %s already exists, skipping.\n", + fprintf ( stderr, "%s: Output file %s already exists.\n", progName, outName ); return; } + if ( srcMode == SM_F2F && !forceOverwrite && + (n=countHardLinks ( inName ) ) > 0) { + fprintf ( stderr, "%s: Input file %s has %d other link%s.\n", + progName, inName, n, n > 1 ? "s" : "" ); + return; + } switch ( srcMode ) { @@ -1032,8 +1127,9 @@ void uncompress ( Char *name ) inStr = fopen ( inName, "rb" ); outStr = stdout; if ( inStr == NULL ) { - fprintf ( stderr, "%s: Can't open input file %s, skipping.\n", - progName, inName ); + fprintf ( stderr, "%s: Can't open input file %s:%s.\n", + progName, inName, strerror(errno) ); + if ( inStr != NULL ) fclose ( inStr ); return; }; break; @@ -1042,13 +1138,15 @@ void uncompress ( Char *name ) inStr = fopen ( inName, "rb" ); outStr = fopen ( outName, "wb" ); if ( outStr == NULL) { - fprintf ( stderr, "%s: Can't create output file %s, skipping.\n", - progName, outName ); + fprintf ( stderr, "%s: Can't create output file %s: %s.\n", + progName, outName, strerror(errno) ); + if ( inStr != NULL ) fclose ( inStr ); return; } if ( inStr == NULL ) { - fprintf ( stderr, "%s: Can't open input file %s, skipping.\n", - progName, inName ); + fprintf ( stderr, "%s: Can't open input file %s: %s.\n", + progName, inName, strerror(errno) ); + if ( outStr != NULL ) fclose ( outStr ); return; }; setInterimPermissions ( outName ); @@ -1091,9 +1189,9 @@ void uncompress ( Char *name ) fprintf ( stderr, "done\n" ); } else { if (verbosity >= 1) - fprintf ( stderr, "not a bzip2 file, skipping.\n" ); else + fprintf ( stderr, "not a bzip2 file.\n" ); else fprintf ( stderr, - "%s: %s is not a bzip2 file, skipping.\n", + "%s: %s is not a bzip2 file.\n", progName, inName ); } @@ -1117,24 +1215,14 @@ void testf ( Char *name ) } if ( srcMode != SM_I2O && containsDubiousChars ( inName ) ) { + if (noisy) fprintf ( stderr, "%s: There are no files matching `%s'.\n", progName, inName ); return; } if ( srcMode != SM_I2O && !fileExists ( inName ) ) { - fprintf ( stderr, "%s: Input file %s doesn't exist, skipping.\n", - progName, inName ); - return; - } - if ( srcMode != SM_I2O && !endsInBz2 ( inName )) { - fprintf ( stderr, - "%s: Input file name %s doesn't end in `.bz2', skipping.\n", - progName, inName ); - return; - } - if ( srcMode != SM_I2O && notAStandardFile ( inName )) { - fprintf ( stderr, "%s: Input file %s is not a normal file, skipping.\n", - progName, inName ); + fprintf ( stderr, "%s: Can't open input %s: %s.\n", + progName, inName, strerror(errno) ); return; } @@ -1155,8 +1243,8 @@ void testf ( Char *name ) case SM_F2O: case SM_F2F: inStr = fopen ( inName, "rb" ); if ( inStr == NULL ) { - fprintf ( stderr, "%s: Can't open input file %s, skipping.\n", - progName, inName ); + fprintf ( stderr, "%s: Can't open input file %s:%s.\n", + progName, inName, strerror(errno) ); return; }; break; @@ -1186,13 +1274,13 @@ void license ( void ) fprintf ( stderr, "bzip2, a block-sorting file compressor. " - "Version 0.9.0c, 18-Oct-98.\n" + "Version 0.9.5d, 4-Sept-99.\n" " \n" - " Copyright (C) 1996, 1997, 1998 by Julian Seward.\n" + " Copyright (C) 1996, 1997, 1998, 1999 by Julian Seward.\n" " \n" " This program is free software; you can redistribute it and/or modify\n" " it under the terms set out in the LICENSE file, which is included\n" - " in the bzip2-0.9.0c source distribution.\n" + " in the bzip2-0.9.5 source distribution.\n" " \n" " This program is distributed in the hope that it will be useful,\n" " but WITHOUT ANY WARRANTY; without even the implied warranty of\n" @@ -1209,27 +1297,26 @@ void usage ( Char *fullProgName ) fprintf ( stderr, "bzip2, a block-sorting file compressor. " - "Version 0.9.0c, 18-Oct-98.\n" + "Version 0.9.5d, 4-Sept-99.\n" "\n usage: %s [flags and input files in any order]\n" "\n" " -h --help print this message\n" " -d --decompress force decompression\n" " -z --compress force compression\n" " -k --keep keep (don't delete) input files\n" - " -f --force overwrite existing output filess\n" + " -f --force overwrite existing output files\n" " -t --test test compressed file integrity\n" " -c --stdout output to standard out\n" + " -q --quiet suppress noncritical error messages\n" " -v --verbose be verbose (a 2nd -v gives more)\n" " -L --license display software version & license\n" " -V --version display software version & license\n" " -s --small use less memory (at most 2500k)\n" " -1 .. -9 set block size to 100k .. 900k\n" - " --repetitive-fast compress repetitive blocks faster\n" - " --repetitive-best compress repetitive blocks better\n" "\n" " If invoked as `bzip2', default action is to compress.\n" " as `bunzip2', default action is to decompress.\n" - " as `bz2cat', default action is to decompress to stdout.\n" + " as `bzcat', default action is to decompress to stdout.\n" "\n" " If no file names are given, bzip2 compresses or decompresses\n" " from standard input to standard output. You can combine\n" @@ -1244,19 +1331,29 @@ void usage ( Char *fullProgName ) } +/*---------------------------------------------*/ +void redundant ( Char* flag ) +{ + fprintf ( + stderr, + "%s: %s is redundant in versions 0.9.5 and above\n", + progName, flag ); +} + + /*---------------------------------------------*/ /*-- All the garbage from here to main() is purely to implement a linked list of command-line arguments, into which main() copies argv[1 .. argc-1]. - The purpose of this ridiculous exercise is to - facilitate the expansion of wildcard characters - * and ? in filenames for halfwitted OSs like - MSDOS, Windows 95 and NT. + The purpose of this exercise is to facilitate + the expansion of wildcard characters * and ? in + filenames for OSs which don't know how to do it + themselves, like MSDOS, Windows 95 and NT. - The actual Dirty Work is done by the platform-specific - macro APPEND_FILESPEC. + The actual Dirty Work is done by the platform- + specific macro APPEND_FILESPEC. --*/ typedef @@ -1308,15 +1405,42 @@ Cell *snocString ( Cell *root, Char *name ) /*---------------------------------------------*/ -#define ISFLAG(s) (strcmp(aa->name, (s))==0) +void addFlagsFromEnvVar ( Cell** argList, Char* varName ) +{ + Int32 i, j, k; + Char *envbase, *p; + + envbase = getenv(varName); + if (envbase != NULL) { + p = envbase; + i = 0; + while (True) { + if (p[i] == 0) break; + p += i; + i = 0; + while (isspace((Int32)(p[0]))) p++; + while (p[i] != 0 && !isspace((Int32)(p[i]))) i++; + if (i > 0) { + k = i; if (k > FILE_NAME_LEN-10) k = FILE_NAME_LEN-10; + for (j = 0; j < k; j++) tmpName[j] = p[j]; + tmpName[k] = 0; + APPEND_FLAG(*argList, tmpName); + } + } + } +} +/*---------------------------------------------*/ +#define ISFLAG(s) (strcmp(aa->name, (s))==0) + IntNative main ( IntNative argc, Char *argv[] ) { Int32 i, j; Char *tmp; Cell *argList; Cell *aa; + Bool decode; /*-- Be really really really paranoid :-) --*/ if (sizeof(Int32) != 4 || sizeof(UInt32) != 4 || @@ -1332,27 +1456,27 @@ IntNative main ( IntNative argc, Char *argv[] ) } - /*-- Set up signal handlers --*/ - signal (SIGINT, mySignalCatcher); - signal (SIGTERM, mySignalCatcher); - signal (SIGSEGV, mySIGSEGVorSIGBUScatcher); -#if BZ_UNIX - signal (SIGHUP, mySignalCatcher); - signal (SIGBUS, mySIGSEGVorSIGBUScatcher); -#endif - - /*-- Initialise --*/ outputHandleJustInCase = NULL; smallMode = False; keepInputFiles = False; forceOverwrite = False; + noisy = True; verbosity = 0; blockSize100k = 9; testFailsExist = False; numFileNames = 0; numFilesProcessed = 0; workFactor = 30; + i = j = 0; /* avoid bogus warning from egcs-1.1.X */ + + /*-- Set up signal handlers for mem access errors --*/ + signal (SIGSEGV, mySIGSEGVorSIGBUScatcher); +#if BZ_UNIX +#ifndef __DJGPP__ + signal (SIGBUS, mySIGSEGVorSIGBUScatcher); +#endif +#endif copyFileName ( inName, "(none)" ); copyFileName ( outName, "(none)" ); @@ -1363,8 +1487,12 @@ IntNative main ( IntNative argc, Char *argv[] ) if (*tmp == PATH_SEP) progName = tmp + 1; - /*-- Expand filename wildcards in arg list --*/ + /*-- Copy flags from env var BZIP2, and + expand filename wildcards in arg list. + --*/ argList = NULL; + addFlagsFromEnvVar ( &argList, "BZIP2" ); + addFlagsFromEnvVar ( &argList, "BZIP" ); for (i = 1; i <= argc-1; i++) APPEND_FILESPEC(argList, argv[i]); @@ -1372,12 +1500,14 @@ IntNative main ( IntNative argc, Char *argv[] ) /*-- Find the length of the longest filename --*/ longestFileName = 7; numFileNames = 0; - for (aa = argList; aa != NULL; aa = aa->link) - if (aa->name[0] != '-') { - numFileNames++; - if (longestFileName < (Int32)strlen(aa->name) ) - longestFileName = (Int32)strlen(aa->name); - } + decode = True; + for (aa = argList; aa != NULL; aa = aa->link) { + if (ISFLAG("--")) { decode = False; continue; } + if (aa->name[0] == '-' && decode) continue; + numFileNames++; + if (longestFileName < (Int32)strlen(aa->name) ) + longestFileName = (Int32)strlen(aa->name); + } /*-- Determine source modes; flag handling may change this too. --*/ @@ -1403,9 +1533,10 @@ IntNative main ( IntNative argc, Char *argv[] ) /*-- Look at the flags. --*/ - for (aa = argList; aa != NULL; aa = aa->link) - if (aa->name[0] == '-' && aa->name[1] != '-') - for (j = 1; aa->name[j] != '\0'; j++) + for (aa = argList; aa != NULL; aa = aa->link) { + if (ISFLAG("--")) break; + if (aa->name[0] == '-' && aa->name[1] != '-') { + for (j = 1; aa->name[j] != '\0'; j++) { switch (aa->name[j]) { case 'c': srcMode = SM_F2O; break; case 'd': opMode = OM_UNZ; break; @@ -1414,6 +1545,7 @@ IntNative main ( IntNative argc, Char *argv[] ) case 't': opMode = OM_TEST; break; case 'k': keepInputFiles = True; break; case 's': smallMode = True; break; + case 'q': noisy = False; break; case '1': blockSize100k = 1; break; case '2': blockSize100k = 2; break; case '3': blockSize100k = 3; break; @@ -1427,17 +1559,21 @@ IntNative main ( IntNative argc, Char *argv[] ) case 'L': license(); break; case 'v': verbosity++; break; case 'h': usage ( progName ); - exit ( 1 ); + exit ( 0 ); break; default: fprintf ( stderr, "%s: Bad flag `%s'\n", progName, aa->name ); usage ( progName ); exit ( 1 ); break; + } } - + } + } + /*-- And again ... --*/ for (aa = argList; aa != NULL; aa = aa->link) { + if (ISFLAG("--")) break; if (ISFLAG("--stdout")) srcMode = SM_F2O; else if (ISFLAG("--decompress")) opMode = OM_UNZ; else if (ISFLAG("--compress")) opMode = OM_Z; else @@ -1445,12 +1581,14 @@ IntNative main ( IntNative argc, Char *argv[] ) if (ISFLAG("--test")) opMode = OM_TEST; else if (ISFLAG("--keep")) keepInputFiles = True; else if (ISFLAG("--small")) smallMode = True; else + if (ISFLAG("--quiet")) noisy = False; else if (ISFLAG("--version")) license(); else if (ISFLAG("--license")) license(); else - if (ISFLAG("--repetitive-fast")) workFactor = 5; else - if (ISFLAG("--repetitive-best")) workFactor = 150; else + if (ISFLAG("--exponential")) workFactor = 1; else + if (ISFLAG("--repetitive-best")) redundant(aa->name); else + if (ISFLAG("--repetitive-fast")) redundant(aa->name); else if (ISFLAG("--verbose")) verbosity++; else - if (ISFLAG("--help")) { usage ( progName ); exit ( 1 ); } + if (ISFLAG("--help")) { usage ( progName ); exit ( 0 ); } else if (strncmp ( aa->name, "--", 2) == 0) { fprintf ( stderr, "%s: Bad flag `%s'\n", progName, aa->name ); @@ -1460,13 +1598,8 @@ IntNative main ( IntNative argc, Char *argv[] ) } if (verbosity > 4) verbosity = 4; - if (opMode == OM_Z && smallMode) blockSize100k = 2; - - if (srcMode == SM_F2O && numFileNames == 0) { - fprintf ( stderr, "%s: -c expects at least one filename.\n", - progName ); - exit ( 1 ); - } + if (opMode == OM_Z && smallMode && blockSize100k > 2) + blockSize100k = 2; if (opMode == OM_TEST && srcMode == SM_F2O) { fprintf ( stderr, "%s: -c and -t cannot be used together.\n", @@ -1474,46 +1607,82 @@ IntNative main ( IntNative argc, Char *argv[] ) exit ( 1 ); } + if (srcMode == SM_F2O && numFileNames == 0) + srcMode = SM_I2O; + if (opMode != OM_Z) blockSize100k = 0; + if (srcMode == SM_F2F) { + signal (SIGINT, mySignalCatcher); + signal (SIGTERM, mySignalCatcher); +# if BZ_UNIX + signal (SIGHUP, mySignalCatcher); +# endif + } + if (opMode == OM_Z) { - if (srcMode == SM_I2O) - compress ( NULL ); - else - for (aa = argList; aa != NULL; aa = aa->link) - if (aa->name[0] != '-') { - numFilesProcessed++; - compress ( aa->name ); - } - } else + if (srcMode == SM_I2O) { + compress ( NULL ); + } else { + decode = True; + for (aa = argList; aa != NULL; aa = aa->link) { + if (ISFLAG("--")) { decode = False; continue; } + if (aa->name[0] == '-' && decode) continue; + numFilesProcessed++; + compress ( aa->name ); + } + } + } + else + if (opMode == OM_UNZ) { - if (srcMode == SM_I2O) + if (srcMode == SM_I2O) { uncompress ( NULL ); - else - for (aa = argList; aa != NULL; aa = aa->link) - if (aa->name[0] != '-') { - numFilesProcessed++; - uncompress ( aa->name ); - } - } else { + } else { + decode = True; + for (aa = argList; aa != NULL; aa = aa->link) { + if (ISFLAG("--")) { decode = False; continue; } + if (aa->name[0] == '-' && decode) continue; + numFilesProcessed++; + uncompress ( aa->name ); + } + } + } + + else { testFailsExist = False; - if (srcMode == SM_I2O) + if (srcMode == SM_I2O) { testf ( NULL ); - else - for (aa = argList; aa != NULL; aa = aa->link) - if (aa->name[0] != '-') { - numFilesProcessed++; - testf ( aa->name ); - } - if (testFailsExist) { + } else { + decode = True; + for (aa = argList; aa != NULL; aa = aa->link) { + if (ISFLAG("--")) { decode = False; continue; } + if (aa->name[0] == '-' && decode) continue; + numFilesProcessed++; + testf ( aa->name ); + } + } + if (testFailsExist && noisy) { fprintf ( stderr, "\n" - "You can use the `bzip2recover' program to *attempt* to recover\n" + "You can use the `bzip2recover' program to attempt to recover\n" "data from undamaged sections of corrupted files.\n\n" ); exit(2); } } + + /* Free the argument list memory to mollify leak detectors + (eg) Purify, Checker. Serves no other useful purpose. + */ + aa = argList; + while (aa != NULL) { + Cell* aa2 = aa->link; + if (aa->name) free(aa->name); + free(aa); + aa = aa2; + } + return 0; } diff --git a/bzip2.txt b/bzip2.txt index 898dfe8..da23c64 100644 --- a/bzip2.txt +++ b/bzip2.txt @@ -1,22 +1,20 @@ -bzip2(1) bzip2(1) - NAME - bzip2, bunzip2 - a block-sorting file compressor, v0.9.0 + bzip2, bunzip2 - a block-sorting file compressor, v0.9.5 bzcat - decompresses files to stdout bzip2recover - recovers data from damaged bzip2 files SYNOPSIS - bzip2 [ -cdfkstvzVL123456789 ] [ filenames ... ] + bzip2 [ -cdfkqstvzVL123456789 ] [ filenames ... ] bunzip2 [ -fkvsVL ] [ filenames ... ] bzcat [ -s ] [ filenames ... ] bzip2recover filename DESCRIPTION - bzip2 compresses files using the Burrows-Wheeler block- + bzip2 compresses files using the Burrows-Wheeler block sorting text compression algorithm, and Huffman coding. Compression is generally considerably better than that achieved by more conventional LZ77/LZ78-based compressors, @@ -24,22 +22,22 @@ DESCRIPTION tistical compressors. The command-line options are deliberately very similar to - those of GNU Gzip, but they are not identical. + those of GNU gzip, but they are not identical. bzip2 expects a list of file names to accompany the com- mand-line flags. Each file is replaced by a compressed version of itself, with the name "original_name.bz2". - Each compressed file has the same modification date and - permissions as the corresponding original, so that these - properties can be correctly restored at decompression - time. File name handling is naive in the sense that there - is no mechanism for preserving original file names, per- - missions and dates in filesystems which lack these con- - cepts, or have serious file name length restrictions, such - as MS-DOS. + Each compressed file has the same modification date, per- + missions, and, when possible, ownership as the correspond- + ing original, so that these properties can be correctly + restored at decompression time. File name handling is + naive in the sense that there is no mechanism for preserv- + ing original file names, permissions, ownerships or dates + in filesystems which lack these concepts, or have serious + file name length restrictions, such as MS-DOS. bzip2 and bunzip2 will by default not overwrite existing - files; if you want this to happen, specify the -f flag. + files. If you want this to happen, specify the -f flag. If no file names are specified, bzip2 compresses from standard input to standard output. In this case, bzip2 @@ -47,10 +45,25 @@ DESCRIPTION this would be entirely incomprehensible and therefore pointless. - bunzip2 (or bzip2 -d ) decompresses and restores all spec- - ified files whose names end in ".bz2". Files without this - suffix are ignored. Again, supplying no filenames causes - decompression from standard input to standard output. + bunzip2 (or bzip2 -d) decompresses all specified files. + Files which were not created by bzip2 will be detected and + ignored, and a warning issued. bzip2 attempts to guess + the filename for the decompressed file from that of the + compressed file as follows: + + filename.bz2 becomes filename + filename.bz becomes filename + filename.tbz2 becomes filename.tar + filename.tbz becomes filename.tar + anyothername becomes anyothername.out + + If the file does not end in one of the recognised endings, + .bz2, .bz, .tbz2 or .tbz, bzip2 complains that it cannot + guess the name of the original file, and uses the original + name with .out appended. + + As with compression, supplying no filenames causes decom- + pression from standard input to standard output. bunzip2 will correctly decompress a file which is the con- catenation of two or more compressed files. The result is @@ -58,19 +71,24 @@ DESCRIPTION Integrity testing (-t) of concatenated compressed files is also supported. - You can also compress or decompress files to the standard - output by giving the -c flag. Multiple files may be com- + You can also compress or decompress files to the standard + output by giving the -c flag. Multiple files may be com- pressed and decompressed like this. The resulting outputs - are fed sequentially to stdout. Compression of multiple - files in this manner generates a stream containing multi- + are fed sequentially to stdout. Compression of multiple + files in this manner generates a stream containing multi- ple compressed file representations. Such a stream can be - decompressed correctly only by bzip2 version 0.9.0 or - later. Earlier versions of bzip2 will stop after decom- + decompressed correctly only by bzip2 version 0.9.0 or + later. Earlier versions of bzip2 will stop after decom- pressing the first file in the stream. - bzcat (or bzip2 -dc ) decompresses all specified files to + bzcat (or bzip2 -dc) decompresses all specified files to the standard output. + bzip2 will read arguments from the environment variables + BZIP2 and BZIP, in that order, and will process them + before any arguments read from the command line. This + gives a convenient way to supply default arguments. + Compression is always performed, even if the compressed file is slightly larger than the original. Files of less than about one hundred bytes tend to get larger, since the @@ -87,98 +105,19 @@ DESCRIPTION corruption going undetected is microscopic, about one chance in four billion for each file processed. Be aware, though, that the check occurs upon decompression, so it - can only tell you that that something is wrong. It can't - help you recover the original uncompressed data. You can - use bzip2recover to try to recover data from damaged - files. + can only tell you that something is wrong. It can't help + you recover the original uncompressed data. You can use + bzip2recover to try to recover data from damaged files. - Return values: 0 for a normal exit, 1 for environmental - problems (file not found, invalid flags, I/O errors, &c), + Return values: 0 for a normal exit, 1 for environmental + problems (file not found, invalid flags, I/O errors, &c), 2 to indicate a corrupt compressed file, 3 for an internal consistency error (eg, bug) which caused bzip2 to panic. -MEMORY MANAGEMENT - Bzip2 compresses large files in blocks. The block size - affects both the compression ratio achieved, and the - amount of memory needed both for compression and decom- - pression. The flags -1 through -9 specify the block size - to be 100,000 bytes through 900,000 bytes (the default) - respectively. At decompression-time, the block size used - for compression is read from the header of the compressed - file, and bunzip2 then allocates itself just enough memory - to decompress the file. Since block sizes are stored in - compressed files, it follows that the flags -1 to -9 are - irrelevant to and so ignored during decompression. - - Compression and decompression requirements, in bytes, can - be estimated as: - - Compression: 400k + ( 7 x block size ) - - Decompression: 100k + ( 4 x block size ), or - 100k + ( 2.5 x block size ) - - Larger block sizes give rapidly diminishing marginal - returns; most of the compression comes from the first two - or three hundred k of block size, a fact worth bearing in - mind when using bzip2 on small machines. It is also - important to appreciate that the decompression memory - requirement is set at compression-time by the choice of - block size. - - For files compressed with the default 900k block size, - bunzip2 will require about 3700 kbytes to decompress. To - support decompression of any file on a 4 megabyte machine, - bunzip2 has an option to decompress using approximately - half this amount of memory, about 2300 kbytes. Decompres- - sion speed is also halved, so you should use this option - only where necessary. The relevant flag is -s. - - In general, try and use the largest block size memory con- - straints allow, since that maximises the compression - achieved. Compression and decompression speed are virtu- - ally unaffected by block size. - - Another significant point applies to files which fit in a - single block -- that means most files you'd encounter - using a large block size. The amount of real memory - touched is proportional to the size of the file, since the - file is smaller than a block. For example, compressing a - file 20,000 bytes long with the flag -9 will cause the - compressor to allocate around 6700k of memory, but only - touch 400k + 20000 * 7 = 540 kbytes of it. Similarly, the - decompressor will allocate 3700k but only touch 100k + - 20000 * 4 = 180 kbytes. - - Here is a table which summarises the maximum memory usage - for different block sizes. Also recorded is the total - compressed size for 14 files of the Calgary Text Compres- - sion Corpus totalling 3,141,622 bytes. This column gives - some feel for how compression varies with block size. - These figures tend to understate the advantage of larger - block sizes for larger files, since the Corpus is domi- - nated by smaller files. - - Compress Decompress Decompress Corpus - Flag usage usage -s usage Size - - -1 1100k 500k 350k 914704 - -2 1800k 900k 600k 877703 - -3 2500k 1300k 850k 860338 - -4 3200k 1700k 1100k 846899 - -5 3900k 2100k 1350k 845160 - -6 4600k 2500k 1600k 838626 - -7 5400k 2900k 1850k 834096 - -8 6000k 3300k 2100k 828642 - -9 6700k 3700k 2350k 828642 - - OPTIONS -c --stdout - Compress or decompress to standard output. -c will - decompress multiple files to stdout, but will only - compress a single file to stdout. + Compress or decompress to standard output. -d --decompress Force decompression. bzip2, bunzip2 and bzcat are @@ -198,7 +137,9 @@ OPTIONS -f --force Force overwrite of output files. Normally, bzip2 - will not overwrite existing output files. + will not overwrite existing output files. Also + forces bzip2 to break hard links to files, which it + otherwise wouldn't do. -k --keep Keep (don't delete) input files during compression @@ -217,7 +158,12 @@ OPTIONS figure, at the expense of your compression ratio. In short, if your machine is low on memory (8 megabytes or less), use -s for everything. See - MEMORY MANAGEMENT above. + MEMORY MANAGEMENT below. + + -q --quiet + Suppress non-essential warning messages. Messages + pertaining to I/O errors and other critical events + will not be suppressed. -v --verbose Verbose mode -- show the compression ratio for each @@ -232,21 +178,96 @@ OPTIONS -1 to -9 Set the block size to 100 k, 200 k .. 900 k when compressing. Has no effect when decompressing. - See MEMORY MANAGEMENT above. + See MEMORY MANAGEMENT below. - --repetitive-fast - bzip2 injects some small pseudo-random variations - into very repetitive blocks to limit worst-case - performance during compression. If sorting runs - into difficulties, the block is randomised, and - sorting is restarted. Very roughly, bzip2 persists - for three times as long as a well-behaved input - would take before resorting to randomisation. This - flag makes it give up much sooner. + -- Treats all subsequent arguments as file names, even + if they start with a dash. This is so you can han- + dle files with names beginning with a dash, for + example: bzip2 -- -myfilename. - --repetitive-best - Opposite of --repetitive-fast; try a lot harder - before resorting to randomisation. + --repetitive-fast --repetitive-best + These flags are redundant in versions 0.9.5 and + above. They provided some coarse control over the + behaviour of the sorting algorithm in earlier ver- + sions, which was sometimes useful. 0.9.5 and above + have an improved algorithm which renders these + flags irrelevant. + + +MEMORY MANAGEMENT + bzip2 compresses large files in blocks. The block size + affects both the compression ratio achieved, and the + amount of memory needed for compression and decompression. + The flags -1 through -9 specify the block size to be + 100,000 bytes through 900,000 bytes (the default) respec- + tively. At decompression time, the block size used for + compression is read from the header of the compressed + file, and bunzip2 then allocates itself just enough memory + to decompress the file. Since block sizes are stored in + compressed files, it follows that the flags -1 to -9 are + irrelevant to and so ignored during decompression. + + Compression and decompression requirements, in bytes, can + be estimated as: + + Compression: 400k + ( 8 x block size ) + + Decompression: 100k + ( 4 x block size ), or + 100k + ( 2.5 x block size ) + + Larger block sizes give rapidly diminishing marginal + returns. Most of the compression comes from the first two + or three hundred k of block size, a fact worth bearing in + mind when using bzip2 on small machines. It is also + important to appreciate that the decompression memory + requirement is set at compression time by the choice of + block size. + + For files compressed with the default 900k block size, + bunzip2 will require about 3700 kbytes to decompress. To + support decompression of any file on a 4 megabyte machine, + bunzip2 has an option to decompress using approximately + half this amount of memory, about 2300 kbytes. Decompres- + sion speed is also halved, so you should use this option + only where necessary. The relevant flag is -s. + + In general, try and use the largest block size memory con- + straints allow, since that maximises the compression + achieved. Compression and decompression speed are virtu- + ally unaffected by block size. + + Another significant point applies to files which fit in a + single block -- that means most files you'd encounter + using a large block size. The amount of real memory + touched is proportional to the size of the file, since the + file is smaller than a block. For example, compressing a + file 20,000 bytes long with the flag -9 will cause the + compressor to allocate around 7600k of memory, but only + touch 400k + 20000 * 8 = 560 kbytes of it. Similarly, the + decompressor will allocate 3700k but only touch 100k + + 20000 * 4 = 180 kbytes. + + Here is a table which summarises the maximum memory usage + for different block sizes. Also recorded is the total + compressed size for 14 files of the Calgary Text Compres- + sion Corpus totalling 3,141,622 bytes. This column gives + some feel for how compression varies with block size. + These figures tend to understate the advantage of larger + block sizes for larger files, since the Corpus is domi- + nated by smaller files. + + Compress Decompress Decompress Corpus + Flag usage usage -s usage Size + + -1 1200k 500k 350k 914704 + -2 2000k 900k 600k 877703 + -3 2800k 1300k 850k 860338 + -4 3600k 1700k 1100k 846899 + -5 4400k 2100k 1350k 845160 + -6 5200k 2500k 1600k 838626 + -7 6100k 2900k 1850k 834096 + -8 6800k 3300k 2100k 828642 + -9 7600k 3700k 2350k 828642 RECOVERING DATA FROM DAMAGED FILES @@ -273,8 +294,8 @@ RECOVERING DATA FROM DAMAGED FILES "rec0002file.bz2", etc, containing the extracted blocks. The output filenames are designed so that the use of wildcards in subsequent processing -- for example, "bzip2 - -dc rec*file.bz2 > recovered_data" -- lists the files in - the "right" order. + -dc rec*file.bz2 > recovered_data" -- lists the files in + the correct order. bzip2recover should be of most use dealing with large .bz2 files, as these will contain many blocks. It is clearly @@ -289,17 +310,15 @@ PERFORMANCE NOTES The sorting phase of compression gathers together similar strings in the file. Because of this, files containing very long runs of repeated symbols, like "aabaabaabaab - ..." (repeated several hundred times) may compress - extraordinarily slowly. You can use the -vvvvv option to - monitor progress in great detail, if you want. Decompres- - sion speed is unaffected. - - Such pathological cases seem rare in practice, appearing - mostly in artificially-constructed test files, and in low- - level disk images. It may be inadvisable to use bzip2 to - compress the latter. If you do get a file which causes - severe slowness in compression, try making the block size - as small as possible, with flag -1. + ..." (repeated several hundred times) may compress more + slowly than normal. Versions 0.9.5 and above fare much + better than previous versions in this respect. The ratio + between worst-case and average-case compression time is in + the region of 10:1. For previous versions, this figure + was more like 100:1. You can use the -vvvv option to mon- + itor progress in great detail, if you want. + + Decompression speed is unaffected by these phenomena. bzip2 usually allocates several megabytes of memory to operate in, and then charges all over it in a fairly ran- @@ -314,42 +333,43 @@ PERFORMANCE NOTES CAVEATS I/O error messages are not as helpful as they could be. - Bzip2 tries hard to detect I/O errors and exit cleanly, + bzip2 tries hard to detect I/O errors and exit cleanly, but the details of what the problem is sometimes seem rather misleading. - This manual page pertains to version 0.9.0 of bzip2. Com- + This manual page pertains to version 0.9.5 of bzip2. Com- pressed data created by this version is entirely forwards - and backwards compatible with the previous public release, - version 0.1pl2, but with the following exception: 0.9.0 - can correctly decompress multiple concatenated compressed - files. 0.1pl2 cannot do this; it will stop after decom- - pressing just the first file in the stream. - - Wildcard expansion for Windows 95 and NT is flaky. - - bzip2recover uses 32-bit integers to represent bit posi- - tions in compressed files, so it cannot handle compressed - files more than 512 megabytes long. This could easily be + and backwards compatible with the previous public + releases, versions 0.1pl2 and 0.9.0, but with the follow- + ing exception: 0.9.0 and above can correctly decompress + multiple concatenated compressed files. 0.1pl2 cannot do + this; it will stop after decompressing just the first file + in the stream. + + bzip2recover uses 32-bit integers to represent bit posi- + tions in compressed files, so it cannot handle compressed + files more than 512 megabytes long. This could easily be fixed. AUTHOR Julian Seward, jseward@acm.org. + http://www.muraroa.demon.co.uk The ideas embodied in bzip2 are due to (at least) the fol- - lowing people: Michael Burrows and David Wheeler (for the - block sorting transformation), David Wheeler (again, for + lowing people: Michael Burrows and David Wheeler (for the + block sorting transformation), David Wheeler (again, for the Huffman coder), Peter Fenwick (for the structured cod- ing model in the original bzip, and many refinements), and - Alistair Moffat, Radford Neal and Ian Witten (for the + Alistair Moffat, Radford Neal and Ian Witten (for the arithmetic coder in the original bzip). I am much indebted for their help, support and advice. See the man- - ual in the source distribution for pointers to sources of + ual in the source distribution for pointers to sources of documentation. Christian von Roques encouraged me to look - for faster sorting algorithms, so as to speed up compres- + for faster sorting algorithms, so as to speed up compres- sion. Bela Lubkin encouraged me to improve the worst-case compression performance. Many people sent patches, helped - with portability problems, lent machines, gave advice and + with portability problems, lent machines, gave advice and were generally helpful. + diff --git a/bzip2recover.c b/bzip2recover.c index 0e2822b..1323b36 100644 --- a/bzip2recover.c +++ b/bzip2recover.c @@ -7,9 +7,9 @@ /*-- This program is bzip2recover, a program to attempt data salvage from damaged files created by the accompanying - bzip2-0.9.0c program. + bzip2-0.9.5 program. - Copyright (C) 1996-1998 Julian R Seward. All rights reserved. + Copyright (C) 1996-1999 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -42,9 +42,9 @@ NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - Julian Seward, Guildford, Surrey, UK. + Julian Seward, Cambridge, UK. jseward@acm.org - bzip2/libbzip2 version 0.9.0c of 18 October 1998 + bzip2/libbzip2 version 0.9.5 of 24 May 1999 --*/ /*-- @@ -260,6 +260,12 @@ Bool endsInBz2 ( Char* name ) #define BLOCK_ENDMARK_HI 0x00001772UL #define BLOCK_ENDMARK_LO 0x45385090UL + +UInt32 bStart[20000]; +UInt32 bEnd[20000]; +UInt32 rbStart[20000]; +UInt32 rbEnd[20000]; + Int32 main ( Int32 argc, Char** argv ) { FILE* inFile; @@ -267,11 +273,6 @@ Int32 main ( Int32 argc, Char** argv ) BitStream* bsIn, *bsWr; Int32 currBlock, b, wrBlock; UInt32 bitsRead; - UInt32 bStart[20000]; - UInt32 bEnd[20000]; - - UInt32 rbStart[20000]; - UInt32 rbEnd[20000]; Int32 rbCtr; @@ -281,7 +282,7 @@ Int32 main ( Int32 argc, Char** argv ) strcpy ( progName, argv[0] ); inFileName[0] = outFileName[0] = 0; - fprintf ( stderr, "bzip2recover v0.9.0c: extracts blocks from damaged .bz2 files.\n" ); + fprintf ( stderr, "bzip2recover 0.9.5d: extracts blocks from damaged .bz2 files.\n" ); if (argc != 2) { fprintf ( stderr, "%s: usage is `%s damaged_file_name'.\n", diff --git a/bzlib.c b/bzlib.c index 362e8ff..24e8bd5 100644 --- a/bzlib.c +++ b/bzlib.c @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-1998 Julian R Seward. All rights reserved. + Copyright (C) 1996-1999 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -41,9 +41,9 @@ NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - Julian Seward, Guildford, Surrey, UK. + Julian Seward, Cambridge, UK. jseward@acm.org - bzip2/libbzip2 version 0.9.0c of 18 October 1998 + bzip2/libbzip2 version 0.9.5 of 24 May 1999 This program is based on (at least) the work of: Mike Burrows @@ -86,14 +86,14 @@ void bz__AssertH__fail ( int errcode ) { fprintf(stderr, - "\n\nbzip2/libbzip2, v0.9.0c: internal error number %d.\n" - "This is a bug in bzip2/libbzip2, v0.9.0c. Please report\n" + "\n\nbzip2/libbzip2, v0.9.5d: internal error number %d.\n" + "This is a bug in bzip2/libbzip2, v0.9.5d. Please report\n" "it to me at: jseward@acm.org. If this happened when\n" "you were using some program which uses libbzip2 as a\n" "component, you should also report this bug to the author(s)\n" "of that program. Please make an effort to report this bug;\n" "timely and accurate bug reports eventually lead to higher\n" - "quality software. Thx. Julian Seward, 18 October 1998.\n\n", + "quality software. Thanks. Julian Seward, 4 Sept 1999.\n\n", errcode ); exit(3); @@ -171,29 +171,23 @@ int BZ_API(bzCompressInit) if (s == NULL) return BZ_MEM_ERROR; s->strm = strm; - s->block = NULL; - s->quadrant = NULL; - s->zptr = NULL; - s->ftab = NULL; - - n = 100000 * blockSize100k; - s->block = BZALLOC( (n + BZ_NUM_OVERSHOOT_BYTES) * sizeof(UChar) ); - s->quadrant = BZALLOC( (n + BZ_NUM_OVERSHOOT_BYTES) * sizeof(Int16) ); - s->zptr = BZALLOC( n * sizeof(Int32) ); - s->ftab = BZALLOC( 65537 * sizeof(Int32) ); - - if (s->block == NULL || s->quadrant == NULL || - s->zptr == NULL || s->ftab == NULL) { - if (s->block != NULL) BZFREE(s->block); - if (s->quadrant != NULL) BZFREE(s->quadrant); - if (s->zptr != NULL) BZFREE(s->zptr); - if (s->ftab != NULL) BZFREE(s->ftab); - if (s != NULL) BZFREE(s); + s->arr1 = NULL; + s->arr2 = NULL; + s->ftab = NULL; + + n = 100000 * blockSize100k; + s->arr1 = BZALLOC( n * sizeof(UInt32) ); + s->arr2 = BZALLOC( (n+BZ_N_OVERSHOOT) * sizeof(UInt32) ); + s->ftab = BZALLOC( 65537 * sizeof(UInt32) ); + + if (s->arr1 == NULL || s->arr2 == NULL || s->ftab == NULL) { + if (s->arr1 != NULL) BZFREE(s->arr1); + if (s->arr2 != NULL) BZFREE(s->arr2); + if (s->ftab != NULL) BZFREE(s->ftab); + if (s != NULL) BZFREE(s); return BZ_MEM_ERROR; } - s->szptr = (UInt16*)(s->zptr); - s->blockNo = 0; s->state = BZ_S_INPUT; s->mode = BZ_M_RUNNING; @@ -202,7 +196,12 @@ int BZ_API(bzCompressInit) s->nblockMAX = 100000 * blockSize100k - 19; s->verbosity = verbosity; s->workFactor = workFactor; - s->nBlocksRandomised = 0; + + s->block = (UInt16*)s->arr2; + s->mtfv = (UInt16*)s->arr1; + s->zbits = NULL; + s->ptr = (UInt32*)s->arr1; + strm->state = s; strm->total_in = 0; strm->total_out = 0; @@ -224,24 +223,24 @@ void add_pair_to_block ( EState* s ) s->inUse[s->state_in_ch] = True; switch (s->state_in_len) { case 1: - s->block[s->nblock] = (UChar)ch; s->nblock++; + s->block[s->nblock] = (UInt16)ch; s->nblock++; break; case 2: - s->block[s->nblock] = (UChar)ch; s->nblock++; - s->block[s->nblock] = (UChar)ch; s->nblock++; + s->block[s->nblock] = (UInt16)ch; s->nblock++; + s->block[s->nblock] = (UInt16)ch; s->nblock++; break; case 3: - s->block[s->nblock] = (UChar)ch; s->nblock++; - s->block[s->nblock] = (UChar)ch; s->nblock++; - s->block[s->nblock] = (UChar)ch; s->nblock++; + s->block[s->nblock] = (UInt16)ch; s->nblock++; + s->block[s->nblock] = (UInt16)ch; s->nblock++; + s->block[s->nblock] = (UInt16)ch; s->nblock++; break; default: s->inUse[s->state_in_len-4] = True; - s->block[s->nblock] = (UChar)ch; s->nblock++; - s->block[s->nblock] = (UChar)ch; s->nblock++; - s->block[s->nblock] = (UChar)ch; s->nblock++; - s->block[s->nblock] = (UChar)ch; s->nblock++; - s->block[s->nblock] = (UChar)(s->state_in_len-4); + s->block[s->nblock] = (UInt16)ch; s->nblock++; + s->block[s->nblock] = (UInt16)ch; s->nblock++; + s->block[s->nblock] = (UInt16)ch; s->nblock++; + s->block[s->nblock] = (UInt16)ch; s->nblock++; + s->block[s->nblock] = ((UInt16)(s->state_in_len-4)); s->nblock++; break; } @@ -267,7 +266,7 @@ void flush_RL ( EState* s ) UChar ch = (UChar)(zs->state_in_ch); \ BZ_UPDATE_CRC( zs->blockCRC, ch ); \ zs->inUse[zs->state_in_ch] = True; \ - zs->block[zs->nblock] = (UChar)ch; \ + zs->block[zs->nblock] = (UInt16)ch; \ zs->nblock++; \ zs->state_in_ch = zchh; \ } \ @@ -343,7 +342,7 @@ Bool copy_output_until_stop ( EState* s ) if (s->state_out_pos >= s->numZ) break; progress_out = True; - *(s->strm->next_out) = ((UChar*)(s->quadrant))[s->state_out_pos]; + *(s->strm->next_out) = s->zbits[s->state_out_pos]; s->state_out_pos++; s->strm->avail_out--; s->strm->next_out++; @@ -382,7 +381,7 @@ Bool handle_compress ( bz_stream* strm ) progress_in |= copy_input_until_stop ( s ); if (s->mode != BZ_M_RUNNING && s->avail_in_expect == 0) { flush_RL ( s ); - compressBlock ( s, s->mode == BZ_M_FINISHING ); + compressBlock ( s, (Bool)(s->mode == BZ_M_FINISHING) ); s->state = BZ_S_OUTPUT; } else @@ -470,10 +469,9 @@ int BZ_API(bzCompressEnd) ( bz_stream *strm ) if (s == NULL) return BZ_PARAM_ERROR; if (s->strm != strm) return BZ_PARAM_ERROR; - if (s->block != NULL) BZFREE(s->block); - if (s->quadrant != NULL) BZFREE(s->quadrant); - if (s->zptr != NULL) BZFREE(s->zptr); - if (s->ftab != NULL) BZFREE(s->ftab); + if (s->arr1 != NULL) BZFREE(s->arr1); + if (s->arr2 != NULL) BZFREE(s->arr2); + if (s->ftab != NULL) BZFREE(s->ftab); BZFREE(strm->state); strm->state = NULL; @@ -816,7 +814,8 @@ int BZ_API(bzDecompress) ( bz_stream *strm ) } AssertH ( 0, 6001 ); - /*notreached*/ + + return 0; /*NOTREACHED*/ } @@ -1284,7 +1283,7 @@ int BZ_API(bzBuffToBuffDecompress) errhandler: bzDecompressEnd ( &strm ); - return BZ_SEQUENCE_ERROR; + return ret; } @@ -1339,24 +1338,18 @@ BZFILE * bzopen_or_bzdopen int smallMode = 0; int nUnused = 0; - if(mode==NULL){return NULL;} - while(*mode){ - switch(*mode){ + if (mode == NULL) return NULL; + while (*mode) { + switch (*mode) { case 'r': - writing = 0;break; + writing = 0; break; case 'w': - writing = 1;break; + writing = 1; break; case 's': - smallMode = 1;break; + smallMode = 1; break; default: - if(isdigit(*mode)){ - blockSize100k = 0; - while(isdigit(*mode)){ - blockSize100k = blockSize100k*10 + *mode-'0'; - mode++; - } - }else{ - /* ignore */ + if (isdigit((int)(*mode))) { + blockSize100k = *mode-'0'; } } mode++; @@ -1364,29 +1357,32 @@ BZFILE * bzopen_or_bzdopen strcat(mode2, writing ? "w" : "r" ); strcat(mode2,"b"); /* binary mode */ - if(open_mode==0){ - if(path==NULL || strcmp(path,"")==0){ + if (open_mode==0) { + if (path==NULL || strcmp(path,"")==0) { fp = (writing ? stdout : stdin); SET_BINARY_MODE(fp); - }else{ + } else { fp = fopen(path,mode2); } - }else{ + } else { #ifdef BZ_STRICT_ANSI fp = NULL; #else fp = fdopen(fd,mode2); #endif } - if(fp==NULL){return NULL;} + if (fp == NULL) return NULL; - if(writing){ + if (writing) { + /* Guard against total chaos and anarchy -- JRS */ + if (blockSize100k < 1) blockSize100k = 1; + if (blockSize100k > 9) blockSize100k = 9; bzfp = bzWriteOpen(&bzerr,fp,blockSize100k,verbosity,workFactor); - }else{ + } else { bzfp = bzReadOpen(&bzerr,fp,verbosity,smallMode,unused,nUnused); } - if(bzfp==NULL){ - if(fp!=stdin && fp!=stdout) fclose(fp); + if (bzfp == NULL) { + if (fp != stdin && fp != stdout) fclose(fp); return NULL; } return bzfp; @@ -1458,7 +1454,7 @@ void BZ_API(bzclose) (BZFILE* b) int bzerr; FILE *fp = ((bzFile *)b)->handle; - if(b==NULL){return;} + if (b==NULL) {return;} if(((bzFile*)b)->writing){ bzWriteClose(&bzerr,b,0,NULL,NULL); if(bzerr != BZ_OK){ diff --git a/bzlib.h b/bzlib.h index bb62273..d74938d 100644 --- a/bzlib.h +++ b/bzlib.h @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-1998 Julian R Seward. All rights reserved. + Copyright (C) 1996-1999 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -41,9 +41,9 @@ NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - Julian Seward, Guildford, Surrey, UK. + Julian Seward, Cambridge, UK. jseward@acm.org - bzip2/libbzip2 version 0.9.0c of 18 October 1998 + bzip2/libbzip2 version 0.9.5 of 24 May 1999 This program is based on (at least) the work of: Mike Burrows @@ -62,6 +62,10 @@ #ifndef _BZLIB_H #define _BZLIB_H +#ifdef __cplusplus +extern "C" { +#endif + #define BZ_RUN 0 #define BZ_FLUSH 1 #define BZ_FINISH 2 @@ -291,6 +295,9 @@ BZ_EXTERN const char * BZ_API(bzerror) ( ); #endif +#ifdef __cplusplus +} +#endif #endif diff --git a/bzlib_private.h b/bzlib_private.h index 4044aef..8e93480 100644 --- a/bzlib_private.h +++ b/bzlib_private.h @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-1998 Julian R Seward. All rights reserved. + Copyright (C) 1996-1999 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -41,9 +41,9 @@ NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - Julian Seward, Guildford, Surrey, UK. + Julian Seward, Cambridge, UK. jseward@acm.org - bzip2/libbzip2 version 0.9.0c of 18 October 1998 + bzip2/libbzip2 version 0.9.5 of 24 May 1999 This program is based on (at least) the work of: Mike Burrows @@ -76,7 +76,7 @@ /*-- General stuff. --*/ -#define BZ_VERSION "0.9.0c" +#define BZ_VERSION "0.9.5d" typedef char Char; typedef unsigned char Bool; @@ -85,7 +85,7 @@ typedef int Int32; typedef unsigned int UInt32; typedef short Int16; typedef unsigned short UInt16; - + #define True ((Bool)1) #define False ((Bool)0) @@ -210,7 +210,11 @@ extern UInt32 crc32Table[256]; #define BZ_S_OUTPUT 1 #define BZ_S_INPUT 2 -#define BZ_NUM_OVERSHOOT_BYTES 20 +#define BZ_N_RADIX 2 +#define BZ_N_QSORT 12 +#define BZ_N_SHELL 18 +#define BZ_N_OVERSHOOT (BZ_N_RADIX + BZ_N_QSORT + BZ_N_SHELL + 2) + @@ -230,18 +234,20 @@ typedef UInt32 avail_in_expect; /* for doing the block sorting */ - UChar* block; - UInt16* quadrant; - UInt32* zptr; - UInt16* szptr; - Int32* ftab; - Int32 workDone; - Int32 workLimit; - Int32 workFactor; - Bool firstAttempt; - Bool blockRandomised; + UInt32* arr1; + UInt32* arr2; + UInt32* ftab; Int32 origPtr; + /* aliases for arr1 and arr2 */ + UInt32* ptr; + UInt16* block; + UInt16* mtfv; + UChar* zbits; + + /* for deciding when to use the fallback sorting algorithm */ + Int32 workFactor; + /* run-length-encoding of the input */ UInt32 state_in_ch; Int32 state_in_len; @@ -269,7 +275,6 @@ typedef /* misc administratium */ Int32 verbosity; Int32 blockNo; - Int32 nBlocksRandomised; Int32 blockSize100k; /* stuff for coding the MTF values */ @@ -478,17 +483,17 @@ typedef } #define GET_LL4(i) \ - (((UInt32)(s->ll4[(i) >> 1])) >> (((i) << 2) & 0x4) & 0xF) + ((((UInt32)(s->ll4[(i) >> 1])) >> (((i) << 2) & 0x4)) & 0xF) -#define SET_LL(i,n) \ +#define SET_LL(i,n) \ { s->ll16[i] = (UInt16)(n & 0x0000ffff); \ - SET_LL4(i, n >> 16); \ + SET_LL4(i, n >> 16); \ } #define GET_LL(i) \ (((UInt32)s->ll16[i]) | (GET_LL4(i) << 16)) -#define BZ_GET_SMALL(cccc) \ +#define BZ_GET_SMALL(cccc) \ cccc = indexIntoF ( s->tPos, s->cftab ); \ s->tPos = GET_LL(s->tPos); diff --git a/compress.c b/compress.c index 23abd43..7b192c3 100644 --- a/compress.c +++ b/compress.c @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-1998 Julian R Seward. All rights reserved. + Copyright (C) 1996-1999 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -41,9 +41,9 @@ NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - Julian Seward, Guildford, Surrey, UK. + Julian Seward, Cambridge, UK. jseward@acm.org - bzip2/libbzip2 version 0.9.0 of 28 June 1998 + bzip2/libbzip2 version 0.9.5 of 24 May 1999 This program is based on (at least) the work of: Mike Burrows @@ -90,7 +90,7 @@ static void bsFinishWrite ( EState* s ) { while (s->bsLive > 0) { - ((UChar*)(s->quadrant))[s->numZ] = (UChar)(s->bsBuff >> 24); + s->zbits[s->numZ] = (UChar)(s->bsBuff >> 24); s->numZ++; s->bsBuff <<= 8; s->bsLive -= 8; @@ -102,7 +102,7 @@ void bsFinishWrite ( EState* s ) #define bsNEEDW(nz) \ { \ while (s->bsLive >= 8) { \ - ((UChar*)(s->quadrant))[s->numZ] \ + s->zbits[s->numZ] \ = (UChar)(s->bsBuff >> 24); \ s->numZ++; \ s->bsBuff <<= 8; \ @@ -162,13 +162,39 @@ void makeMaps_e ( EState* s ) static void generateMTFValues ( EState* s ) { - UChar yy[256]; - Int32 i, j; - UChar tmp; - UChar tmp2; - Int32 zPend; - Int32 wr; - Int32 EOB; + UChar yy[256]; + Int32 i, j; + UChar tmp; + UChar tmp2; + Int32 zPend; + Int32 wr; + Int32 EOB; + + /* + After sorting (eg, here), + s->arr1 [ 0 .. s->nblock-1 ] holds sorted order, + and + ((UInt16*)s->arr2) [ 0 .. s->nblock-1 ] [15:8] + holds the original block data. + + The first thing to do is generate the MTF values, + and put them in + ((UInt16*)s->arr1) [ 0 .. s->nblock-1 ]. + Because there are strictly fewer or equal MTF values + than block values, ptr values in this area are overwritten + with MTF values only when they are no longer needed. + + The final compressed bitstream is generated into the + area starting at + (UChar*) (&((UInt16)s->arr2)[s->nblock]) + + These storage aliases are set up in bzCompressInit(), + except for the last one, which is arranged in + compressBlock(). + */ + UInt32* ptr = s->ptr; + UInt16* block = s->block; + UInt16* mtfv = s->mtfv; makeMaps_e ( s ); EOB = s->nInUse+1; @@ -183,52 +209,61 @@ void generateMTFValues ( EState* s ) UChar ll_i; AssertD ( wr <= i, "generateMTFValues(1)" ); - j = s->zptr[i]-1; if (j < 0) j += s->nblock; - ll_i = s->unseqToSeq[s->block[j]]; + j = ptr[i]-1; if (j < 0) j += s->nblock; + ll_i = s->unseqToSeq[block[j] >> 8]; AssertD ( ll_i < s->nInUse, "generateMTFValues(2a)" ); - j = 0; - tmp = yy[j]; - while ( ll_i != tmp ) { - j++; - tmp2 = tmp; - tmp = yy[j]; - yy[j] = tmp2; - }; - yy[0] = tmp; - - if (j == 0) { + tmp = yy[0]; + if (tmp == ll_i) { zPend++; } else { + tmp2 = tmp; + tmp = yy[1]; + yy[1] = tmp2; + j = 1; + while ( ll_i != tmp ) { + j++; + tmp2 = tmp; + tmp = yy[j]; + yy[j] = tmp2; + }; + yy[0] = tmp; + if (zPend > 0) { zPend--; while (True) { - switch (zPend % 2) { - case 0: s->szptr[wr] = BZ_RUNA; wr++; s->mtfFreq[BZ_RUNA]++; break; - case 1: s->szptr[wr] = BZ_RUNB; wr++; s->mtfFreq[BZ_RUNB]++; break; - }; + if (zPend & 1) { + mtfv[wr] = BZ_RUNB; wr++; + s->mtfFreq[BZ_RUNB]++; + } else { + mtfv[wr] = BZ_RUNA; wr++; + s->mtfFreq[BZ_RUNA]++; + } if (zPend < 2) break; zPend = (zPend - 2) / 2; }; zPend = 0; } - s->szptr[wr] = j+1; wr++; s->mtfFreq[j+1]++; + mtfv[wr] = j+1; wr++; s->mtfFreq[j+1]++; } } if (zPend > 0) { zPend--; while (True) { - switch (zPend % 2) { - case 0: s->szptr[wr] = BZ_RUNA; wr++; s->mtfFreq[BZ_RUNA]++; break; - case 1: s->szptr[wr] = BZ_RUNB; wr++; s->mtfFreq[BZ_RUNB]++; break; - }; + if (zPend & 1) { + mtfv[wr] = BZ_RUNB; wr++; + s->mtfFreq[BZ_RUNB]++; + } else { + mtfv[wr] = BZ_RUNA; wr++; + s->mtfFreq[BZ_RUNA]++; + } if (zPend < 2) break; zPend = (zPend - 2) / 2; }; } - s->szptr[wr] = EOB; wr++; s->mtfFreq[EOB]++; + mtfv[wr] = EOB; wr++; s->mtfFreq[EOB]++; s->nMTF = wr; } @@ -259,6 +294,8 @@ void sendMTFValues ( EState* s ) UInt16 cost[BZ_N_GROUPS]; Int32 fave[BZ_N_GROUPS]; + UInt16* mtfv = s->mtfv; + if (s->verbosity >= 3) VPrintf3( " %d in block, %d after MTF & 1-2 coding, " "%d+2 syms in use\n", @@ -348,7 +385,7 @@ void sendMTFValues ( EState* s ) register UInt16 cost0, cost1, cost2, cost3, cost4, cost5; cost0 = cost1 = cost2 = cost3 = cost4 = cost5 = 0; for (i = gs; i <= ge; i++) { - UInt16 icv = s->szptr[i]; + UInt16 icv = mtfv[i]; cost0 += s->len[0][icv]; cost1 += s->len[1][icv]; cost2 += s->len[2][icv]; @@ -360,7 +397,7 @@ void sendMTFValues ( EState* s ) cost[3] = cost3; cost[4] = cost4; cost[5] = cost5; } else { for (i = gs; i <= ge; i++) { - UInt16 icv = s->szptr[i]; + UInt16 icv = mtfv[i]; for (t = 0; t < nGroups; t++) cost[t] += s->len[t][icv]; } } @@ -381,7 +418,7 @@ void sendMTFValues ( EState* s ) Increment the symbol frequencies for the selected table. --*/ for (i = gs; i <= ge; i++) - s->rfreq[bt][ s->szptr[i] ]++; + s->rfreq[bt][ mtfv[i] ]++; gs = ge+1; } @@ -502,8 +539,8 @@ void sendMTFValues ( EState* s ) for (i = gs; i <= ge; i++) { AssertH ( s->selector[selCtr] < nGroups, 3006 ); bsW ( s, - s->len [s->selector[selCtr]] [s->szptr[i]], - s->code [s->selector[selCtr]] [s->szptr[i]] ); + s->len [s->selector[selCtr]] [mtfv[i]], + s->code [s->selector[selCtr]] [mtfv[i]] ); } gs = ge+1; @@ -534,13 +571,15 @@ void compressBlock ( EState* s, Bool is_last_block ) blockSort ( s ); } + s->zbits = (UChar*) (&((UInt16*)s->arr2)[s->nblock]); + /*-- If this is the first block, create the stream header. --*/ if (s->blockNo == 1) { bsInitWrite ( s ); bsPutUChar ( s, 'B' ); bsPutUChar ( s, 'Z' ); bsPutUChar ( s, 'h' ); - bsPutUChar ( s, '0' + s->blockSize100k ); + bsPutUChar ( s, (UChar)('0' + s->blockSize100k) ); } if (s->nblock > 0) { @@ -552,11 +591,16 @@ void compressBlock ( EState* s, Bool is_last_block ) /*-- Now the block's CRC, so it is in a known place. --*/ bsPutUInt32 ( s, s->blockCRC ); - /*-- Now a single bit indicating randomisation. --*/ - if (s->blockRandomised) { - bsW(s,1,1); s->nBlocksRandomised++; - } else - bsW(s,1,0); + /*-- + Now a single bit indicating (non-)randomisation. + As of version 0.9.5, we use a better sorting algorithm + which makes randomisation unnecessary. So always set + the randomised bit to 'no'. Of course, the decoder + still needs to be able to handle randomised blocks + so as to maintain backwards compatibility with + older versions of bzip2. + --*/ + bsW(s,1,0); bsW ( s, 24, s->origPtr ); generateMTFValues ( s ); @@ -567,11 +611,6 @@ void compressBlock ( EState* s, Bool is_last_block ) /*-- If this is the last block, add the stream trailer. --*/ if (is_last_block) { - if (s->verbosity >= 2 && s->nBlocksRandomised > 0) - VPrintf2 ( " %d block%s needed randomisation\n", - s->nBlocksRandomised, - s->nBlocksRandomised == 1 ? "" : "s" ); - bsPutUChar ( s, 0x17 ); bsPutUChar ( s, 0x72 ); bsPutUChar ( s, 0x45 ); bsPutUChar ( s, 0x38 ); bsPutUChar ( s, 0x50 ); bsPutUChar ( s, 0x90 ); diff --git a/crctable.c b/crctable.c index 2f3eacb..ab53df6 100644 --- a/crctable.c +++ b/crctable.c @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-1998 Julian R Seward. All rights reserved. + Copyright (C) 1996-1999 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -41,9 +41,9 @@ NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - Julian Seward, Guildford, Surrey, UK. + Julian Seward, Cambridge, UK. jseward@acm.org - bzip2/libbzip2 version 0.9.0c of 18 October 1998 + bzip2/libbzip2 version 0.9.5 of 24 May 1999 This program is based on (at least) the work of: Mike Burrows diff --git a/decompress.c b/decompress.c index ac2b0a5..31f8b67 100644 --- a/decompress.c +++ b/decompress.c @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-1998 Julian R Seward. All rights reserved. + Copyright (C) 1996-1999 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -41,9 +41,9 @@ NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - Julian Seward, Guildford, Surrey, UK. + Julian Seward, Cambridge, UK. jseward@acm.org - bzip2/libbzip2 version 0.9.0c of 18 October 1998 + bzip2/libbzip2 version 0.9.5 of 24 May 1999 This program is based on (at least) the work of: Mike Burrows @@ -141,7 +141,7 @@ Int32 decompress ( DState* s ) bz_stream* strm = s->strm; /* stuff that needs to be saved/restored */ - Int32 i ; + Int32 i; Int32 j; Int32 t; Int32 alphaSize; diff --git a/dlltest.c b/dlltest.c index ee81bcd..e5639f1 100644 --- a/dlltest.c +++ b/dlltest.c @@ -1,17 +1,17 @@ /* - minibz2 - libbz2.dll test program. - by Yoshioka Tsuneo(QWF00133@nifty.ne.jp/tsuneo-y@is.aist-nara.ac.jp) - This file is Public Domain. - welcome any email to me. + minibz2 + libbz2.dll test program. + by Yoshioka Tsuneo(QWF00133@nifty.ne.jp/tsuneo-y@is.aist-nara.ac.jp) + This file is Public Domain. + welcome any email to me. - usage: minibz2 [-d] [-{1,2,..9}] [[srcfilename] destfilename] + usage: minibz2 [-d] [-{1,2,..9}] [[srcfilename] destfilename] */ #define BZ_IMPORT -#include "bzlib.h" #include #include +#include "bzlib.h" #ifdef _WIN32 #include #endif @@ -24,140 +24,142 @@ static int BZ2DLLLoaded = 0; static HINSTANCE BZ2DLLhLib; int BZ2DLLLoadLibrary(void) { - HINSTANCE hLib; + HINSTANCE hLib; - if(BZ2DLLLoaded==1){return 0;} - hLib=LoadLibrary("libbz2.dll"); - if(hLib == NULL){ - puts("Can't load libbz2.dll"); - return -1; - } - BZ2DLLLoaded=1; - BZ2DLLhLib=hLib; - bzlibVersion=GetProcAddress(hLib,"bzlibVersion"); - bzopen=GetProcAddress(hLib,"bzopen"); - bzdopen=GetProcAddress(hLib,"bzdopen"); - bzread=GetProcAddress(hLib,"bzread"); - bzwrite=GetProcAddress(hLib,"bzwrite"); - bzflush=GetProcAddress(hLib,"bzflush"); - bzclose=GetProcAddress(hLib,"bzclose"); - bzerror=GetProcAddress(hLib,"bzerror"); - return 0; + if(BZ2DLLLoaded==1){return 0;} + hLib=LoadLibrary("libbz2.dll"); + if(hLib == NULL){ + puts("Can't load libbz2.dll"); + return -1; + } + BZ2DLLLoaded=1; + BZ2DLLhLib=hLib; + bzlibVersion=GetProcAddress(hLib,"bzlibVersion"); + bzopen=GetProcAddress(hLib,"bzopen"); + bzdopen=GetProcAddress(hLib,"bzdopen"); + bzread=GetProcAddress(hLib,"bzread"); + bzwrite=GetProcAddress(hLib,"bzwrite"); + bzflush=GetProcAddress(hLib,"bzflush"); + bzclose=GetProcAddress(hLib,"bzclose"); + bzerror=GetProcAddress(hLib,"bzerror"); + return 0; } int BZ2DLLFreeLibrary(void) { - if(BZ2DLLLoaded==0){return 0;} - FreeLibrary(BZ2DLLhLib); - BZ2DLLLoaded=0; + if(BZ2DLLLoaded==0){return 0;} + FreeLibrary(BZ2DLLhLib); + BZ2DLLLoaded=0; } #endif /* WIN32 */ void usage(void) { - puts("usage: minibz2 [-d] [-{1,2,..9}] [[srcfilename] destfilename]"); + puts("usage: minibz2 [-d] [-{1,2,..9}] [[srcfilename] destfilename]"); } -void main(int argc,char *argv[]) +int main(int argc,char *argv[]) { - int decompress = 0; - int level = 9; - char *fn_r,*fn_w; + int decompress = 0; + int level = 9; + char *fn_r = NULL; + char *fn_w = NULL; #ifdef _WIN32 - if(BZ2DLLLoadLibrary()<0){ - puts("can't load dll"); - exit(1); - } + if(BZ2DLLLoadLibrary()<0){ + puts("can't load dll"); + exit(1); + } #endif - while(++argv,--argc){ - if(**argv =='-' || **argv=='/'){ - char *p; + while(++argv,--argc){ + if(**argv =='-' || **argv=='/'){ + char *p; - for(p=*argv+1;*p;p++){ - if(*p=='d'){ - decompress = 1; - }else if('1'<=*p && *p<='9'){ - level = *p - '0'; - }else{ - usage(); - exit(1); - } - } - }else{ - break; - } - } - if(argc>=1){ - fn_r = *argv; - argc--;argv++; - }else{ - fn_r = NULL; - } - if(argc>=1){ - fn_w = *argv; - argc--;argv++; - }else{ - fn_w = NULL; - } - { - int len; - char buff[0x1000]; - char mode[10]; + for(p=*argv+1;*p;p++){ + if(*p=='d'){ + decompress = 1; + }else if('1'<=*p && *p<='9'){ + level = *p - '0'; + }else{ + usage(); + exit(1); + } + } + }else{ + break; + } + } + if(argc>=1){ + fn_r = *argv; + argc--;argv++; + }else{ + fn_r = NULL; + } + if(argc>=1){ + fn_w = *argv; + argc--;argv++; + }else{ + fn_w = NULL; + } + { + int len; + char buff[0x1000]; + char mode[10]; - if(decompress){ - BZFILE *BZ2fp_r; - FILE *fp_w; + if(decompress){ + BZFILE *BZ2fp_r = NULL; + FILE *fp_w = NULL; - if(fn_w){ - if((fp_w = fopen(fn_w,"wb"))==NULL){ - printf("can't open [%s]\n",fn_w); - perror("reason:"); - exit(1); - } - }else{ - fp_w = stdout; - } - if((BZ2fp_r == NULL && (BZ2fp_r = bzdopen(fileno(stdin),"rb"))==NULL) - || (BZ2fp_r != NULL && (BZ2fp_r = bzopen(fn_r,"rb"))==NULL)){ - printf("can't bz2openstream\n"); - exit(1); - } - while((len=bzread(BZ2fp_r,buff,0x1000))>0){ - fwrite(buff,1,len,fp_w); - } - bzclose(BZ2fp_r); - if(fp_w != stdout) fclose(fp_w); - }else{ - BZFILE *BZ2fp_w; - FILE *fp_r; + if(fn_w){ + if((fp_w = fopen(fn_w,"wb"))==NULL){ + printf("can't open [%s]\n",fn_w); + perror("reason:"); + exit(1); + } + }else{ + fp_w = stdout; + } + if((BZ2fp_r == NULL && (BZ2fp_r = bzdopen(fileno(stdin),"rb"))==NULL) + || (BZ2fp_r != NULL && (BZ2fp_r = bzopen(fn_r,"rb"))==NULL)){ + printf("can't bz2openstream\n"); + exit(1); + } + while((len=bzread(BZ2fp_r,buff,0x1000))>0){ + fwrite(buff,1,len,fp_w); + } + bzclose(BZ2fp_r); + if(fp_w != stdout) fclose(fp_w); + }else{ + BZFILE *BZ2fp_w = NULL; + FILE *fp_r = NULL; - if(fn_r){ - if((fp_r = fopen(fn_r,"rb"))==NULL){ - printf("can't open [%s]\n",fn_r); - perror("reason:"); - exit(1); - } - }else{ - fp_r = stdin; - } - mode[0]='w'; - mode[1] = '0' + level; - mode[2] = '\0'; + if(fn_r){ + if((fp_r = fopen(fn_r,"rb"))==NULL){ + printf("can't open [%s]\n",fn_r); + perror("reason:"); + exit(1); + } + }else{ + fp_r = stdin; + } + mode[0]='w'; + mode[1] = '0' + level; + mode[2] = '\0'; - if((fn_w == NULL && (BZ2fp_w = bzdopen(fileno(stdout),mode))==NULL) - || (fn_w !=NULL && (BZ2fp_w = bzopen(fn_w,mode))==NULL)){ - printf("can't bz2openstream\n"); - exit(1); - } - while((len=fread(buff,1,0x1000,fp_r))>0){ - bzwrite(BZ2fp_w,buff,len); - } - bzclose(BZ2fp_w); - if(fp_r!=stdin)fclose(fp_r); - } - } + if((fn_w == NULL && (BZ2fp_w = bzdopen(fileno(stdout),mode))==NULL) + || (fn_w !=NULL && (BZ2fp_w = bzopen(fn_w,mode))==NULL)){ + printf("can't bz2openstream\n"); + exit(1); + } + while((len=fread(buff,1,0x1000,fp_r))>0){ + bzwrite(BZ2fp_w,buff,len); + } + bzclose(BZ2fp_w); + if(fp_r!=stdin)fclose(fp_r); + } + } #ifdef _WIN32 - BZ2DLLFreeLibrary(); + BZ2DLLFreeLibrary(); #endif + return 0; } diff --git a/howbig.c b/howbig.c deleted file mode 100644 index 9f2ad7c..0000000 --- a/howbig.c +++ /dev/null @@ -1,37 +0,0 @@ - -#include -#include -#include "bzlib.h" - -unsigned char ibuff[1000000]; -unsigned char obuff[1000000]; - -void doone ( int n ) -{ - int i, j, k, q, nobuff; - q = 0; - - for (k = 0; k < 1; k++) { - for (i = 0; i < n; i++) - ibuff[i] = ((unsigned long)(random())) & 0xff; - nobuff = 1000000; - j = bzBuffToBuffCompress ( obuff, &nobuff, ibuff, n, 9,0,0 ); - assert (j == BZ_OK); - if (nobuff > q) q = nobuff; - } - printf ( "%d %d(%d)\n", n, q, (int)((float)n * 1.01 - (float)q) ); -} - -int main ( int argc, char** argv ) -{ - int i; - i = 0; - while (1) { - if (i >= 900000) break; - doone(i); - if ( (int)(1.10 * i) > i ) - i = (int)(1.10 * i); else i++; - } - - return 0; -} \ No newline at end of file diff --git a/huffman.c b/huffman.c index 8254990..bb2a6cc 100644 --- a/huffman.c +++ b/huffman.c @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-1998 Julian R Seward. All rights reserved. + Copyright (C) 1996-1999 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -41,9 +41,9 @@ NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - Julian Seward, Guildford, Surrey, UK. + Julian Seward, Cambridge, UK. jseward@acm.org - bzip2/libbzip2 version 0.9.0c of 18 October 1998 + bzip2/libbzip2 version 0.9.5 of 24 May 1999 This program is based on (at least) the work of: Mike Burrows diff --git a/makefile.msc b/makefile.msc new file mode 100644 index 0000000..4b49f78 --- /dev/null +++ b/makefile.msc @@ -0,0 +1,65 @@ +# Makefile for Microsoft Visual C++ 6.0 +# usage: nmake -f makefile.msc +# K.M. Syring (syring@gsf.de) +# Fixed up by JRS for bzip2-0.9.5d release. + +CC=cl +CFLAGS= -DWIN32 -MD -Ox + +OBJS= blocksort.obj \ + huffman.obj \ + crctable.obj \ + randtable.obj \ + compress.obj \ + decompress.obj \ + bzlib.obj + +all: lib bzip2 test + +bzip2: lib + $(CC) $(CFLAGS) -o bzip2 bzip2.c libbz2.lib setargv.obj + $(CC) $(CFLAGS) -o bzip2recover bzip2recover.c + +lib: $(OBJS) + del libbz2.lib + lib /out:libbz2.lib $(OBJS) + +test: bzip2 + type words1 + .\\bzip2 -1 < sample1.ref > sample1.rb2 + .\\bzip2 -2 < sample2.ref > sample2.rb2 + .\\bzip2 -3 < sample3.ref > sample3.rb2 + .\\bzip2 -d < sample1.bz2 > sample1.tst + .\\bzip2 -d < sample2.bz2 > sample2.tst + .\\bzip2 -ds < sample3.bz2 > sample3.tst + fc sample1.bz2 sample1.rb2 + fc sample2.bz2 sample2.rb2 + fc sample3.bz2 sample3.rb2 + fc sample1.tst sample1.ref + fc sample2.tst sample2.ref + fc sample3.tst sample3.ref + @echo All six of the fc's should find no differences. + @echo If fc finds an error on sample3.tst, this could be + @echo because WinZips 'TAR file smart CR/LF conversion' + @echo is too clever for its own good. Disable this option. + @echo The correct size for sample3.ref is 120,244. If it + @echo is around 150k, WinZip has stuffed it up. + @echo Also remember to set BZ_UNIX to 0 and BZ_LCCWIN32 + @echo to 1 in bzip2.c. + + +clean: + del *.obj + del libbz2.lib + del bzip2.exe + del bzip2recover.exe + del sample1.rb2 + del sample2.rb2 + del sample3.rb2 + del sample1.tst + del sample2.tst + del sample3.tst + +.c.obj: + $(CC) $(CFLAGS) -c $*.c -o $*.obj + diff --git a/manual.texi b/manual.texi index 99ce661..e48e656 100644 --- a/manual.texi +++ b/manual.texi @@ -2,10 +2,10 @@ @setfilename bzip2.info @ignore -This file documents bzip2 version 0.9.0c, and associated library +This file documents bzip2 version 0.9.5, and associated library libbzip2, written by Julian Seward (jseward@acm.org). -Copyright (C) 1996-1998 Julian R Seward +Copyright (C) 1996-1999 Julian R Seward Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice @@ -30,21 +30,21 @@ END-INFO-DIR-ENTRY @titlepage @title bzip2 and libbzip2 @subtitle a program and library for data compression -@subtitle copyright (C) 1996-1998 Julian Seward -@subtitle version 0.9.0c of 18 October 1998 +@subtitle copyright (C) 1996-1999 Julian Seward +@subtitle version 0.9.5d of 4 September 1999 @author Julian Seward @end titlepage -@end iftex - @parindent 0mm @parskip 2mm +@end iftex +@node Top, Overview, (dir), (dir) This program, @code{bzip2}, and associated library @code{libbzip2}, are -Copyright (C) 1996-1998 Julian R Seward. All rights reserved. +Copyright (C) 1996-1999 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -78,13 +78,13 @@ WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -Julian Seward, Guildford, Surrey, UK. +Julian Seward, Cambridge, UK. @code{jseward@@acm.org} @code{http://www.muraroa.demon.co.uk} -@code{bzip2}/@code{libbzip2} version 0.9.0c of 18 October 1998. +@code{bzip2}/@code{libbzip2} version 0.9.5 of 24 May 1999. PATENTS: To the best of my knowledge, @code{bzip2} does not use any patented algorithms. However, I do not have the resources available to carry out @@ -124,360 +124,345 @@ ought to be recorded somewhere. This chapter contains a copy of the @code{bzip2} man page, and nothing else. + +@quotation + +@unnumberedsubsubsec NAME +@itemize +@item @code{bzip2}, @code{bunzip2} +- a block-sorting file compressor, v0.9.5 +@item @code{bzcat} +- decompresses files to stdout +@item @code{bzip2recover} +- recovers data from damaged bzip2 files +@end itemize + +@unnumberedsubsubsec SYNOPSIS +@itemize +@item @code{bzip2} [ -cdfkqstvzVL123456789 ] [ filenames ... ] +@item @code{bunzip2} [ -fkvsVL ] [ filenames ... ] +@item @code{bzcat} [ -s ] [ filenames ... ] +@item @code{bzip2recover} filename +@end itemize + +@unnumberedsubsubsec DESCRIPTION + +@code{bzip2} compresses files using the Burrows-Wheeler block sorting +text compression algorithm, and Huffman coding. Compression is +generally considerably better than that achieved by more conventional +LZ77/LZ78-based compressors, and approaches the performance of the PPM +family of statistical compressors. + +The command-line options are deliberately very similar to those of GNU +@code{gzip}, but they are not identical. + +@code{bzip2} expects a list of file names to accompany the command-line +flags. Each file is replaced by a compressed version of itself, with +the name @code{original_name.bz2}. Each compressed file has the same +modification date, permissions, and, when possible, ownership as the +corresponding original, so that these properties can be correctly +restored at decompression time. File name handling is naive in the +sense that there is no mechanism for preserving original file names, +permissions, ownerships or dates in filesystems which lack these +concepts, or have serious file name length restrictions, such as MS-DOS. + +@code{bzip2} and @code{bunzip2} will by default not overwrite existing +files. If you want this to happen, specify the @code{-f} flag. + +If no file names are specified, @code{bzip2} compresses from standard +input to standard output. In this case, @code{bzip2} will decline to +write compressed output to a terminal, as this would be entirely +incomprehensible and therefore pointless. + +@code{bunzip2} (or @code{bzip2 -d}) decompresses all +specified files. Files which were not created by @code{bzip2} +will be detected and ignored, and a warning issued. +@code{bzip2} attempts to guess the filename for the decompressed file +from that of the compressed file as follows: +@itemize +@item @code{filename.bz2 } becomes @code{filename} +@item @code{filename.bz } becomes @code{filename} +@item @code{filename.tbz2} becomes @code{filename.tar} +@item @code{filename.tbz } becomes @code{filename.tar} +@item @code{anyothername } becomes @code{anyothername.out} +@end itemize +If the file does not end in one of the recognised endings, +@code{.bz2}, @code{.bz}, +@code{.tbz2} or @code{.tbz}, @code{bzip2} complains that it cannot +guess the name of the original file, and uses the original name +with @code{.out} appended. + +As with compression, supplying no +filenames causes decompression from standard input to standard output. + +@code{bunzip2} will correctly decompress a file which is the +concatenation of two or more compressed files. The result is the +concatenation of the corresponding uncompressed files. Integrity +testing (@code{-t}) of concatenated compressed files is also supported. + +You can also compress or decompress files to the standard output by +giving the @code{-c} flag. Multiple files may be compressed and +decompressed like this. The resulting outputs are fed sequentially to +stdout. Compression of multiple files in this manner generates a stream +containing multiple compressed file representations. Such a stream +can be decompressed correctly only by @code{bzip2} version 0.9.0 or +later. Earlier versions of @code{bzip2} will stop after decompressing +the first file in the stream. + +@code{bzcat} (or @code{bzip2 -dc}) decompresses all specified files to +the standard output. + +@code{bzip2} will read arguments from the environment variables +@code{BZIP2} and @code{BZIP}, in that order, and will process them +before any arguments read from the command line. This gives a +convenient way to supply default arguments. + +Compression is always performed, even if the compressed file is slightly +larger than the original. Files of less than about one hundred bytes +tend to get larger, since the compression mechanism has a constant +overhead in the region of 50 bytes. Random data (including the output +of most file compressors) is coded at about 8.05 bits per byte, giving +an expansion of around 0.5%. + +As a self-check for your protection, @code{bzip2} uses 32-bit CRCs to +make sure that the decompressed version of a file is identical to the +original. This guards against corruption of the compressed data, and +against undetected bugs in @code{bzip2} (hopefully very unlikely). The +chances of data corruption going undetected is microscopic, about one +chance in four billion for each file processed. Be aware, though, that +the check occurs upon decompression, so it can only tell you that +something is wrong. It can't help you recover the original uncompressed +data. You can use @code{bzip2recover} to try to recover data from +damaged files. + +Return values: 0 for a normal exit, 1 for environmental problems (file +not found, invalid flags, I/O errors, &c), 2 to indicate a corrupt +compressed file, 3 for an internal consistency error (eg, bug) which +caused @code{bzip2} to panic. + + +@unnumberedsubsubsec OPTIONS +@table @code +@item -c --stdout +Compress or decompress to standard output. +@item -d --decompress +Force decompression. @code{bzip2}, @code{bunzip2} and @code{bzcat} are +really the same program, and the decision about what actions to take is +done on the basis of which name is used. This flag overrides that +mechanism, and forces bzip2 to decompress. +@item -z --compress +The complement to @code{-d}: forces compression, regardless of the +invokation name. +@item -t --test +Check integrity of the specified file(s), but don't decompress them. +This really performs a trial decompression and throws away the result. +@item -f --force +Force overwrite of output files. Normally, @code{bzip2} will not overwrite +existing output files. Also forces @code{bzip2} to break hard links +to files, which it otherwise wouldn't do. +@item -k --keep +Keep (don't delete) input files during compression +or decompression. +@item -s --small +Reduce memory usage, for compression, decompression and testing. Files +are decompressed and tested using a modified algorithm which only +requires 2.5 bytes per block byte. This means any file can be +decompressed in 2300k of memory, albeit at about half the normal speed. + +During compression, @code{-s} selects a block size of 200k, which limits +memory use to around the same figure, at the expense of your compression +ratio. In short, if your machine is low on memory (8 megabytes or +less), use -s for everything. See MEMORY MANAGEMENT below. +@item -q --quiet +Suppress non-essential warning messages. Messages pertaining to +I/O errors and other critical events will not be suppressed. +@item -v --verbose +Verbose mode -- show the compression ratio for each file processed. +Further @code{-v}'s increase the verbosity level, spewing out lots of +information which is primarily of interest for diagnostic purposes. +@item -L --license -V --version +Display the software version, license terms and conditions. +@item -1 to -9 +Set the block size to 100 k, 200 k .. 900 k when compressing. Has no +effect when decompressing. See MEMORY MANAGEMENT below. +@item -- +Treats all subsequent arguments as file names, even if they start +with a dash. This is so you can handle files with names beginning +with a dash, for example: @code{bzip2 -- -myfilename}. +@item --repetitive-fast +@item --repetitive-best +These flags are redundant in versions 0.9.5 and above. They provided +some coarse control over the behaviour of the sorting algorithm in +earlier versions, which was sometimes useful. 0.9.5 and above have an +improved algorithm which renders these flags irrelevant. +@end table + + +@unnumberedsubsubsec MEMORY MANAGEMENT + +@code{bzip2} compresses large files in blocks. The block size affects +both the compression ratio achieved, and the amount of memory needed for +compression and decompression. The flags @code{-1} through @code{-9} +specify the block size to be 100,000 bytes through 900,000 bytes (the +default) respectively. At decompression time, the block size used for +compression is read from the header of the compressed file, and +@code{bunzip2} then allocates itself just enough memory to decompress +the file. Since block sizes are stored in compressed files, it follows +that the flags @code{-1} to @code{-9} are irrelevant to and so ignored +during decompression. + +Compression and decompression requirements, in bytes, can be estimated +as: @example -NAME - bzip2, bunzip2 - a block-sorting file compressor, v0.9.0 - bzcat - decompresses files to stdout - bzip2recover - recovers data from damaged bzip2 files - - -SYNOPSIS - bzip2 [ -cdfkstvzVL123456789 ] [ filenames ... ] - bunzip2 [ -fkvsVL ] [ filenames ... ] - bzcat [ -s ] [ filenames ... ] - bzip2recover filename - - -DESCRIPTION - bzip2 compresses files using the Burrows-Wheeler block- - sorting text compression algorithm, and Huffman coding. - Compression is generally considerably better than that - achieved by more conventional LZ77/LZ78-based compressors, - and approaches the performance of the PPM family of sta- - tistical compressors. - - The command-line options are deliberately very similar to - those of GNU Gzip, but they are not identical. - - bzip2 expects a list of file names to accompany the com- - mand-line flags. Each file is replaced by a compressed - version of itself, with the name "original_name.bz2". - Each compressed file has the same modification date and - permissions as the corresponding original, so that these - properties can be correctly restored at decompression - time. File name handling is naive in the sense that there - is no mechanism for preserving original file names, per- - missions and dates in filesystems which lack these con- - cepts, or have serious file name length restrictions, such - as MS-DOS. - - bzip2 and bunzip2 will by default not overwrite existing - files; if you want this to happen, specify the -f flag. - - If no file names are specified, bzip2 compresses from - standard input to standard output. In this case, bzip2 - will decline to write compressed output to a terminal, as - this would be entirely incomprehensible and therefore - pointless. - - bunzip2 (or bzip2 -d ) decompresses and restores all spec- - ified files whose names end in ".bz2". Files without this - suffix are ignored. Again, supplying no filenames causes - decompression from standard input to standard output. - - bunzip2 will correctly decompress a file which is the con- - catenation of two or more compressed files. The result is - the concatenation of the corresponding uncompressed files. - Integrity testing (-t) of concatenated compressed files is - also supported. - - You can also compress or decompress files to the standard - output by giving the -c flag. Multiple files may be com- - pressed and decompressed like this. The resulting outputs - are fed sequentially to stdout. Compression of multiple - files in this manner generates a stream containing multi- - ple compressed file representations. Such a stream can be - decompressed correctly only by bzip2 version 0.9.0 or - later. Earlier versions of bzip2 will stop after decom- - pressing the first file in the stream. - - bzcat (or bzip2 -dc ) decompresses all specified files to - the standard output. - - Compression is always performed, even if the compressed - file is slightly larger than the original. Files of less - than about one hundred bytes tend to get larger, since the - compression mechanism has a constant overhead in the - region of 50 bytes. Random data (including the output of - most file compressors) is coded at about 8.05 bits per - byte, giving an expansion of around 0.5%. - - As a self-check for your protection, bzip2 uses 32-bit - CRCs to make sure that the decompressed version of a file - is identical to the original. This guards against corrup- - tion of the compressed data, and against undetected bugs - in bzip2 (hopefully very unlikely). The chances of data - corruption going undetected is microscopic, about one - chance in four billion for each file processed. Be aware, - though, that the check occurs upon decompression, so it - can only tell you that that something is wrong. It can't - help you recover the original uncompressed data. You can - use bzip2recover to try to recover data from damaged - files. - - Return values: 0 for a normal exit, 1 for environmental - problems (file not found, invalid flags, I/O errors, &c), - 2 to indicate a corrupt compressed file, 3 for an internal - consistency error (eg, bug) which caused bzip2 to panic. - - -MEMORY MANAGEMENT - Bzip2 compresses large files in blocks. The block size - affects both the compression ratio achieved, and the - amount of memory needed both for compression and decom- - pression. The flags -1 through -9 specify the block size - to be 100,000 bytes through 900,000 bytes (the default) - respectively. At decompression-time, the block size used - for compression is read from the header of the compressed - file, and bunzip2 then allocates itself just enough memory - to decompress the file. Since block sizes are stored in - compressed files, it follows that the flags -1 to -9 are - irrelevant to and so ignored during decompression. - - Compression and decompression requirements, in bytes, can - be estimated as: - - Compression: 400k + ( 7 x block size ) - - Decompression: 100k + ( 4 x block size ), or - 100k + ( 2.5 x block size ) - - Larger block sizes give rapidly diminishing marginal - returns; most of the compression comes from the first two - or three hundred k of block size, a fact worth bearing in - mind when using bzip2 on small machines. It is also - important to appreciate that the decompression memory - requirement is set at compression-time by the choice of - block size. + Compression: 400k + ( 8 x block size ) + + Decompression: 100k + ( 4 x block size ), or + 100k + ( 2.5 x block size ) +@end example +Larger block sizes give rapidly diminishing marginal returns. Most of +the compression comes from the first two or three hundred k of block +size, a fact worth bearing in mind when using @code{bzip2} on small machines. +It is also important to appreciate that the decompression memory +requirement is set at compression time by the choice of block size. + +For files compressed with the default 900k block size, @code{bunzip2} +will require about 3700 kbytes to decompress. To support decompression +of any file on a 4 megabyte machine, @code{bunzip2} has an option to +decompress using approximately half this amount of memory, about 2300 +kbytes. Decompression speed is also halved, so you should use this +option only where necessary. The relevant flag is @code{-s}. + +In general, try and use the largest block size memory constraints allow, +since that maximises the compression achieved. Compression and +decompression speed are virtually unaffected by block size. + +Another significant point applies to files which fit in a single block +-- that means most files you'd encounter using a large block size. The +amount of real memory touched is proportional to the size of the file, +since the file is smaller than a block. For example, compressing a file +20,000 bytes long with the flag @code{-9} will cause the compressor to +allocate around 7600k of memory, but only touch 400k + 20000 * 8 = 560 +kbytes of it. Similarly, the decompressor will allocate 3700k but only +touch 100k + 20000 * 4 = 180 kbytes. + +Here is a table which summarises the maximum memory usage for different +block sizes. Also recorded is the total compressed size for 14 files of +the Calgary Text Compression Corpus totalling 3,141,622 bytes. This +column gives some feel for how compression varies with block size. +These figures tend to understate the advantage of larger block sizes for +larger files, since the Corpus is dominated by smaller files. +@example + Compress Decompress Decompress Corpus + Flag usage usage -s usage Size + + -1 1200k 500k 350k 914704 + -2 2000k 900k 600k 877703 + -3 2800k 1300k 850k 860338 + -4 3600k 1700k 1100k 846899 + -5 4400k 2100k 1350k 845160 + -6 5200k 2500k 1600k 838626 + -7 6100k 2900k 1850k 834096 + -8 6800k 3300k 2100k 828642 + -9 7600k 3700k 2350k 828642 +@end example + +@unnumberedsubsubsec RECOVERING DATA FROM DAMAGED FILES - For files compressed with the default 900k block size, - bunzip2 will require about 3700 kbytes to decompress. To - support decompression of any file on a 4 megabyte machine, - bunzip2 has an option to decompress using approximately - half this amount of memory, about 2300 kbytes. Decompres- - sion speed is also halved, so you should use this option - only where necessary. The relevant flag is -s. - - In general, try and use the largest block size memory con- - straints allow, since that maximises the compression - achieved. Compression and decompression speed are virtu- - ally unaffected by block size. - - Another significant point applies to files which fit in a - single block -- that means most files you'd encounter - using a large block size. The amount of real memory - touched is proportional to the size of the file, since the - file is smaller than a block. For example, compressing a - file 20,000 bytes long with the flag -9 will cause the - compressor to allocate around 6700k of memory, but only - touch 400k + 20000 * 7 = 540 kbytes of it. Similarly, the - decompressor will allocate 3700k but only touch 100k + - 20000 * 4 = 180 kbytes. - - Here is a table which summarises the maximum memory usage - for different block sizes. Also recorded is the total - compressed size for 14 files of the Calgary Text Compres- - sion Corpus totalling 3,141,622 bytes. This column gives - some feel for how compression varies with block size. - These figures tend to understate the advantage of larger - block sizes for larger files, since the Corpus is domi- - nated by smaller files. - - Compress Decompress Decompress Corpus - Flag usage usage -s usage Size - - -1 1100k 500k 350k 914704 - -2 1800k 900k 600k 877703 - -3 2500k 1300k 850k 860338 - -4 3200k 1700k 1100k 846899 - -5 3900k 2100k 1350k 845160 - -6 4600k 2500k 1600k 838626 - -7 5400k 2900k 1850k 834096 - -8 6000k 3300k 2100k 828642 - -9 6700k 3700k 2350k 828642 - - -OPTIONS - -c --stdout - Compress or decompress to standard output. -c will - decompress multiple files to stdout, but will only - compress a single file to stdout. - - -d --decompress - Force decompression. bzip2, bunzip2 and bzcat are - really the same program, and the decision about - what actions to take is done on the basis of which - name is used. This flag overrides that mechanism, - and forces bzip2 to decompress. - - -z --compress - The complement to -d: forces compression, regard- - less of the invokation name. - - -t --test - Check integrity of the specified file(s), but don't - decompress them. This really performs a trial - decompression and throws away the result. - - -f --force - Force overwrite of output files. Normally, bzip2 - will not overwrite existing output files. - - -k --keep - Keep (don't delete) input files during compression - or decompression. - - -s --small - Reduce memory usage, for compression, decompression - and testing. Files are decompressed and tested - using a modified algorithm which only requires 2.5 - bytes per block byte. This means any file can be - decompressed in 2300k of memory, albeit at about - half the normal speed. - - During compression, -s selects a block size of - 200k, which limits memory use to around the same - figure, at the expense of your compression ratio. - In short, if your machine is low on memory (8 - megabytes or less), use -s for everything. See - MEMORY MANAGEMENT above. - - -v --verbose - Verbose mode -- show the compression ratio for each - file processed. Further -v's increase the ver- - bosity level, spewing out lots of information which - is primarily of interest for diagnostic purposes. - - -L --license -V --version - Display the software version, license terms and - conditions. - - -1 to -9 - Set the block size to 100 k, 200 k .. 900 k when - compressing. Has no effect when decompressing. - See MEMORY MANAGEMENT above. - - --repetitive-fast - bzip2 injects some small pseudo-random variations - into very repetitive blocks to limit worst-case - performance during compression. If sorting runs - into difficulties, the block is randomised, and - sorting is restarted. Very roughly, bzip2 persists - for three times as long as a well-behaved input - would take before resorting to randomisation. This - flag makes it give up much sooner. - - --repetitive-best - Opposite of --repetitive-fast; try a lot harder - before resorting to randomisation. - - -RECOVERING DATA FROM DAMAGED FILES - bzip2 compresses files in blocks, usually 900kbytes long. - Each block is handled independently. If a media or trans- - mission error causes a multi-block .bz2 file to become - damaged, it may be possible to recover data from the - undamaged blocks in the file. - - The compressed representation of each block is delimited - by a 48-bit pattern, which makes it possible to find the - block boundaries with reasonable certainty. Each block - also carries its own 32-bit CRC, so damaged blocks can be - distinguished from undamaged ones. - - bzip2recover is a simple program whose purpose is to - search for blocks in .bz2 files, and write each block out - into its own .bz2 file. You can then use bzip2 -t to test - the integrity of the resulting files, and decompress those - which are undamaged. - - bzip2recover takes a single argument, the name of the dam- - aged file, and writes a number of files "rec0001file.bz2", - "rec0002file.bz2", etc, containing the extracted blocks. +@code{bzip2} compresses files in blocks, usually 900kbytes long. Each +block is handled independently. If a media or transmission error causes +a multi-block @code{.bz2} file to become damaged, it may be possible to +recover data from the undamaged blocks in the file. + +The compressed representation of each block is delimited by a 48-bit +pattern, which makes it possible to find the block boundaries with +reasonable certainty. Each block also carries its own 32-bit CRC, so +damaged blocks can be distinguished from undamaged ones. + +@code{bzip2recover} is a simple program whose purpose is to search for +blocks in @code{.bz2} files, and write each block out into its own +@code{.bz2} file. You can then use @code{bzip2 -t} to test the +integrity of the resulting files, and decompress those which are +undamaged. + +@code{bzip2recover} +takes a single argument, the name of the damaged file, +and writes a number of files @code{rec0001file.bz2}, + @code{rec0002file.bz2}, etc, containing the extracted blocks. The output filenames are designed so that the use of - wildcards in subsequent processing -- for example, "bzip2 - -dc rec*file.bz2 > recovered_data" -- lists the files in - the "right" order. + wildcards in subsequent processing -- for example, +@code{bzip2 -dc rec*file.bz2 > recovered_data} -- lists the files in + the correct order. - bzip2recover should be of most use dealing with large .bz2 +@code{bzip2recover} should be of most use dealing with large @code{.bz2} files, as these will contain many blocks. It is clearly futile to use it on damaged single-block files, since a - damaged block cannot be recovered. If you wish to min- - imise any potential data loss through media or transmis- - sion errors, you might consider compressing with a smaller + damaged block cannot be recovered. If you wish to minimise +any potential data loss through media or transmission errors, +you might consider compressing with a smaller block size. -PERFORMANCE NOTES - The sorting phase of compression gathers together similar - strings in the file. Because of this, files containing - very long runs of repeated symbols, like "aabaabaabaab - ..." (repeated several hundred times) may compress - extraordinarily slowly. You can use the -vvvvv option to - monitor progress in great detail, if you want. Decompres- - sion speed is unaffected. - - Such pathological cases seem rare in practice, appearing - mostly in artificially-constructed test files, and in low- - level disk images. It may be inadvisable to use bzip2 to - compress the latter. If you do get a file which causes - severe slowness in compression, try making the block size - as small as possible, with flag -1. - - bzip2 usually allocates several megabytes of memory to - operate in, and then charges all over it in a fairly ran- - dom fashion. This means that performance, both for com- - pressing and decompressing, is largely determined by the - speed at which your machine can service cache misses. - Because of this, small changes to the code to reduce the - miss rate have been observed to give disproportionately - large performance improvements. I imagine bzip2 will per- - form best on machines with very large caches. - - -CAVEATS - I/O error messages are not as helpful as they could be. - Bzip2 tries hard to detect I/O errors and exit cleanly, - but the details of what the problem is sometimes seem - rather misleading. - - This manual page pertains to version 0.9.0 of bzip2. Com- - pressed data created by this version is entirely forwards - and backwards compatible with the previous public release, - version 0.1pl2, but with the following exception: 0.9.0 - can correctly decompress multiple concatenated compressed - files. 0.1pl2 cannot do this; it will stop after decom- - pressing just the first file in the stream. - - Wildcard expansion for Windows 95 and NT is flaky. - - bzip2recover uses 32-bit integers to represent bit posi- - tions in compressed files, so it cannot handle compressed - files more than 512 megabytes long. This could easily be - fixed. - - -AUTHOR - Julian Seward, jseward@@acm.org. - - The ideas embodied in bzip2 are due to (at least) the fol- - lowing people: Michael Burrows and David Wheeler (for the - block sorting transformation), David Wheeler (again, for - the Huffman coder), Peter Fenwick (for the structured cod- - ing model in the original bzip, and many refinements), and - Alistair Moffat, Radford Neal and Ian Witten (for the - arithmetic coder in the original bzip). I am much - indebted for their help, support and advice. See the man- - ual in the source distribution for pointers to sources of - documentation. Christian von Roques encouraged me to look - for faster sorting algorithms, so as to speed up compres- - sion. Bela Lubkin encouraged me to improve the worst-case - compression performance. Many people sent patches, helped - with portability problems, lent machines, gave advice and - were generally helpful. -@end example +@unnumberedsubsubsec PERFORMANCE NOTES + +The sorting phase of compression gathers together similar strings in the +file. Because of this, files containing very long runs of repeated +symbols, like "aabaabaabaab ..." (repeated several hundred times) may +compress more slowly than normal. Versions 0.9.5 and above fare much +better than previous versions in this respect. The ratio between +worst-case and average-case compression time is in the region of 10:1. +For previous versions, this figure was more like 100:1. You can use the +@code{-vvvv} option to monitor progress in great detail, if you want. + +Decompression speed is unaffected by these phenomena. + +@code{bzip2} usually allocates several megabytes of memory to operate +in, and then charges all over it in a fairly random fashion. This means +that performance, both for compressing and decompressing, is largely +determined by the speed at which your machine can service cache misses. +Because of this, small changes to the code to reduce the miss rate have +been observed to give disproportionately large performance improvements. +I imagine @code{bzip2} will perform best on machines with very large +caches. +@unnumberedsubsubsec CAVEATS + +I/O error messages are not as helpful as they could be. @code{bzip2} +tries hard to detect I/O errors and exit cleanly, but the details of +what the problem is sometimes seem rather misleading. + +This manual page pertains to version 0.9.5 of @code{bzip2}. Compressed +data created by this version is entirely forwards and backwards +compatible with the previous public releases, versions 0.1pl2 and 0.9.0, +but with the following exception: 0.9.0 and above can correctly +decompress multiple concatenated compressed files. 0.1pl2 cannot do +this; it will stop after decompressing just the first file in the +stream. + +@code{bzip2recover} uses 32-bit integers to represent bit positions in +compressed files, so it cannot handle compressed files more than 512 +megabytes long. This could easily be fixed. + + +@unnumberedsubsubsec AUTHOR +Julian Seward, @code{jseward@@acm.org}. + +The ideas embodied in @code{bzip2} are due to (at least) the following +people: Michael Burrows and David Wheeler (for the block sorting +transformation), David Wheeler (again, for the Huffman coder), Peter +Fenwick (for the structured coding model in the original @code{bzip}, +and many refinements), and Alistair Moffat, Radford Neal and Ian Witten +(for the arithmetic coder in the original @code{bzip}). I am much +indebted for their help, support and advice. See the manual in the +source distribution for pointers to sources of documentation. Christian +von Roques encouraged me to look for faster sorting algorithms, so as to +speed up compression. Bela Lubkin encouraged me to improve the +worst-case compression performance. Many people sent patches, helped +with portability problems, lent machines, gave advice and were generally +helpful. + +@end quotation + @@ -579,7 +564,7 @@ give better @code{zlib} compatibility. These functions are @code{bzerror} and @code{bzlibVersion}. You may find these functions more convenient for simple file reading and writing, than those in the high-level interface. These functions are not (yet) officially part of -the library, and are not further documented here. If they break, you +the library, and are minimally documented here. If they break, you get to keep all the pieces. I hope to document them properly when time permits. @@ -748,25 +733,31 @@ monitoring/debugging output. If the library has been compiled with setting. Parameter @code{workFactor} controls how the compression phase behaves -when presented with worst case, highly repetitive, input data. -If compression runs into difficulties caused by repetitive data, -some pseudo-random variations are inserted into the block, and -compression is restarted. Lower values of @code{workFactor} -reduce the tolerance of compression to repetitive data. -You should set this parameter carefully; too low, and -compression ratio suffers, too high, and your average-to-worst -case compression times can become very large. -The default value of 30 -gives reasonable behaviour over a wide range of circumstances. - -Allowable values range from 0 to 250 inclusive. 0 is a special -case, equivalent to using the default value of 30. - -Note that the randomisation process is entirely transparent. -If the library decides to randomise and restart compression on a -block, it does so without comment. Randomised blocks are -automatically de-randomised during decompression, so data -integrity is never compromised. +when presented with worst case, highly repetitive, input data. If +compression runs into difficulties caused by repetitive data, the +library switches from the standard sorting algorithm to a fallback +algorithm. The fallback is slower than the standard algorithm by +perhaps a factor of three, but always behaves reasonably, no matter how +bad the input. + +Lower values of @code{workFactor} reduce the amount of effort the +standard algorithm will expend before resorting to the fallback. You +should set this parameter carefully; too low, and many inputs will be +handled by the fallback algorithm and so compress rather slowly, too +high, and your average-to-worst case compression times can become very +large. The default value of 30 gives reasonable behaviour over a wide +range of circumstances. + +Allowable values range from 0 to 250 inclusive. 0 is a special case, +equivalent to using the default value of 30. + +Note that the compressed output generated is the same regardless of +whether or not the fallback algorithm is used. + +Be aware also that this parameter may disappear entirely in future +versions of the library. In principle it should be possible to devise a +good way to automatically choose which algorithm to use. Such a +mechanism would render the parameter obsolete. Possible return values: @display @@ -1652,6 +1643,48 @@ Possible return values: +@section @code{zlib} compatibility functions +Yoshioka Tsuneo has contributed some functions to +give better @code{zlib} compatibility. These functions are +@code{bzopen}, @code{bzread}, @code{bzwrite}, @code{bzflush}, +@code{bzclose}, +@code{bzerror} and @code{bzlibVersion}. +These functions are not (yet) officially part of +the library. If they break, you get to keep all the pieces. +Nevertheless, I think they work ok. +@example +typedef void BZFILE; + +const char * bzlibVersion ( void ); +@end example +Returns a string indicating the library version. +@example +BZFILE * bzopen ( const char *path, const char *mode ); +BZFILE * bzdopen ( int fd, const char *mode ); +@end example +Opens a @code{.bz2} file for reading or writing, using either its name +or a pre-existing file descriptor. +Analogous to @code{fopen} and @code{fdopen}. +@example +int bzread ( BZFILE* b, void* buf, int len ); +int bzwrite ( BZFILE* b, void* buf, int len ); +@end example +Reads/writes data from/to a previously opened @code{BZFILE}. +Analogous to @code{fread} and @code{fwrite}. +@example +int bzflush ( BZFILE* b ); +void bzclose ( BZFILE* b ); +@end example +Flushes/closes a @code{BZFILE}. @code{bzflush} doesn't actually do +anything. Analogous to @code{fflush} and @code{fclose}. + +@example +const char * bzerror ( BZFILE *b, int *errnum ) +@end example +Returns a string describing the more recent error status of +@code{b}, and also sets @code{*errnum} to its numerical value. + + @section Using the library in a @code{stdio}-free environment @subsection Getting rid of @code{stdio} @@ -1677,14 +1710,14 @@ was compiled with @code{BZ_NO_STDIO} set. For a normal compile, an assertion failure yields the message @example - bzip2/libbzip2, v0.9.0: internal error number N. - This is a bug in bzip2/libbzip2, v0.9.0. Please report + bzip2/libbzip2, v0.9.5: internal error number N. + This is a bug in bzip2/libbzip2, v0.9.5. Please report it to me at: jseward@@acm.org. If this happened when you were using some program which uses libbzip2 as a component, you should also report this bug to the author(s) of that program. Please make an effort to report this bug; timely and accurate bug reports eventually lead to higher - quality software. Thx. Julian Seward, 27 June 1998. + quality software. Thanks. Julian Seward, 24 May 1999. @end example where @code{N} is some error code number. @code{exit(3)} is then called. @@ -1721,7 +1754,7 @@ If you can't open the project file for some reason, make a new one, naming these files: @code{blocksort.c}, @code{bzlib.c}, @code{compress.c}, @code{crctable.c}, @code{decompress.c}, @code{huffman.c}, @* -@code{randtable.c} and @code{libbz2.def}. You might also need +@code{randtable.c} and @code{libbz2.def}. You will also need to name the header files @code{bzlib.h} and @code{bzlib_private.h}. If you don't use VC++, you may need to define the proprocessor symbol @@ -1730,6 +1763,14 @@ If you don't use VC++, you may need to define the proprocessor symbol Finally, @code{dlltest.c} is a sample program using the DLL. It has a project file, @code{dlltest.dsp}. +If you just want a makefile for Visual C, have a look at +@code{makefile.msc}. + +Be aware that if you compile @code{bzip2} itself on Win32, you must set +@code{BZ_UNIX} to 0 and @code{BZ_LCCWIN32} to 1, in the file +@code{bzip2.c}, before compiling. Otherwise the resulting binary won't +work correctly. + I haven't tried any of this stuff myself, but it all looks plausible. @@ -1740,7 +1781,8 @@ These are just some random thoughts of mine. Your mileage may vary. @section Limitations of the compressed file format -@code{bzip2-0.9.0} uses exactly the same file format as the previous +@code{bzip2-0.9.5} and @code{0.9.0} +use exactly the same file format as the previous version, @code{bzip2-0.1}. This decision was made in the interests of stability. Creating yet another incompatible compressed file format would create further confusion and disruption for users. @@ -1776,12 +1818,11 @@ decompression and, in retrospect, are unnecessary. These are: @code{bzip2-0.1}'s performance on repetitive data, so perhaps it isn't a problem for real inputs. - Probably the best long-term solution + Probably the best long-term solution, + and the one I have incorporated into 0.9.5 and above, is to use the existing sorting algorithm initially, and fall back to a O(N (log N)^2) algorithm if the standard algorithm gets into difficulties. - This can be done without much difficulty; I made - a prototype implementation of it some months now. @item The compressed file format was never designed to be handled by a library, and I have had to jump though some hoops to produce an efficient implementation of @@ -1797,7 +1838,7 @@ It would be fair to say that the @code{bzip2} format was frozen before I properly and fully understood the performance consequences of doing so. -Improvements which I have been able to incorporate into +Improvements which I was able to incorporate into 0.9.0, despite using the same file format, are: @itemize @bullet @item Single array implementation of the inverse BWT. This @@ -1808,8 +1849,7 @@ Improvements which I have been able to incorporate into of values. @item @code{bzip2-0.9.0} now reads and writes files with @code{fread} and @code{fwrite}; version 0.1 used @code{putc} and @code{getc}. - Duh! I'm embarrassed at my own moronicness (moronicity?) on this - one. + Duh! Well, you live and learn. @end itemize Further ahead, it would be nice @@ -1820,7 +1860,7 @@ require some careful design of compressed file formats. @section Portability issues After some consideration, I have decided not to use -GNU @code{autoconf} to configure 0.9.0. +GNU @code{autoconf} to configure 0.9.5. @code{autoconf}, admirable and wonderful though it is, mainly assists with portability problems between Unix-like @@ -1835,8 +1875,10 @@ under Unix straight out-of-the-box, so to speak, especially if you have a version of GNU C available. There are a couple of @code{__inline__} directives in the code. GNU C -(@code{gcc}) should be able to handle them. If your compiler doesn't -like them, just @code{#define} @code{__inline__} to be null. One +(@code{gcc}) should be able to handle them. If you're not using +GNU C, your C compiler shouldn't see them at all. +If your compiler does, for some reason, see them and doesn't +like them, just @code{#define} @code{__inline__} to be @code{/* */}. One easy way to do this is to compile with the flag @code{-D__inline__=}, which should be understood by most Unix compilers. @@ -1853,6 +1895,11 @@ distribution, please try and link it statically (@code{gcc -s}). This avoids all sorts of library-version issues that others may encounter later on. +If you build @code{bzip2} on Win32, you must set @code{BZ_UNIX} to 0 and +@code{BZ_LCCWIN32} to 1, in the file @code{bzip2.c}, before compiling. +Otherwise the resulting binary won't work correctly. + + @section Reporting bugs I tried pretty hard to make sure @code{bzip2} is @@ -1976,45 +2023,39 @@ A record of the tests I've done. First, some data sets: @itemize @bullet -@item B: a directory containing a 6001 files, one for every length in the +@item B: a directory containing 6001 files, one for every length in the range 0 to 6000 bytes. The files contain random lowercase letters. 18.7 megabytes. @item H: my home directory tree. Documents, source code, mail files, compressed data. H contains B, and also a directory of files designed as boundary cases for the sorting; mostly very - repetitive, nasty files. 445 megabytes. + repetitive, nasty files. 565 megabytes. @item A: directory tree holding various applications built from source: - @code{egcs-1.0.2}, @code{gcc-2.8.1}, KDE Beta 4, GTK, Octave, etc. - 827 megabytes. -@item P: directory tree holding large amounts of source code (@code{.tar} - files) of the entire GNU distribution, plus a couple of - Linux distributions. 2400 megabytes. + @code{egcs}, @code{gcc-2.8.1}, KDE, GTK, Octave, etc. + 2200 megabytes. @end itemize The tests conducted are as follows. Each test means compressing (a copy of) each file in the data set, decompressing it and comparing it against the original. -First, a bunch of tests with block sizes, internal buffer -sizes and randomisation lengths set very small, +First, a bunch of tests with block sizes and internal buffer +sizes set very small, to detect any problems with the -blocking, buffering and randomisation mechanisms. +blocking and buffering mechanisms. This required modifying the source code so as to try to break it. @enumerate @item Data set H, with buffer size of 1 byte, and block size of 23 bytes. @item Data set B, buffer sizes 1 byte, block size 1 byte. -@item As (2) but small-mode decompression (first 1700 files). +@item As (2) but small-mode decompression. @item As (2) with block size 2 bytes. @item As (2) with block size 3 bytes. @item As (2) with block size 4 bytes. @item As (2) with block size 5 bytes. @item As (2) with block size 6 bytes and small-mode decompression. -@item H with normal buffer sizes (5000 bytes), normal block - size (up to 900000 bytes), but with randomisation - mechanism running intensely (randomising approximately every - third byte). -@item As (9) with small-mode decompression. +@item H with buffer size of 1 byte, but normal block + size (up to 900000 bytes). @end enumerate Then some tests with unmodified source code. @enumerate @@ -2023,18 +2064,26 @@ Then some tests with unmodified source code. @item H, compress with flag @code{-1}. @item H, compress with flag @code{-s}, decompress with flag @code{-s}. @item Forwards compatibility: H, @code{bzip2-0.1pl2} compressing, - @code{bzip2-0.9.0} decompressing, all settings normal. -@item Backwards compatibility: H, @code{bzip2-0.9.0} compressing, + @code{bzip2-0.9.5} decompressing, all settings normal. +@item Backwards compatibility: H, @code{bzip2-0.9.5} compressing, @code{bzip2-0.1pl2} decompressing, all settings normal. @item Bigger tests: A, all settings normal. -@item P, all settings normal. -@item Misc test: about 100 megabytes of @code{.tar} files with - @code{bzip2} compiled with Purify. +@item As (7), using the fallback (Sadakane-like) sorting algorithm. +@item As (8), compress with flag @code{-1}, decompress with flag + @code{-s}. +@item H, using the fallback sorting algorithm. +@item Forwards compatibility: A, @code{bzip2-0.1pl2} compressing, + @code{bzip2-0.9.5} decompressing, all settings normal. +@item Backwards compatibility: A, @code{bzip2-0.9.5} compressing, + @code{bzip2-0.1pl2} decompressing, all settings normal. +@item Misc test: about 400 megabytes of @code{.tar} files with + @code{bzip2} compiled with Checker (a memory access error + detector, like Purify). @item Misc tests to make sure it builds and runs ok on non-Linux/x86 platforms. @end enumerate -These tests were conducted on a 205 MHz Cyrix 6x86MX machine, running -Linux 2.0.32. They represent nearly a week of continuous computation. +These tests were conducted on a 225 MHz IDT WinChip machine, running +Linux 2.0.36. They represent nearly a week of continuous computation. All tests completed successfully. @@ -2063,7 +2112,7 @@ David J. Wheeler Program bred3.c and accompanying document bred3.ps. This contains the idea behind the multi-table Huffman coding scheme. - ftp://ftp.cl.cam.ac.uk/pub/user/djw3/ + ftp://ftp.cl.cam.ac.uk/users/djw3/ Jon L. Bentley and Robert Sedgewick "Fast Algorithms for Sorting and Searching Strings" diff --git a/randtable.c b/randtable.c index 27b34af..8f6266f 100644 --- a/randtable.c +++ b/randtable.c @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-1998 Julian R Seward. All rights reserved. + Copyright (C) 1996-1999 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -41,9 +41,9 @@ NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - Julian Seward, Guildford, Surrey, UK. + Julian Seward, Cambridge, UK. jseward@acm.org - bzip2/libbzip2 version 0.9.0c of 18 October 1998 + bzip2/libbzip2 version 0.9.5 of 24 May 1999 This program is based on (at least) the work of: Mike Burrows diff --git a/sample3.bz2 b/sample3.bz2 new file mode 100644 index 0000000..d90cff9 Binary files /dev/null and b/sample3.bz2 differ diff --git a/sample3.ref b/sample3.ref new file mode 100644 index 0000000..775a2f6 --- /dev/null +++ b/sample3.ref @@ -0,0 +1,30007 @@ +This file is exceedingly boring. If you find yourself +reading it, please (1) take it from me that you can safely +guess what the rest of the file says, and (2) seek professional +help. + +ps. there are no further sarcastic remarks in this file. + +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh +ugh diff --git a/words1 b/words1 index a891431..2e83de9 100644 --- a/words1 +++ b/words1 @@ -1,4 +1,4 @@ -Doing 4 tests (2 compress, 2 uncompress) ... +Doing 6 tests (3 compress, 3 uncompress) ... If there's a problem, things might stop at this point. diff --git a/words3 b/words3 index 10bb2e9..8486a84 100644 --- a/words3 +++ b/words3 @@ -1,12 +1,17 @@ -If you got this far and the "cmp"s didn't find anything amiss, looks -like you're in business. You should install bzip2, bunzip2 and bzcat: +If you got this far and the "cmp"s didn't complain, it looks +like you're in business. - Copy bzip2 and bzip2recover to a public place, maybe /usr/bin. - In that public place, make bunzip2 and bzcat be - symbolic links to the bzip2 you just copied there. - Put the manual page, bzip2.1, somewhere appropriate; - perhaps in /usr/man/man1. +To install in /usr/bin, /usr/lib, /usr/man and /usr/include, type + make install +To install somewhere else, eg, /xxx/yyy/{bin,lib,man,include}, type + make install PREFIX=/xxx/yyy +If you are (justifiably) paranoid and want to see what 'make install' +is going to do, you can first do + make -n install or + make -n install PREFIX=/xxx/yyy respectively. +The -n instructs make to show the commands it would execute, but +not actually execute them. Instructions for use are in the preformatted manual page, in the file bzip2.txt. For more detailed documentation, read the full manual. @@ -16,5 +21,3 @@ It is available in Postscript form (manual.ps) and HTML form You can also do "bzip2 --help" to see some helpful information. "bzip2 -L" displays the software license. -Happy compressing. -- JRS, 30 August 1998. - -- cgit v1.2.3 From 795b859eee96c700e8f3c3fe68e6a9a39d95797c Mon Sep 17 00:00:00 2001 From: Julian Seward Date: Sat, 24 Jun 2000 22:13:13 +0200 Subject: bzip2-1.0.1 --- CHANGES | 67 ++++++ LICENSE | 4 +- Makefile | 58 ++++- Makefile-libbz2_so | 43 ++++ README | 43 +++- README.COMPILATION.PROBLEMS | 130 +++++++++++ blocksort.c | 335 ++++++++++++++++------------ bzip2.1 | 8 +- bzip2.1.preformatted | 113 ++++++++-- bzip2.c | 508 ++++++++++++++++++++++++++++++++++++------- bzip2.txt | 15 +- bzip2recover.c | 8 +- bzlib.c | 248 ++++++++++++--------- bzlib.h | 67 +++--- bzlib_private.h | 52 ++--- compress.c | 193 ++++++++++++----- crctable.c | 6 +- decompress.c | 44 +++- dlltest.c | 341 +++++++++++++++-------------- huffman.c | 36 ++-- libbz2.def | 46 ++-- makefile.msc | 18 +- manual.texi | 516 +++++++++++++++++++++++++------------------- randtable.c | 6 +- spewG.c | 39 ++++ unzcrash.c | 126 +++++++++++ words0 | 5 + 27 files changed, 2160 insertions(+), 915 deletions(-) create mode 100644 Makefile-libbz2_so create mode 100644 README.COMPILATION.PROBLEMS create mode 100644 spewG.c create mode 100644 unzcrash.c create mode 100644 words0 diff --git a/CHANGES b/CHANGES index 0acb1c2..ecaf417 100644 --- a/CHANGES +++ b/CHANGES @@ -98,3 +98,70 @@ functioning of the bzip2 program or library. Added a couple of casts so the library compiles without warnings at level 3 in MS Visual Studio 6.0. Included a Y2K statement in the file Y2K_INFO. All other changes are minor documentation changes. + +1.0 +~~~ +Several minor bugfixes and enhancements: + +* Large file support. The library uses 64-bit counters to + count the volume of data passing through it. bzip2.c + is now compiled with -D_FILE_OFFSET_BITS=64 to get large + file support from the C library. -v correctly prints out + file sizes greater than 4 gigabytes. All these changes have + been made without assuming a 64-bit platform or a C compiler + which supports 64-bit ints, so, except for the C library + aspect, they are fully portable. + +* Decompression robustness. The library/program should be + robust to any corruption of compressed data, detecting and + handling _all_ corruption, instead of merely relying on + the CRCs. What this means is that the program should + never crash, given corrupted data, and the library should + always return BZ_DATA_ERROR. + +* Fixed an obscure race-condition bug only ever observed on + Solaris, in which, if you were very unlucky and issued + control-C at exactly the wrong time, both input and output + files would be deleted. + +* Don't run out of file handles on test/decompression when + large numbers of files have invalid magic numbers. + +* Avoid library namespace pollution. Prefix all exported + symbols with BZ2_. + +* Minor sorting enhancements from my DCC2000 paper. + +* Advance the version number to 1.0, so as to counteract the + (false-in-this-case) impression some people have that programs + with version numbers less than 1.0 are in someway, experimental, + pre-release versions. + +* Create an initial Makefile-libbz2_so to build a shared library. + Yes, I know I should really use libtool et al ... + +* Make the program exit with 2 instead of 0 when decompression + fails due to a bad magic number (ie, an invalid bzip2 header). + Also exit with 1 (as the manual claims :-) whenever a diagnostic + message would have been printed AND the corresponding operation + is aborted, for example + bzip2: Output file xx already exists. + When a diagnostic message is printed but the operation is not + aborted, for example + bzip2: Can't guess original name for wurble -- using wurble.out + then the exit value 0 is returned, unless some other problem is + also detected. + + I think it corresponds more closely to what the manual claims now. + + +1.0.1 +~~~~~ +* Modified dlltest.c so it uses the new BZ2_ naming scheme. +* Modified makefile-msc to fix minor build probs on Win2k. +* Updated README.COMPILATION.PROBLEMS. + +There are no functionality changes or bug fixes relative to version +1.0.0. This is just a documentation update + a fix for minor Win32 +build problems. For almost everyone, upgrading from 1.0.0 to 1.0.1 is +utterly pointless. Don't bother. diff --git a/LICENSE b/LICENSE index bc0069a..88fa6d8 100644 --- a/LICENSE +++ b/LICENSE @@ -1,6 +1,6 @@ This program, "bzip2" and associated library "libbzip2", are -copyright (C) 1996-1999 Julian R Seward. All rights reserved. +copyright (C) 1996-2000 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -35,5 +35,5 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. Julian Seward, Cambridge, UK. jseward@acm.org -bzip2/libbzip2 version 0.9.5 of 24 May 1999 +bzip2/libbzip2 version 1.0 of 21 March 2000 diff --git a/Makefile b/Makefile index 8a1235d..ab17f49 100644 --- a/Makefile +++ b/Makefile @@ -1,7 +1,8 @@ SHELL=/bin/sh CC=gcc -CFLAGS=-Wall -Winline -O2 -fomit-frame-pointer -fno-strength-reduce +BIGFILES=-D_FILE_OFFSET_BITS=64 +CFLAGS=-Wall -Winline -O2 -fomit-frame-pointer -fno-strength-reduce $(BIGFILES) OBJS= blocksort.o \ huffman.o \ @@ -73,6 +74,7 @@ clean: sample1.tst sample2.tst sample3.tst blocksort.o: blocksort.c + @cat words0 $(CC) $(CFLAGS) -c blocksort.c huffman.o: huffman.c $(CC) $(CFLAGS) -c huffman.c @@ -91,13 +93,49 @@ bzip2.o: bzip2.c bzip2recover.o: bzip2recover.c $(CC) $(CFLAGS) -c bzip2recover.c +DISTNAME=bzip2-1.0.1 tarfile: - tar cvf interim.tar blocksort.c huffman.c crctable.c \ - randtable.c compress.c decompress.c bzlib.c bzip2.c \ - bzip2recover.c bzlib.h bzlib_private.h Makefile manual.texi \ - manual.ps LICENSE bzip2.1 bzip2.1.preformatted bzip2.txt \ - words1 words2 words3 sample1.ref sample2.ref sample3.ref \ - sample1.bz2 sample2.bz2 sample3.bz2 dlltest.c \ - *.html README CHANGES libbz2.def libbz2.dsp \ - dlltest.dsp makefile.msc Y2K_INFO - + rm -f $(DISTNAME) + ln -sf . $(DISTNAME) + tar cvf $(DISTNAME).tar \ + $(DISTNAME)/blocksort.c \ + $(DISTNAME)/huffman.c \ + $(DISTNAME)/crctable.c \ + $(DISTNAME)/randtable.c \ + $(DISTNAME)/compress.c \ + $(DISTNAME)/decompress.c \ + $(DISTNAME)/bzlib.c \ + $(DISTNAME)/bzip2.c \ + $(DISTNAME)/bzip2recover.c \ + $(DISTNAME)/bzlib.h \ + $(DISTNAME)/bzlib_private.h \ + $(DISTNAME)/Makefile \ + $(DISTNAME)/manual.texi \ + $(DISTNAME)/manual.ps \ + $(DISTNAME)/LICENSE \ + $(DISTNAME)/bzip2.1 \ + $(DISTNAME)/bzip2.1.preformatted \ + $(DISTNAME)/bzip2.txt \ + $(DISTNAME)/words0 \ + $(DISTNAME)/words1 \ + $(DISTNAME)/words2 \ + $(DISTNAME)/words3 \ + $(DISTNAME)/sample1.ref \ + $(DISTNAME)/sample2.ref \ + $(DISTNAME)/sample3.ref \ + $(DISTNAME)/sample1.bz2 \ + $(DISTNAME)/sample2.bz2 \ + $(DISTNAME)/sample3.bz2 \ + $(DISTNAME)/dlltest.c \ + $(DISTNAME)/*.html \ + $(DISTNAME)/README \ + $(DISTNAME)/README.COMPILATION.PROBLEMS \ + $(DISTNAME)/CHANGES \ + $(DISTNAME)/libbz2.def \ + $(DISTNAME)/libbz2.dsp \ + $(DISTNAME)/dlltest.dsp \ + $(DISTNAME)/makefile.msc \ + $(DISTNAME)/Y2K_INFO \ + $(DISTNAME)/unzcrash.c \ + $(DISTNAME)/spewG.c \ + $(DISTNAME)/Makefile-libbz2_so diff --git a/Makefile-libbz2_so b/Makefile-libbz2_so new file mode 100644 index 0000000..a347c50 --- /dev/null +++ b/Makefile-libbz2_so @@ -0,0 +1,43 @@ + +# This Makefile builds a shared version of the library, +# libbz2.so.1.0.1, with soname libbz2.so.1.0, +# at least on x86-Linux (RedHat 5.2), +# with gcc-2.7.2.3. Please see the README file for some +# important info about building the library like this. + +SHELL=/bin/sh +CC=gcc +BIGFILES=-D_FILE_OFFSET_BITS=64 +CFLAGS=-fpic -fPIC -Wall -Winline -O2 -fomit-frame-pointer -fno-strength-reduce $(BIGFILES) + +OBJS= blocksort.o \ + huffman.o \ + crctable.o \ + randtable.o \ + compress.o \ + decompress.o \ + bzlib.o + +all: $(OBJS) + $(CC) -shared -Wl,-soname -Wl,libbz2.so.1.0 -o libbz2.so.1.0.1 $(OBJS) + $(CC) $(CFLAGS) -o bzip2-shared bzip2.c libbz2.so.1.0.1 + rm -f libbz2.so.1.0 + ln -s libbz2.so.1.0.1 libbz2.so.1.0 + +clean: + rm -f $(OBJS) bzip2.o libbz2.so.1.0.1 libbz2.so.1.0 bzip2-shared + +blocksort.o: blocksort.c + $(CC) $(CFLAGS) -c blocksort.c +huffman.o: huffman.c + $(CC) $(CFLAGS) -c huffman.c +crctable.o: crctable.c + $(CC) $(CFLAGS) -c crctable.c +randtable.o: randtable.c + $(CC) $(CFLAGS) -c randtable.c +compress.o: compress.c + $(CC) $(CFLAGS) -c compress.c +decompress.o: decompress.c + $(CC) $(CFLAGS) -c decompress.c +bzlib.o: bzlib.c + $(CC) $(CFLAGS) -c bzlib.c diff --git a/README b/README index ee70649..22945a2 100644 --- a/README +++ b/README @@ -1,9 +1,9 @@ This is the README for bzip2, a block-sorting file compressor, version -0.9.5d. This version is fully compatible with the previous public -releases, bzip2-0.1pl2 and bzip2-0.9.0. +1.0. This version is fully compatible with the previous public +releases, bzip2-0.1pl2, bzip2-0.9.0 and bzip2-0.9.5. -bzip2-0.9.5 is distributed under a BSD-style license. For details, +bzip2-1.0 is distributed under a BSD-style license. For details, see the file LICENSE. Complete documentation is available in Postscript form (manual.ps) or @@ -30,15 +30,37 @@ The -n instructs make to show the commands it would execute, but not actually execute them. +HOW TO BUILD -- UNIX, shared library libbz2.so. + +Do 'make -f Makefile-libbz2_so'. This Makefile seems to work for +Linux-ELF (RedHat 5.2 on an x86 box), with gcc. I make no claims +that it works for any other platform, though I suspect it probably +will work for most platforms employing both ELF and gcc. + +bzip2-shared, a client of the shared library, is also build, but +not self-tested. So I suggest you also build using the normal +Makefile, since that conducts a self-test. + +Important note for people upgrading .so's from 0.9.0/0.9.5 to +version 1.0. All the functions in the library have been renamed, +from (eg) bzCompress to BZ2_bzCompress, to avoid namespace pollution. +Unfortunately this means that the libbz2.so created by +Makefile-libbz2_so will not work with any program which used an +older version of the library. Sorry. I do encourage library +clients to make the effort to upgrade to use version 1.0, since +it is both faster and more robust than previous versions. + + HOW TO BUILD -- Windows 95, NT, DOS, Mac, etc. It's difficult for me to support compilation on all these platforms. My approach is to collect binaries for these platforms, and put them -on my web page (http://www.muraroa.demon.co.uk). Look there. However -(FWIW), bzip2-0.9.5 is very standard ANSI C and should compile -unmodified with MS Visual C. For Win32, there is one important -caveat: in bzip2.c, you must set BZ_UNIX to 0 and BZ_LCCWIN32 to 1 -before building. +on the master web page (http://sourceware.cygnus.com/bzip2). Look +there. However (FWIW), bzip2-1.0 is very standard ANSI C and should +compile unmodified with MS Visual C. For Win32, there is one +important caveat: in bzip2.c, you must set BZ_UNIX to 0 and +BZ_LCCWIN32 to 1 before building. If you have difficulties building, +you might want to read README.COMPILATION.PROBLEMS. VALIDATION @@ -116,6 +138,10 @@ WHAT'S NEW IN 0.9.5 ? * Many small improvements in file and flag handling. * A Y2K statement. +WHAT'S NEW IN 1.0 + + See the CHANGES file. + I hope you find bzip2 useful. Feel free to contact me at jseward@acm.org if you have any suggestions or queries. Many people mailed me with @@ -137,3 +163,4 @@ Cambridge, UK 23 August 1998 (bzip2, version 0.9.0) 8 June 1999 (bzip2, version 0.9.5) 4 Sept 1999 (bzip2, version 0.9.5d) + 5 May 2000 (bzip2, version 1.0pre8) diff --git a/README.COMPILATION.PROBLEMS b/README.COMPILATION.PROBLEMS new file mode 100644 index 0000000..d621ad5 --- /dev/null +++ b/README.COMPILATION.PROBLEMS @@ -0,0 +1,130 @@ + +bzip2-1.0 should compile without problems on the vast majority of +platforms. Using the supplied Makefile, I've built and tested it +myself for x86-linux, sparc-solaris, alpha-linux, x86-cygwin32 and +alpha-tru64unix. With makefile.msc, Visual C++ 6.0 and nmake, you can +build a native Win32 version too. Large file support seems to work +correctly on at least alpha-tru64unix and x86-cygwin32 (on Windows +2000). + +When I say "large file" I mean a file of size 2,147,483,648 (2^31) +bytes or above. Many older OSs can't handle files above this size, +but many newer ones can. Large files are pretty huge -- most files +you'll encounter are not Large Files. + +Earlier versions of bzip2 (0.1, 0.9.0, 0.9.5) compiled on a wide +variety of platforms without difficulty, and I hope this version will +continue in that tradition. However, in order to support large files, +I've had to include the define -D_FILE_OFFSET_BITS=64 in the Makefile. +This can cause problems. + +The technique of adding -D_FILE_OFFSET_BITS=64 to get large file +support is, as far as I know, the Recommended Way to get correct large +file support. For more details, see the Large File Support +Specification, published by the Large File Summit, at + http://www.sas.com/standard/large.file/ + +As a general comment, if you get compilation errors which you think +are related to large file support, try removing the above define from +the Makefile, ie, delete the line + BIGFILES=-D_FILE_OFFSET_BITS=64 +from the Makefile, and do 'make clean ; make'. This will give you a +version of bzip2 without large file support, which, for most +applications, is probably not a problem. + +Alternatively, try some of the platform-specific hints listed below. + +You can use the spewG.c program to generate huge files to test bzip2's +large file support, if you are feeling paranoid. Be aware though that +any compilation problems which affect bzip2 will also affect spewG.c, +alas. + + +Known problems as of 1.0pre8: +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +* HP/UX 10.20 and 11.00, using gcc (2.7.2.3 and 2.95.2): A large + number of warnings appear, including the following: + + /usr/include/sys/resource.h: In function `getrlimit': + /usr/include/sys/resource.h:168: + warning: implicit declaration of function `__getrlimit64' + /usr/include/sys/resource.h: In function `setrlimit': + /usr/include/sys/resource.h:170: + warning: implicit declaration of function `__setrlimit64' + + This would appear to be a problem with large file support, header + files and gcc. gcc may or may not give up at this point. If it + fails, you might be able to improve matters by adding + -D__STDC_EXT__=1 + to the BIGFILES variable in the Makefile (ie, change its definition + to + BIGFILES=-D_FILE_OFFSET_BITS=64 -D__STDC_EXT__=1 + + Even if gcc does produce a binary which appears to work (ie passes + its self-tests), you might want to test it to see if it works properly + on large files. + + +* HP/UX 10.20 and 11.00, using HP's cc compiler. + + No specific problems for this combination, except that you'll need to + specify the -Ae flag, and zap the gcc-specific stuff + -Wall -Winline -O2 -fomit-frame-pointer -fno-strength-reduce. + You should retain -D_FILE_OFFSET_BITS=64 in order to get large + file support -- which is reported to work ok for this HP/UX + cc + combination. + + +* SunOS 4.1.X. + + Amazingly, there are still people out there using this venerable old + banger. I shouldn't be too rude -- I started life on SunOS, and + it was a pretty darn good OS, way back then. Anyway: + + SunOS doesn't seem to have strerror(), so you'll have to use + perror(), perhaps by doing adding this (warning: UNTESTED CODE): + + char* strerror ( int errnum ) + { + if (errnum < 0 || errnum >= sys_nerr) + return "Unknown error"; + else + return sys_errlist[errnum]; + } + + Or you could comment out the relevant calls to strerror; they're + not mission-critical. Or you could upgrade to Solaris. Ha ha ha! + (what?? you think I've got Bad Attitude?) + + +* Making a shared library on Solaris. (Not really a compilation + problem, but many people ask ...) + + Firstly, if you have Solaris 8, either you have libbz2.so already + on your system, or you can install it from the Solaris CD. + + Secondly, be aware that there are potential naming conflicts + between the .so file supplied with Solaris 8, and the .so file + which Makefile-libbz2_so will make. Makefile-libbz2_so creates + a .so which has the names which I intend to be "official" as + of version 1.0.0 and onwards. Unfortunately, the .so in + Solaris 8 appeared before I decided on the final names, so + the two libraries are incompatible. We have since communicated + and I hope that the problems will have been solved in the next + version of Solaris, whenever that might appear. + + All that said: you might be able to get somewhere + by finding the line in Makefile-libbz2_so which says + + $(CC) -shared -Wl,-soname -Wl,libbz2.so.1.0 -o libbz2.so.1.0.1 $(OBJS) + + and replacing with + + ($CC) -G -shared -o libbz2.so.1.0.1 -h libbz2.so.1.0 $(OBJS) + + If gcc objects to the combination -fpic -fPIC, get rid of + the second one, leaving just "-fpic". + + +That's the end of the currently known compilation problems. diff --git a/blocksort.c b/blocksort.c index 85a02de..ec42672 100644 --- a/blocksort.c +++ b/blocksort.c @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-1999 Julian R Seward. All rights reserved. + Copyright (C) 1996-2000 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -43,7 +43,7 @@ Julian Seward, Cambridge, UK. jseward@acm.org - bzip2/libbzip2 version 0.9.5 of 24 May 1999 + bzip2/libbzip2 version 1.0 of 21 March 2000 This program is based on (at least) the work of: Mike Burrows @@ -56,6 +56,13 @@ Jon L. Bentley For more information on these sources, see the manual. + + To get some idea how the block sorting algorithms in this file + work, read my paper + On the Performance of BWT Sorting Algorithms + in Proceedings of the IEEE Data Compression Conference 2000, + Snowbird, Utah, USA, 27-30 March 2000. The main sort in this + file implements the algorithm called cache in the paper. --*/ @@ -232,11 +239,11 @@ void fallbackQSort3 ( UInt32* fmap, /* Pre: nblock > 0 eclass exists for [0 .. nblock-1] - ((UInt16*)eclass) [0 .. nblock-1] [15:8] holds block + ((UChar*)eclass) [0 .. nblock-1] holds block ptr exists for [0 .. nblock-1] Post: - ((UInt16*)eclass) [0 .. nblock-1] [15:8] holds block + ((UChar*)eclass) [0 .. nblock-1] holds block All other areas of eclass destroyed fmap [0 .. nblock-1] holds sorted order bhtab [ 0 .. 2+(nblock/32) ] destroyed @@ -260,7 +267,7 @@ void fallbackSort ( UInt32* fmap, Int32 H, i, j, k, l, r, cc, cc1; Int32 nNotDone; Int32 nBhtab; - UInt16* eclass16 = (UInt16*)eclass; + UChar* eclass8 = (UChar*)eclass; /*-- Initial 1-char radix sort to generate @@ -269,12 +276,12 @@ void fallbackSort ( UInt32* fmap, if (verb >= 4) VPrintf0 ( " bucket sorting ...\n" ); for (i = 0; i < 257; i++) ftab[i] = 0; - for (i = 0; i < nblock; i++) ftab[eclass16[i] >> 8]++; + for (i = 0; i < nblock; i++) ftab[eclass8[i]]++; for (i = 0; i < 256; i++) ftabCopy[i] = ftab[i]; for (i = 1; i < 257; i++) ftab[i] += ftab[i-1]; for (i = 0; i < nblock; i++) { - j = eclass16[i] >> 8; + j = eclass8[i]; k = ftab[j] - 1; ftab[j] = k; fmap[k] = i; @@ -354,7 +361,7 @@ void fallbackSort ( UInt32* fmap, /*-- Reconstruct the original block in - eclass16 [0 .. nblock-1] [15:8], since the + eclass8 [0 .. nblock-1], since the previous phase destroyed it. --*/ if (verb >= 4) @@ -363,7 +370,7 @@ void fallbackSort ( UInt32* fmap, for (i = 0; i < nblock; i++) { while (ftabCopy[j] == 0) j++; ftabCopy[j]--; - eclass16[fmap[i]] = j << 8; + eclass8[fmap[i]] = (UChar)j; } AssertH ( j < 256, 1005 ); } @@ -386,67 +393,116 @@ static __inline__ Bool mainGtU ( UInt32 i1, UInt32 i2, - UInt16* block, + UChar* block, UInt16* quadrant, UInt32 nblock, Int32* budget ) { - Int32 k; + Int32 k; + UChar c1, c2; UInt16 s1, s2; AssertD ( i1 != i2, "mainGtU" ); - - s1 = block[i1]; s2 = block[i2]; - if (s1 != s2) return (s1 > s2); - i1 += 2; i2 += 2; - - s1 = block[i1]; s2 = block[i2]; - if (s1 != s2) return (s1 > s2); - i1 += 2; i2 += 2; - - s1 = block[i1]; s2 = block[i2]; - if (s1 != s2) return (s1 > s2); - i1 += 2; i2 += 2; - - s1 = block[i1]; s2 = block[i2]; - if (s1 != s2) return (s1 > s2); - i1 += 2; i2 += 2; - - s1 = block[i1]; s2 = block[i2]; - if (s1 != s2) return (s1 > s2); - i1 += 2; i2 += 2; - - s1 = block[i1]; s2 = block[i2]; - if (s1 != s2) return (s1 > s2); - i1 += 2; i2 += 2; + /* 1 */ + c1 = block[i1]; c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + i1++; i2++; + /* 2 */ + c1 = block[i1]; c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + i1++; i2++; + /* 3 */ + c1 = block[i1]; c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + i1++; i2++; + /* 4 */ + c1 = block[i1]; c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + i1++; i2++; + /* 5 */ + c1 = block[i1]; c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + i1++; i2++; + /* 6 */ + c1 = block[i1]; c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + i1++; i2++; + /* 7 */ + c1 = block[i1]; c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + i1++; i2++; + /* 8 */ + c1 = block[i1]; c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + i1++; i2++; + /* 9 */ + c1 = block[i1]; c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + i1++; i2++; + /* 10 */ + c1 = block[i1]; c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + i1++; i2++; + /* 11 */ + c1 = block[i1]; c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + i1++; i2++; + /* 12 */ + c1 = block[i1]; c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + i1++; i2++; k = nblock + 8; do { - - s1 = block[i1]; s2 = block[i2]; + /* 1 */ + c1 = block[i1]; c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + s1 = quadrant[i1]; s2 = quadrant[i2]; if (s1 != s2) return (s1 > s2); + i1++; i2++; + /* 2 */ + c1 = block[i1]; c2 = block[i2]; + if (c1 != c2) return (c1 > c2); s1 = quadrant[i1]; s2 = quadrant[i2]; if (s1 != s2) return (s1 > s2); - i1 += 2; i2 += 2; - - s1 = block[i1]; s2 = block[i2]; + i1++; i2++; + /* 3 */ + c1 = block[i1]; c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + s1 = quadrant[i1]; s2 = quadrant[i2]; if (s1 != s2) return (s1 > s2); + i1++; i2++; + /* 4 */ + c1 = block[i1]; c2 = block[i2]; + if (c1 != c2) return (c1 > c2); s1 = quadrant[i1]; s2 = quadrant[i2]; if (s1 != s2) return (s1 > s2); - i1 += 2; i2 += 2; - - s1 = block[i1]; s2 = block[i2]; + i1++; i2++; + /* 5 */ + c1 = block[i1]; c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + s1 = quadrant[i1]; s2 = quadrant[i2]; if (s1 != s2) return (s1 > s2); + i1++; i2++; + /* 6 */ + c1 = block[i1]; c2 = block[i2]; + if (c1 != c2) return (c1 > c2); s1 = quadrant[i1]; s2 = quadrant[i2]; if (s1 != s2) return (s1 > s2); - i1 += 2; i2 += 2; - - s1 = block[i1]; s2 = block[i2]; + i1++; i2++; + /* 7 */ + c1 = block[i1]; c2 = block[i2]; + if (c1 != c2) return (c1 > c2); + s1 = quadrant[i1]; s2 = quadrant[i2]; if (s1 != s2) return (s1 > s2); + i1++; i2++; + /* 8 */ + c1 = block[i1]; c2 = block[i2]; + if (c1 != c2) return (c1 > c2); s1 = quadrant[i1]; s2 = quadrant[i2]; if (s1 != s2) return (s1 > s2); - i1 += 2; i2 += 2; + i1++; i2++; if (i1 >= nblock) i1 -= nblock; if (i2 >= nblock) i2 -= nblock; @@ -467,13 +523,14 @@ Bool mainGtU ( UInt32 i1, because the number of elems to sort is usually small, typically <= 20. --*/ +static Int32 incs[14] = { 1, 4, 13, 40, 121, 364, 1093, 3280, 9841, 29524, 88573, 265720, 797161, 2391484 }; static void mainSimpleSort ( UInt32* ptr, - UInt16* block, + UChar* block, UInt16* quadrant, Int32 nblock, Int32 lo, @@ -568,19 +625,19 @@ void mainSimpleSort ( UInt32* ptr, } \ } - static __inline__ -UInt16 mmed3 ( UInt16 a, UInt16 b, UInt16 c ) +UChar mmed3 ( UChar a, UChar b, UChar c ) { - UInt16 t; + UChar t; if (a > b) { t = a; a = b; b = t; }; - if (b > c) { t = b; b = c; c = t; }; - if (a > b) b = a; + if (b > c) { + b = c; + if (a > b) b = a; + } return b; } - #define mmin(a,b) ((a) < (b)) ? (a) : (b) #define mpush(lz,hz,dz) { stackLo[sp] = lz; \ @@ -609,7 +666,7 @@ UInt16 mmed3 ( UInt16 a, UInt16 b, UInt16 c ) static void mainQSort3 ( UInt32* ptr, - UInt16* block, + UChar* block, UInt16* quadrant, Int32 nblock, Int32 loSt, @@ -679,7 +736,7 @@ void mainQSort3 ( UInt32* ptr, AssertD ( unHi == unLo-1, "mainQSort3(2)" ); if (gtHi < ltLo) { - mpush(lo, hi, d+2 ); + mpush(lo, hi, d+1 ); continue; } @@ -691,7 +748,7 @@ void mainQSort3 ( UInt32* ptr, nextLo[0] = lo; nextHi[0] = n; nextD[0] = d; nextLo[1] = m; nextHi[1] = hi; nextD[1] = d; - nextLo[2] = n+1; nextHi[2] = m-1; nextD[2] = d+2; + nextLo[2] = n+1; nextHi[2] = m-1; nextD[2] = d+1; if (mnextsize(0) < mnextsize(1)) mnextswap(0,1); if (mnextsize(1) < mnextsize(2)) mnextswap(1,2); @@ -722,11 +779,11 @@ void mainQSort3 ( UInt32* ptr, /* Pre: nblock > N_OVERSHOOT block32 exists for [0 .. nblock-1 +N_OVERSHOOT] - ((UInt16*)block32) [0 .. nblock-1] [15:8] holds block + ((UChar*)block32) [0 .. nblock-1] holds block ptr exists for [0 .. nblock-1] Post: - ((UInt16*)block32) [0 .. nblock-1] [15:8] holds block + ((UChar*)block32) [0 .. nblock-1] holds block All other areas of block32 destroyed ftab [0 .. 65536 ] destroyed ptr [0 .. nblock-1] holds sorted order @@ -739,40 +796,47 @@ void mainQSort3 ( UInt32* ptr, static void mainSort ( UInt32* ptr, - UInt16* block, + UChar* block, UInt16* quadrant, UInt32* ftab, Int32 nblock, Int32 verb, Int32* budget ) { - Int32 i, j, k, m, ss, sb; + Int32 i, j, k, ss, sb; Int32 runningOrder[256]; - Int32 copy[256]; Bool bigDone[256]; + Int32 copyStart[256]; + Int32 copyEnd [256]; UChar c1; Int32 numQSorted; - Int32 biggestSoFar; UInt16 s; - if (verb >= 4) VPrintf0 ( " main sort initialise ...\n" ); - /*-- Stripe the block data into 16 bits, and at the - same time set up the 2-byte frequency table - --*/ + /*-- set up the 2-byte frequency table --*/ for (i = 65536; i >= 0; i--) ftab[i] = 0; - s = block[0]; - for (i = 1; i < nblock; i++) { + j = block[0] << 8; + i = nblock-1; + for (; i >= 3; i -= 4) { + quadrant[i] = 0; + j = (j >> 8) | ( ((UInt16)block[i]) << 8); + ftab[j]++; + quadrant[i-1] = 0; + j = (j >> 8) | ( ((UInt16)block[i-1]) << 8); + ftab[j]++; + quadrant[i-2] = 0; + j = (j >> 8) | ( ((UInt16)block[i-2]) << 8); + ftab[j]++; + quadrant[i-3] = 0; + j = (j >> 8) | ( ((UInt16)block[i-3]) << 8); + ftab[j]++; + } + for (; i >= 0; i--) { quadrant[i] = 0; - s = (s << 8) | block[i]; - block[i-1] = s; - ftab[s]++; + j = (j >> 8) | ( ((UInt16)block[i]) << 8); + ftab[j]++; } - quadrant[0] = 0; - s = (s << 8) | (block[0] >> 8); - block[nblock-1] = s; - ftab[s]++; /*-- (emphasises close relationship of block & quadrant) --*/ for (i = 0; i < BZ_N_OVERSHOOT; i++) { @@ -785,9 +849,29 @@ void mainSort ( UInt32* ptr, /*-- Complete the initial radix sort --*/ for (i = 1; i <= 65536; i++) ftab[i] += ftab[i-1]; - for (i = 0; i < nblock; i++) { - s = block[i]; - j = ftab[s] - 1; + s = block[0] << 8; + i = nblock-1; + for (; i >= 3; i -= 4) { + s = (s >> 8) | (block[i] << 8); + j = ftab[s] -1; + ftab[s] = j; + ptr[j] = i; + s = (s >> 8) | (block[i-1] << 8); + j = ftab[s] -1; + ftab[s] = j; + ptr[j] = i-1; + s = (s >> 8) | (block[i-2] << 8); + j = ftab[s] -1; + ftab[s] = j; + ptr[j] = i-2; + s = (s >> 8) | (block[i-3] << 8); + j = ftab[s] -1; + ftab[s] = j; + ptr[j] = i-3; + } + for (; i >= 0; i--) { + s = (s >> 8) | (block[i] << 8); + j = ftab[s] -1; ftab[s] = j; ptr[j] = i; } @@ -826,13 +910,13 @@ void mainSort ( UInt32* ptr, The main sorting loop. --*/ - biggestSoFar = numQSorted = 0; + numQSorted = 0; for (i = 0; i <= 255; i++) { /*-- Process big buckets, starting with the least full. - Basically this is a 4-step process in which we call + Basically this is a 3-step process in which we call mainQSort3 to sort the small buckets [ss, j], but also make a big effort to avoid the calls if we can. --*/ @@ -869,39 +953,38 @@ void mainSort ( UInt32* ptr, } } + AssertH ( !bigDone[ss], 1006 ); + /*-- Step 2: - Deal specially with case [ss, ss]. This establishes the - sorted order for [ss, ss] without any comparisons. - A clever trick, cryptically described as steps Q6b and Q6c - in SRC-124 (aka BW94). Compared to bzip2, this makes it - practical not to use a preliminary run-length coder. + Now scan this big bucket [ss] so as to synthesise the + sorted order for small buckets [t, ss] for all t, + including, magically, the bucket [ss,ss] too. + This will avoid doing Real Work in subsequent Step 1's. --*/ { - Int32 put0, get0, put1, get1; - Int32 sbn = (ss << 8) + ss; - Int32 lo = ftab[sbn] & CLEARMASK; - Int32 hi = (ftab[sbn+1] & CLEARMASK) - 1; - UChar ssc = (UChar)ss; - put0 = lo; - get0 = ftab[ss << 8] & CLEARMASK; - put1 = hi; - get1 = (ftab[(ss+1) << 8] & CLEARMASK) - 1; - while (get0 < put0) { - j = ptr[get0]-1; if (j < 0) j += nblock; - c1 = (UChar)(block[j] >> 8); - if (c1 == ssc) { ptr[put0] = j; put0++; }; - get0++; + for (j = 0; j <= 255; j++) { + copyStart[j] = ftab[(j << 8) + ss] & CLEARMASK; + copyEnd [j] = (ftab[(j << 8) + ss + 1] & CLEARMASK) - 1; + } + for (j = ftab[ss << 8] & CLEARMASK; j < copyStart[ss]; j++) { + k = ptr[j]-1; if (k < 0) k += nblock; + c1 = block[k]; + if (!bigDone[c1]) + ptr[ copyStart[c1]++ ] = k; } - while (get1 > put1) { - j = ptr[get1]-1; if (j < 0) j += nblock; - c1 = (UChar)(block[j] >> 8); - if (c1 == ssc) { ptr[put1] = j; put1--; }; - get1--; + for (j = (ftab[(ss+1) << 8] & CLEARMASK) - 1; j > copyEnd[ss]; j--) { + k = ptr[j]-1; if (k < 0) k += nblock; + c1 = block[k]; + if (!bigDone[c1]) + ptr[ copyEnd[c1]-- ] = k; } - ftab[sbn] |= SETMASK; } + AssertH ( copyStart[ss]-1 == copyEnd[ss], 1007 ); + + for (j = 0; j <= 255; j++) ftab[(j << 8) + ss] |= SETMASK; + /*-- Step 3: The [ss] big bucket is now done. Record this fact, @@ -950,7 +1033,7 @@ void mainSort ( UInt32* ptr, while ((bbSize >> shifts) > 65534) shifts++; - for (j = 0; j < bbSize; j++) { + for (j = bbSize-1; j >= 0; j--) { Int32 a2update = ptr[bbStart + j]; UInt16 qVal = (UInt16)(j >> shifts); quadrant[a2update] = qVal; @@ -960,26 +1043,6 @@ void mainSort ( UInt32* ptr, AssertH ( ((bbSize-1) >> shifts) <= 65535, 1002 ); } - /*-- - Step 4: - Now scan this big bucket [ss] so as to synthesise the - sorted order for small buckets [t, ss] for all t != ss. - This will avoid doing Real Work in subsequent Step 1's. - --*/ - for (j = 0; j <= 255; j++) - copy[j] = ftab[(j << 8) + ss] & CLEARMASK; - - m = ftab[(ss+1) << 8] & CLEARMASK; - for (j = ftab[ss << 8] & CLEARMASK; j < m; j++) { - k = ptr[j] - 1; if (k < 0) k += nblock; - c1 = (UChar)(block[k] >> 8); - if ( ! bigDone[c1] ) { - ptr[copy[c1]] = k; - copy[c1] ++; - } - } - - for (j = 0; j <= 255; j++) ftab[(j << 8) + ss] |= SETMASK; } if (verb >= 4) @@ -996,19 +1059,19 @@ void mainSort ( UInt32* ptr, /* Pre: nblock > 0 arr2 exists for [0 .. nblock-1 +N_OVERSHOOT] - ((UInt16*)arr2) [0 .. nblock-1] [15:8] holds block + ((UChar*)arr2) [0 .. nblock-1] holds block arr1 exists for [0 .. nblock-1] Post: - ((UInt16*)arr2) [0 .. nblock-1] [15:8] holds block + ((UChar*)arr2) [0 .. nblock-1] holds block All other areas of block destroyed ftab [ 0 .. 65536 ] destroyed arr1 [0 .. nblock-1] holds sorted order */ -void blockSort ( EState* s ) +void BZ2_blockSort ( EState* s ) { UInt32* ptr = s->ptr; - UInt16* block = s->block; + UChar* block = s->block; UInt32* ftab = s->ftab; Int32 nblock = s->nblock; Int32 verb = s->verbosity; @@ -1019,10 +1082,16 @@ void blockSort ( EState* s ) Int32 i; if (nblock < 10000) { - for (i = 0; i < nblock; i++) block[i] <<= 8; fallbackSort ( s->arr1, s->arr2, ftab, nblock, verb ); } else { - quadrant = &(block[nblock+BZ_N_OVERSHOOT]); + /* Calculate the location for quadrant, remembering to get + the alignment right. Assumes that &(block[0]) is at least + 2-byte aligned -- this should be ok since block is really + the first section of arr2. + */ + i = nblock+BZ_N_OVERSHOOT; + if (i & 1) i++; + quadrant = (UInt16*)(&(block[i])); /* (wfact-1) / 3 puts the default-factor-30 transition point at very roughly the same place as diff --git a/bzip2.1 b/bzip2.1 index 99eda9b..7de54a0 100644 --- a/bzip2.1 +++ b/bzip2.1 @@ -1,7 +1,7 @@ .PU .TH bzip2 1 .SH NAME -bzip2, bunzip2 \- a block-sorting file compressor, v0.9.5 +bzip2, bunzip2 \- a block-sorting file compressor, v1.0 .br bzcat \- decompresses files to stdout .br @@ -397,11 +397,12 @@ I/O error messages are not as helpful as they could be. tries hard to detect I/O errors and exit cleanly, but the details of what the problem is sometimes seem rather misleading. -This manual page pertains to version 0.9.5 of +This manual page pertains to version 1.0 of .I bzip2. Compressed data created by this version is entirely forwards and backwards -compatible with the previous public releases, versions 0.1pl2 and 0.9.0, +compatible with the previous public releases, versions 0.1pl2, 0.9.0 +and 0.9.5, but with the following exception: 0.9.0 and above can correctly decompress multiple concatenated compressed files. 0.1pl2 cannot do this; it will stop after decompressing just the first file in the @@ -415,6 +416,7 @@ megabytes long. This could easily be fixed. .SH AUTHOR Julian Seward, jseward@acm.org. +http://sourceware.cygnus.com/bzip2 http://www.muraroa.demon.co.uk The ideas embodied in diff --git a/bzip2.1.preformatted b/bzip2.1.preformatted index 96b44be..9f18339 100644 --- a/bzip2.1.preformatted +++ b/bzip2.1.preformatted @@ -1,7 +1,11 @@ + +bzip2(1) bzip2(1) + + NNAAMMEE - bzip2, bunzip2 - a block-sorting file compressor, v0.9.5 + bzip2, bunzip2 - a block-sorting file compressor, v1.0 bzcat - decompresses files to stdout bzip2recover - recovers data from damaged bzip2 files @@ -54,6 +58,18 @@ DDEESSCCRRIIPPTTIIOONN filename.bz2 becomes filename filename.bz becomes filename filename.tbz2 becomes filename.tar + + + + 1 + + + + + +bzip2(1) bzip2(1) + + filename.tbz becomes filename.tar anyothername becomes anyothername.out @@ -109,6 +125,17 @@ DDEESSCCRRIIPPTTIIOONN you recover the original uncompressed data. You can use _b_z_i_p_2_r_e_c_o_v_e_r to try to recover data from damaged files. + + + 2 + + + + + +bzip2(1) bzip2(1) + + Return values: 0 for a normal exit, 1 for environmental problems (file not found, invalid flags, I/O errors, &c), 2 to indicate a corrupt compressed file, 3 for an internal @@ -163,6 +190,18 @@ OOPPTTIIOONNSS --qq ----qquuiieett Suppress non-essential warning messages. Messages pertaining to I/O errors and other critical events + + + + 3 + + + + + +bzip2(1) bzip2(1) + + will not be suppressed. --vv ----vveerrbboossee @@ -217,6 +256,18 @@ MMEEMMOORRYY MMAANNAAGGEEMMEENNTT Larger block sizes give rapidly diminishing marginal returns. Most of the compression comes from the first two + + + + 4 + + + + + +bzip2(1) bzip2(1) + + or three hundred k of block size, a fact worth bearing in mind when using _b_z_i_p_2 on small machines. It is also important to appreciate that the decompression memory @@ -270,6 +321,19 @@ MMEEMMOORRYY MMAANNAAGGEEMMEENNTT -9 7600k 3700k 2350k 828642 + + + + + 5 + + + + + +bzip2(1) bzip2(1) + + RREECCOOVVEERRIINNGG DDAATTAA FFRROOMM DDAAMMAAGGEEDD FFIILLEESS _b_z_i_p_2 compresses files in blocks, usually 900kbytes long. Each block is handled independently. If a media or trans- @@ -324,6 +388,18 @@ PPEERRFFOORRMMAANNCCEE NNOOTTEESS operate in, and then charges all over it in a fairly ran- dom fashion. This means that performance, both for com- pressing and decompressing, is largely determined by the + + + + 6 + + + + + +bzip2(1) bzip2(1) + + speed at which your machine can service cache misses. Because of this, small changes to the code to reduce the miss rate have been observed to give disproportionately @@ -337,14 +413,14 @@ CCAAVVEEAATTSS but the details of what the problem is sometimes seem rather misleading. - This manual page pertains to version 0.9.5 of _b_z_i_p_2_. Com- + This manual page pertains to version 1.0 of _b_z_i_p_2_. Com- pressed data created by this version is entirely forwards and backwards compatible with the previous public - releases, versions 0.1pl2 and 0.9.0, but with the follow- - ing exception: 0.9.0 and above can correctly decompress - multiple concatenated compressed files. 0.1pl2 cannot do - this; it will stop after decompressing just the first file - in the stream. + releases, versions 0.1pl2, 0.9.0 and 0.9.5, but with the + following exception: 0.9.0 and above can correctly decom- + press multiple concatenated compressed files. 0.1pl2 can- + not do this; it will stop after decompressing just the + first file in the stream. _b_z_i_p_2_r_e_c_o_v_e_r uses 32-bit integers to represent bit posi- tions in compressed files, so it cannot handle compressed @@ -355,21 +431,32 @@ CCAAVVEEAATTSS AAUUTTHHOORR Julian Seward, jseward@acm.org. + http://sourceware.cygnus.com/bzip2 http://www.muraroa.demon.co.uk The ideas embodied in _b_z_i_p_2 are due to (at least) the fol- - lowing people: Michael Burrows and David Wheeler (for the - block sorting transformation), David Wheeler (again, for + lowing people: Michael Burrows and David Wheeler (for the + block sorting transformation), David Wheeler (again, for the Huffman coder), Peter Fenwick (for the structured cod- ing model in the original _b_z_i_p_, and many refinements), and - Alistair Moffat, Radford Neal and Ian Witten (for the + Alistair Moffat, Radford Neal and Ian Witten (for the arithmetic coder in the original _b_z_i_p_)_. I am much indebted for their help, support and advice. See the man- - ual in the source distribution for pointers to sources of + ual in the source distribution for pointers to sources of documentation. Christian von Roques encouraged me to look - for faster sorting algorithms, so as to speed up compres- + for faster sorting algorithms, so as to speed up compres- sion. Bela Lubkin encouraged me to improve the worst-case compression performance. Many people sent patches, helped - with portability problems, lent machines, gave advice and + with portability problems, lent machines, gave advice and were generally helpful. + + + + + + + + 7 + + diff --git a/bzip2.c b/bzip2.c index abb9530..56adfdc 100644 --- a/bzip2.c +++ b/bzip2.c @@ -7,7 +7,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-1999 Julian R Seward. All rights reserved. + Copyright (C) 1996-2000 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -42,7 +42,7 @@ Julian Seward, Cambridge, UK. jseward@acm.org - bzip2/libbzip2 version 0.9.5 of 24 May 1999 + bzip2/libbzip2 version 1.0 of 21 March 2000 This program is based on (at least) the work of: Mike Burrows @@ -123,10 +123,10 @@ --*/ #define BZ_LCCWIN32 0 -#if defined(_WIN32) && !defined(__CYGWIN32__) -#undef BZ_LCCWIN32 +#if defined(_WIN32) && !defined(__CYGWIN__) +#undef BZ_LCCWIN32 #define BZ_LCCWIN32 1 -#undef BZ_UNIX +#undef BZ_UNIX #define BZ_UNIX 0 #endif @@ -193,6 +193,17 @@ ERROR_IF_MINUS_ONE ( retVal ); \ } while ( 0 ) # endif +# ifdef __CYGWIN__ +# include +# include +# undef SET_BINARY_MODE +# define SET_BINARY_MODE(fd) \ + do { \ + int retVal = setmode ( fileno ( fd ), \ + O_BINARY ); \ + ERROR_IF_MINUS_ONE ( retVal ); \ + } while ( 0 ) +# endif #endif @@ -276,10 +287,10 @@ typedef int IntNative; /*---------------------------------------------------*/ Int32 verbosity; -Bool keepInputFiles, smallMode; -Bool forceOverwrite, testFailsExist, noisy; +Bool keepInputFiles, smallMode, deleteOutputOnInterrupt; +Bool forceOverwrite, testFailsExist, unzFailsExist, noisy; Int32 numFileNames, numFilesProcessed, blockSize100k; - +Int32 exitValue; /*-- source modes; F==file, I==stdin, O==stdout --*/ #define SM_I2O 1 @@ -305,27 +316,204 @@ Char progNameReally[FILE_NAME_LEN]; FILE *outputHandleJustInCase; Int32 workFactor; -void panic ( Char* ) NORETURN; -void ioError ( void ) NORETURN; -void outOfMemory ( void ) NORETURN; -void blockOverrun ( void ) NORETURN; -void badBlockHeader ( void ) NORETURN; -void badBGLengths ( void ) NORETURN; -void crcError ( void ) NORETURN; -void bitStreamEOF ( void ) NORETURN; -void cleanUpAndFail ( Int32 ) NORETURN; -void compressedStreamEOF ( void ) NORETURN; +static void panic ( Char* ) NORETURN; +static void ioError ( void ) NORETURN; +static void outOfMemory ( void ) NORETURN; +static void configError ( void ) NORETURN; +static void crcError ( void ) NORETURN; +static void cleanUpAndFail ( Int32 ) NORETURN; +static void compressedStreamEOF ( void ) NORETURN; -void copyFileName ( Char*, Char* ); -void* myMalloc ( Int32 ); +static void copyFileName ( Char*, Char* ); +static void* myMalloc ( Int32 ); +/*---------------------------------------------------*/ +/*--- An implementation of 64-bit ints. Sigh. ---*/ +/*--- Roll on widespread deployment of ANSI C9X ! ---*/ +/*---------------------------------------------------*/ + +typedef + struct { UChar b[8]; } + UInt64; + +static +void uInt64_from_UInt32s ( UInt64* n, UInt32 lo32, UInt32 hi32 ) +{ + n->b[7] = (UChar)((hi32 >> 24) & 0xFF); + n->b[6] = (UChar)((hi32 >> 16) & 0xFF); + n->b[5] = (UChar)((hi32 >> 8) & 0xFF); + n->b[4] = (UChar) (hi32 & 0xFF); + n->b[3] = (UChar)((lo32 >> 24) & 0xFF); + n->b[2] = (UChar)((lo32 >> 16) & 0xFF); + n->b[1] = (UChar)((lo32 >> 8) & 0xFF); + n->b[0] = (UChar) (lo32 & 0xFF); +} + +static +double uInt64_to_double ( UInt64* n ) +{ + Int32 i; + double base = 1.0; + double sum = 0.0; + for (i = 0; i < 8; i++) { + sum += base * (double)(n->b[i]); + base *= 256.0; + } + return sum; +} + +static +void uInt64_add ( UInt64* src, UInt64* dst ) +{ + Int32 i; + Int32 carry = 0; + for (i = 0; i < 8; i++) { + carry += ( ((Int32)src->b[i]) + ((Int32)dst->b[i]) ); + dst->b[i] = (UChar)(carry & 0xFF); + carry >>= 8; + } +} + +static +void uInt64_sub ( UInt64* src, UInt64* dst ) +{ + Int32 t, i; + Int32 borrow = 0; + for (i = 0; i < 8; i++) { + t = ((Int32)dst->b[i]) - ((Int32)src->b[i]) - borrow; + if (t < 0) { + dst->b[i] = (UChar)(t + 256); + borrow = 1; + } else { + dst->b[i] = (UChar)t; + borrow = 0; + } + } +} + +static +void uInt64_mul ( UInt64* a, UInt64* b, UInt64* r_hi, UInt64* r_lo ) +{ + UChar sum[16]; + Int32 ia, ib, carry; + for (ia = 0; ia < 16; ia++) sum[ia] = 0; + for (ia = 0; ia < 8; ia++) { + carry = 0; + for (ib = 0; ib < 8; ib++) { + carry += ( ((Int32)sum[ia+ib]) + + ((Int32)a->b[ia]) * ((Int32)b->b[ib]) ); + sum[ia+ib] = (UChar)(carry & 0xFF); + carry >>= 8; + } + sum[ia+8] = (UChar)(carry & 0xFF); + if ((carry >>= 8) != 0) panic ( "uInt64_mul" ); + } + + for (ia = 0; ia < 8; ia++) r_hi->b[ia] = sum[ia+8]; + for (ia = 0; ia < 8; ia++) r_lo->b[ia] = sum[ia]; +} + + +static +void uInt64_shr1 ( UInt64* n ) +{ + Int32 i; + for (i = 0; i < 8; i++) { + n->b[i] >>= 1; + if (i < 7 && (n->b[i+1] & 1)) n->b[i] |= 0x80; + } +} + +static +void uInt64_shl1 ( UInt64* n ) +{ + Int32 i; + for (i = 7; i >= 0; i--) { + n->b[i] <<= 1; + if (i > 0 && (n->b[i-1] & 0x80)) n->b[i]++; + } +} + +static +Bool uInt64_isZero ( UInt64* n ) +{ + Int32 i; + for (i = 0; i < 8; i++) + if (n->b[i] != 0) return 0; + return 1; +} + +static +Int32 uInt64_qrm10 ( UInt64* n ) +{ + /* Divide *n by 10, and return the remainder. Long division + is difficult, so we cheat and instead multiply by + 0xCCCC CCCC CCCC CCCD, which is 0.8 (viz, 0.1 << 3). + */ + Int32 i; + UInt64 tmp1, tmp2, n_orig, zero_point_eight; + + zero_point_eight.b[1] = zero_point_eight.b[2] = + zero_point_eight.b[3] = zero_point_eight.b[4] = + zero_point_eight.b[5] = zero_point_eight.b[6] = + zero_point_eight.b[7] = 0xCC; + zero_point_eight.b[0] = 0xCD; + + n_orig = *n; + + /* divide n by 10, + by multiplying by 0.8 and then shifting right 3 times */ + uInt64_mul ( n, &zero_point_eight, &tmp1, &tmp2 ); + uInt64_shr1(&tmp1); uInt64_shr1(&tmp1); uInt64_shr1(&tmp1); + *n = tmp1; + + /* tmp1 = 8*n, tmp2 = 2*n */ + uInt64_shl1(&tmp1); uInt64_shl1(&tmp1); uInt64_shl1(&tmp1); + tmp2 = *n; uInt64_shl1(&tmp2); + + /* tmp1 = 10*n */ + uInt64_add ( &tmp2, &tmp1 ); + + /* n_orig = n_orig - 10*n */ + uInt64_sub ( &tmp1, &n_orig ); + + /* n_orig should now hold quotient, in range 0 .. 9 */ + for (i = 7; i >= 1; i--) + if (n_orig.b[i] != 0) panic ( "uInt64_qrm10(1)" ); + if (n_orig.b[0] > 9) + panic ( "uInt64_qrm10(2)" ); + + return (int)n_orig.b[0]; +} + +/* ... and the Whole Entire Point of all this UInt64 stuff is + so that we can supply the following function. +*/ +static +void uInt64_toAscii ( char* outbuf, UInt64* n ) +{ + Int32 i, q; + UChar buf[32]; + Int32 nBuf = 0; + UInt64 n_copy = *n; + do { + q = uInt64_qrm10 ( &n_copy ); + buf[nBuf] = q + '0'; + nBuf++; + } while (!uInt64_isZero(&n_copy)); + outbuf[nBuf] = 0; + for (i = 0; i < nBuf; i++) outbuf[i] = buf[nBuf-i-1]; +} + + /*---------------------------------------------------*/ /*--- Processing of complete files and streams ---*/ /*---------------------------------------------------*/ /*---------------------------------------------*/ +static Bool myfeof ( FILE* f ) { Int32 c = fgetc ( f ); @@ -336,12 +524,14 @@ Bool myfeof ( FILE* f ) /*---------------------------------------------*/ +static void compressStream ( FILE *stream, FILE *zStream ) { BZFILE* bzf = NULL; UChar ibuf[5000]; Int32 nIbuf; - UInt32 nbytes_in, nbytes_out; + UInt32 nbytes_in_lo32, nbytes_in_hi32; + UInt32 nbytes_out_lo32, nbytes_out_hi32; Int32 bzerr, bzerr_dummy, ret; SET_BINARY_MODE(stream); @@ -350,8 +540,8 @@ void compressStream ( FILE *stream, FILE *zStream ) if (ferror(stream)) goto errhandler_io; if (ferror(zStream)) goto errhandler_io; - bzf = bzWriteOpen ( &bzerr, zStream, - blockSize100k, verbosity, workFactor ); + bzf = BZ2_bzWriteOpen ( &bzerr, zStream, + blockSize100k, verbosity, workFactor ); if (bzerr != BZ_OK) goto errhandler; if (verbosity >= 2) fprintf ( stderr, "\n" ); @@ -361,12 +551,14 @@ void compressStream ( FILE *stream, FILE *zStream ) if (myfeof(stream)) break; nIbuf = fread ( ibuf, sizeof(UChar), 5000, stream ); if (ferror(stream)) goto errhandler_io; - if (nIbuf > 0) bzWrite ( &bzerr, bzf, (void*)ibuf, nIbuf ); + if (nIbuf > 0) BZ2_bzWrite ( &bzerr, bzf, (void*)ibuf, nIbuf ); if (bzerr != BZ_OK) goto errhandler; } - bzWriteClose ( &bzerr, bzf, 0, &nbytes_in, &nbytes_out ); + BZ2_bzWriteClose64 ( &bzerr, bzf, 0, + &nbytes_in_lo32, &nbytes_in_hi32, + &nbytes_out_lo32, &nbytes_out_hi32 ); if (bzerr != BZ_OK) goto errhandler; if (ferror(zStream)) goto errhandler_io; @@ -380,25 +572,42 @@ void compressStream ( FILE *stream, FILE *zStream ) ret = fclose ( stream ); if (ret == EOF) goto errhandler_io; - if (nbytes_in == 0) nbytes_in = 1; + if (nbytes_in_lo32 == 0 && nbytes_in_hi32 == 0) + nbytes_in_lo32 = 1; - if (verbosity >= 1) + if (verbosity >= 1) { + Char buf_nin[32], buf_nout[32]; + UInt64 nbytes_in, nbytes_out; + double nbytes_in_d, nbytes_out_d; + uInt64_from_UInt32s ( &nbytes_in, + nbytes_in_lo32, nbytes_in_hi32 ); + uInt64_from_UInt32s ( &nbytes_out, + nbytes_out_lo32, nbytes_out_hi32 ); + nbytes_in_d = uInt64_to_double ( &nbytes_in ); + nbytes_out_d = uInt64_to_double ( &nbytes_out ); + uInt64_toAscii ( buf_nin, &nbytes_in ); + uInt64_toAscii ( buf_nout, &nbytes_out ); fprintf ( stderr, "%6.3f:1, %6.3f bits/byte, " - "%5.2f%% saved, %d in, %d out.\n", - (float)nbytes_in / (float)nbytes_out, - (8.0 * (float)nbytes_out) / (float)nbytes_in, - 100.0 * (1.0 - (float)nbytes_out / (float)nbytes_in), - nbytes_in, - nbytes_out + "%5.2f%% saved, %s in, %s out.\n", + nbytes_in_d / nbytes_out_d, + (8.0 * nbytes_out_d) / nbytes_in_d, + 100.0 * (1.0 - nbytes_out_d / nbytes_in_d), + buf_nin, + buf_nout ); + } return; errhandler: - bzWriteClose ( &bzerr_dummy, bzf, 1, &nbytes_in, &nbytes_out ); + BZ2_bzWriteClose64 ( &bzerr_dummy, bzf, 1, + &nbytes_in_lo32, &nbytes_in_hi32, + &nbytes_out_lo32, &nbytes_out_hi32 ); switch (bzerr) { + case BZ_CONFIG_ERROR: + configError(); break; case BZ_MEM_ERROR: - outOfMemory (); + outOfMemory (); break; case BZ_IO_ERROR: errhandler_io: ioError(); break; @@ -413,6 +622,7 @@ void compressStream ( FILE *stream, FILE *zStream ) /*---------------------------------------------*/ +static Bool uncompressStream ( FILE *zStream, FILE *stream ) { BZFILE* bzf = NULL; @@ -433,7 +643,7 @@ Bool uncompressStream ( FILE *zStream, FILE *stream ) while (True) { - bzf = bzReadOpen ( + bzf = BZ2_bzReadOpen ( &bzerr, zStream, verbosity, (int)smallMode, unused, nUnused ); @@ -441,7 +651,7 @@ Bool uncompressStream ( FILE *zStream, FILE *stream ) streamNo++; while (bzerr == BZ_OK) { - nread = bzRead ( &bzerr, bzf, obuf, 5000 ); + nread = BZ2_bzRead ( &bzerr, bzf, obuf, 5000 ); if (bzerr == BZ_DATA_ERROR_MAGIC) goto errhandler; if ((bzerr == BZ_OK || bzerr == BZ_STREAM_END) && nread > 0) fwrite ( obuf, sizeof(UChar), nread, stream ); @@ -449,12 +659,12 @@ Bool uncompressStream ( FILE *zStream, FILE *stream ) } if (bzerr != BZ_STREAM_END) goto errhandler; - bzReadGetUnused ( &bzerr, bzf, (void**)(&unusedTmp), &nUnused ); + BZ2_bzReadGetUnused ( &bzerr, bzf, (void**)(&unusedTmp), &nUnused ); if (bzerr != BZ_OK) panic ( "decompress:bzReadGetUnused" ); for (i = 0; i < nUnused; i++) unused[i] = unusedTmp[i]; - bzReadClose ( &bzerr, bzf ); + BZ2_bzReadClose ( &bzerr, bzf ); if (bzerr != BZ_OK) panic ( "decompress:bzReadGetUnused" ); if (nUnused == 0 && myfeof(zStream)) break; @@ -476,8 +686,10 @@ Bool uncompressStream ( FILE *zStream, FILE *stream ) return True; errhandler: - bzReadClose ( &bzerr_dummy, bzf ); + BZ2_bzReadClose ( &bzerr_dummy, bzf ); switch (bzerr) { + case BZ_CONFIG_ERROR: + configError(); break; case BZ_IO_ERROR: errhandler_io: ioError(); break; @@ -488,6 +700,8 @@ Bool uncompressStream ( FILE *zStream, FILE *stream ) case BZ_UNEXPECTED_EOF: compressedStreamEOF(); case BZ_DATA_ERROR_MAGIC: + if (zStream != stdin) fclose(zStream); + if (stream != stdout) fclose(stream); if (streamNo == 1) { return False; } else { @@ -507,6 +721,7 @@ Bool uncompressStream ( FILE *zStream, FILE *stream ) /*---------------------------------------------*/ +static Bool testStream ( FILE *zStream ) { BZFILE* bzf = NULL; @@ -524,7 +739,7 @@ Bool testStream ( FILE *zStream ) while (True) { - bzf = bzReadOpen ( + bzf = BZ2_bzReadOpen ( &bzerr, zStream, verbosity, (int)smallMode, unused, nUnused ); @@ -532,17 +747,17 @@ Bool testStream ( FILE *zStream ) streamNo++; while (bzerr == BZ_OK) { - nread = bzRead ( &bzerr, bzf, obuf, 5000 ); + nread = BZ2_bzRead ( &bzerr, bzf, obuf, 5000 ); if (bzerr == BZ_DATA_ERROR_MAGIC) goto errhandler; } if (bzerr != BZ_STREAM_END) goto errhandler; - bzReadGetUnused ( &bzerr, bzf, (void**)(&unusedTmp), &nUnused ); + BZ2_bzReadGetUnused ( &bzerr, bzf, (void**)(&unusedTmp), &nUnused ); if (bzerr != BZ_OK) panic ( "test:bzReadGetUnused" ); for (i = 0; i < nUnused; i++) unused[i] = unusedTmp[i]; - bzReadClose ( &bzerr, bzf ); + BZ2_bzReadClose ( &bzerr, bzf ); if (bzerr != BZ_OK) panic ( "test:bzReadGetUnused" ); if (nUnused == 0 && myfeof(zStream)) break; @@ -556,10 +771,12 @@ Bool testStream ( FILE *zStream ) return True; errhandler: - bzReadClose ( &bzerr_dummy, bzf ); + BZ2_bzReadClose ( &bzerr_dummy, bzf ); if (verbosity == 0) fprintf ( stderr, "%s: %s: ", progName, inName ); switch (bzerr) { + case BZ_CONFIG_ERROR: + configError(); break; case BZ_IO_ERROR: errhandler_io: ioError(); break; @@ -574,6 +791,7 @@ Bool testStream ( FILE *zStream ) "file ends unexpectedly\n" ); return False; case BZ_DATA_ERROR_MAGIC: + if (zStream != stdin) fclose(zStream); if (streamNo == 1) { fprintf ( stderr, "bad magic number (file not created by bzip2)\n" ); @@ -598,6 +816,15 @@ Bool testStream ( FILE *zStream ) /*---------------------------------------------------*/ /*---------------------------------------------*/ +static +void setExit ( Int32 v ) +{ + if (v > exitValue) exitValue = v; +} + + +/*---------------------------------------------*/ +static void cadvise ( void ) { if (noisy) @@ -612,6 +839,7 @@ void cadvise ( void ) /*---------------------------------------------*/ +static void showFileNames ( void ) { if (noisy) @@ -624,11 +852,14 @@ void showFileNames ( void ) /*---------------------------------------------*/ +static void cleanUpAndFail ( Int32 ec ) { IntNative retVal; - if ( srcMode == SM_F2F && opMode != OM_TEST ) { + if ( srcMode == SM_F2F + && opMode != OM_TEST + && deleteOutputOnInterrupt ) { if (noisy) fprintf ( stderr, "%s: Deleting output file %s, if it exists.\n", progName, outName ); @@ -647,11 +878,13 @@ void cleanUpAndFail ( Int32 ec ) progName, numFileNames, numFileNames - numFilesProcessed ); } - exit ( ec ); + setExit(ec); + exit(exitValue); } /*---------------------------------------------*/ +static void panic ( Char* s ) { fprintf ( stderr, @@ -666,6 +899,7 @@ void panic ( Char* s ) /*---------------------------------------------*/ +static void crcError ( void ) { fprintf ( stderr, @@ -678,6 +912,7 @@ void crcError ( void ) /*---------------------------------------------*/ +static void compressedStreamEOF ( void ) { fprintf ( stderr, @@ -692,10 +927,12 @@ void compressedStreamEOF ( void ) /*---------------------------------------------*/ +static void ioError ( void ) { fprintf ( stderr, - "\n%s: I/O or other error, bailing out. Possible reason follows.\n", + "\n%s: I/O or other error, bailing out. " + "Possible reason follows.\n", progName ); perror ( progName ); showFileNames(); @@ -704,6 +941,7 @@ void ioError ( void ) /*---------------------------------------------*/ +static void mySignalCatcher ( IntNative n ) { fprintf ( stderr, @@ -714,20 +952,53 @@ void mySignalCatcher ( IntNative n ) /*---------------------------------------------*/ +static void mySIGSEGVorSIGBUScatcher ( IntNative n ) { if (opMode == OM_Z) - fprintf ( stderr, - "\n%s: Caught a SIGSEGV or SIGBUS whilst compressing,\n" - "\twhich probably indicates a bug in bzip2. Please\n" - "\treport it to me at: jseward@acm.org\n", - progName ); + fprintf ( + stderr, + "\n%s: Caught a SIGSEGV or SIGBUS whilst compressing.\n" + "\n" + " Possible causes are (most likely first):\n" + " (1) This computer has unreliable memory or cache hardware\n" + " (a surprisingly common problem; try a different machine.)\n" + " (2) A bug in the compiler used to create this executable\n" + " (unlikely, if you didn't compile bzip2 yourself.)\n" + " (3) A real bug in bzip2 -- I hope this should never be the case.\n" + " The user's manual, Section 4.3, has more info on (1) and (2).\n" + " \n" + " If you suspect this is a bug in bzip2, or are unsure about (1)\n" + " or (2), feel free to report it to me at: jseward@acm.org.\n" + " Section 4.3 of the user's manual describes the info a useful\n" + " bug report should have. If the manual is available on your\n" + " system, please try and read it before mailing me. If you don't\n" + " have the manual or can't be bothered to read it, mail me anyway.\n" + "\n", + progName ); else - fprintf ( stderr, - "\n%s: Caught a SIGSEGV or SIGBUS whilst decompressing,\n" - "\twhich probably indicates that the compressed data\n" - "\tis corrupted.\n", - progName ); + fprintf ( + stderr, + "\n%s: Caught a SIGSEGV or SIGBUS whilst decompressing.\n" + "\n" + " Possible causes are (most likely first):\n" + " (1) The compressed data is corrupted, and bzip2's usual checks\n" + " failed to detect this. Try bzip2 -tvv my_file.bz2.\n" + " (2) This computer has unreliable memory or cache hardware\n" + " (a surprisingly common problem; try a different machine.)\n" + " (3) A bug in the compiler used to create this executable\n" + " (unlikely, if you didn't compile bzip2 yourself.)\n" + " (4) A real bug in bzip2 -- I hope this should never be the case.\n" + " The user's manual, Section 4.3, has more info on (2) and (3).\n" + " \n" + " If you suspect this is a bug in bzip2, or are unsure about (2)\n" + " or (3), feel free to report it to me at: jseward@acm.org.\n" + " Section 4.3 of the user's manual describes the info a useful\n" + " bug report should have. If the manual is available on your\n" + " system, please try and read it before mailing me. If you don't\n" + " have the manual or can't be bothered to read it, mail me anyway.\n" + "\n", + progName ); showFileNames(); if (opMode == OM_Z) @@ -737,6 +1008,7 @@ void mySIGSEGVorSIGBUScatcher ( IntNative n ) /*---------------------------------------------*/ +static void outOfMemory ( void ) { fprintf ( stderr, @@ -747,11 +1019,27 @@ void outOfMemory ( void ) } +/*---------------------------------------------*/ +static +void configError ( void ) +{ + fprintf ( stderr, + "bzip2: I'm not configured correctly for this platform!\n" + "\tI require Int32, Int16 and Char to have sizes\n" + "\tof 4, 2 and 1 bytes to run properly, and they don't.\n" + "\tProbably you can fix this by defining them correctly,\n" + "\tand recompiling. Bye!\n" ); + setExit(3); + exit(exitValue); +} + + /*---------------------------------------------------*/ /*--- The main driver machinery ---*/ /*---------------------------------------------------*/ /*---------------------------------------------*/ +static void pad ( Char *s ) { Int32 i; @@ -762,6 +1050,7 @@ void pad ( Char *s ) /*---------------------------------------------*/ +static void copyFileName ( Char* to, Char* from ) { if ( strlen(from) > FILE_NAME_LEN-10 ) { @@ -772,7 +1061,8 @@ void copyFileName ( Char* to, Char* from ) "Try using a reasonable file name instead. Sorry! :-)\n", from, FILE_NAME_LEN-10 ); - exit(1); + setExit(1); + exit(exitValue); } strncpy(to,from,FILE_NAME_LEN-10); @@ -781,6 +1071,7 @@ void copyFileName ( Char* to, Char* from ) /*---------------------------------------------*/ +static Bool fileExists ( Char* name ) { FILE *tmp = fopen ( name, "rb" ); @@ -794,6 +1085,7 @@ Bool fileExists ( Char* name ) /*-- if in doubt, return True --*/ +static Bool notAStandardFile ( Char* name ) { IntNative i; @@ -810,6 +1102,7 @@ Bool notAStandardFile ( Char* name ) /*-- rac 11/21/98 see if file has hard links to it --*/ +static Int32 countHardLinks ( Char* name ) { IntNative i; @@ -822,6 +1115,7 @@ Int32 countHardLinks ( Char* name ) /*---------------------------------------------*/ +static void copyDatePermissionsAndOwner ( Char *srcName, Char *dstName ) { #if BZ_UNIX @@ -849,6 +1143,7 @@ void copyDatePermissionsAndOwner ( Char *srcName, Char *dstName ) /*---------------------------------------------*/ +static void setInterimPermissions ( Char *dstName ) { #if BZ_UNIX @@ -860,6 +1155,7 @@ void setInterimPermissions ( Char *dstName ) /*---------------------------------------------*/ +static Bool containsDubiousChars ( Char* name ) { Bool cdc = False; @@ -877,6 +1173,7 @@ Char* zSuffix[BZ_N_SUFFIX_PAIRS] Char* unzSuffix[BZ_N_SUFFIX_PAIRS] = { "", "", ".tar", ".tar" }; +static Bool hasSuffix ( Char* s, Char* suffix ) { Int32 ns = strlen(s); @@ -886,6 +1183,7 @@ Bool hasSuffix ( Char* s, Char* suffix ) return False; } +static Bool mapSuffix ( Char* name, Char* oldSuffix, Char* newSuffix ) { @@ -897,11 +1195,15 @@ Bool mapSuffix ( Char* name, /*---------------------------------------------*/ +static void compress ( Char *name ) { FILE *inStr; FILE *outStr; Int32 n, i; + + deleteOutputOnInterrupt = False; + if (name == NULL && srcMode != SM_I2O) panic ( "compress: bad modes\n" ); @@ -924,12 +1226,14 @@ void compress ( Char *name ) if ( srcMode != SM_I2O && containsDubiousChars ( inName ) ) { if (noisy) fprintf ( stderr, "%s: There are no files matching `%s'.\n", - progName, inName ); + progName, inName ); + setExit(1); return; } if ( srcMode != SM_I2O && !fileExists ( inName ) ) { fprintf ( stderr, "%s: Can't open input file %s: %s.\n", progName, inName, strerror(errno) ); + setExit(1); return; } for (i = 0; i < BZ_N_SUFFIX_PAIRS; i++) { @@ -938,6 +1242,7 @@ void compress ( Char *name ) fprintf ( stderr, "%s: Input file %s already has %s suffix.\n", progName, inName, zSuffix[i] ); + setExit(1); return; } } @@ -945,17 +1250,20 @@ void compress ( Char *name ) if (noisy) fprintf ( stderr, "%s: Input file %s is not a normal file.\n", progName, inName ); + setExit(1); return; } if ( srcMode == SM_F2F && !forceOverwrite && fileExists ( outName ) ) { fprintf ( stderr, "%s: Output file %s already exists.\n", progName, outName ); + setExit(1); return; } if ( srcMode == SM_F2F && !forceOverwrite && (n=countHardLinks ( inName )) > 0) { fprintf ( stderr, "%s: Input file %s has %d other link%s.\n", progName, inName, n, n > 1 ? "s" : "" ); + setExit(1); return; } @@ -970,6 +1278,7 @@ void compress ( Char *name ) progName ); fprintf ( stderr, "%s: For help, type: `%s --help'.\n", progName, progName ); + setExit(1); return; }; break; @@ -984,11 +1293,13 @@ void compress ( Char *name ) fprintf ( stderr, "%s: For help, type: `%s --help'.\n", progName, progName ); if ( inStr != NULL ) fclose ( inStr ); + setExit(1); return; }; if ( inStr == NULL ) { fprintf ( stderr, "%s: Can't open input file %s: %s.\n", progName, inName, strerror(errno) ); + setExit(1); return; }; break; @@ -1000,12 +1311,14 @@ void compress ( Char *name ) fprintf ( stderr, "%s: Can't create output file %s: %s.\n", progName, outName, strerror(errno) ); if ( inStr != NULL ) fclose ( inStr ); + setExit(1); return; } if ( inStr == NULL ) { fprintf ( stderr, "%s: Can't open input file %s: %s.\n", progName, inName, strerror(errno) ); if ( outStr != NULL ) fclose ( outStr ); + setExit(1); return; }; setInterimPermissions ( outName ); @@ -1024,21 +1337,26 @@ void compress ( Char *name ) /*--- Now the input and output handles are sane. Do the Biz. ---*/ outputHandleJustInCase = outStr; + deleteOutputOnInterrupt = True; compressStream ( inStr, outStr ); outputHandleJustInCase = NULL; /*--- If there was an I/O error, we won't get here. ---*/ if ( srcMode == SM_F2F ) { copyDatePermissionsAndOwner ( inName, outName ); + deleteOutputOnInterrupt = False; if ( !keepInputFiles ) { IntNative retVal = remove ( inName ); ERROR_IF_NOT_ZERO ( retVal ); } } + + deleteOutputOnInterrupt = False; } /*---------------------------------------------*/ +static void uncompress ( Char *name ) { FILE *inStr; @@ -1047,6 +1365,8 @@ void uncompress ( Char *name ) Bool magicNumberOK; Bool cantGuess; + deleteOutputOnInterrupt = False; + if (name == NULL && srcMode != SM_I2O) panic ( "uncompress: bad modes\n" ); @@ -1076,17 +1396,20 @@ void uncompress ( Char *name ) if (noisy) fprintf ( stderr, "%s: There are no files matching `%s'.\n", progName, inName ); + setExit(1); return; } if ( srcMode != SM_I2O && !fileExists ( inName ) ) { fprintf ( stderr, "%s: Can't open input file %s: %s.\n", progName, inName, strerror(errno) ); + setExit(1); return; } if ( srcMode == SM_F2F && !forceOverwrite && notAStandardFile ( inName )) { if (noisy) fprintf ( stderr, "%s: Input file %s is not a normal file.\n", progName, inName ); + setExit(1); return; } if ( /* srcMode == SM_F2F implied && */ cantGuess ) { @@ -1099,12 +1422,14 @@ void uncompress ( Char *name ) if ( srcMode == SM_F2F && !forceOverwrite && fileExists ( outName ) ) { fprintf ( stderr, "%s: Output file %s already exists.\n", progName, outName ); + setExit(1); return; } if ( srcMode == SM_F2F && !forceOverwrite && (n=countHardLinks ( inName ) ) > 0) { fprintf ( stderr, "%s: Input file %s has %d other link%s.\n", progName, inName, n, n > 1 ? "s" : "" ); + setExit(1); return; } @@ -1119,6 +1444,7 @@ void uncompress ( Char *name ) progName ); fprintf ( stderr, "%s: For help, type: `%s --help'.\n", progName, progName ); + setExit(1); return; }; break; @@ -1130,6 +1456,7 @@ void uncompress ( Char *name ) fprintf ( stderr, "%s: Can't open input file %s:%s.\n", progName, inName, strerror(errno) ); if ( inStr != NULL ) fclose ( inStr ); + setExit(1); return; }; break; @@ -1141,12 +1468,14 @@ void uncompress ( Char *name ) fprintf ( stderr, "%s: Can't create output file %s: %s.\n", progName, outName, strerror(errno) ); if ( inStr != NULL ) fclose ( inStr ); + setExit(1); return; } if ( inStr == NULL ) { fprintf ( stderr, "%s: Can't open input file %s: %s.\n", progName, inName, strerror(errno) ); if ( outStr != NULL ) fclose ( outStr ); + setExit(1); return; }; setInterimPermissions ( outName ); @@ -1165,6 +1494,7 @@ void uncompress ( Char *name ) /*--- Now the input and output handles are sane. Do the Biz. ---*/ outputHandleJustInCase = outStr; + deleteOutputOnInterrupt = True; magicNumberOK = uncompressStream ( inStr, outStr ); outputHandleJustInCase = NULL; @@ -1172,22 +1502,27 @@ void uncompress ( Char *name ) if ( magicNumberOK ) { if ( srcMode == SM_F2F ) { copyDatePermissionsAndOwner ( inName, outName ); + deleteOutputOnInterrupt = False; if ( !keepInputFiles ) { IntNative retVal = remove ( inName ); ERROR_IF_NOT_ZERO ( retVal ); } } } else { + unzFailsExist = True; + deleteOutputOnInterrupt = False; if ( srcMode == SM_F2F ) { IntNative retVal = remove ( outName ); ERROR_IF_NOT_ZERO ( retVal ); } } + deleteOutputOnInterrupt = False; if ( magicNumberOK ) { if (verbosity >= 1) fprintf ( stderr, "done\n" ); } else { + setExit(2); if (verbosity >= 1) fprintf ( stderr, "not a bzip2 file.\n" ); else fprintf ( stderr, @@ -1199,11 +1534,14 @@ void uncompress ( Char *name ) /*---------------------------------------------*/ +static void testf ( Char *name ) { FILE *inStr; Bool allOK; + deleteOutputOnInterrupt = False; + if (name == NULL && srcMode != SM_I2O) panic ( "testf: bad modes\n" ); @@ -1218,11 +1556,13 @@ void testf ( Char *name ) if (noisy) fprintf ( stderr, "%s: There are no files matching `%s'.\n", progName, inName ); + setExit(1); return; } if ( srcMode != SM_I2O && !fileExists ( inName ) ) { fprintf ( stderr, "%s: Can't open input %s: %s.\n", progName, inName, strerror(errno) ); + setExit(1); return; } @@ -1235,6 +1575,7 @@ void testf ( Char *name ) progName ); fprintf ( stderr, "%s: For help, type: `%s --help'.\n", progName, progName ); + setExit(1); return; }; inStr = stdin; @@ -1245,6 +1586,7 @@ void testf ( Char *name ) if ( inStr == NULL ) { fprintf ( stderr, "%s: Can't open input file %s:%s.\n", progName, inName, strerror(errno) ); + setExit(1); return; }; break; @@ -1269,35 +1611,38 @@ void testf ( Char *name ) /*---------------------------------------------*/ +static void license ( void ) { fprintf ( stderr, "bzip2, a block-sorting file compressor. " - "Version 0.9.5d, 4-Sept-99.\n" + "Version %s.\n" " \n" - " Copyright (C) 1996, 1997, 1998, 1999 by Julian Seward.\n" + " Copyright (C) 1996-2000 by Julian Seward.\n" " \n" " This program is free software; you can redistribute it and/or modify\n" " it under the terms set out in the LICENSE file, which is included\n" - " in the bzip2-0.9.5 source distribution.\n" + " in the bzip2-1.0 source distribution.\n" " \n" " This program is distributed in the hope that it will be useful,\n" " but WITHOUT ANY WARRANTY; without even the implied warranty of\n" " MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\n" " LICENSE file for more details.\n" - " \n" + " \n", + BZ2_bzlibVersion() ); } /*---------------------------------------------*/ +static void usage ( Char *fullProgName ) { fprintf ( stderr, "bzip2, a block-sorting file compressor. " - "Version 0.9.5d, 4-Sept-99.\n" + "Version %s.\n" "\n usage: %s [flags and input files in any order]\n" "\n" " -h --help print this message\n" @@ -1326,12 +1671,14 @@ void usage ( Char *fullProgName ) #endif , + BZ2_bzlibVersion(), fullProgName ); } /*---------------------------------------------*/ +static void redundant ( Char* flag ) { fprintf ( @@ -1365,6 +1712,7 @@ typedef /*---------------------------------------------*/ +static void *myMalloc ( Int32 n ) { void* p; @@ -1376,6 +1724,7 @@ void *myMalloc ( Int32 n ) /*---------------------------------------------*/ +static Cell *mkCell ( void ) { Cell *c; @@ -1388,6 +1737,7 @@ Cell *mkCell ( void ) /*---------------------------------------------*/ +static Cell *snocString ( Cell *root, Char *name ) { if (root == NULL) { @@ -1405,6 +1755,7 @@ Cell *snocString ( Cell *root, Char *name ) /*---------------------------------------------*/ +static void addFlagsFromEnvVar ( Cell** argList, Char* varName ) { Int32 i, j, k; @@ -1445,16 +1796,8 @@ IntNative main ( IntNative argc, Char *argv[] ) /*-- Be really really really paranoid :-) --*/ if (sizeof(Int32) != 4 || sizeof(UInt32) != 4 || sizeof(Int16) != 2 || sizeof(UInt16) != 2 || - sizeof(Char) != 1 || sizeof(UChar) != 1) { - fprintf ( stderr, - "bzip2: I'm not configured correctly for this platform!\n" - "\tI require Int32, Int16 and Char to have sizes\n" - "\tof 4, 2 and 1 bytes to run properly, and they don't.\n" - "\tProbably you can fix this by defining them correctly,\n" - "\tand recompiling. Bye!\n" ); - exit(3); - } - + sizeof(Char) != 1 || sizeof(UChar) != 1) + configError(); /*-- Initialise --*/ outputHandleJustInCase = NULL; @@ -1465,9 +1808,12 @@ IntNative main ( IntNative argc, Char *argv[] ) verbosity = 0; blockSize100k = 9; testFailsExist = False; + unzFailsExist = False; numFileNames = 0; numFilesProcessed = 0; workFactor = 30; + deleteOutputOnInterrupt = False; + exitValue = 0; i = j = 0; /* avoid bogus warning from egcs-1.1.X */ /*-- Set up signal handlers for mem access errors --*/ @@ -1636,6 +1982,7 @@ IntNative main ( IntNative argc, Char *argv[] ) else if (opMode == OM_UNZ) { + unzFailsExist = False; if (srcMode == SM_I2O) { uncompress ( NULL ); } else { @@ -1647,6 +1994,10 @@ IntNative main ( IntNative argc, Char *argv[] ) uncompress ( aa->name ); } } + if (unzFailsExist) { + setExit(2); + exit(exitValue); + } } else { @@ -1668,7 +2019,8 @@ IntNative main ( IntNative argc, Char *argv[] ) "You can use the `bzip2recover' program to attempt to recover\n" "data from undamaged sections of corrupted files.\n\n" ); - exit(2); + setExit(2); + exit(exitValue); } } @@ -1678,12 +2030,12 @@ IntNative main ( IntNative argc, Char *argv[] ) aa = argList; while (aa != NULL) { Cell* aa2 = aa->link; - if (aa->name) free(aa->name); + if (aa->name != NULL) free(aa->name); free(aa); aa = aa2; } - return 0; + return exitValue; } diff --git a/bzip2.txt b/bzip2.txt index da23c64..4f1ae86 100644 --- a/bzip2.txt +++ b/bzip2.txt @@ -1,7 +1,7 @@ NAME - bzip2, bunzip2 - a block-sorting file compressor, v0.9.5 + bzip2, bunzip2 - a block-sorting file compressor, v1.0 bzcat - decompresses files to stdout bzip2recover - recovers data from damaged bzip2 files @@ -337,14 +337,14 @@ CAVEATS but the details of what the problem is sometimes seem rather misleading. - This manual page pertains to version 0.9.5 of bzip2. Com- + This manual page pertains to version 1.0 of bzip2. Com- pressed data created by this version is entirely forwards and backwards compatible with the previous public - releases, versions 0.1pl2 and 0.9.0, but with the follow- - ing exception: 0.9.0 and above can correctly decompress - multiple concatenated compressed files. 0.1pl2 cannot do - this; it will stop after decompressing just the first file - in the stream. + releases, versions 0.1pl2, 0.9.0 and 0.9.5, but with the + following exception: 0.9.0 and above can correctly decom- + press multiple concatenated compressed files. 0.1pl2 can- + not do this; it will stop after decompressing just the + first file in the stream. bzip2recover uses 32-bit integers to represent bit posi- tions in compressed files, so it cannot handle compressed @@ -355,6 +355,7 @@ CAVEATS AUTHOR Julian Seward, jseward@acm.org. + http://sourceware.cygnus.com/bzip2 http://www.muraroa.demon.co.uk The ideas embodied in bzip2 are due to (at least) the fol- diff --git a/bzip2recover.c b/bzip2recover.c index 1323b36..ba3d175 100644 --- a/bzip2recover.c +++ b/bzip2recover.c @@ -7,9 +7,9 @@ /*-- This program is bzip2recover, a program to attempt data salvage from damaged files created by the accompanying - bzip2-0.9.5 program. + bzip2-1.0 program. - Copyright (C) 1996-1999 Julian R Seward. All rights reserved. + Copyright (C) 1996-2000 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -44,7 +44,7 @@ Julian Seward, Cambridge, UK. jseward@acm.org - bzip2/libbzip2 version 0.9.5 of 24 May 1999 + bzip2/libbzip2 version 1.0 of 21 March 2000 --*/ /*-- @@ -282,7 +282,7 @@ Int32 main ( Int32 argc, Char** argv ) strcpy ( progName, argv[0] ); inFileName[0] = outFileName[0] = 0; - fprintf ( stderr, "bzip2recover 0.9.5d: extracts blocks from damaged .bz2 files.\n" ); + fprintf ( stderr, "bzip2recover 1.0: extracts blocks from damaged .bz2 files.\n" ); if (argc != 2) { fprintf ( stderr, "%s: usage is `%s damaged_file_name'.\n", diff --git a/bzlib.c b/bzlib.c index 24e8bd5..4a06d9f 100644 --- a/bzlib.c +++ b/bzlib.c @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-1999 Julian R Seward. All rights reserved. + Copyright (C) 1996-2000 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -43,7 +43,7 @@ Julian Seward, Cambridge, UK. jseward@acm.org - bzip2/libbzip2 version 0.9.5 of 24 May 1999 + bzip2/libbzip2 version 1.0 of 21 March 2000 This program is based on (at least) the work of: Mike Burrows @@ -83,24 +83,36 @@ /*---------------------------------------------------*/ #ifndef BZ_NO_STDIO -void bz__AssertH__fail ( int errcode ) +void BZ2_bz__AssertH__fail ( int errcode ) { fprintf(stderr, - "\n\nbzip2/libbzip2, v0.9.5d: internal error number %d.\n" - "This is a bug in bzip2/libbzip2, v0.9.5d. Please report\n" - "it to me at: jseward@acm.org. If this happened when\n" - "you were using some program which uses libbzip2 as a\n" + "\n\nbzip2/libbzip2: internal error number %d.\n" + "This is a bug in bzip2/libbzip2, %s.\n" + "Please report it to me at: jseward@acm.org. If this happened\n" + "when you were using some program which uses libbzip2 as a\n" "component, you should also report this bug to the author(s)\n" "of that program. Please make an effort to report this bug;\n" "timely and accurate bug reports eventually lead to higher\n" - "quality software. Thanks. Julian Seward, 4 Sept 1999.\n\n", - errcode + "quality software. Thanks. Julian Seward, 21 March 2000.\n\n", + errcode, + BZ2_bzlibVersion() ); exit(3); } #endif +/*---------------------------------------------------*/ +static +int bz_config_ok ( void ) +{ + if (sizeof(int) != 4) return 0; + if (sizeof(short) != 2) return 0; + if (sizeof(char) != 1) return 0; + return 1; +} + + /*---------------------------------------------------*/ static void* default_bzalloc ( void* opaque, Int32 items, Int32 size ) @@ -149,7 +161,7 @@ Bool isempty_RL ( EState* s ) /*---------------------------------------------------*/ -int BZ_API(bzCompressInit) +int BZ_API(BZ2_bzCompressInit) ( bz_stream* strm, int blockSize100k, int verbosity, @@ -158,6 +170,8 @@ int BZ_API(bzCompressInit) Int32 n; EState* s; + if (!bz_config_ok()) return BZ_CONFIG_ERROR; + if (strm == NULL || blockSize100k < 1 || blockSize100k > 9 || workFactor < 0 || workFactor > 250) @@ -197,14 +211,16 @@ int BZ_API(bzCompressInit) s->verbosity = verbosity; s->workFactor = workFactor; - s->block = (UInt16*)s->arr2; + s->block = (UChar*)s->arr2; s->mtfv = (UInt16*)s->arr1; s->zbits = NULL; s->ptr = (UInt32*)s->arr1; strm->state = s; - strm->total_in = 0; - strm->total_out = 0; + strm->total_in_lo32 = 0; + strm->total_in_hi32 = 0; + strm->total_out_lo32 = 0; + strm->total_out_hi32 = 0; init_RL ( s ); prepare_new_block ( s ); return BZ_OK; @@ -223,24 +239,24 @@ void add_pair_to_block ( EState* s ) s->inUse[s->state_in_ch] = True; switch (s->state_in_len) { case 1: - s->block[s->nblock] = (UInt16)ch; s->nblock++; + s->block[s->nblock] = (UChar)ch; s->nblock++; break; case 2: - s->block[s->nblock] = (UInt16)ch; s->nblock++; - s->block[s->nblock] = (UInt16)ch; s->nblock++; + s->block[s->nblock] = (UChar)ch; s->nblock++; + s->block[s->nblock] = (UChar)ch; s->nblock++; break; case 3: - s->block[s->nblock] = (UInt16)ch; s->nblock++; - s->block[s->nblock] = (UInt16)ch; s->nblock++; - s->block[s->nblock] = (UInt16)ch; s->nblock++; + s->block[s->nblock] = (UChar)ch; s->nblock++; + s->block[s->nblock] = (UChar)ch; s->nblock++; + s->block[s->nblock] = (UChar)ch; s->nblock++; break; default: s->inUse[s->state_in_len-4] = True; - s->block[s->nblock] = (UInt16)ch; s->nblock++; - s->block[s->nblock] = (UInt16)ch; s->nblock++; - s->block[s->nblock] = (UInt16)ch; s->nblock++; - s->block[s->nblock] = (UInt16)ch; s->nblock++; - s->block[s->nblock] = ((UInt16)(s->state_in_len-4)); + s->block[s->nblock] = (UChar)ch; s->nblock++; + s->block[s->nblock] = (UChar)ch; s->nblock++; + s->block[s->nblock] = (UChar)ch; s->nblock++; + s->block[s->nblock] = (UChar)ch; s->nblock++; + s->block[s->nblock] = ((UChar)(s->state_in_len-4)); s->nblock++; break; } @@ -266,7 +282,7 @@ void flush_RL ( EState* s ) UChar ch = (UChar)(zs->state_in_ch); \ BZ_UPDATE_CRC( zs->blockCRC, ch ); \ zs->inUse[zs->state_in_ch] = True; \ - zs->block[zs->nblock] = (UInt16)ch; \ + zs->block[zs->nblock] = (UChar)ch; \ zs->nblock++; \ zs->state_in_ch = zchh; \ } \ @@ -302,7 +318,8 @@ Bool copy_input_until_stop ( EState* s ) ADD_CHAR_TO_BLOCK ( s, (UInt32)(*((UChar*)(s->strm->next_in))) ); s->strm->next_in++; s->strm->avail_in--; - s->strm->total_in++; + s->strm->total_in_lo32++; + if (s->strm->total_in_lo32 == 0) s->strm->total_in_hi32++; } } else { @@ -319,7 +336,8 @@ Bool copy_input_until_stop ( EState* s ) ADD_CHAR_TO_BLOCK ( s, (UInt32)(*((UChar*)(s->strm->next_in))) ); s->strm->next_in++; s->strm->avail_in--; - s->strm->total_in++; + s->strm->total_in_lo32++; + if (s->strm->total_in_lo32 == 0) s->strm->total_in_hi32++; s->avail_in_expect--; } } @@ -346,8 +364,8 @@ Bool copy_output_until_stop ( EState* s ) s->state_out_pos++; s->strm->avail_out--; s->strm->next_out++; - s->strm->total_out++; - + s->strm->total_out_lo32++; + if (s->strm->total_out_lo32 == 0) s->strm->total_out_hi32++; } return progress_out; @@ -381,12 +399,12 @@ Bool handle_compress ( bz_stream* strm ) progress_in |= copy_input_until_stop ( s ); if (s->mode != BZ_M_RUNNING && s->avail_in_expect == 0) { flush_RL ( s ); - compressBlock ( s, (Bool)(s->mode == BZ_M_FINISHING) ); + BZ2_compressBlock ( s, (Bool)(s->mode == BZ_M_FINISHING) ); s->state = BZ_S_OUTPUT; } else if (s->nblock >= s->nblockMAX) { - compressBlock ( s, False ); + BZ2_compressBlock ( s, False ); s->state = BZ_S_OUTPUT; } else @@ -402,7 +420,7 @@ Bool handle_compress ( bz_stream* strm ) /*---------------------------------------------------*/ -int BZ_API(bzCompress) ( bz_stream *strm, int action ) +int BZ_API(BZ2_bzCompress) ( bz_stream *strm, int action ) { Bool progress; EState* s; @@ -439,7 +457,8 @@ int BZ_API(bzCompress) ( bz_stream *strm, int action ) case BZ_M_FLUSHING: if (action != BZ_FLUSH) return BZ_SEQUENCE_ERROR; - if (s->avail_in_expect != s->strm->avail_in) return BZ_SEQUENCE_ERROR; + if (s->avail_in_expect != s->strm->avail_in) + return BZ_SEQUENCE_ERROR; progress = handle_compress ( strm ); if (s->avail_in_expect > 0 || !isempty_RL(s) || s->state_out_pos < s->numZ) return BZ_FLUSH_OK; @@ -448,7 +467,8 @@ int BZ_API(bzCompress) ( bz_stream *strm, int action ) case BZ_M_FINISHING: if (action != BZ_FINISH) return BZ_SEQUENCE_ERROR; - if (s->avail_in_expect != s->strm->avail_in) return BZ_SEQUENCE_ERROR; + if (s->avail_in_expect != s->strm->avail_in) + return BZ_SEQUENCE_ERROR; progress = handle_compress ( strm ); if (!progress) return BZ_SEQUENCE_ERROR; if (s->avail_in_expect > 0 || !isempty_RL(s) || @@ -461,7 +481,7 @@ int BZ_API(bzCompress) ( bz_stream *strm, int action ) /*---------------------------------------------------*/ -int BZ_API(bzCompressEnd) ( bz_stream *strm ) +int BZ_API(BZ2_bzCompressEnd) ( bz_stream *strm ) { EState* s; if (strm == NULL) return BZ_PARAM_ERROR; @@ -485,13 +505,15 @@ int BZ_API(bzCompressEnd) ( bz_stream *strm ) /*---------------------------------------------------*/ /*---------------------------------------------------*/ -int BZ_API(bzDecompressInit) +int BZ_API(BZ2_bzDecompressInit) ( bz_stream* strm, int verbosity, int small ) { DState* s; + if (!bz_config_ok()) return BZ_CONFIG_ERROR; + if (strm == NULL) return BZ_PARAM_ERROR; if (small != 0 && small != 1) return BZ_PARAM_ERROR; if (verbosity < 0 || verbosity > 4) return BZ_PARAM_ERROR; @@ -507,8 +529,10 @@ int BZ_API(bzDecompressInit) s->bsLive = 0; s->bsBuff = 0; s->calculatedCombinedCRC = 0; - strm->total_in = 0; - strm->total_out = 0; + strm->total_in_lo32 = 0; + strm->total_in_hi32 = 0; + strm->total_out_lo32 = 0; + strm->total_out_hi32 = 0; s->smallDecompress = (Bool)small; s->ll4 = NULL; s->ll16 = NULL; @@ -538,7 +562,8 @@ void unRLE_obuf_to_output_FAST ( DState* s ) s->state_out_len--; s->strm->next_out++; s->strm->avail_out--; - s->strm->total_out++; + s->strm->total_out_lo32++; + if (s->strm->total_out_lo32 == 0) s->strm->total_out_hi32++; } /* can a new run be started? */ @@ -585,8 +610,9 @@ void unRLE_obuf_to_output_FAST ( DState* s ) unsigned int cs_avail_out = s->strm->avail_out; /* end restore */ - UInt32 avail_out_INIT = cs_avail_out; - Int32 s_save_nblockPP = s->save_nblock+1; + UInt32 avail_out_INIT = cs_avail_out; + Int32 s_save_nblockPP = s->save_nblock+1; + unsigned int total_out_lo32_old; while (True) { @@ -640,7 +666,10 @@ void unRLE_obuf_to_output_FAST ( DState* s ) } return_notr: - s->strm->total_out += (avail_out_INIT - cs_avail_out); + total_out_lo32_old = s->strm->total_out_lo32; + s->strm->total_out_lo32 += (avail_out_INIT - cs_avail_out); + if (s->strm->total_out_lo32 < total_out_lo32_old) + s->strm->total_out_hi32++; /* save */ s->calculatedBlockCRC = c_calculatedBlockCRC; @@ -659,7 +688,7 @@ void unRLE_obuf_to_output_FAST ( DState* s ) /*---------------------------------------------------*/ -__inline__ Int32 indexIntoF ( Int32 indx, Int32 *cftab ) +__inline__ Int32 BZ2_indexIntoF ( Int32 indx, Int32 *cftab ) { Int32 nb, na, mid; nb = 0; @@ -691,7 +720,8 @@ void unRLE_obuf_to_output_SMALL ( DState* s ) s->state_out_len--; s->strm->next_out++; s->strm->avail_out--; - s->strm->total_out++; + s->strm->total_out_lo32++; + if (s->strm->total_out_lo32 == 0) s->strm->total_out_hi32++; } /* can a new run be started? */ @@ -736,7 +766,8 @@ void unRLE_obuf_to_output_SMALL ( DState* s ) s->state_out_len--; s->strm->next_out++; s->strm->avail_out--; - s->strm->total_out++; + s->strm->total_out_lo32++; + if (s->strm->total_out_lo32 == 0) s->strm->total_out_hi32++; } /* can a new run be started? */ @@ -768,7 +799,7 @@ void unRLE_obuf_to_output_SMALL ( DState* s ) /*---------------------------------------------------*/ -int BZ_API(bzDecompress) ( bz_stream *strm ) +int BZ_API(BZ2_bzDecompress) ( bz_stream *strm ) { DState* s; if (strm == NULL) return BZ_PARAM_ERROR; @@ -800,7 +831,7 @@ int BZ_API(bzDecompress) ( bz_stream *strm ) } } if (s->state >= BZ_X_MAGIC_1) { - Int32 r = decompress ( s ); + Int32 r = BZ2_decompress ( s ); if (r == BZ_STREAM_END) { if (s->verbosity >= 3) VPrintf2 ( "\n combined CRCs: stored = 0x%x, computed = 0x%x", @@ -820,7 +851,7 @@ int BZ_API(bzDecompress) ( bz_stream *strm ) /*---------------------------------------------------*/ -int BZ_API(bzDecompressEnd) ( bz_stream *strm ) +int BZ_API(BZ2_bzDecompressEnd) ( bz_stream *strm ) { DState* s; if (strm == NULL) return BZ_PARAM_ERROR; @@ -874,7 +905,7 @@ static Bool myfeof ( FILE* f ) /*---------------------------------------------------*/ -BZFILE* BZ_API(bzWriteOpen) +BZFILE* BZ_API(BZ2_bzWriteOpen) ( int* bzerror, FILE* f, int blockSize100k, @@ -909,8 +940,8 @@ BZFILE* BZ_API(bzWriteOpen) bzf->strm.opaque = NULL; if (workFactor == 0) workFactor = 30; - ret = bzCompressInit ( &(bzf->strm), blockSize100k, - verbosity, workFactor ); + ret = BZ2_bzCompressInit ( &(bzf->strm), blockSize100k, + verbosity, workFactor ); if (ret != BZ_OK) { BZ_SETERR(ret); free(bzf); return NULL; }; @@ -922,7 +953,7 @@ BZFILE* BZ_API(bzWriteOpen) /*---------------------------------------------------*/ -void BZ_API(bzWrite) +void BZ_API(BZ2_bzWrite) ( int* bzerror, BZFILE* b, void* buf, @@ -948,7 +979,7 @@ void BZ_API(bzWrite) while (True) { bzf->strm.avail_out = BZ_MAX_UNUSED; bzf->strm.next_out = bzf->buf; - ret = bzCompress ( &(bzf->strm), BZ_RUN ); + ret = BZ2_bzCompress ( &(bzf->strm), BZ_RUN ); if (ret != BZ_RUN_OK) { BZ_SETERR(ret); return; }; @@ -967,12 +998,26 @@ void BZ_API(bzWrite) /*---------------------------------------------------*/ -void BZ_API(bzWriteClose) +void BZ_API(BZ2_bzWriteClose) ( int* bzerror, BZFILE* b, int abandon, unsigned int* nbytes_in, unsigned int* nbytes_out ) +{ + BZ2_bzWriteClose64 ( bzerror, b, abandon, + nbytes_in, NULL, nbytes_out, NULL ); +} + + +void BZ_API(BZ2_bzWriteClose64) + ( int* bzerror, + BZFILE* b, + int abandon, + unsigned int* nbytes_in_lo32, + unsigned int* nbytes_in_hi32, + unsigned int* nbytes_out_lo32, + unsigned int* nbytes_out_hi32 ) { Int32 n, n2, ret; bzFile* bzf = (bzFile*)b; @@ -984,14 +1029,16 @@ void BZ_API(bzWriteClose) if (ferror(bzf->handle)) { BZ_SETERR(BZ_IO_ERROR); return; }; - if (nbytes_in != NULL) *nbytes_in = 0; - if (nbytes_out != NULL) *nbytes_out = 0; + if (nbytes_in_lo32 != NULL) *nbytes_in_lo32 = 0; + if (nbytes_in_hi32 != NULL) *nbytes_in_hi32 = 0; + if (nbytes_out_lo32 != NULL) *nbytes_out_lo32 = 0; + if (nbytes_out_hi32 != NULL) *nbytes_out_hi32 = 0; if ((!abandon) && bzf->lastErr == BZ_OK) { while (True) { bzf->strm.avail_out = BZ_MAX_UNUSED; bzf->strm.next_out = bzf->buf; - ret = bzCompress ( &(bzf->strm), BZ_FINISH ); + ret = BZ2_bzCompress ( &(bzf->strm), BZ_FINISH ); if (ret != BZ_FINISH_OK && ret != BZ_STREAM_END) { BZ_SETERR(ret); return; }; @@ -1013,17 +1060,23 @@ void BZ_API(bzWriteClose) { BZ_SETERR(BZ_IO_ERROR); return; }; } - if (nbytes_in != NULL) *nbytes_in = bzf->strm.total_in; - if (nbytes_out != NULL) *nbytes_out = bzf->strm.total_out; + if (nbytes_in_lo32 != NULL) + *nbytes_in_lo32 = bzf->strm.total_in_lo32; + if (nbytes_in_hi32 != NULL) + *nbytes_in_hi32 = bzf->strm.total_in_hi32; + if (nbytes_out_lo32 != NULL) + *nbytes_out_lo32 = bzf->strm.total_out_lo32; + if (nbytes_out_hi32 != NULL) + *nbytes_out_hi32 = bzf->strm.total_out_hi32; BZ_SETERR(BZ_OK); - bzCompressEnd ( &(bzf->strm) ); + BZ2_bzCompressEnd ( &(bzf->strm) ); free ( bzf ); } /*---------------------------------------------------*/ -BZFILE* BZ_API(bzReadOpen) +BZFILE* BZ_API(BZ2_bzReadOpen) ( int* bzerror, FILE* f, int verbosity, @@ -1066,7 +1119,7 @@ BZFILE* BZ_API(bzReadOpen) nUnused--; } - ret = bzDecompressInit ( &(bzf->strm), verbosity, small ); + ret = BZ2_bzDecompressInit ( &(bzf->strm), verbosity, small ); if (ret != BZ_OK) { BZ_SETERR(ret); free(bzf); return NULL; }; @@ -1079,7 +1132,7 @@ BZFILE* BZ_API(bzReadOpen) /*---------------------------------------------------*/ -void BZ_API(bzReadClose) ( int *bzerror, BZFILE *b ) +void BZ_API(BZ2_bzReadClose) ( int *bzerror, BZFILE *b ) { bzFile* bzf = (bzFile*)b; @@ -1091,13 +1144,13 @@ void BZ_API(bzReadClose) ( int *bzerror, BZFILE *b ) { BZ_SETERR(BZ_SEQUENCE_ERROR); return; }; if (bzf->initialisedOk) - (void)bzDecompressEnd ( &(bzf->strm) ); + (void)BZ2_bzDecompressEnd ( &(bzf->strm) ); free ( bzf ); } /*---------------------------------------------------*/ -int BZ_API(bzRead) +int BZ_API(BZ2_bzRead) ( int* bzerror, BZFILE* b, void* buf, @@ -1135,7 +1188,7 @@ int BZ_API(bzRead) bzf->strm.next_in = bzf->buf; } - ret = bzDecompress ( &(bzf->strm) ); + ret = BZ2_bzDecompress ( &(bzf->strm) ); if (ret != BZ_OK && ret != BZ_STREAM_END) { BZ_SETERR(ret); return 0; }; @@ -1157,7 +1210,7 @@ int BZ_API(bzRead) /*---------------------------------------------------*/ -void BZ_API(bzReadGetUnused) +void BZ_API(BZ2_bzReadGetUnused) ( int* bzerror, BZFILE* b, void** unused, @@ -1183,7 +1236,7 @@ void BZ_API(bzReadGetUnused) /*---------------------------------------------------*/ /*---------------------------------------------------*/ -int BZ_API(bzBuffToBuffCompress) +int BZ_API(BZ2_bzBuffToBuffCompress) ( char* dest, unsigned int* destLen, char* source, @@ -1206,8 +1259,8 @@ int BZ_API(bzBuffToBuffCompress) strm.bzalloc = NULL; strm.bzfree = NULL; strm.opaque = NULL; - ret = bzCompressInit ( &strm, blockSize100k, - verbosity, workFactor ); + ret = BZ2_bzCompressInit ( &strm, blockSize100k, + verbosity, workFactor ); if (ret != BZ_OK) return ret; strm.next_in = source; @@ -1215,27 +1268,27 @@ int BZ_API(bzBuffToBuffCompress) strm.avail_in = sourceLen; strm.avail_out = *destLen; - ret = bzCompress ( &strm, BZ_FINISH ); + ret = BZ2_bzCompress ( &strm, BZ_FINISH ); if (ret == BZ_FINISH_OK) goto output_overflow; if (ret != BZ_STREAM_END) goto errhandler; /* normal termination */ *destLen -= strm.avail_out; - bzCompressEnd ( &strm ); + BZ2_bzCompressEnd ( &strm ); return BZ_OK; output_overflow: - bzCompressEnd ( &strm ); + BZ2_bzCompressEnd ( &strm ); return BZ_OUTBUFF_FULL; errhandler: - bzCompressEnd ( &strm ); + BZ2_bzCompressEnd ( &strm ); return ret; } /*---------------------------------------------------*/ -int BZ_API(bzBuffToBuffDecompress) +int BZ_API(BZ2_bzBuffToBuffDecompress) ( char* dest, unsigned int* destLen, char* source, @@ -1255,7 +1308,7 @@ int BZ_API(bzBuffToBuffDecompress) strm.bzalloc = NULL; strm.bzfree = NULL; strm.opaque = NULL; - ret = bzDecompressInit ( &strm, verbosity, small ); + ret = BZ2_bzDecompressInit ( &strm, verbosity, small ); if (ret != BZ_OK) return ret; strm.next_in = source; @@ -1263,26 +1316,26 @@ int BZ_API(bzBuffToBuffDecompress) strm.avail_in = sourceLen; strm.avail_out = *destLen; - ret = bzDecompress ( &strm ); + ret = BZ2_bzDecompress ( &strm ); if (ret == BZ_OK) goto output_overflow_or_eof; if (ret != BZ_STREAM_END) goto errhandler; /* normal termination */ *destLen -= strm.avail_out; - bzDecompressEnd ( &strm ); + BZ2_bzDecompressEnd ( &strm ); return BZ_OK; output_overflow_or_eof: if (strm.avail_out > 0) { - bzDecompressEnd ( &strm ); + BZ2_bzDecompressEnd ( &strm ); return BZ_UNEXPECTED_EOF; } else { - bzDecompressEnd ( &strm ); + BZ2_bzDecompressEnd ( &strm ); return BZ_OUTBUFF_FULL; }; errhandler: - bzDecompressEnd ( &strm ); + BZ2_bzDecompressEnd ( &strm ); return ret; } @@ -1303,7 +1356,7 @@ int BZ_API(bzBuffToBuffDecompress) /*-- return version like "0.9.0c". --*/ -const char * BZ_API(bzlibVersion)(void) +const char * BZ_API(BZ2_bzlibVersion)(void) { return BZ_VERSION; } @@ -1377,9 +1430,11 @@ BZFILE * bzopen_or_bzdopen /* Guard against total chaos and anarchy -- JRS */ if (blockSize100k < 1) blockSize100k = 1; if (blockSize100k > 9) blockSize100k = 9; - bzfp = bzWriteOpen(&bzerr,fp,blockSize100k,verbosity,workFactor); + bzfp = BZ2_bzWriteOpen(&bzerr,fp,blockSize100k, + verbosity,workFactor); } else { - bzfp = bzReadOpen(&bzerr,fp,verbosity,smallMode,unused,nUnused); + bzfp = BZ2_bzReadOpen(&bzerr,fp,verbosity,smallMode, + unused,nUnused); } if (bzfp == NULL) { if (fp != stdin && fp != stdout) fclose(fp); @@ -1395,7 +1450,7 @@ BZFILE * bzopen_or_bzdopen ex) bzopen("file","w9") case path="" or NULL => use stdin or stdout. --*/ -BZFILE * BZ_API(bzopen) +BZFILE * BZ_API(BZ2_bzopen) ( const char *path, const char *mode ) { @@ -1404,7 +1459,7 @@ BZFILE * BZ_API(bzopen) /*---------------------------------------------------*/ -BZFILE * BZ_API(bzdopen) +BZFILE * BZ_API(BZ2_bzdopen) ( int fd, const char *mode ) { @@ -1413,11 +1468,11 @@ BZFILE * BZ_API(bzdopen) /*---------------------------------------------------*/ -int BZ_API(bzread) (BZFILE* b, void* buf, int len ) +int BZ_API(BZ2_bzread) (BZFILE* b, void* buf, int len ) { int bzerr, nread; if (((bzFile*)b)->lastErr == BZ_STREAM_END) return 0; - nread = bzRead(&bzerr,b,buf,len); + nread = BZ2_bzRead(&bzerr,b,buf,len); if (bzerr == BZ_OK || bzerr == BZ_STREAM_END) { return nread; } else { @@ -1427,11 +1482,11 @@ int BZ_API(bzread) (BZFILE* b, void* buf, int len ) /*---------------------------------------------------*/ -int BZ_API(bzwrite) (BZFILE* b, void* buf, int len ) +int BZ_API(BZ2_bzwrite) (BZFILE* b, void* buf, int len ) { int bzerr; - bzWrite(&bzerr,b,buf,len); + BZ2_bzWrite(&bzerr,b,buf,len); if(bzerr == BZ_OK){ return len; }else{ @@ -1441,7 +1496,7 @@ int BZ_API(bzwrite) (BZFILE* b, void* buf, int len ) /*---------------------------------------------------*/ -int BZ_API(bzflush) (BZFILE *b) +int BZ_API(BZ2_bzflush) (BZFILE *b) { /* do nothing now... */ return 0; @@ -1449,19 +1504,19 @@ int BZ_API(bzflush) (BZFILE *b) /*---------------------------------------------------*/ -void BZ_API(bzclose) (BZFILE* b) +void BZ_API(BZ2_bzclose) (BZFILE* b) { int bzerr; FILE *fp = ((bzFile *)b)->handle; if (b==NULL) {return;} if(((bzFile*)b)->writing){ - bzWriteClose(&bzerr,b,0,NULL,NULL); + BZ2_bzWriteClose(&bzerr,b,0,NULL,NULL); if(bzerr != BZ_OK){ - bzWriteClose(NULL,b,1,NULL,NULL); + BZ2_bzWriteClose(NULL,b,1,NULL,NULL); } }else{ - bzReadClose(&bzerr,b); + BZ2_bzReadClose(&bzerr,b); } if(fp!=stdin && fp!=stdout){ fclose(fp); @@ -1483,6 +1538,7 @@ static char *bzerrorstrings[] = { ,"IO_ERROR" ,"UNEXPECTED_EOF" ,"OUTBUFF_FULL" + ,"CONFIG_ERROR" ,"???" /* for future */ ,"???" /* for future */ ,"???" /* for future */ @@ -1492,7 +1548,7 @@ static char *bzerrorstrings[] = { }; -const char * BZ_API(bzerror) (BZFILE *b, int *errnum) +const char * BZ_API(BZ2_bzerror) (BZFILE *b, int *errnum) { int err = ((bzFile *)b)->lastErr; diff --git a/bzlib.h b/bzlib.h index d74938d..c9447a2 100644 --- a/bzlib.h +++ b/bzlib.h @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-1999 Julian R Seward. All rights reserved. + Copyright (C) 1996-2000 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -43,7 +43,7 @@ Julian Seward, Cambridge, UK. jseward@acm.org - bzip2/libbzip2 version 0.9.5 of 24 May 1999 + bzip2/libbzip2 version 1.0 of 21 March 2000 This program is based on (at least) the work of: Mike Burrows @@ -83,16 +83,19 @@ extern "C" { #define BZ_IO_ERROR (-6) #define BZ_UNEXPECTED_EOF (-7) #define BZ_OUTBUFF_FULL (-8) +#define BZ_CONFIG_ERROR (-9) typedef struct { char *next_in; unsigned int avail_in; - unsigned int total_in; + unsigned int total_in_lo32; + unsigned int total_in_hi32; char *next_out; unsigned int avail_out; - unsigned int total_out; + unsigned int total_out_lo32; + unsigned int total_out_hi32; void *state; @@ -130,33 +133,33 @@ typedef /*-- Core (low-level) library functions --*/ -BZ_EXTERN int BZ_API(bzCompressInit) ( +BZ_EXTERN int BZ_API(BZ2_bzCompressInit) ( bz_stream* strm, int blockSize100k, int verbosity, int workFactor ); -BZ_EXTERN int BZ_API(bzCompress) ( +BZ_EXTERN int BZ_API(BZ2_bzCompress) ( bz_stream* strm, int action ); -BZ_EXTERN int BZ_API(bzCompressEnd) ( +BZ_EXTERN int BZ_API(BZ2_bzCompressEnd) ( bz_stream* strm ); -BZ_EXTERN int BZ_API(bzDecompressInit) ( +BZ_EXTERN int BZ_API(BZ2_bzDecompressInit) ( bz_stream *strm, int verbosity, int small ); -BZ_EXTERN int BZ_API(bzDecompress) ( +BZ_EXTERN int BZ_API(BZ2_bzDecompress) ( bz_stream* strm ); -BZ_EXTERN int BZ_API(bzDecompressEnd) ( +BZ_EXTERN int BZ_API(BZ2_bzDecompressEnd) ( bz_stream *strm ); @@ -169,7 +172,7 @@ BZ_EXTERN int BZ_API(bzDecompressEnd) ( typedef void BZFILE; -BZ_EXTERN BZFILE* BZ_API(bzReadOpen) ( +BZ_EXTERN BZFILE* BZ_API(BZ2_bzReadOpen) ( int* bzerror, FILE* f, int verbosity, @@ -178,26 +181,26 @@ BZ_EXTERN BZFILE* BZ_API(bzReadOpen) ( int nUnused ); -BZ_EXTERN void BZ_API(bzReadClose) ( +BZ_EXTERN void BZ_API(BZ2_bzReadClose) ( int* bzerror, BZFILE* b ); -BZ_EXTERN void BZ_API(bzReadGetUnused) ( +BZ_EXTERN void BZ_API(BZ2_bzReadGetUnused) ( int* bzerror, BZFILE* b, void** unused, int* nUnused ); -BZ_EXTERN int BZ_API(bzRead) ( +BZ_EXTERN int BZ_API(BZ2_bzRead) ( int* bzerror, BZFILE* b, void* buf, int len ); -BZ_EXTERN BZFILE* BZ_API(bzWriteOpen) ( +BZ_EXTERN BZFILE* BZ_API(BZ2_bzWriteOpen) ( int* bzerror, FILE* f, int blockSize100k, @@ -205,26 +208,36 @@ BZ_EXTERN BZFILE* BZ_API(bzWriteOpen) ( int workFactor ); -BZ_EXTERN void BZ_API(bzWrite) ( +BZ_EXTERN void BZ_API(BZ2_bzWrite) ( int* bzerror, BZFILE* b, void* buf, int len ); -BZ_EXTERN void BZ_API(bzWriteClose) ( +BZ_EXTERN void BZ_API(BZ2_bzWriteClose) ( int* bzerror, BZFILE* b, int abandon, unsigned int* nbytes_in, unsigned int* nbytes_out ); + +BZ_EXTERN void BZ_API(BZ2_bzWriteClose64) ( + int* bzerror, + BZFILE* b, + int abandon, + unsigned int* nbytes_in_lo32, + unsigned int* nbytes_in_hi32, + unsigned int* nbytes_out_lo32, + unsigned int* nbytes_out_hi32 + ); #endif /*-- Utility functions --*/ -BZ_EXTERN int BZ_API(bzBuffToBuffCompress) ( +BZ_EXTERN int BZ_API(BZ2_bzBuffToBuffCompress) ( char* dest, unsigned int* destLen, char* source, @@ -234,7 +247,7 @@ BZ_EXTERN int BZ_API(bzBuffToBuffCompress) ( int workFactor ); -BZ_EXTERN int BZ_API(bzBuffToBuffDecompress) ( +BZ_EXTERN int BZ_API(BZ2_bzBuffToBuffDecompress) ( char* dest, unsigned int* destLen, char* source, @@ -254,42 +267,42 @@ BZ_EXTERN int BZ_API(bzBuffToBuffDecompress) ( If this code breaks, please contact both Yoshioka and me. --*/ -BZ_EXTERN const char * BZ_API(bzlibVersion) ( +BZ_EXTERN const char * BZ_API(BZ2_bzlibVersion) ( void ); #ifndef BZ_NO_STDIO -BZ_EXTERN BZFILE * BZ_API(bzopen) ( +BZ_EXTERN BZFILE * BZ_API(BZ2_bzopen) ( const char *path, const char *mode ); -BZ_EXTERN BZFILE * BZ_API(bzdopen) ( +BZ_EXTERN BZFILE * BZ_API(BZ2_bzdopen) ( int fd, const char *mode ); -BZ_EXTERN int BZ_API(bzread) ( +BZ_EXTERN int BZ_API(BZ2_bzread) ( BZFILE* b, void* buf, int len ); -BZ_EXTERN int BZ_API(bzwrite) ( +BZ_EXTERN int BZ_API(BZ2_bzwrite) ( BZFILE* b, void* buf, int len ); -BZ_EXTERN int BZ_API(bzflush) ( +BZ_EXTERN int BZ_API(BZ2_bzflush) ( BZFILE* b ); -BZ_EXTERN void BZ_API(bzclose) ( +BZ_EXTERN void BZ_API(BZ2_bzclose) ( BZFILE* b ); -BZ_EXTERN const char * BZ_API(bzerror) ( +BZ_EXTERN const char * BZ_API(BZ2_bzerror) ( BZFILE *b, int *errnum ); diff --git a/bzlib_private.h b/bzlib_private.h index 8e93480..fb51c7a 100644 --- a/bzlib_private.h +++ b/bzlib_private.h @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-1999 Julian R Seward. All rights reserved. + Copyright (C) 1996-2000 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -43,7 +43,7 @@ Julian Seward, Cambridge, UK. jseward@acm.org - bzip2/libbzip2 version 0.9.5 of 24 May 1999 + bzip2/libbzip2 version 1.0 of 21 March 2000 This program is based on (at least) the work of: Mike Burrows @@ -76,7 +76,7 @@ /*-- General stuff. --*/ -#define BZ_VERSION "0.9.5d" +#define BZ_VERSION "1.0.1, 23-June-2000" typedef char Char; typedef unsigned char Bool; @@ -94,9 +94,9 @@ typedef unsigned short UInt16; #endif #ifndef BZ_NO_STDIO -extern void bz__AssertH__fail ( int errcode ); +extern void BZ2_bz__AssertH__fail ( int errcode ); #define AssertH(cond,errcode) \ - { if (!(cond)) bz__AssertH__fail ( errcode ); } + { if (!(cond)) BZ2_bz__AssertH__fail ( errcode ); } #if BZ_DEBUG #define AssertD(cond,msg) \ { if (!(cond)) { \ @@ -155,7 +155,7 @@ extern void bz_internal_error ( int errcode ); /*-- Stuff for randomising repetitive blocks. --*/ -extern Int32 rNums[512]; +extern Int32 BZ2_rNums[512]; #define BZ_RAND_DECLS \ Int32 rNToGo; \ @@ -169,7 +169,7 @@ extern Int32 rNums[512]; #define BZ_RAND_UPD_MASK \ if (s->rNToGo == 0) { \ - s->rNToGo = rNums[s->rTPos]; \ + s->rNToGo = BZ2_rNums[s->rTPos]; \ s->rTPos++; \ if (s->rTPos == 512) s->rTPos = 0; \ } \ @@ -179,7 +179,7 @@ extern Int32 rNums[512]; /*-- Stuff for doing CRCs. --*/ -extern UInt32 crc32Table[256]; +extern UInt32 BZ2_crc32Table[256]; #define BZ_INITIALISE_CRC(crcVar) \ { \ @@ -194,8 +194,8 @@ extern UInt32 crc32Table[256]; #define BZ_UPDATE_CRC(crcVar,cha) \ { \ crcVar = (crcVar << 8) ^ \ - crc32Table[(crcVar >> 24) ^ \ - ((UChar)cha)]; \ + BZ2_crc32Table[(crcVar >> 24) ^ \ + ((UChar)cha)]; \ } @@ -241,7 +241,7 @@ typedef /* aliases for arr1 and arr2 */ UInt32* ptr; - UInt16* block; + UChar* block; UInt16* mtfv; UChar* zbits; @@ -283,9 +283,11 @@ typedef UChar selector [BZ_MAX_SELECTORS]; UChar selectorMtf[BZ_MAX_SELECTORS]; - UChar len [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE]; - Int32 code [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE]; - Int32 rfreq[BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE]; + UChar len [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE]; + Int32 code [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE]; + Int32 rfreq [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE]; + /* second dimension: only 3 needed; 4 makes index calculations faster */ + UInt32 len_pack[BZ_MAX_ALPHA_SIZE][4]; } EState; @@ -295,19 +297,19 @@ typedef /*-- externs for compression. --*/ extern void -blockSort ( EState* ); +BZ2_blockSort ( EState* ); extern void -compressBlock ( EState*, Bool ); +BZ2_compressBlock ( EState*, Bool ); extern void -bsInitWrite ( EState* ); +BZ2_bsInitWrite ( EState* ); extern void -hbAssignCodes ( Int32*, UChar*, Int32, Int32, Int32 ); +BZ2_hbAssignCodes ( Int32*, UChar*, Int32, Int32, Int32 ); extern void -hbMakeCodeLengths ( UChar*, Int32*, Int32, Int32 ); +BZ2_hbMakeCodeLengths ( UChar*, Int32*, Int32, Int32 ); @@ -493,22 +495,22 @@ typedef #define GET_LL(i) \ (((UInt32)s->ll16[i]) | (GET_LL4(i) << 16)) -#define BZ_GET_SMALL(cccc) \ - cccc = indexIntoF ( s->tPos, s->cftab ); \ +#define BZ_GET_SMALL(cccc) \ + cccc = BZ2_indexIntoF ( s->tPos, s->cftab ); \ s->tPos = GET_LL(s->tPos); /*-- externs for decompression. --*/ extern Int32 -indexIntoF ( Int32, Int32* ); +BZ2_indexIntoF ( Int32, Int32* ); extern Int32 -decompress ( DState* ); +BZ2_decompress ( DState* ); extern void -hbCreateDecodeTables ( Int32*, Int32*, Int32*, UChar*, - Int32, Int32, Int32 ); +BZ2_hbCreateDecodeTables ( Int32*, Int32*, Int32*, UChar*, + Int32, Int32, Int32 ); #endif diff --git a/compress.c b/compress.c index 7b192c3..cc5e31d 100644 --- a/compress.c +++ b/compress.c @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-1999 Julian R Seward. All rights reserved. + Copyright (C) 1996-2000 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -43,7 +43,7 @@ Julian Seward, Cambridge, UK. jseward@acm.org - bzip2/libbzip2 version 0.9.5 of 24 May 1999 + bzip2/libbzip2 version 1.0 of 21 March 2000 This program is based on (at least) the work of: Mike Burrows @@ -78,7 +78,7 @@ /*---------------------------------------------------*/ /*---------------------------------------------------*/ -void bsInitWrite ( EState* s ) +void BZ2_bsInitWrite ( EState* s ) { s->bsLive = 0; s->bsBuff = 0; @@ -113,6 +113,7 @@ void bsFinishWrite ( EState* s ) /*---------------------------------------------------*/ static +__inline__ void bsW ( EState* s, Int32 n, UInt32 v ) { bsNEEDW ( n ); @@ -164,8 +165,6 @@ void generateMTFValues ( EState* s ) { UChar yy[256]; Int32 i, j; - UChar tmp; - UChar tmp2; Int32 zPend; Int32 wr; Int32 EOB; @@ -174,7 +173,7 @@ void generateMTFValues ( EState* s ) After sorting (eg, here), s->arr1 [ 0 .. s->nblock-1 ] holds sorted order, and - ((UInt16*)s->arr2) [ 0 .. s->nblock-1 ] [15:8] + ((UChar*)s->arr2) [ 0 .. s->nblock-1 ] holds the original block data. The first thing to do is generate the MTF values, @@ -186,14 +185,14 @@ void generateMTFValues ( EState* s ) The final compressed bitstream is generated into the area starting at - (UChar*) (&((UInt16)s->arr2)[s->nblock]) + (UChar*) (&((UChar*)s->arr2)[s->nblock]) These storage aliases are set up in bzCompressInit(), except for the last one, which is arranged in compressBlock(). */ UInt32* ptr = s->ptr; - UInt16* block = s->block; + UChar* block = s->block; UInt16* mtfv = s->mtfv; makeMaps_e ( s ); @@ -207,27 +206,14 @@ void generateMTFValues ( EState* s ) for (i = 0; i < s->nblock; i++) { UChar ll_i; - AssertD ( wr <= i, "generateMTFValues(1)" ); j = ptr[i]-1; if (j < 0) j += s->nblock; - ll_i = s->unseqToSeq[block[j] >> 8]; + ll_i = s->unseqToSeq[block[j]]; AssertD ( ll_i < s->nInUse, "generateMTFValues(2a)" ); - tmp = yy[0]; - if (tmp == ll_i) { + if (yy[0] == ll_i) { zPend++; } else { - tmp2 = tmp; - tmp = yy[1]; - yy[1] = tmp2; - j = 1; - while ( ll_i != tmp ) { - j++; - tmp2 = tmp; - tmp = yy[j]; - yy[j] = tmp2; - }; - yy[0] = tmp; if (zPend > 0) { zPend--; @@ -244,7 +230,26 @@ void generateMTFValues ( EState* s ) }; zPend = 0; } - mtfv[wr] = j+1; wr++; s->mtfFreq[j+1]++; + { + register UChar rtmp; + register UChar* ryy_j; + register UChar rll_i; + rtmp = yy[1]; + yy[1] = yy[0]; + ryy_j = &(yy[1]); + rll_i = ll_i; + while ( rll_i != rtmp ) { + register UChar rtmp2; + ryy_j++; + rtmp2 = rtmp; + rtmp = *ryy_j; + *ryy_j = rtmp2; + }; + yy[0] = rtmp; + j = ryy_j - &(yy[0]); + mtfv[wr] = j+1; wr++; s->mtfFreq[j+1]++; + } + } } @@ -261,6 +266,7 @@ void generateMTFValues ( EState* s ) if (zPend < 2) break; zPend = (zPend - 2) / 2; }; + zPend = 0; } mtfv[wr] = EOB; wr++; s->mtfFreq[EOB]++; @@ -365,6 +371,18 @@ void sendMTFValues ( EState* s ) for (v = 0; v < alphaSize; v++) s->rfreq[t][v] = 0; + /*--- + Set up an auxiliary length table which is used to fast-track + the common case (nGroups == 6). + ---*/ + if (nGroups == 6) { + for (v = 0; v < alphaSize; v++) { + s->len_pack[v][0] = (s->len[1][v] << 16) | s->len[0][v]; + s->len_pack[v][1] = (s->len[3][v] << 16) | s->len[2][v]; + s->len_pack[v][2] = (s->len[5][v] << 16) | s->len[4][v]; + } + } + nSelectors = 0; totc = 0; gs = 0; @@ -381,21 +399,37 @@ void sendMTFValues ( EState* s ) --*/ for (t = 0; t < nGroups; t++) cost[t] = 0; - if (nGroups == 6) { - register UInt16 cost0, cost1, cost2, cost3, cost4, cost5; - cost0 = cost1 = cost2 = cost3 = cost4 = cost5 = 0; - for (i = gs; i <= ge; i++) { - UInt16 icv = mtfv[i]; - cost0 += s->len[0][icv]; - cost1 += s->len[1][icv]; - cost2 += s->len[2][icv]; - cost3 += s->len[3][icv]; - cost4 += s->len[4][icv]; - cost5 += s->len[5][icv]; - } - cost[0] = cost0; cost[1] = cost1; cost[2] = cost2; - cost[3] = cost3; cost[4] = cost4; cost[5] = cost5; + if (nGroups == 6 && 50 == ge-gs+1) { + /*--- fast track the common case ---*/ + register UInt32 cost01, cost23, cost45; + register UInt16 icv; + cost01 = cost23 = cost45 = 0; + +# define BZ_ITER(nn) \ + icv = mtfv[gs+(nn)]; \ + cost01 += s->len_pack[icv][0]; \ + cost23 += s->len_pack[icv][1]; \ + cost45 += s->len_pack[icv][2]; \ + + BZ_ITER(0); BZ_ITER(1); BZ_ITER(2); BZ_ITER(3); BZ_ITER(4); + BZ_ITER(5); BZ_ITER(6); BZ_ITER(7); BZ_ITER(8); BZ_ITER(9); + BZ_ITER(10); BZ_ITER(11); BZ_ITER(12); BZ_ITER(13); BZ_ITER(14); + BZ_ITER(15); BZ_ITER(16); BZ_ITER(17); BZ_ITER(18); BZ_ITER(19); + BZ_ITER(20); BZ_ITER(21); BZ_ITER(22); BZ_ITER(23); BZ_ITER(24); + BZ_ITER(25); BZ_ITER(26); BZ_ITER(27); BZ_ITER(28); BZ_ITER(29); + BZ_ITER(30); BZ_ITER(31); BZ_ITER(32); BZ_ITER(33); BZ_ITER(34); + BZ_ITER(35); BZ_ITER(36); BZ_ITER(37); BZ_ITER(38); BZ_ITER(39); + BZ_ITER(40); BZ_ITER(41); BZ_ITER(42); BZ_ITER(43); BZ_ITER(44); + BZ_ITER(45); BZ_ITER(46); BZ_ITER(47); BZ_ITER(48); BZ_ITER(49); + +# undef BZ_ITER + + cost[0] = cost01 & 0xffff; cost[1] = cost01 >> 16; + cost[2] = cost23 & 0xffff; cost[3] = cost23 >> 16; + cost[4] = cost45 & 0xffff; cost[5] = cost45 >> 16; + } else { + /*--- slow version which correctly handles all situations ---*/ for (i = gs; i <= ge; i++) { UInt16 icv = mtfv[i]; for (t = 0; t < nGroups; t++) cost[t] += s->len[t][icv]; @@ -417,8 +451,29 @@ void sendMTFValues ( EState* s ) /*-- Increment the symbol frequencies for the selected table. --*/ - for (i = gs; i <= ge; i++) - s->rfreq[bt][ mtfv[i] ]++; + if (nGroups == 6 && 50 == ge-gs+1) { + /*--- fast track the common case ---*/ + +# define BZ_ITUR(nn) s->rfreq[bt][ mtfv[gs+(nn)] ]++ + + BZ_ITUR(0); BZ_ITUR(1); BZ_ITUR(2); BZ_ITUR(3); BZ_ITUR(4); + BZ_ITUR(5); BZ_ITUR(6); BZ_ITUR(7); BZ_ITUR(8); BZ_ITUR(9); + BZ_ITUR(10); BZ_ITUR(11); BZ_ITUR(12); BZ_ITUR(13); BZ_ITUR(14); + BZ_ITUR(15); BZ_ITUR(16); BZ_ITUR(17); BZ_ITUR(18); BZ_ITUR(19); + BZ_ITUR(20); BZ_ITUR(21); BZ_ITUR(22); BZ_ITUR(23); BZ_ITUR(24); + BZ_ITUR(25); BZ_ITUR(26); BZ_ITUR(27); BZ_ITUR(28); BZ_ITUR(29); + BZ_ITUR(30); BZ_ITUR(31); BZ_ITUR(32); BZ_ITUR(33); BZ_ITUR(34); + BZ_ITUR(35); BZ_ITUR(36); BZ_ITUR(37); BZ_ITUR(38); BZ_ITUR(39); + BZ_ITUR(40); BZ_ITUR(41); BZ_ITUR(42); BZ_ITUR(43); BZ_ITUR(44); + BZ_ITUR(45); BZ_ITUR(46); BZ_ITUR(47); BZ_ITUR(48); BZ_ITUR(49); + +# undef BZ_ITUR + + } else { + /*--- slow version which correctly handles all situations ---*/ + for (i = gs; i <= ge; i++) + s->rfreq[bt][ mtfv[i] ]++; + } gs = ge+1; } @@ -434,8 +489,8 @@ void sendMTFValues ( EState* s ) Recompute the tables based on the accumulated frequencies. --*/ for (t = 0; t < nGroups; t++) - hbMakeCodeLengths ( &(s->len[t][0]), &(s->rfreq[t][0]), - alphaSize, 20 ); + BZ2_hbMakeCodeLengths ( &(s->len[t][0]), &(s->rfreq[t][0]), + alphaSize, 20 ); } @@ -474,8 +529,8 @@ void sendMTFValues ( EState* s ) } AssertH ( !(maxLen > 20), 3004 ); AssertH ( !(minLen < 1), 3005 ); - hbAssignCodes ( &(s->code[t][0]), &(s->len[t][0]), - minLen, maxLen, alphaSize ); + BZ2_hbAssignCodes ( &(s->code[t][0]), &(s->len[t][0]), + minLen, maxLen, alphaSize ); } /*--- Transmit the mapping table. ---*/ @@ -536,13 +591,45 @@ void sendMTFValues ( EState* s ) if (gs >= s->nMTF) break; ge = gs + BZ_G_SIZE - 1; if (ge >= s->nMTF) ge = s->nMTF-1; - for (i = gs; i <= ge; i++) { - AssertH ( s->selector[selCtr] < nGroups, 3006 ); - bsW ( s, - s->len [s->selector[selCtr]] [mtfv[i]], - s->code [s->selector[selCtr]] [mtfv[i]] ); + AssertH ( s->selector[selCtr] < nGroups, 3006 ); + + if (nGroups == 6 && 50 == ge-gs+1) { + /*--- fast track the common case ---*/ + UInt16 mtfv_i; + UChar* s_len_sel_selCtr + = &(s->len[s->selector[selCtr]][0]); + Int32* s_code_sel_selCtr + = &(s->code[s->selector[selCtr]][0]); + +# define BZ_ITAH(nn) \ + mtfv_i = mtfv[gs+(nn)]; \ + bsW ( s, \ + s_len_sel_selCtr[mtfv_i], \ + s_code_sel_selCtr[mtfv_i] ) + + BZ_ITAH(0); BZ_ITAH(1); BZ_ITAH(2); BZ_ITAH(3); BZ_ITAH(4); + BZ_ITAH(5); BZ_ITAH(6); BZ_ITAH(7); BZ_ITAH(8); BZ_ITAH(9); + BZ_ITAH(10); BZ_ITAH(11); BZ_ITAH(12); BZ_ITAH(13); BZ_ITAH(14); + BZ_ITAH(15); BZ_ITAH(16); BZ_ITAH(17); BZ_ITAH(18); BZ_ITAH(19); + BZ_ITAH(20); BZ_ITAH(21); BZ_ITAH(22); BZ_ITAH(23); BZ_ITAH(24); + BZ_ITAH(25); BZ_ITAH(26); BZ_ITAH(27); BZ_ITAH(28); BZ_ITAH(29); + BZ_ITAH(30); BZ_ITAH(31); BZ_ITAH(32); BZ_ITAH(33); BZ_ITAH(34); + BZ_ITAH(35); BZ_ITAH(36); BZ_ITAH(37); BZ_ITAH(38); BZ_ITAH(39); + BZ_ITAH(40); BZ_ITAH(41); BZ_ITAH(42); BZ_ITAH(43); BZ_ITAH(44); + BZ_ITAH(45); BZ_ITAH(46); BZ_ITAH(47); BZ_ITAH(48); BZ_ITAH(49); + +# undef BZ_ITAH + + } else { + /*--- slow version which correctly handles all situations ---*/ + for (i = gs; i <= ge; i++) { + bsW ( s, + s->len [s->selector[selCtr]] [mtfv[i]], + s->code [s->selector[selCtr]] [mtfv[i]] ); + } } + gs = ge+1; selCtr++; } @@ -554,7 +641,7 @@ void sendMTFValues ( EState* s ) /*---------------------------------------------------*/ -void compressBlock ( EState* s, Bool is_last_block ) +void BZ2_compressBlock ( EState* s, Bool is_last_block ) { if (s->nblock > 0) { @@ -568,14 +655,14 @@ void compressBlock ( EState* s, Bool is_last_block ) "combined CRC = 0x%8x, size = %d\n", s->blockNo, s->blockCRC, s->combinedCRC, s->nblock ); - blockSort ( s ); + BZ2_blockSort ( s ); } - s->zbits = (UChar*) (&((UInt16*)s->arr2)[s->nblock]); + s->zbits = (UChar*) (&((UChar*)s->arr2)[s->nblock]); /*-- If this is the first block, create the stream header. --*/ if (s->blockNo == 1) { - bsInitWrite ( s ); + BZ2_bsInitWrite ( s ); bsPutUChar ( s, 'B' ); bsPutUChar ( s, 'Z' ); bsPutUChar ( s, 'h' ); diff --git a/crctable.c b/crctable.c index ab53df6..61c040c 100644 --- a/crctable.c +++ b/crctable.c @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-1999 Julian R Seward. All rights reserved. + Copyright (C) 1996-2000 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -43,7 +43,7 @@ Julian Seward, Cambridge, UK. jseward@acm.org - bzip2/libbzip2 version 0.9.5 of 24 May 1999 + bzip2/libbzip2 version 1.0 of 21 March 2000 This program is based on (at least) the work of: Mike Burrows @@ -68,7 +68,7 @@ comp.compression FAQ. --*/ -UInt32 crc32Table[256] = { +UInt32 BZ2_crc32Table[256] = { /*-- Ugly, innit? --*/ diff --git a/decompress.c b/decompress.c index 31f8b67..cdced18 100644 --- a/decompress.c +++ b/decompress.c @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-1999 Julian R Seward. All rights reserved. + Copyright (C) 1996-2000 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -43,7 +43,7 @@ Julian Seward, Cambridge, UK. jseward@acm.org - bzip2/libbzip2 version 0.9.5 of 24 May 1999 + bzip2/libbzip2 version 1.0 of 21 March 2000 This program is based on (at least) the work of: Mike Burrows @@ -99,7 +99,9 @@ void makeMaps_d ( DState* s ) s->bsLive += 8; \ s->strm->next_in++; \ s->strm->avail_in--; \ - s->strm->total_in++; \ + s->strm->total_in_lo32++; \ + if (s->strm->total_in_lo32 == 0) \ + s->strm->total_in_hi32++; \ } #define GET_UCHAR(lll,uuu) \ @@ -113,6 +115,8 @@ void makeMaps_d ( DState* s ) { \ if (groupPos == 0) { \ groupNo++; \ + if (groupNo >= nSelectors) \ + RETURN(BZ_DATA_ERROR); \ groupPos = BZ_G_SIZE; \ gSel = s->selector[groupNo]; \ gMinlen = s->minLens[gSel]; \ @@ -123,17 +127,23 @@ void makeMaps_d ( DState* s ) groupPos--; \ zn = gMinlen; \ GET_BITS(label1, zvec, zn); \ - while (zvec > gLimit[zn]) { \ + while (1) { \ + if (zn > 20 /* the longest code */) \ + RETURN(BZ_DATA_ERROR); \ + if (zvec <= gLimit[zn]) break; \ zn++; \ GET_BIT(label2, zj); \ zvec = (zvec << 1) | zj; \ }; \ + if (zvec - gBase[zn] < 0 \ + || zvec - gBase[zn] >= BZ_MAX_ALPHA_SIZE) \ + RETURN(BZ_DATA_ERROR); \ lval = gPerm[zvec - gBase[zn]]; \ } /*---------------------------------------------------*/ -Int32 decompress ( DState* s ) +Int32 BZ2_decompress ( DState* s ) { UChar uc; Int32 retVal; @@ -288,6 +298,11 @@ Int32 decompress ( DState* s ) GET_UCHAR(BZ_X_ORIGPTR_3, uc); s->origPtr = (s->origPtr << 8) | ((Int32)uc); + if (s->origPtr < 0) + RETURN(BZ_DATA_ERROR); + if (s->origPtr > 10 + 100000*s->blockSize100k) + RETURN(BZ_DATA_ERROR); + /*--- Receive the mapping table ---*/ for (i = 0; i < 16; i++) { GET_BIT(BZ_X_MAPPING_1, uc); @@ -305,18 +320,21 @@ Int32 decompress ( DState* s ) if (uc == 1) s->inUse[i * 16 + j] = True; } makeMaps_d ( s ); + if (s->nInUse == 0) RETURN(BZ_DATA_ERROR); alphaSize = s->nInUse+2; /*--- Now the selectors ---*/ GET_BITS(BZ_X_SELECTOR_1, nGroups, 3); + if (nGroups < 2 || nGroups > 6) RETURN(BZ_DATA_ERROR); GET_BITS(BZ_X_SELECTOR_2, nSelectors, 15); + if (nSelectors < 1) RETURN(BZ_DATA_ERROR); for (i = 0; i < nSelectors; i++) { j = 0; while (True) { GET_BIT(BZ_X_SELECTOR_3, uc); if (uc == 0) break; j++; - if (j > 5) RETURN(BZ_DATA_ERROR); + if (j >= nGroups) RETURN(BZ_DATA_ERROR); } s->selectorMtf[i] = j; } @@ -358,7 +376,7 @@ Int32 decompress ( DState* s ) if (s->len[t][i] > maxLen) maxLen = s->len[t][i]; if (s->len[t][i] < minLen) minLen = s->len[t][i]; } - hbCreateDecodeTables ( + BZ2_hbCreateDecodeTables ( &(s->limit[t][0]), &(s->base[t][0]), &(s->perm[t][0]), @@ -392,7 +410,6 @@ Int32 decompress ( DState* s ) /*-- end MTF init --*/ nblock = 0; - GET_MTF_VAL(BZ_X_MTF_1, BZ_X_MTF_2, nextSym); while (True) { @@ -417,23 +434,24 @@ Int32 decompress ( DState* s ) if (s->smallDecompress) while (es > 0) { + if (nblock >= nblockMAX) RETURN(BZ_DATA_ERROR); s->ll16[nblock] = (UInt16)uc; nblock++; es--; } else while (es > 0) { + if (nblock >= nblockMAX) RETURN(BZ_DATA_ERROR); s->tt[nblock] = (UInt32)uc; nblock++; es--; }; - if (nblock > nblockMAX) RETURN(BZ_DATA_ERROR); continue; } else { - if (nblock > nblockMAX) RETURN(BZ_DATA_ERROR); + if (nblock >= nblockMAX) RETURN(BZ_DATA_ERROR); /*-- uc = MTF ( nextSym-1 ) --*/ { @@ -500,6 +518,12 @@ Int32 decompress ( DState* s ) } } + /* Now we know what nblock is, we can do a better sanity + check on s->origPtr. + */ + if (s->origPtr < 0 || s->origPtr >= nblock) + RETURN(BZ_DATA_ERROR); + s->state_out_len = 0; s->state_out_ch = 0; BZ_INITIALISE_CRC ( s->calculatedBlockCRC ); diff --git a/dlltest.c b/dlltest.c index e5639f1..f79279c 100644 --- a/dlltest.c +++ b/dlltest.c @@ -1,165 +1,176 @@ -/* - minibz2 - libbz2.dll test program. - by Yoshioka Tsuneo(QWF00133@nifty.ne.jp/tsuneo-y@is.aist-nara.ac.jp) - This file is Public Domain. - welcome any email to me. - - usage: minibz2 [-d] [-{1,2,..9}] [[srcfilename] destfilename] -*/ - -#define BZ_IMPORT -#include -#include -#include "bzlib.h" -#ifdef _WIN32 -#include -#endif - - -#ifdef _WIN32 - -#include -static int BZ2DLLLoaded = 0; -static HINSTANCE BZ2DLLhLib; -int BZ2DLLLoadLibrary(void) -{ - HINSTANCE hLib; - - if(BZ2DLLLoaded==1){return 0;} - hLib=LoadLibrary("libbz2.dll"); - if(hLib == NULL){ - puts("Can't load libbz2.dll"); - return -1; - } - BZ2DLLLoaded=1; - BZ2DLLhLib=hLib; - bzlibVersion=GetProcAddress(hLib,"bzlibVersion"); - bzopen=GetProcAddress(hLib,"bzopen"); - bzdopen=GetProcAddress(hLib,"bzdopen"); - bzread=GetProcAddress(hLib,"bzread"); - bzwrite=GetProcAddress(hLib,"bzwrite"); - bzflush=GetProcAddress(hLib,"bzflush"); - bzclose=GetProcAddress(hLib,"bzclose"); - bzerror=GetProcAddress(hLib,"bzerror"); - return 0; - -} -int BZ2DLLFreeLibrary(void) -{ - if(BZ2DLLLoaded==0){return 0;} - FreeLibrary(BZ2DLLhLib); - BZ2DLLLoaded=0; -} -#endif /* WIN32 */ - -void usage(void) -{ - puts("usage: minibz2 [-d] [-{1,2,..9}] [[srcfilename] destfilename]"); -} - -int main(int argc,char *argv[]) -{ - int decompress = 0; - int level = 9; - char *fn_r = NULL; - char *fn_w = NULL; - -#ifdef _WIN32 - if(BZ2DLLLoadLibrary()<0){ - puts("can't load dll"); - exit(1); - } -#endif - while(++argv,--argc){ - if(**argv =='-' || **argv=='/'){ - char *p; - - for(p=*argv+1;*p;p++){ - if(*p=='d'){ - decompress = 1; - }else if('1'<=*p && *p<='9'){ - level = *p - '0'; - }else{ - usage(); - exit(1); - } - } - }else{ - break; - } - } - if(argc>=1){ - fn_r = *argv; - argc--;argv++; - }else{ - fn_r = NULL; - } - if(argc>=1){ - fn_w = *argv; - argc--;argv++; - }else{ - fn_w = NULL; - } - { - int len; - char buff[0x1000]; - char mode[10]; - - if(decompress){ - BZFILE *BZ2fp_r = NULL; - FILE *fp_w = NULL; - - if(fn_w){ - if((fp_w = fopen(fn_w,"wb"))==NULL){ - printf("can't open [%s]\n",fn_w); - perror("reason:"); - exit(1); - } - }else{ - fp_w = stdout; - } - if((BZ2fp_r == NULL && (BZ2fp_r = bzdopen(fileno(stdin),"rb"))==NULL) - || (BZ2fp_r != NULL && (BZ2fp_r = bzopen(fn_r,"rb"))==NULL)){ - printf("can't bz2openstream\n"); - exit(1); - } - while((len=bzread(BZ2fp_r,buff,0x1000))>0){ - fwrite(buff,1,len,fp_w); - } - bzclose(BZ2fp_r); - if(fp_w != stdout) fclose(fp_w); - }else{ - BZFILE *BZ2fp_w = NULL; - FILE *fp_r = NULL; - - if(fn_r){ - if((fp_r = fopen(fn_r,"rb"))==NULL){ - printf("can't open [%s]\n",fn_r); - perror("reason:"); - exit(1); - } - }else{ - fp_r = stdin; - } - mode[0]='w'; - mode[1] = '0' + level; - mode[2] = '\0'; - - if((fn_w == NULL && (BZ2fp_w = bzdopen(fileno(stdout),mode))==NULL) - || (fn_w !=NULL && (BZ2fp_w = bzopen(fn_w,mode))==NULL)){ - printf("can't bz2openstream\n"); - exit(1); - } - while((len=fread(buff,1,0x1000,fp_r))>0){ - bzwrite(BZ2fp_w,buff,len); - } - bzclose(BZ2fp_w); - if(fp_r!=stdin)fclose(fp_r); - } - } -#ifdef _WIN32 - BZ2DLLFreeLibrary(); -#endif - return 0; -} +/* + minibz2 + libbz2.dll test program. + by Yoshioka Tsuneo(QWF00133@nifty.ne.jp/tsuneo-y@is.aist-nara.ac.jp) + This file is Public Domain. + welcome any email to me. + + usage: minibz2 [-d] [-{1,2,..9}] [[srcfilename] destfilename] +*/ + +#define BZ_IMPORT +#include +#include +#include "bzlib.h" +#ifdef _WIN32 +#include +#endif + + +#ifdef _WIN32 + +#define BZ2_LIBNAME "libbz2-1.0.0.DLL" + +#include +static int BZ2DLLLoaded = 0; +static HINSTANCE BZ2DLLhLib; +int BZ2DLLLoadLibrary(void) +{ + HINSTANCE hLib; + + if(BZ2DLLLoaded==1){return 0;} + hLib=LoadLibrary(BZ2_LIBNAME); + if(hLib == NULL){ + fprintf(stderr,"Can't load %s\n",BZ2_LIBNAME); + return -1; + } + BZ2_bzlibVersion=GetProcAddress(hLib,"BZ2_bzlibVersion"); + BZ2_bzopen=GetProcAddress(hLib,"BZ2_bzopen"); + BZ2_bzdopen=GetProcAddress(hLib,"BZ2_bzdopen"); + BZ2_bzread=GetProcAddress(hLib,"BZ2_bzread"); + BZ2_bzwrite=GetProcAddress(hLib,"BZ2_bzwrite"); + BZ2_bzflush=GetProcAddress(hLib,"BZ2_bzflush"); + BZ2_bzclose=GetProcAddress(hLib,"BZ2_bzclose"); + BZ2_bzerror=GetProcAddress(hLib,"BZ2_bzerror"); + + if (!BZ2_bzlibVersion || !BZ2_bzopen || !BZ2_bzdopen + || !BZ2_bzread || !BZ2_bzwrite || !BZ2_bzflush + || !BZ2_bzclose || !BZ2_bzerror) { + fprintf(stderr,"GetProcAddress failed.\n"); + return -1; + } + BZ2DLLLoaded=1; + BZ2DLLhLib=hLib; + return 0; + +} +int BZ2DLLFreeLibrary(void) +{ + if(BZ2DLLLoaded==0){return 0;} + FreeLibrary(BZ2DLLhLib); + BZ2DLLLoaded=0; +} +#endif /* WIN32 */ + +void usage(void) +{ + puts("usage: minibz2 [-d] [-{1,2,..9}] [[srcfilename] destfilename]"); +} + +int main(int argc,char *argv[]) +{ + int decompress = 0; + int level = 9; + char *fn_r = NULL; + char *fn_w = NULL; + +#ifdef _WIN32 + if(BZ2DLLLoadLibrary()<0){ + fprintf(stderr,"Loading of %s failed. Giving up.\n", BZ2_LIBNAME); + exit(1); + } + printf("Loading of %s succeeded. Library version is %s.\n", + BZ2_LIBNAME, BZ2_bzlibVersion() ); +#endif + while(++argv,--argc){ + if(**argv =='-' || **argv=='/'){ + char *p; + + for(p=*argv+1;*p;p++){ + if(*p=='d'){ + decompress = 1; + }else if('1'<=*p && *p<='9'){ + level = *p - '0'; + }else{ + usage(); + exit(1); + } + } + }else{ + break; + } + } + if(argc>=1){ + fn_r = *argv; + argc--;argv++; + }else{ + fn_r = NULL; + } + if(argc>=1){ + fn_w = *argv; + argc--;argv++; + }else{ + fn_w = NULL; + } + { + int len; + char buff[0x1000]; + char mode[10]; + + if(decompress){ + BZFILE *BZ2fp_r = NULL; + FILE *fp_w = NULL; + + if(fn_w){ + if((fp_w = fopen(fn_w,"wb"))==NULL){ + printf("can't open [%s]\n",fn_w); + perror("reason:"); + exit(1); + } + }else{ + fp_w = stdout; + } + if((BZ2fp_r == NULL && (BZ2fp_r = BZ2_bzdopen(fileno(stdin),"rb"))==NULL) + || (BZ2fp_r != NULL && (BZ2fp_r = BZ2_bzopen(fn_r,"rb"))==NULL)){ + printf("can't bz2openstream\n"); + exit(1); + } + while((len=BZ2_bzread(BZ2fp_r,buff,0x1000))>0){ + fwrite(buff,1,len,fp_w); + } + BZ2_bzclose(BZ2fp_r); + if(fp_w != stdout) fclose(fp_w); + }else{ + BZFILE *BZ2fp_w = NULL; + FILE *fp_r = NULL; + + if(fn_r){ + if((fp_r = fopen(fn_r,"rb"))==NULL){ + printf("can't open [%s]\n",fn_r); + perror("reason:"); + exit(1); + } + }else{ + fp_r = stdin; + } + mode[0]='w'; + mode[1] = '0' + level; + mode[2] = '\0'; + + if((fn_w == NULL && (BZ2fp_w = BZ2_bzdopen(fileno(stdout),mode))==NULL) + || (fn_w !=NULL && (BZ2fp_w = BZ2_bzopen(fn_w,mode))==NULL)){ + printf("can't bz2openstream\n"); + exit(1); + } + while((len=fread(buff,1,0x1000,fp_r))>0){ + BZ2_bzwrite(BZ2fp_w,buff,len); + } + BZ2_bzclose(BZ2fp_w); + if(fp_r!=stdin)fclose(fp_r); + } + } +#ifdef _WIN32 + BZ2DLLFreeLibrary(); +#endif + return 0; +} diff --git a/huffman.c b/huffman.c index bb2a6cc..9b446c4 100644 --- a/huffman.c +++ b/huffman.c @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-1999 Julian R Seward. All rights reserved. + Copyright (C) 1996-2000 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -43,7 +43,7 @@ Julian Seward, Cambridge, UK. jseward@acm.org - bzip2/libbzip2 version 0.9.5 of 24 May 1999 + bzip2/libbzip2 version 1.0 of 21 March 2000 This program is based on (at least) the work of: Mike Burrows @@ -100,10 +100,10 @@ /*---------------------------------------------------*/ -void hbMakeCodeLengths ( UChar *len, - Int32 *freq, - Int32 alphaSize, - Int32 maxLen ) +void BZ2_hbMakeCodeLengths ( UChar *len, + Int32 *freq, + Int32 alphaSize, + Int32 maxLen ) { /*-- Nodes and heap entries run from 1. Entry 0 @@ -172,11 +172,11 @@ void hbMakeCodeLengths ( UChar *len, /*---------------------------------------------------*/ -void hbAssignCodes ( Int32 *code, - UChar *length, - Int32 minLen, - Int32 maxLen, - Int32 alphaSize ) +void BZ2_hbAssignCodes ( Int32 *code, + UChar *length, + Int32 minLen, + Int32 maxLen, + Int32 alphaSize ) { Int32 n, vec, i; @@ -190,13 +190,13 @@ void hbAssignCodes ( Int32 *code, /*---------------------------------------------------*/ -void hbCreateDecodeTables ( Int32 *limit, - Int32 *base, - Int32 *perm, - UChar *length, - Int32 minLen, - Int32 maxLen, - Int32 alphaSize ) +void BZ2_hbCreateDecodeTables ( Int32 *limit, + Int32 *base, + Int32 *perm, + UChar *length, + Int32 minLen, + Int32 maxLen, + Int32 alphaSize ) { Int32 pp, i, j, vec; diff --git a/libbz2.def b/libbz2.def index ba0f54e..2dc0dd8 100644 --- a/libbz2.def +++ b/libbz2.def @@ -1,25 +1,27 @@ LIBRARY LIBBZ2 DESCRIPTION "libbzip2: library for data compression" EXPORTS - bzCompressInit - bzCompress - bzCompressEnd - bzDecompressInit - bzDecompress - bzDecompressEnd - bzReadOpen - bzReadClose - bzReadGetUnused - bzRead - bzWriteOpen - bzWrite - bzWriteClose - bzBuffToBuffCompress - bzBuffToBuffDecompress - bzlibVersion - bzopen - bzdopen - bzread - bzwrite - bzflush - bzclose + BZ2_bzCompressInit + BZ2_bzCompress + BZ2_bzCompressEnd + BZ2_bzDecompressInit + BZ2_bzDecompress + BZ2_bzDecompressEnd + BZ2_bzReadOpen + BZ2_bzReadClose + BZ2_bzReadGetUnused + BZ2_bzRead + BZ2_bzWriteOpen + BZ2_bzWrite + BZ2_bzWriteClose + BZ2_bzWriteClose64 + BZ2_bzBuffToBuffCompress + BZ2_bzBuffToBuffDecompress + BZ2_bzlibVersion + BZ2_bzopen + BZ2_bzdopen + BZ2_bzread + BZ2_bzwrite + BZ2_bzflush + BZ2_bzclose + BZ2_bzerror diff --git a/makefile.msc b/makefile.msc index 4b49f78..3fe4232 100644 --- a/makefile.msc +++ b/makefile.msc @@ -4,7 +4,7 @@ # Fixed up by JRS for bzip2-0.9.5d release. CC=cl -CFLAGS= -DWIN32 -MD -Ox +CFLAGS= -DWIN32 -MD -Ox -D_FILE_OFFSET_BITS=64 OBJS= blocksort.obj \ huffman.obj \ @@ -21,7 +21,6 @@ bzip2: lib $(CC) $(CFLAGS) -o bzip2recover bzip2recover.c lib: $(OBJS) - del libbz2.lib lib /out:libbz2.lib $(OBJS) test: bzip2 @@ -32,20 +31,19 @@ test: bzip2 .\\bzip2 -d < sample1.bz2 > sample1.tst .\\bzip2 -d < sample2.bz2 > sample2.tst .\\bzip2 -ds < sample3.bz2 > sample3.tst + @echo All six of the fc's should find no differences. + @echo If fc finds an error on sample3.bz2, this could be + @echo because WinZip's 'TAR file smart CR/LF conversion' + @echo is too clever for its own good. Disable this option. + @echo The correct size for sample3.ref is 120,244. If it + @echo is 150,251, WinZip has messed it up. fc sample1.bz2 sample1.rb2 fc sample2.bz2 sample2.rb2 fc sample3.bz2 sample3.rb2 fc sample1.tst sample1.ref fc sample2.tst sample2.ref fc sample3.tst sample3.ref - @echo All six of the fc's should find no differences. - @echo If fc finds an error on sample3.tst, this could be - @echo because WinZips 'TAR file smart CR/LF conversion' - @echo is too clever for its own good. Disable this option. - @echo The correct size for sample3.ref is 120,244. If it - @echo is around 150k, WinZip has stuffed it up. - @echo Also remember to set BZ_UNIX to 0 and BZ_LCCWIN32 - @echo to 1 in bzip2.c. + clean: diff --git a/manual.texi b/manual.texi index e48e656..336776a 100644 --- a/manual.texi +++ b/manual.texi @@ -2,10 +2,10 @@ @setfilename bzip2.info @ignore -This file documents bzip2 version 0.9.5, and associated library +This file documents bzip2 version 1.0, and associated library libbzip2, written by Julian Seward (jseward@acm.org). -Copyright (C) 1996-1999 Julian R Seward +Copyright (C) 1996-2000 Julian R Seward Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice @@ -30,8 +30,8 @@ END-INFO-DIR-ENTRY @titlepage @title bzip2 and libbzip2 @subtitle a program and library for data compression -@subtitle copyright (C) 1996-1999 Julian Seward -@subtitle version 0.9.5d of 4 September 1999 +@subtitle copyright (C) 1996-2000 Julian Seward +@subtitle version 1.0 of 21 March 2000 @author Julian Seward @end titlepage @@ -44,7 +44,7 @@ END-INFO-DIR-ENTRY This program, @code{bzip2}, and associated library @code{libbzip2}, are -Copyright (C) 1996-1999 Julian R Seward. All rights reserved. +Copyright (C) 1996-2000 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -82,9 +82,13 @@ Julian Seward, Cambridge, UK. @code{jseward@@acm.org} +@code{http://sourceware.cygnus.com/bzip2} + +@code{http://www.cacheprof.org} + @code{http://www.muraroa.demon.co.uk} -@code{bzip2}/@code{libbzip2} version 0.9.5 of 24 May 1999. +@code{bzip2}/@code{libbzip2} version 1.0 of 21 March 2000. PATENTS: To the best of my knowledge, @code{bzip2} does not use any patented algorithms. However, I do not have the resources available to carry out @@ -130,7 +134,7 @@ and nothing else. @unnumberedsubsubsec NAME @itemize @item @code{bzip2}, @code{bunzip2} -- a block-sorting file compressor, v0.9.5 +- a block-sorting file compressor, v1.0 @item @code{bzcat} - decompresses files to stdout @item @code{bzip2recover} @@ -431,10 +435,10 @@ I/O error messages are not as helpful as they could be. @code{bzip2} tries hard to detect I/O errors and exit cleanly, but the details of what the problem is sometimes seem rather misleading. -This manual page pertains to version 0.9.5 of @code{bzip2}. Compressed +This manual page pertains to version 1.0 of @code{bzip2}. Compressed data created by this version is entirely forwards and backwards -compatible with the previous public releases, versions 0.1pl2 and 0.9.0, -but with the following exception: 0.9.0 and above can correctly +compatible with the previous public releases, versions 0.1pl2, 0.9.0 and +0.9.5, but with the following exception: 0.9.0 and above can correctly decompress multiple concatenated compressed files. 0.1pl2 cannot do this; it will stop after decompressing just the first file in the stream. @@ -486,6 +490,10 @@ The structure of @code{libbzip2}'s interfaces is similar to that of Jean-loup Gailly's and Mark Adler's excellent @code{zlib} library. +All externally visible symbols have names beginning @code{BZ2_}. +This is new in version 1.0. The intention is to minimise pollution +of the namespaces of library clients. + @subsection Low-level summary This interface provides services for compressing and decompressing @@ -498,17 +506,17 @@ The low-level part of the library has no global variables and is therefore thread-safe. Six routines make up the low level interface: -@code{bzCompressInit}, @code{bzCompress}, and @* @code{bzCompressEnd} +@code{BZ2_bzCompressInit}, @code{BZ2_bzCompress}, and @* @code{BZ2_bzCompressEnd} for compression, -and a corresponding trio @code{bzDecompressInit}, @* @code{bzDecompress} -and @code{bzDecompressEnd} for decompression. +and a corresponding trio @code{BZ2_bzDecompressInit}, @* @code{BZ2_bzDecompress} +and @code{BZ2_bzDecompressEnd} for decompression. The @code{*Init} functions allocate memory for compression/decompression and do other initialisations, whilst the @code{*End} functions close down operations and release memory. -The real work is done by @code{bzCompress} and @code{bzDecompress}. -These compress/decompress data from a user-supplied input buffer +The real work is done by @code{BZ2_bzCompress} and @code{BZ2_bzDecompress}. +These compress and decompress data from a user-supplied input buffer to a user-supplied output buffer. These buffers can be any size; arbitrary quantities of data are handled by making repeated calls to these functions. This is a flexible mechanism allowing a @@ -526,10 +534,10 @@ reading files in which the @code{bzip2} data stream is embedded within some larger-scale file structure, or where there are multiple @code{bzip2} data streams concatenated end-to-end. -For reading files, @code{bzReadOpen}, @code{bzRead}, @code{bzReadClose} -and @code{bzReadGetUnused} are supplied. For writing files, -@code{bzWriteOpen}, @code{bzWrite} and @code{bzWriteFinish} are -available. +For reading files, @code{BZ2_bzReadOpen}, @code{BZ2_bzRead}, +@code{BZ2_bzReadClose} and @* @code{BZ2_bzReadGetUnused} are supplied. For +writing files, @code{BZ2_bzWriteOpen}, @code{BZ2_bzWrite} and +@code{BZ2_bzWriteFinish} are available. As with the low-level library, no global variables are used so the library is per se thread-safe. However, if I/O errors @@ -539,7 +547,7 @@ the error. In that case, you'd need a C library which correctly supports @code{errno} in a multithreaded environment. To make the library a little simpler and more portable, -@code{bzReadOpen} and @code{bzWriteOpen} require you to pass them file +@code{BZ2_bzReadOpen} and @code{BZ2_bzWriteOpen} require you to pass them file handles (@code{FILE*}s) which have previously been opened for reading or writing respectively. That avoids portability problems associated with file operations and file attributes, whilst not being much of an @@ -548,8 +556,8 @@ imposition on the programmer. @subsection Utility functions summary -For very simple needs, @code{bzBuffToBuffCompress} and -@code{bzBuffToBuffDecompress} are provided. These compress +For very simple needs, @code{BZ2_bzBuffToBuffCompress} and +@code{BZ2_bzBuffToBuffDecompress} are provided. These compress data in memory from one buffer to another buffer in a single function call. You should assess whether these functions fulfill your memory-to-memory compression/decompression @@ -559,9 +567,9 @@ general but more complex low-level interface. Yoshioka Tsuneo (@code{QWF00133@@niftyserve.or.jp} / @code{tsuneo-y@@is.aist-nara.ac.jp}) has contributed some functions to give better @code{zlib} compatibility. These functions are -@code{bzopen}, @code{bzread}, @code{bzwrite}, @code{bzflush}, -@code{bzclose}, -@code{bzerror} and @code{bzlibVersion}. You may find these functions +@code{BZ2_bzopen}, @code{BZ2_bzread}, @code{BZ2_bzwrite}, @code{BZ2_bzflush}, +@code{BZ2_bzclose}, +@code{BZ2_bzerror} and @code{BZ2_bzlibVersion}. You may find these functions more convenient for simple file reading and writing, than those in the high-level interface. These functions are not (yet) officially part of the library, and are minimally documented here. If they break, you @@ -582,6 +590,15 @@ if you are feeling especially paranoid. I would be interested in hearing more about the robustness of the library to corrupted compressed data. +Version 1.0 is much more robust in this respect than +0.9.0 or 0.9.5. Investigations with Checker (a tool for +detecting problems with memory management, similar to Purify) +indicate that, at least for the few files I tested, all single-bit +errors in the decompressed data are caught properly, with no +segmentation faults, no reads of uninitialised data and no +out of range reads or writes. So it's certainly much improved, +although I wouldn't claim it to be totally bombproof. + The file @code{bzlib.h} contains all definitions needed to use the library. In particular, you should definitely not include @code{bzlib_private.h}. @@ -598,7 +615,7 @@ The requested action was completed successfully. @item BZ_RUN_OK @itemx BZ_FLUSH_OK @itemx BZ_FINISH_OK -In @code{bzCompress}, the requested flush/finish/nothing-special action +In @code{BZ2_bzCompress}, the requested flush/finish/nothing-special action was completed successfully. @item BZ_STREAM_END Compression of data was completed, or the logical stream end was @@ -607,6 +624,16 @@ detected during decompression. The following return values indicate an error of some kind. @table @code +@item BZ_CONFIG_ERROR +Indicates that the library has been improperly compiled on your +platform -- a major configuration error. Specifically, it means +that @code{sizeof(char)}, @code{sizeof(short)} and @code{sizeof(int)} +are not 1, 2 and 4 respectively, as they should be. Note that the +library should still work properly on 64-bit platforms which follow +the LP64 programming model -- that is, where @code{sizeof(long)} +and @code{sizeof(void*)} are 8. Under LP64, @code{sizeof(int)} is +still 4, so @code{libbzip2}, which doesn't use the @code{long} type, +is OK. @item BZ_SEQUENCE_ERROR When using the library, it is important to call the functions in the correct sequence and with data structures (buffers etc) in the correct @@ -624,10 +651,10 @@ making. @item BZ_MEM_ERROR Returned when a request to allocate memory failed. Note that the quantity of memory needed to decompress a stream cannot be determined -until the stream's header has been read. So @code{bzDecompress} and -@code{bzRead} may return @code{BZ_MEM_ERROR} even though some of +until the stream's header has been read. So @code{BZ2_bzDecompress} and +@code{BZ2_bzRead} may return @code{BZ_MEM_ERROR} even though some of the compressed data has been read. The same is not true for -compression; once @code{bzCompressInit} or @code{bzWriteOpen} have +compression; once @code{BZ2_bzCompressInit} or @code{BZ2_bzWriteOpen} have successfully completed, @code{BZ_MEM_ERROR} cannot occur. @item BZ_DATA_ERROR Returned when a data integrity error is detected during decompression. @@ -639,19 +666,19 @@ As a special case of @code{BZ_DATA_ERROR}, it is sometimes useful to know when the compressed stream does not start with the correct magic bytes (@code{'B' 'Z' 'h'}). @item BZ_IO_ERROR -Returned by @code{bzRead} and @code{bzRead} when there is an error -reading or writing in the compressed file, and by @code{bzReadOpen} -and @code{bzWriteOpen} for attempts to use a file for which the +Returned by @code{BZ2_bzRead} and @code{BZ2_bzWrite} when there is an error +reading or writing in the compressed file, and by @code{BZ2_bzReadOpen} +and @code{BZ2_bzWriteOpen} for attempts to use a file for which the error indicator (viz, @code{ferror(f)}) is set. On receipt of @code{BZ_IO_ERROR}, the caller should consult @code{errno} and/or @code{perror} to acquire operating-system specific information about the problem. @item BZ_UNEXPECTED_EOF -Returned by @code{bzRead} when the compressed file finishes +Returned by @code{BZ2_bzRead} when the compressed file finishes before the logical end of stream is detected. @item BZ_OUTBUFF_FULL -Returned by @code{bzBuffToBuffCompress} and -@code{bzBuffToBuffDecompress} to indicate that the output data +Returned by @code{BZ2_bzBuffToBuffCompress} and +@code{BZ2_bzBuffToBuffDecompress} to indicate that the output data will not fit into the output buffer provided. @end table @@ -659,17 +686,19 @@ will not fit into the output buffer provided. @section Low-level interface -@subsection @code{bzCompressInit} +@subsection @code{BZ2_bzCompressInit} @example typedef struct @{ char *next_in; unsigned int avail_in; - unsigned int total_in; + unsigned int total_in_lo32; + unsigned int total_in_hi32; char *next_out; unsigned int avail_out; - unsigned int total_out; + unsigned int total_out_lo32; + unsigned int total_out_hi32; void *state; @@ -679,10 +708,10 @@ typedef @} bz_stream; -int bzCompressInit ( bz_stream *strm, - int blockSize100k, - int verbosity, - int workFactor ); +int BZ2_bzCompressInit ( bz_stream *strm, + int blockSize100k, + int verbosity, + int workFactor ); @end example @@ -712,14 +741,19 @@ If you don't want to use a custom memory allocator, set @code{bzalloc}, and the library will then use the standard @code{malloc}/@code{free} routines. -Before calling @code{bzCompressInit}, fields @code{bzalloc}, +Before calling @code{BZ2_bzCompressInit}, fields @code{bzalloc}, @code{bzfree} and @code{opaque} should be filled appropriately, as just described. Upon return, the internal -state will have been allocated and initialised, and @code{total_in} and -@code{total_out} will have been set to zero. -These last two fields are used by the library +state will have been allocated and initialised, and @code{total_in_lo32}, +@code{total_in_hi32}, @code{total_out_lo32} and +@code{total_out_hi32} will have been set to zero. +These four fields are used by the library to inform the caller of the total amount of data passed into and out of the library, respectively. You should not try to change them. +As of version 1.0, 64-bit counts are maintained, even on 32-bit +platforms, using the @code{_hi32} fields to store the upper 32 bits +of the count. So, for example, the total amount of data in +is @code{(total_in_hi32 << 32) + total_in_lo32}. Parameter @code{blockSize100k} specifies the block size to be used for compression. It should be a value between 1 and 9 inclusive, and the @@ -761,6 +795,8 @@ mechanism would render the parameter obsolete. Possible return values: @display + @code{BZ_CONFIG_ERROR} + if the library has been mis-compiled @code{BZ_PARAM_ERROR} if @code{strm} is @code{NULL} or @code{blockSize} < 1 or @code{blockSize} > 9 @@ -773,86 +809,86 @@ Possible return values: @end display Allowable next actions: @display - @code{bzCompress} + @code{BZ2_bzCompress} if @code{BZ_OK} is returned no specific action needed in case of error @end display -@subsection @code{bzCompress} +@subsection @code{BZ2_bzCompress} @example - int bzCompress ( bz_stream *strm, int action ); + int BZ2_bzCompress ( bz_stream *strm, int action ); @end example Provides more input and/or output buffer space for the library. The -caller maintains input and output buffers, and calls @code{bzCompress} to +caller maintains input and output buffers, and calls @code{BZ2_bzCompress} to transfer data between them. -Before each call to @code{bzCompress}, @code{next_in} should point at +Before each call to @code{BZ2_bzCompress}, @code{next_in} should point at the data to be compressed, and @code{avail_in} should indicate how many -bytes the library may read. @code{bzCompress} updates @code{next_in}, +bytes the library may read. @code{BZ2_bzCompress} updates @code{next_in}, @code{avail_in} and @code{total_in} to reflect the number of bytes it has read. Similarly, @code{next_out} should point to a buffer in which the compressed data is to be placed, with @code{avail_out} indicating how -much output space is available. @code{bzCompress} updates +much output space is available. @code{BZ2_bzCompress} updates @code{next_out}, @code{avail_out} and @code{total_out} to reflect the number of bytes output. You may provide and remove as little or as much data as you like on each -call of @code{bzCompress}. In the limit, it is acceptable to supply and +call of @code{BZ2_bzCompress}. In the limit, it is acceptable to supply and remove data one byte at a time, although this would be terribly inefficient. You should always ensure that at least one byte of output space is available at each call. -A second purpose of @code{bzCompress} is to request a change of mode of the +A second purpose of @code{BZ2_bzCompress} is to request a change of mode of the compressed stream. Conceptually, a compressed stream can be in one of four states: IDLE, RUNNING, FLUSHING and FINISHING. Before initialisation -(@code{bzCompressInit}) and after termination (@code{bzCompressEnd}), a +(@code{BZ2_bzCompressInit}) and after termination (@code{BZ2_bzCompressEnd}), a stream is regarded as IDLE. -Upon initialisation (@code{bzCompressInit}), the stream is placed in the -RUNNING state. Subsequent calls to @code{bzCompress} should pass +Upon initialisation (@code{BZ2_bzCompressInit}), the stream is placed in the +RUNNING state. Subsequent calls to @code{BZ2_bzCompress} should pass @code{BZ_RUN} as the requested action; other actions are illegal and will result in @code{BZ_SEQUENCE_ERROR}. At some point, the calling program will have provided all the input data it wants to. It will then want to finish up -- in effect, asking the library to process any data it might have buffered internally. In this -state, @code{bzCompress} will no longer attempt to read data from +state, @code{BZ2_bzCompress} will no longer attempt to read data from @code{next_in}, but it will want to write data to @code{next_out}. Because the output buffer supplied by the user can be arbitrarily small, the finishing-up operation cannot necessarily be done with a single call -of @code{bzCompress}. +of @code{BZ2_bzCompress}. Instead, the calling program passes @code{BZ_FINISH} as an action to -@code{bzCompress}. This changes the stream's state to FINISHING. Any +@code{BZ2_bzCompress}. This changes the stream's state to FINISHING. Any remaining input (ie, @code{next_in[0 .. avail_in-1]}) is compressed and -transferred to the output buffer. To do this, @code{bzCompress} must be +transferred to the output buffer. To do this, @code{BZ2_bzCompress} must be called repeatedly until all the output has been consumed. At that -point, @code{bzCompress} returns @code{BZ_STREAM_END}, and the stream's -state is set back to IDLE. @code{bzCompressEnd} should then be +point, @code{BZ2_bzCompress} returns @code{BZ_STREAM_END}, and the stream's +state is set back to IDLE. @code{BZ2_bzCompressEnd} should then be called. Just to make sure the calling program does not cheat, the library makes a note of @code{avail_in} at the time of the first call to -@code{bzCompress} which has @code{BZ_FINISH} as an action (ie, at the +@code{BZ2_bzCompress} which has @code{BZ_FINISH} as an action (ie, at the time the program has announced its intention to not supply any more input). By comparing this value with that of @code{avail_in} over -subsequent calls to @code{bzCompress}, the library can detect any +subsequent calls to @code{BZ2_bzCompress}, the library can detect any attempts to slip in more data to compress. Any calls for which this is detected will return @code{BZ_SEQUENCE_ERROR}. This indicates a programming mistake which should be corrected. Instead of asking to finish, the calling program may ask -@code{bzCompress} to take all the remaining input, compress it and +@code{BZ2_bzCompress} to take all the remaining input, compress it and terminate the current (Burrows-Wheeler) compression block. This could be useful for error control purposes. The mechanism is analogous to -that for finishing: call @code{bzCompress} with an action of +that for finishing: call @code{BZ2_bzCompress} with an action of @code{BZ_FLUSH}, remove output data, and persist with the @code{BZ_FLUSH} action until the value @code{BZ_RUN} is returned. As -with finishing, @code{bzCompress} detects any attempt to provide more +with finishing, @code{BZ2_bzCompress} detects any attempt to provide more input data once the flush has begun. Once the flush is complete, the stream returns to the normal RUNNING @@ -863,11 +899,11 @@ which shows which actions are allowable in each state, what action will be taken, what the next state is, and what the non-error return values are. Note that you can't explicitly ask what state the stream is in, but nor do you need to -- it can be inferred from the -values returned by @code{bzCompress}. +values returned by @code{BZ2_bzCompress}. @display IDLE/@code{any} - Illegal. IDLE state only exists after @code{bzCompressEnd} or - before @code{bzCompressInit}. + Illegal. IDLE state only exists after @code{BZ2_bzCompressEnd} or + before @code{BZ2_bzCompressInit}. Return value = @code{BZ_SEQUENCE_ERROR} RUNNING/@code{BZ_RUN} @@ -917,21 +953,21 @@ FINISHING/other That still looks complicated? Well, fair enough. The usual sequence of calls for compressing a load of data is: @itemize @bullet -@item Get started with @code{bzCompressInit}. +@item Get started with @code{BZ2_bzCompressInit}. @item Shovel data in and shlurp out its compressed form using zero or more -calls of @code{bzCompress} with action = @code{BZ_RUN}. +calls of @code{BZ2_bzCompress} with action = @code{BZ_RUN}. @item Finish up. -Repeatedly call @code{bzCompress} with action = @code{BZ_FINISH}, +Repeatedly call @code{BZ2_bzCompress} with action = @code{BZ_FINISH}, copying out the compressed output, until @code{BZ_STREAM_END} is returned. -@item Close up and go home. Call @code{bzCompressEnd}. +@item Close up and go home. Call @code{BZ2_bzCompressEnd}. @end itemize If the data you want to compress fits into your input buffer all -at once, you can skip the calls of @code{bzCompress ( ..., BZ_RUN )} and -just do the @code{bzCompress ( ..., BZ_FINISH )} calls. +at once, you can skip the calls of @code{BZ2_bzCompress ( ..., BZ_RUN )} and +just do the @code{BZ2_bzCompress ( ..., BZ_FINISH )} calls. -All required memory is allocated by @code{bzCompressInit}. The +All required memory is allocated by @code{BZ2_bzCompressInit}. The compression library can accept any data at all (obviously). So you -shouldn't get any error return values from the @code{bzCompress} calls. +shouldn't get any error return values from the @code{BZ2_bzCompress} calls. If you do, they will be @code{BZ_SEQUENCE_ERROR}, and indicate a bug in your programming. @@ -941,9 +977,9 @@ Trivial other possible return values: if @code{strm} is @code{NULL}, or @code{strm->s} is @code{NULL} @end display -@subsection @code{bzCompressEnd} +@subsection @code{BZ2_bzCompressEnd} @example -int bzCompressEnd ( bz_stream *strm ); +int BZ2_bzCompressEnd ( bz_stream *strm ); @end example Releases all memory associated with a compression stream. @@ -954,11 +990,11 @@ Possible return values: @end display -@subsection @code{bzDecompressInit} +@subsection @code{BZ2_bzDecompressInit} @example -int bzDecompressInit ( bz_stream *strm, int verbosity, int small ); +int BZ2_bzDecompressInit ( bz_stream *strm, int verbosity, int small ); @end example -Prepares for decompression. As with @code{bzCompressInit}, a +Prepares for decompression. As with @code{BZ2_bzCompressInit}, a @code{bz_stream} record should be allocated and initialised before the call. Fields @code{bzalloc}, @code{bzfree} and @code{opaque} should be set if a custom memory allocator is required, or made @code{NULL} for @@ -966,7 +1002,7 @@ the normal @code{malloc}/@code{free} routines. Upon return, the internal state will have been initialised, and @code{total_in} and @code{total_out} will be zero. -For the meaning of parameter @code{verbosity}, see @code{bzCompressInit}. +For the meaning of parameter @code{verbosity}, see @code{BZ2_bzCompressInit}. If @code{small} is nonzero, the library will use an alternative decompression algorithm which uses less memory but at the cost of @@ -976,11 +1012,13 @@ more information on memory management. Note that the amount of memory needed to decompress a stream cannot be determined until the stream's header has been read, -so even if @code{bzDecompressInit} succeeds, a subsequent -@code{bzDecompress} could fail with @code{BZ_MEM_ERROR}. +so even if @code{BZ2_bzDecompressInit} succeeds, a subsequent +@code{BZ2_bzDecompress} could fail with @code{BZ_MEM_ERROR}. Possible return values: @display + @code{BZ_CONFIG_ERROR} + if the library has been mis-compiled @code{BZ_PARAM_ERROR} if @code{(small != 0 && small != 1)} or @code{(verbosity < 0 || verbosity > 4)} @@ -990,54 +1028,54 @@ Possible return values: Allowable next actions: @display - @code{bzDecompress} + @code{BZ2_bzDecompress} if @code{BZ_OK} was returned no specific action required in case of error @end display -@subsection @code{bzDecompress} +@subsection @code{BZ2_bzDecompress} @example -int bzDecompress ( bz_stream *strm ); +int BZ2_bzDecompress ( bz_stream *strm ); @end example Provides more input and/out output buffer space for the library. The -caller maintains input and output buffers, and uses @code{bzDecompress} +caller maintains input and output buffers, and uses @code{BZ2_bzDecompress} to transfer data between them. -Before each call to @code{bzDecompress}, @code{next_in} +Before each call to @code{BZ2_bzDecompress}, @code{next_in} should point at the compressed data, and @code{avail_in} should indicate how many bytes the library -may read. @code{bzDecompress} updates @code{next_in}, @code{avail_in} +may read. @code{BZ2_bzDecompress} updates @code{next_in}, @code{avail_in} and @code{total_in} to reflect the number of bytes it has read. Similarly, @code{next_out} should point to a buffer in which the uncompressed output is to be placed, with @code{avail_out} indicating how much output space -is available. @code{bzCompress} updates @code{next_out}, +is available. @code{BZ2_bzCompress} updates @code{next_out}, @code{avail_out} and @code{total_out} to reflect the number of bytes output. You may provide and remove as little or as much data as you like on -each call of @code{bzDecompress}. +each call of @code{BZ2_bzDecompress}. In the limit, it is acceptable to supply and remove data one byte at a time, although this would be terribly inefficient. You should always ensure that at least one byte of output space is available at each call. -Use of @code{bzDecompress} is simpler than @code{bzCompress}. +Use of @code{BZ2_bzDecompress} is simpler than @code{BZ2_bzCompress}. You should provide input and remove output as described above, and -repeatedly call @code{bzDecompress} until @code{BZ_STREAM_END} is +repeatedly call @code{BZ2_bzDecompress} until @code{BZ_STREAM_END} is returned. Appearance of @code{BZ_STREAM_END} denotes that -@code{bzDecompress} has detected the logical end of the compressed -stream. @code{bzDecompress} will not produce @code{BZ_STREAM_END} until +@code{BZ2_bzDecompress} has detected the logical end of the compressed +stream. @code{BZ2_bzDecompress} will not produce @code{BZ_STREAM_END} until all output data has been placed into the output buffer, so once @code{BZ_STREAM_END} appears, you are guaranteed to have available all -the decompressed output, and @code{bzDecompressEnd} can safely be +the decompressed output, and @code{BZ2_bzDecompressEnd} can safely be called. -If case of an error return value, you should call @code{bzDecompressEnd} +If case of an error return value, you should call @code{BZ2_bzDecompressEnd} to clean up and release memory. Possible return values: @@ -1059,16 +1097,16 @@ Possible return values: @end display Allowable next actions: @display - @code{bzDecompress} + @code{BZ2_bzDecompress} if @code{BZ_OK} was returned - @code{bzDecompressEnd} + @code{BZ2_bzDecompressEnd} otherwise @end display -@subsection @code{bzDecompressEnd} +@subsection @code{BZ2_bzDecompressEnd} @example -int bzDecompressEnd ( bz_stream *strm ); +int BZ2_bzDecompressEnd ( bz_stream *strm ); @end example Releases all memory associated with a decompression stream. @@ -1107,16 +1145,16 @@ This interface provides functions for reading and writing given on a per-function basis below. @item If @code{bzerror} indicates an error (ie, anything except @code{BZ_OK} and @code{BZ_STREAM_END}), - you should immediately call @code{bzReadClose} (or @code{bzWriteClose}, + you should immediately call @code{BZ2_bzReadClose} (or @code{BZ2_bzWriteClose}, depending on whether you are attempting to read or to write) to free up all resources associated with the stream. Once an error has been indicated, behaviour of all calls - except @code{bzReadClose} (@code{bzWriteClose}) is undefined. + except @code{BZ2_bzReadClose} (@code{BZ2_bzWriteClose}) is undefined. The implication is that (1) @code{bzerror} should be checked after each call, and (2) if @code{bzerror} indicates an error, - @code{bzReadClose} (@code{bzWriteClose}) should then be called to clean up. + @code{BZ2_bzReadClose} (@code{BZ2_bzWriteClose}) should then be called to clean up. @item The @code{FILE*} arguments passed to - @code{bzReadOpen}/@code{bzWriteOpen} + @code{BZ2_bzReadOpen}/@code{BZ2_bzWriteOpen} should be set to binary mode. Most Unix systems will do this by default, but other platforms, including Windows and Mac, will not. If you omit this, you may @@ -1130,13 +1168,13 @@ This interface provides functions for reading and writing -@subsection @code{bzReadOpen} +@subsection @code{BZ2_bzReadOpen} @example typedef void BZFILE; - BZFILE *bzReadOpen ( int *bzerror, FILE *f, - int small, int verbosity, - void *unused, int nUnused ); + BZFILE *BZ2_bzReadOpen ( int *bzerror, FILE *f, + int small, int verbosity, + void *unused, int nUnused ); @end example Prepare to read compressed data from file handle @code{f}. @code{f} should refer to a file which has been opened for reading, and for which @@ -1144,7 +1182,7 @@ the error indicator (@code{ferror(f)})is not set. If @code{small} is 1, the library will try to decompress using less memory, at the expense of speed. -For reasons explained below, @code{bzRead} will decompress the +For reasons explained below, @code{BZ2_bzRead} will decompress the @code{nUnused} bytes starting at @code{unused}, before starting to read from the file @code{f}. At most @code{BZ_MAX_UNUSED} bytes may be supplied like this. If this facility is not required, you should pass @@ -1152,15 +1190,17 @@ supplied like this. If this facility is not required, you should pass respectively. For the meaning of parameters @code{small} and @code{verbosity}, -see @code{bzDecompressInit}. +see @code{BZ2_bzDecompressInit}. The amount of memory needed to decompress a file cannot be determined until the file's header has been read. So it is possible that -@code{bzReadOpen} returns @code{BZ_OK} but a subsequent call of -@code{bzRead} will return @code{BZ_MEM_ERROR}. +@code{BZ2_bzReadOpen} returns @code{BZ_OK} but a subsequent call of +@code{BZ2_bzRead} will return @code{BZ_MEM_ERROR}. Possible assignments to @code{bzerror}: @display + @code{BZ_CONFIG_ERROR} + if the library has been mis-compiled @code{BZ_PARAM_ERROR} if @code{f} is @code{NULL} or @code{small} is neither @code{0} nor @code{1} @@ -1184,16 +1224,16 @@ Possible return values: Allowable next actions: @display - @code{bzRead} + @code{BZ2_bzRead} if @code{bzerror} is @code{BZ_OK} - @code{bzClose} + @code{BZ2_bzClose} otherwise @end display -@subsection @code{bzRead} +@subsection @code{BZ2_bzRead} @example - int bzRead ( int *bzerror, BZFILE *b, void *buf, int len ); + int BZ2_bzRead ( int *bzerror, BZFILE *b, void *buf, int len ); @end example Reads up to @code{len} (uncompressed) bytes from the compressed file @code{b} into @@ -1204,7 +1244,7 @@ was detected, @code{bzerror} will be set to @code{BZ_STREAM_END}, and the number of bytes read is returned. All other @code{bzerror} values denote an error. -@code{bzRead} will supply @code{len} bytes, +@code{BZ2_bzRead} will supply @code{len} bytes, unless the logical stream end is detected or an error occurs. Because of this, it is possible to detect the stream end by observing when the number of bytes returned is @@ -1213,20 +1253,20 @@ requested. Nevertheless, this is regarded as inadvisable; you should instead check @code{bzerror} after every call and watch out for @code{BZ_STREAM_END}. -Internally, @code{bzRead} copies data from the compressed file in chunks +Internally, @code{BZ2_bzRead} copies data from the compressed file in chunks of size @code{BZ_MAX_UNUSED} bytes before decompressing it. If the file contains more bytes than strictly -needed to reach the logical end-of-stream, @code{bzRead} will almost certainly +needed to reach the logical end-of-stream, @code{BZ2_bzRead} will almost certainly read some of the trailing data before signalling @code{BZ_SEQUENCE_END}. To collect the read but unused data once @code{BZ_SEQUENCE_END} has -appeared, call @code{bzReadGetUnused} immediately before @code{bzReadClose}. +appeared, call @code{BZ2_bzReadGetUnused} immediately before @code{BZ2_bzReadClose}. Possible assignments to @code{bzerror}: @display @code{BZ_PARAM_ERROR} if @code{b} is @code{NULL} or @code{buf} is @code{NULL} or @code{len < 0} @code{BZ_SEQUENCE_ERROR} - if @code{b} was opened with @code{bzWriteOpen} + if @code{b} was opened with @code{BZ2_bzWriteOpen} @code{BZ_IO_ERROR} if there is an error reading from the compressed file @code{BZ_UNEXPECTED_EOF} @@ -1254,28 +1294,28 @@ Possible return values: Allowable next actions: @display - collect data from @code{buf}, then @code{bzRead} or @code{bzReadClose} + collect data from @code{buf}, then @code{BZ2_bzRead} or @code{BZ2_bzReadClose} if @code{bzerror} is @code{BZ_OK} - collect data from @code{buf}, then @code{bzReadClose} or @code{bzReadGetUnused} + collect data from @code{buf}, then @code{BZ2_bzReadClose} or @code{BZ2_bzReadGetUnused} if @code{bzerror} is @code{BZ_SEQUENCE_END} - @code{bzReadClose} + @code{BZ2_bzReadClose} otherwise @end display -@subsection @code{bzReadGetUnused} +@subsection @code{BZ2_bzReadGetUnused} @example - void bzReadGetUnused ( int* bzerror, BZFILE *b, - void** unused, int* nUnused ); + void BZ2_bzReadGetUnused ( int* bzerror, BZFILE *b, + void** unused, int* nUnused ); @end example Returns data which was read from the compressed file but was not needed to get to the logical end-of-stream. @code{*unused} is set to the address of the data, and @code{*nUnused} to the number of bytes. @code{*nUnused} will be set to a value between @code{0} and @code{BZ_MAX_UNUSED} inclusive. -This function may only be called once @code{bzRead} has signalled -@code{BZ_STREAM_END} but before @code{bzReadClose}. +This function may only be called once @code{BZ2_bzRead} has signalled +@code{BZ_STREAM_END} but before @code{BZ2_bzReadClose}. Possible assignments to @code{bzerror}: @display @@ -1284,31 +1324,31 @@ Possible assignments to @code{bzerror}: or @code{unused} is @code{NULL} or @code{nUnused} is @code{NULL} @code{BZ_SEQUENCE_ERROR} if @code{BZ_STREAM_END} has not been signalled - or if @code{b} was opened with @code{bzWriteOpen} + or if @code{b} was opened with @code{BZ2_bzWriteOpen} @code{BZ_OK} otherwise @end display Allowable next actions: @display - @code{bzReadClose} + @code{BZ2_bzReadClose} @end display -@subsection @code{bzReadClose} +@subsection @code{BZ2_bzReadClose} @example - void bzReadClose ( int *bzerror, BZFILE *b ); + void BZ2_bzReadClose ( int *bzerror, BZFILE *b ); @end example Releases all memory pertaining to the compressed file @code{b}. -@code{bzReadClose} does not call @code{fclose} on the underlying file +@code{BZ2_bzReadClose} does not call @code{fclose} on the underlying file handle, so you should do that yourself if appropriate. -@code{bzReadClose} should be called to clean up after all error +@code{BZ2_bzReadClose} should be called to clean up after all error situations. Possible assignments to @code{bzerror}: @display @code{BZ_SEQUENCE_ERROR} - if @code{b} was opened with @code{bzOpenWrite} + if @code{b} was opened with @code{BZ2_bzOpenWrite} @code{BZ_OK} otherwise @end display @@ -1320,11 +1360,11 @@ Allowable next actions: -@subsection @code{bzWriteOpen} +@subsection @code{BZ2_bzWriteOpen} @example - BZFILE *bzWriteOpen ( int *bzerror, FILE *f, - int blockSize100k, int verbosity, - int workFactor ); + BZFILE *BZ2_bzWriteOpen ( int *bzerror, FILE *f, + int blockSize100k, int verbosity, + int workFactor ); @end example Prepare to write compressed data to file handle @code{f}. @code{f} should refer to @@ -1333,14 +1373,16 @@ indicator (@code{ferror(f)})is not set. For the meaning of parameters @code{blockSize100k}, @code{verbosity} and @code{workFactor}, see -@* @code{bzCompressInit}. +@* @code{BZ2_bzCompressInit}. All required memory is allocated at this stage, so if the call completes successfully, @code{BZ_MEM_ERROR} cannot be signalled by a -subsequent call to @code{bzWrite}. +subsequent call to @code{BZ2_bzWrite}. Possible assignments to @code{bzerror}: @display + @code{BZ_CONFIG_ERROR} + if the library has been mis-compiled @code{BZ_PARAM_ERROR} if @code{f} is @code{NULL} or @code{blockSize100k < 1} or @code{blockSize100k > 9} @@ -1362,18 +1404,18 @@ Possible return values: Allowable next actions: @display - @code{bzWrite} + @code{BZ2_bzWrite} if @code{bzerror} is @code{BZ_OK} - (you could go directly to @code{bzWriteClose}, but this would be pretty pointless) - @code{bzWriteClose} + (you could go directly to @code{BZ2_bzWriteClose}, but this would be pretty pointless) + @code{BZ2_bzWriteClose} otherwise @end display -@subsection @code{bzWrite} +@subsection @code{BZ2_bzWrite} @example - void bzWrite ( int *bzerror, BZFILE *b, void *buf, int len ); + void BZ2_bzWrite ( int *bzerror, BZFILE *b, void *buf, int len ); @end example Absorbs @code{len} bytes from the buffer @code{buf}, eventually to be compressed and written to the file. @@ -1383,7 +1425,7 @@ Possible assignments to @code{bzerror}: @code{BZ_PARAM_ERROR} if @code{b} is @code{NULL} or @code{buf} is @code{NULL} or @code{len < 0} @code{BZ_SEQUENCE_ERROR} - if b was opened with @code{bzReadOpen} + if b was opened with @code{BZ2_bzReadOpen} @code{BZ_IO_ERROR} if there is an error writing the compressed file. @code{BZ_OK} @@ -1393,22 +1435,29 @@ Possible assignments to @code{bzerror}: -@subsection @code{bzWriteClose} +@subsection @code{BZ2_bzWriteClose} @example - int bzWriteClose ( int *bzerror, BZFILE* f, - int abandon, - unsigned int* nbytes_in, - unsigned int* nbytes_out ); + void BZ2_bzWriteClose ( int *bzerror, BZFILE* f, + int abandon, + unsigned int* nbytes_in, + unsigned int* nbytes_out ); + + void BZ2_bzWriteClose64 ( int *bzerror, BZFILE* f, + int abandon, + unsigned int* nbytes_in_lo32, + unsigned int* nbytes_in_hi32, + unsigned int* nbytes_out_lo32, + unsigned int* nbytes_out_hi32 ); @end example Compresses and flushes to the compressed file all data so far supplied -by @code{bzWrite}. The logical end-of-stream markers are also written, so -subsequent calls to @code{bzWrite} are illegal. All memory associated +by @code{BZ2_bzWrite}. The logical end-of-stream markers are also written, so +subsequent calls to @code{BZ2_bzWrite} are illegal. All memory associated with the compressed file @code{b} is released. @code{fflush} is called on the compressed file, but it is not @code{fclose}'d. -If @code{bzWriteClose} is called to clean up after an error, the only +If @code{BZ2_bzWriteClose} is called to clean up after an error, the only action is to release the memory. The library records the error codes issued by previous calls, so this situation will be detected automatically. There is no attempt to complete the compression @@ -1418,12 +1467,17 @@ value to @code{abandon}. If @code{nbytes_in} is non-null, @code{*nbytes_in} will be set to be the total volume of uncompressed data handled. Similarly, @code{nbytes_out} -will be set to the total volume of compressed data written. +will be set to the total volume of compressed data written. For +compatibility with older versions of the library, @code{BZ2_bzWriteClose} +only yields the lower 32 bits of these counts. Use +@code{BZ2_bzWriteClose64} if you want the full 64 bit counts. These +two functions are otherwise absolutely identical. + Possible assignments to @code{bzerror}: @display @code{BZ_SEQUENCE_ERROR} - if @code{b} was opened with @code{bzReadOpen} + if @code{b} was opened with @code{BZ2_bzReadOpen} @code{BZ_IO_ERROR} if there is an error writing the compressed file @code{BZ_OK} @@ -1442,26 +1496,26 @@ The calling application can write its own data before and after the compressed data stream, using that same file handle. @item Reading is more complex, and the facilities are not as general as they could be since generality is hard to reconcile with efficiency. -@code{bzRead} reads from the compressed file in blocks of size +@code{BZ2_bzRead} reads from the compressed file in blocks of size @code{BZ_MAX_UNUSED} bytes, and in doing so probably will overshoot the logical end of compressed stream. To recover this data once decompression has -ended, call @code{bzReadGetUnused} after the last call of @code{bzRead} +ended, call @code{BZ2_bzReadGetUnused} after the last call of @code{BZ2_bzRead} (the one returning @code{BZ_STREAM_END}) but before calling -@code{bzReadClose}. +@code{BZ2_bzReadClose}. @end itemize This mechanism makes it easy to decompress multiple @code{bzip2} -streams placed end-to-end. As the end of one stream, when @code{bzRead} -returns @code{BZ_STREAM_END}, call @code{bzReadGetUnused} to collect the +streams placed end-to-end. As the end of one stream, when @code{BZ2_bzRead} +returns @code{BZ_STREAM_END}, call @code{BZ2_bzReadGetUnused} to collect the unused data (copy it into your own buffer somewhere). That data forms the start of the next compressed stream. -To start uncompressing that next stream, call @code{bzReadOpen} again, +To start uncompressing that next stream, call @code{BZ2_bzReadOpen} again, feeding in the unused data via the @code{unused}/@code{nUnused} parameters. Keep doing this until @code{BZ_STREAM_END} return coincides with the physical end of file (@code{feof(f)}). In this situation -@code{bzReadGetUnused} +@code{BZ2_bzReadGetUnused} will of course return no data. This should give some feel for how the high-level interface can be used. @@ -1482,22 +1536,22 @@ f = fopen ( "myfile.bz2", "w" ); if (!f) @{ /* handle error */ @} -b = bzWriteOpen ( &bzerror, f, 9 ); +b = BZ2_bzWriteOpen ( &bzerror, f, 9 ); if (bzerror != BZ_OK) @{ - bzWriteClose ( b ); + BZ2_bzWriteClose ( b ); /* handle error */ @} while ( /* condition */ ) @{ /* get data to write into buf, and set nBuf appropriately */ - nWritten = bzWrite ( &bzerror, b, buf, nBuf ); + nWritten = BZ2_bzWrite ( &bzerror, b, buf, nBuf ); if (bzerror == BZ_IO_ERROR) @{ - bzWriteClose ( &bzerror, b ); + BZ2_bzWriteClose ( &bzerror, b ); /* handle error */ @} @} -bzWriteClose ( &bzerror, b ); +BZ2_bzWriteClose ( &bzerror, b ); if (bzerror == BZ_IO_ERROR) @{ /* handle error */ @} @@ -1515,39 +1569,39 @@ f = fopen ( "myfile.bz2", "r" ); if (!f) @{ /* handle error */ @} -b = bzReadOpen ( &bzerror, f, 0, NULL, 0 ); +b = BZ2_bzReadOpen ( &bzerror, f, 0, NULL, 0 ); if (bzerror != BZ_OK) @{ - bzReadClose ( &bzerror, b ); + BZ2_bzReadClose ( &bzerror, b ); /* handle error */ @} bzerror = BZ_OK; while (bzerror == BZ_OK && /* arbitrary other conditions */) @{ - nBuf = bzRead ( &bzerror, b, buf, /* size of buf */ ); + nBuf = BZ2_bzRead ( &bzerror, b, buf, /* size of buf */ ); if (bzerror == BZ_OK) @{ /* do something with buf[0 .. nBuf-1] */ @} @} if (bzerror != BZ_STREAM_END) @{ - bzReadClose ( &bzerror, b ); + BZ2_bzReadClose ( &bzerror, b ); /* handle error */ @} else @{ - bzReadClose ( &bzerror ); + BZ2_bzReadClose ( &bzerror ); @} @end example @section Utility functions -@subsection @code{bzBuffToBuffCompress} +@subsection @code{BZ2_bzBuffToBuffCompress} @example - int bzBuffToBuffCompress( char* dest, - unsigned int* destLen, - char* source, - unsigned int sourceLen, - int blockSize100k, - int verbosity, - int workFactor ); + int BZ2_bzBuffToBuffCompress( char* dest, + unsigned int* destLen, + char* source, + unsigned int sourceLen, + int blockSize100k, + int verbosity, + int workFactor ); @end example Attempts to compress the data in @code{source[0 .. sourceLen-1]} into the destination buffer, @code{dest[0 .. *destLen-1]}. @@ -1563,17 +1617,19 @@ additional calls to provide extra input data. If you want that kind of mechanism, use the low-level interface. For the meaning of parameters @code{blockSize100k}, @code{verbosity} -and @code{workFactor}, @* see @code{bzCompressInit}. +and @code{workFactor}, @* see @code{BZ2_bzCompressInit}. To guarantee that the compressed data will fit in its buffer, allocate an output buffer of size 1% larger than the uncompressed data, plus six hundred extra bytes. -@code{bzBuffToBuffDecompress} will not write data at or +@code{BZ2_bzBuffToBuffDecompress} will not write data at or beyond @code{dest[*destLen]}, even in case of buffer overflow. Possible return values: @display + @code{BZ_CONFIG_ERROR} + if the library has been mis-compiled @code{BZ_PARAM_ERROR} if @code{dest} is @code{NULL} or @code{destLen} is @code{NULL} or @code{blockSize100k < 1} or @code{blockSize100k > 9} @@ -1589,14 +1645,14 @@ Possible return values: -@subsection @code{bzBuffToBuffDecompress} +@subsection @code{BZ2_bzBuffToBuffDecompress} @example - int bzBuffToBuffDecompress ( char* dest, - unsigned int* destLen, - char* source, - unsigned int sourceLen, - int small, - int verbosity ); + int BZ2_bzBuffToBuffDecompress ( char* dest, + unsigned int* destLen, + char* source, + unsigned int sourceLen, + int small, + int verbosity ); @end example Attempts to decompress the data in @code{source[0 .. sourceLen-1]} into the destination buffer, @code{dest[0 .. *destLen-1]}. @@ -1606,11 +1662,11 @@ returned. If the compressed data won't fit, @code{*destLen} is unchanged, and @code{BZ_OUTBUFF_FULL} is returned. @code{source} is assumed to hold a complete @code{bzip2} format -data stream. @code{bzBuffToBuffDecompress} tries to decompress +data stream. @* @code{BZ2_bzBuffToBuffDecompress} tries to decompress the entirety of the stream into the output buffer. For the meaning of parameters @code{small} and @code{verbosity}, -see @code{bzDecompressInit}. +see @code{BZ2_bzDecompressInit}. Because the compression ratio of the compressed data cannot be known in advance, there is no easy way to guarantee that the output buffer will @@ -1618,11 +1674,13 @@ be big enough. You may of course make arrangements in your code to record the size of the uncompressed data, but such a mechanism is beyond the scope of this library. -@code{bzBuffToBuffDecompress} will not write data at or +@code{BZ2_bzBuffToBuffDecompress} will not write data at or beyond @code{dest[*destLen]}, even in case of buffer overflow. Possible return values: @display + @code{BZ_CONFIG_ERROR} + if the library has been mis-compiled @code{BZ_PARAM_ERROR} if @code{dest} is @code{NULL} or @code{destLen} is @code{NULL} or @code{small != 0 && small != 1} @@ -1646,40 +1704,40 @@ Possible return values: @section @code{zlib} compatibility functions Yoshioka Tsuneo has contributed some functions to give better @code{zlib} compatibility. These functions are -@code{bzopen}, @code{bzread}, @code{bzwrite}, @code{bzflush}, -@code{bzclose}, -@code{bzerror} and @code{bzlibVersion}. +@code{BZ2_bzopen}, @code{BZ2_bzread}, @code{BZ2_bzwrite}, @code{BZ2_bzflush}, +@code{BZ2_bzclose}, +@code{BZ2_bzerror} and @code{BZ2_bzlibVersion}. These functions are not (yet) officially part of the library. If they break, you get to keep all the pieces. Nevertheless, I think they work ok. @example typedef void BZFILE; -const char * bzlibVersion ( void ); +const char * BZ2_bzlibVersion ( void ); @end example Returns a string indicating the library version. @example -BZFILE * bzopen ( const char *path, const char *mode ); -BZFILE * bzdopen ( int fd, const char *mode ); +BZFILE * BZ2_bzopen ( const char *path, const char *mode ); +BZFILE * BZ2_bzdopen ( int fd, const char *mode ); @end example Opens a @code{.bz2} file for reading or writing, using either its name or a pre-existing file descriptor. Analogous to @code{fopen} and @code{fdopen}. @example -int bzread ( BZFILE* b, void* buf, int len ); -int bzwrite ( BZFILE* b, void* buf, int len ); +int BZ2_bzread ( BZFILE* b, void* buf, int len ); +int BZ2_bzwrite ( BZFILE* b, void* buf, int len ); @end example Reads/writes data from/to a previously opened @code{BZFILE}. Analogous to @code{fread} and @code{fwrite}. @example -int bzflush ( BZFILE* b ); -void bzclose ( BZFILE* b ); +int BZ2_bzflush ( BZFILE* b ); +void BZ2_bzclose ( BZFILE* b ); @end example -Flushes/closes a @code{BZFILE}. @code{bzflush} doesn't actually do +Flushes/closes a @code{BZFILE}. @code{BZ2_bzflush} doesn't actually do anything. Analogous to @code{fflush} and @code{fclose}. @example -const char * bzerror ( BZFILE *b, int *errnum ) +const char * BZ2_bzerror ( BZFILE *b, int *errnum ) @end example Returns a string describing the more recent error status of @code{b}, and also sets @code{*errnum} to its numerical value. @@ -1695,9 +1753,9 @@ by compiling the library with preprocessor symbol @code{BZ_NO_STDIO} defined. Doing this gives you a library containing only the following eight functions: -@code{bzCompressInit}, @code{bzCompress}, @code{bzCompressEnd} @* -@code{bzDecompressInit}, @code{bzDecompress}, @code{bzDecompressEnd} @* -@code{bzBuffToBuffCompress}, @code{bzBuffToBuffDecompress} +@code{BZ2_bzCompressInit}, @code{BZ2_bzCompress}, @code{BZ2_bzCompressEnd} @* +@code{BZ2_bzDecompressInit}, @code{BZ2_bzDecompress}, @code{BZ2_bzDecompressEnd} @* +@code{BZ2_bzBuffToBuffCompress}, @code{BZ2_bzBuffToBuffDecompress} When compiled like this, all functions will ignore @code{verbosity} settings. @@ -1710,14 +1768,14 @@ was compiled with @code{BZ_NO_STDIO} set. For a normal compile, an assertion failure yields the message @example - bzip2/libbzip2, v0.9.5: internal error number N. - This is a bug in bzip2/libbzip2, v0.9.5. Please report - it to me at: jseward@@acm.org. If this happened when - you were using some program which uses libbzip2 as a + bzip2/libbzip2: internal error number N. + This is a bug in bzip2/libbzip2, 1.0 of 21-Mar-2000. + Please report it to me at: jseward@@acm.org. If this happened + when you were using some program which uses libbzip2 as a component, you should also report this bug to the author(s) of that program. Please make an effort to report this bug; timely and accurate bug reports eventually lead to higher - quality software. Thanks. Julian Seward, 24 May 1999. + quality software. Thanks. Julian Seward, 21 March 2000. @end example where @code{N} is some error code number. @code{exit(3)} is then called. @@ -1781,7 +1839,7 @@ These are just some random thoughts of mine. Your mileage may vary. @section Limitations of the compressed file format -@code{bzip2-0.9.5} and @code{0.9.0} +@code{bzip2-1.0}, @code{0.9.5} and @code{0.9.0} use exactly the same file format as the previous version, @code{bzip2-0.1}. This decision was made in the interests of stability. Creating yet another incompatible compressed file format @@ -1860,7 +1918,7 @@ require some careful design of compressed file formats. @section Portability issues After some consideration, I have decided not to use -GNU @code{autoconf} to configure 0.9.5. +GNU @code{autoconf} to configure 0.9.5 or 1.0. @code{autoconf}, admirable and wonderful though it is, mainly assists with portability problems between Unix-like @@ -1925,7 +1983,7 @@ If you get problems, try using the flags @code{-O2} @code{-fomit-frame-pointer} @code{-fno-strength-reduce}. You should specifically @emph{not} use @code{-funroll-loops}. -You may notice that the Makefile runs four tests as part of +You may notice that the Makefile runs six tests as part of the build process. If the program passes all of these, it's a pretty good (but not 100%) indication that the compiler has done its job correctly. @@ -2000,6 +2058,7 @@ memory but gets pretty good compression, and has minimal latency, consider Jean-loup Gailly's and Mark Adler's work, @code{zlib-1.1.2} and @code{gzip-1.2.4}. Look for them at + @code{http://www.cdrom.com/pub/infozip/zlib} and @code{http://www.gzip.org} respectively. @@ -2140,7 +2199,14 @@ available from: @example http://www.cs.arizona.edu/people/gene/PAPERS/suffix.ps @end example - +Finally, the following paper documents some recent investigations +I made into the performance of sorting algorithms: +@example +Julian Seward: + On the Performance of BWT Sorting Algorithms + Proceedings of the IEEE Data Compression Conference 2000 + Snowbird, Utah. 28-30 March 2000. +@end example @contents diff --git a/randtable.c b/randtable.c index 8f6266f..983089d 100644 --- a/randtable.c +++ b/randtable.c @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-1999 Julian R Seward. All rights reserved. + Copyright (C) 1996-2000 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -43,7 +43,7 @@ Julian Seward, Cambridge, UK. jseward@acm.org - bzip2/libbzip2 version 0.9.5 of 24 May 1999 + bzip2/libbzip2 version 1.0 of 21 March 2000 This program is based on (at least) the work of: Mike Burrows @@ -63,7 +63,7 @@ /*---------------------------------------------*/ -Int32 rNums[512] = { +Int32 BZ2_rNums[512] = { 619, 720, 127, 481, 931, 816, 813, 233, 566, 247, 985, 724, 205, 454, 863, 491, 741, 242, 949, 214, 733, 859, 335, 708, 621, 574, 73, 654, 730, 472, diff --git a/spewG.c b/spewG.c new file mode 100644 index 0000000..7934e76 --- /dev/null +++ b/spewG.c @@ -0,0 +1,39 @@ + +/* spew out a thoroughly gigantic file designed so that bzip2 + can compress it reasonably rapidly. This is to help test + support for large files (> 2GB) in a reasonable amount of time. + I suggest you use the undocumented --exponential option to + bzip2 when compressing the resulting file; this saves a bit of + time. Note: *don't* bother with --exponential when compressing + Real Files; it'll just waste a lot of CPU time :-) + (but is otherwise harmless). +*/ + +#define _FILE_OFFSET_BITS 64 + +#include +#include + +/* The number of megabytes of junk to spew out (roughly) */ +#define MEGABYTES 5000 + +#define N_BUF 1000000 +char buf[N_BUF]; + +int main ( int argc, char** argv ) +{ + int ii, kk, p; + srandom(1); + setbuffer ( stdout, buf, N_BUF ); + for (kk = 0; kk < MEGABYTES * 515; kk+=3) { + p = 25+random()%50; + for (ii = 0; ii < p; ii++) + printf ( "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" ); + for (ii = 0; ii < p-1; ii++) + printf ( "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" ); + for (ii = 0; ii < p+1; ii++) + printf ( "ccccccccccccccccccccccccccccccccccccc" ); + } + fflush(stdout); + return 0; +} diff --git a/unzcrash.c b/unzcrash.c new file mode 100644 index 0000000..f0f17fc --- /dev/null +++ b/unzcrash.c @@ -0,0 +1,126 @@ + +/* A test program written to test robustness to decompression of + corrupted data. Usage is + unzcrash filename + and the program will read the specified file, compress it (in memory), + and then repeatedly decompress it, each time with a different bit of + the compressed data inverted, so as to test all possible one-bit errors. + This should not cause any invalid memory accesses. If it does, + I want to know about it! + + p.s. As you can see from the above description, the process is + incredibly slow. A file of size eg 5KB will cause it to run for + many hours. +*/ + +#include +#include +#include "bzlib.h" + +#define M_BLOCK 1000000 + +typedef unsigned char uchar; + +#define M_BLOCK_OUT (M_BLOCK + 1000000) +uchar inbuf[M_BLOCK]; +uchar outbuf[M_BLOCK_OUT]; +uchar zbuf[M_BLOCK + 600 + (M_BLOCK / 100)]; + +int nIn, nOut, nZ; + +static char *bzerrorstrings[] = { + "OK" + ,"SEQUENCE_ERROR" + ,"PARAM_ERROR" + ,"MEM_ERROR" + ,"DATA_ERROR" + ,"DATA_ERROR_MAGIC" + ,"IO_ERROR" + ,"UNEXPECTED_EOF" + ,"OUTBUFF_FULL" + ,"???" /* for future */ + ,"???" /* for future */ + ,"???" /* for future */ + ,"???" /* for future */ + ,"???" /* for future */ + ,"???" /* for future */ +}; + +void flip_bit ( int bit ) +{ + int byteno = bit / 8; + int bitno = bit % 8; + uchar mask = 1 << bitno; + //fprintf ( stderr, "(byte %d bit %d mask %d)", + // byteno, bitno, (int)mask ); + zbuf[byteno] ^= mask; +} + +int main ( int argc, char** argv ) +{ + FILE* f; + int r; + int bit; + int i; + + if (argc != 2) { + fprintf ( stderr, "usage: unzcrash filename\n" ); + return 1; + } + + f = fopen ( argv[1], "r" ); + if (!f) { + fprintf ( stderr, "unzcrash: can't open %s\n", argv[1] ); + return 1; + } + + nIn = fread ( inbuf, 1, M_BLOCK, f ); + fprintf ( stderr, "%d bytes read\n", nIn ); + + nZ = M_BLOCK; + r = BZ2_bzBuffToBuffCompress ( + zbuf, &nZ, inbuf, nIn, 9, 0, 30 ); + + assert (r == BZ_OK); + fprintf ( stderr, "%d after compression\n", nZ ); + + for (bit = 0; bit < nZ*8; bit++) { + fprintf ( stderr, "bit %d ", bit ); + flip_bit ( bit ); + nOut = M_BLOCK_OUT; + r = BZ2_bzBuffToBuffDecompress ( + outbuf, &nOut, zbuf, nZ, 0, 0 ); + fprintf ( stderr, " %d %s ", r, bzerrorstrings[-r] ); + + if (r != BZ_OK) { + fprintf ( stderr, "\n" ); + } else { + if (nOut != nIn) { + fprintf(stderr, "nIn/nOut mismatch %d %d\n", nIn, nOut ); + return 1; + } else { + for (i = 0; i < nOut; i++) + if (inbuf[i] != outbuf[i]) { + fprintf(stderr, "mismatch at %d\n", i ); + return 1; + } + if (i == nOut) fprintf(stderr, "really ok!\n" ); + } + } + + flip_bit ( bit ); + } + +#if 0 + assert (nOut == nIn); + for (i = 0; i < nOut; i++) { + if (inbuf[i] != outbuf[i]) { + fprintf ( stderr, "difference at %d !\n", i ); + return 1; + } + } +#endif + + fprintf ( stderr, "all ok\n" ); + return 0; +} diff --git a/words0 b/words0 new file mode 100644 index 0000000..164a8ed --- /dev/null +++ b/words0 @@ -0,0 +1,5 @@ + +If compilation produces errors, or a large number of warnings, +please read README.COMPILATION.PROBLEMS -- you might be able to +adjust the flags in this Makefile to improve matters. + -- cgit v1.2.3 From 099d844292f60f9d58914da29e5773204dc55e7a Mon Sep 17 00:00:00 2001 From: Julian Seward Date: Sun, 30 Dec 2001 22:13:13 +0100 Subject: bzip2-1.0.2 --- CHANGES | 88 +++++++- LICENSE | 4 +- Makefile | 81 +++++-- Makefile-libbz2_so | 15 +- README | 89 +++++--- README.COMPILATION.PROBLEMS | 4 +- blocksort.c | 11 +- bzdiff | 76 +++++++ bzdiff.1 | 47 ++++ bzgrep | 71 ++++++ bzgrep.1 | 56 +++++ bzip2.1 | 56 +++-- bzip2.1.preformatted | 226 +++++++------------ bzip2.c | 533 ++++++++++++++++++++++++-------------------- bzip2.txt | 134 ++++++----- bzip2recover.c | 161 ++++++++++--- bzlib.c | 35 ++- bzlib.h | 6 +- bzlib_private.h | 11 +- bzmore | 61 +++++ bzmore.1 | 152 +++++++++++++ compress.c | 10 +- crctable.c | 2 +- decompress.c | 14 +- dlltest.c | 6 +- huffman.c | 2 +- makefile.msc | 2 +- manual.texi | 114 ++++++---- mk251.c | 16 ++ randtable.c | 2 +- words3 | 4 +- 31 files changed, 1464 insertions(+), 625 deletions(-) create mode 100644 bzdiff create mode 100644 bzdiff.1 create mode 100644 bzgrep create mode 100644 bzgrep.1 create mode 100644 bzmore create mode 100644 bzmore.1 create mode 100644 mk251.c diff --git a/CHANGES b/CHANGES index ecaf417..d984395 100644 --- a/CHANGES +++ b/CHANGES @@ -134,7 +134,7 @@ Several minor bugfixes and enhancements: * Advance the version number to 1.0, so as to counteract the (false-in-this-case) impression some people have that programs - with version numbers less than 1.0 are in someway, experimental, + with version numbers less than 1.0 are in some way, experimental, pre-release versions. * Create an initial Makefile-libbz2_so to build a shared library. @@ -165,3 +165,89 @@ There are no functionality changes or bug fixes relative to version 1.0.0. This is just a documentation update + a fix for minor Win32 build problems. For almost everyone, upgrading from 1.0.0 to 1.0.1 is utterly pointless. Don't bother. + + +1.0.2 +~~~~~ +A bug fix release, addressing various minor issues which have appeared +in the 18 or so months since 1.0.1 was released. Most of the fixes +are to do with file-handling or documentation bugs. To the best of my +knowledge, there have been no data-loss-causing bugs reported in the +compression/decompression engine of 1.0.0 or 1.0.1. + +Note that this release does not improve the rather crude build system +for Unix platforms. The general plan here is to autoconfiscate/ +libtoolise 1.0.2 soon after release, and release the result as 1.1.0 +or perhaps 1.2.0. That, however, is still just a plan at this point. + +Here are the changes in 1.0.2. Bug-reporters and/or patch-senders in +parentheses. + +* Fix an infinite segfault loop in 1.0.1 when a directory is + encountered in -f (force) mode. + (Trond Eivind Glomsrod, Nicholas Nethercote, Volker Schmidt) + +* Avoid double fclose() of output file on certain I/O error paths. + (Solar Designer) + +* Don't fail with internal error 1007 when fed a long stream (> 48MB) + of byte 251. Also print useful message suggesting that 1007s may be + caused by bad memory. + (noticed by Juan Pedro Vallejo, fixed by me) + +* Fix uninitialised variable silly bug in demo prog dlltest.c. + (Jorj Bauer) + +* Remove 512-MB limitation on recovered file size for bzip2recover + on selected platforms which support 64-bit ints. At the moment + all GCC supported platforms, and Win32. + (me, Alson van der Meulen) + +* Hard-code header byte values, to give correct operation on platforms + using EBCDIC as their native character set (IBM's OS/390). + (Leland Lucius) + +* Copy file access times correctly. + (Marty Leisner) + +* Add distclean and check targets to Makefile. + (Michael Carmack) + +* Parameterise use of ar and ranlib in Makefile. Also add $(LDFLAGS). + (Rich Ireland, Bo Thorsen) + +* Pass -p (create parent dirs as needed) to mkdir during make install. + (Jeremy Fusco) + +* Dereference symlinks when copying file permissions in -f mode. + (Volker Schmidt) + +* Majorly simplify implementation of uInt64_qrm10. + (Bo Lindbergh) + +* Check the input file still exists before deleting the output one, + when aborting in cleanUpAndFail(). + (Joerg Prante, Robert Linden, Matthias Krings) + +Also a bunch of patches courtesy of Philippe Troin, the Debian maintainer +of bzip2: + +* Wrapper scripts (with manpages): bzdiff, bzgrep, bzmore. + +* Spelling changes and minor enhancements in bzip2.1. + +* Avoid race condition between creating the output file and setting its + interim permissions safely, by using fopen_output_safely(). + No changes to bzip2recover since there is no issue with file + permissions there. + +* do not print senseless report with -v when compressing an empty + file. + +* bzcat -f works on non-bzip2 files. + +* do not try to escape shell meta-characters on unix (the shell takes + care of these). + +* added --fast and --best aliases for -1 -9 for gzip compatibility. + diff --git a/LICENSE b/LICENSE index 88fa6d8..9d4fa43 100644 --- a/LICENSE +++ b/LICENSE @@ -1,6 +1,6 @@ This program, "bzip2" and associated library "libbzip2", are -copyright (C) 1996-2000 Julian R Seward. All rights reserved. +copyright (C) 1996-2002 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -35,5 +35,5 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. Julian Seward, Cambridge, UK. jseward@acm.org -bzip2/libbzip2 version 1.0 of 21 March 2000 +bzip2/libbzip2 version 1.0.2 of 30 December 2001 diff --git a/Makefile b/Makefile index ab17f49..8305235 100644 --- a/Makefile +++ b/Makefile @@ -1,9 +1,20 @@ SHELL=/bin/sh + +# To assist in cross-compiling CC=gcc +AR=ar +RANLIB=ranlib +LDFLAGS= + +# Suitably paranoid flags to avoid bugs in gcc-2.7 BIGFILES=-D_FILE_OFFSET_BITS=64 CFLAGS=-Wall -Winline -O2 -fomit-frame-pointer -fno-strength-reduce $(BIGFILES) +# Where you want it installed when you do 'make install' +PREFIX=/usr + + OBJS= blocksort.o \ huffman.o \ crctable.o \ @@ -15,20 +26,21 @@ OBJS= blocksort.o \ all: libbz2.a bzip2 bzip2recover test bzip2: libbz2.a bzip2.o - $(CC) $(CFLAGS) -o bzip2 bzip2.o -L. -lbz2 + $(CC) $(CFLAGS) $(LDFLAGS) -o bzip2 bzip2.o -L. -lbz2 bzip2recover: bzip2recover.o - $(CC) $(CFLAGS) -o bzip2recover bzip2recover.o + $(CC) $(CFLAGS) $(LDFLAGS) -o bzip2recover bzip2recover.o libbz2.a: $(OBJS) rm -f libbz2.a - ar cq libbz2.a $(OBJS) - @if ( test -f /usr/bin/ranlib -o -f /bin/ranlib -o \ - -f /usr/ccs/bin/ranlib ) ; then \ - echo ranlib libbz2.a ; \ - ranlib libbz2.a ; \ + $(AR) cq libbz2.a $(OBJS) + @if ( test -f $(RANLIB) -o -f /usr/bin/ranlib -o \ + -f /bin/ranlib -o -f /usr/ccs/bin/ranlib ) ; then \ + echo $(RANLIB) libbz2.a ; \ + $(RANLIB) libbz2.a ; \ fi +check: test test: bzip2 @cat words1 ./bzip2 -1 < sample1.ref > sample1.rb2 @@ -45,14 +57,12 @@ test: bzip2 cmp sample3.tst sample3.ref @cat words3 -PREFIX=/usr - install: bzip2 bzip2recover - if ( test ! -d $(PREFIX)/bin ) ; then mkdir $(PREFIX)/bin ; fi - if ( test ! -d $(PREFIX)/lib ) ; then mkdir $(PREFIX)/lib ; fi - if ( test ! -d $(PREFIX)/man ) ; then mkdir $(PREFIX)/man ; fi - if ( test ! -d $(PREFIX)/man/man1 ) ; then mkdir $(PREFIX)/man/man1 ; fi - if ( test ! -d $(PREFIX)/include ) ; then mkdir $(PREFIX)/include ; fi + if ( test ! -d $(PREFIX)/bin ) ; then mkdir -p $(PREFIX)/bin ; fi + if ( test ! -d $(PREFIX)/lib ) ; then mkdir -p $(PREFIX)/lib ; fi + if ( test ! -d $(PREFIX)/man ) ; then mkdir -p $(PREFIX)/man ; fi + if ( test ! -d $(PREFIX)/man/man1 ) ; then mkdir -p $(PREFIX)/man/man1 ; fi + if ( test ! -d $(PREFIX)/include ) ; then mkdir -p $(PREFIX)/include ; fi cp -f bzip2 $(PREFIX)/bin/bzip2 cp -f bzip2 $(PREFIX)/bin/bunzip2 cp -f bzip2 $(PREFIX)/bin/bzcat @@ -67,7 +77,26 @@ install: bzip2 bzip2recover chmod a+r $(PREFIX)/include/bzlib.h cp -f libbz2.a $(PREFIX)/lib chmod a+r $(PREFIX)/lib/libbz2.a + cp -f bzgrep $(PREFIX)/bin/bzgrep + ln $(PREFIX)/bin/bzgrep $(PREFIX)/bin/bzegrep + ln $(PREFIX)/bin/bzgrep $(PREFIX)/bin/bzfgrep + chmod a+x $(PREFIX)/bin/bzgrep + cp -f bzmore $(PREFIX)/bin/bzmore + ln $(PREFIX)/bin/bzmore $(PREFIX)/bin/bzless + chmod a+x $(PREFIX)/bin/bzmore + cp -f bzdiff $(PREFIX)/bin/bzdiff + ln $(PREFIX)/bin/bzdiff $(PREFIX)/bin/bzcmp + chmod a+x $(PREFIX)/bin/bzdiff + cp -f bzgrep.1 bzmore.1 bzdiff.1 $(PREFIX)/man/man1 + chmod a+r $(PREFIX)/man/man1/bzgrep.1 + chmod a+r $(PREFIX)/man/man1/bzmore.1 + chmod a+r $(PREFIX)/man/man1/bzdiff.1 + echo ".so man1/bzgrep.1" > $(PREFIX)/man/man1/bzegrep.1 + echo ".so man1/bzgrep.1" > $(PREFIX)/man/man1/bzfgrep.1 + echo ".so man1/bzmore.1" > $(PREFIX)/man/man1/bzless.1 + echo ".so man1/bzdiff.1" > $(PREFIX)/man/man1/bzcmp.1 +distclean: clean clean: rm -f *.o libbz2.a bzip2 bzip2recover \ sample1.rb2 sample2.rb2 sample3.rb2 \ @@ -93,7 +122,7 @@ bzip2.o: bzip2.c bzip2recover.o: bzip2recover.c $(CC) $(CFLAGS) -c bzip2recover.c -DISTNAME=bzip2-1.0.1 +DISTNAME=bzip2-1.0.2 tarfile: rm -f $(DISTNAME) ln -sf . $(DISTNAME) @@ -112,6 +141,7 @@ tarfile: $(DISTNAME)/Makefile \ $(DISTNAME)/manual.texi \ $(DISTNAME)/manual.ps \ + $(DISTNAME)/manual.pdf \ $(DISTNAME)/LICENSE \ $(DISTNAME)/bzip2.1 \ $(DISTNAME)/bzip2.1.preformatted \ @@ -138,4 +168,25 @@ tarfile: $(DISTNAME)/Y2K_INFO \ $(DISTNAME)/unzcrash.c \ $(DISTNAME)/spewG.c \ + $(DISTNAME)/mk251.c \ + $(DISTNAME)/bzdiff \ + $(DISTNAME)/bzdiff.1 \ + $(DISTNAME)/bzmore \ + $(DISTNAME)/bzmore.1 \ + $(DISTNAME)/bzgrep \ + $(DISTNAME)/bzgrep.1 \ $(DISTNAME)/Makefile-libbz2_so + gzip -v $(DISTNAME).tar + +# For rebuilding the manual from sources on my RedHat 7.2 box +manual: manual.ps manual.pdf manual.html + +manual.ps: manual.texi + tex manual.texi + dvips -o manual.ps manual.dvi + +manual.pdf: manual.ps + ps2pdf manual.ps + +manual.html: manual.texi + texi2html -split_chapter manual.texi diff --git a/Makefile-libbz2_so b/Makefile-libbz2_so index a347c50..4986fe2 100644 --- a/Makefile-libbz2_so +++ b/Makefile-libbz2_so @@ -1,8 +1,9 @@ # This Makefile builds a shared version of the library, -# libbz2.so.1.0.1, with soname libbz2.so.1.0, -# at least on x86-Linux (RedHat 5.2), -# with gcc-2.7.2.3. Please see the README file for some +# libbz2.so.1.0.2, with soname libbz2.so.1.0, +# at least on x86-Linux (RedHat 7.2), +# with gcc-2.96 20000731 (Red Hat Linux 7.1 2.96-98). +# Please see the README file for some # important info about building the library like this. SHELL=/bin/sh @@ -19,13 +20,13 @@ OBJS= blocksort.o \ bzlib.o all: $(OBJS) - $(CC) -shared -Wl,-soname -Wl,libbz2.so.1.0 -o libbz2.so.1.0.1 $(OBJS) - $(CC) $(CFLAGS) -o bzip2-shared bzip2.c libbz2.so.1.0.1 + $(CC) -shared -Wl,-soname -Wl,libbz2.so.1.0 -o libbz2.so.1.0.2 $(OBJS) + $(CC) $(CFLAGS) -o bzip2-shared bzip2.c libbz2.so.1.0.2 rm -f libbz2.so.1.0 - ln -s libbz2.so.1.0.1 libbz2.so.1.0 + ln -s libbz2.so.1.0.2 libbz2.so.1.0 clean: - rm -f $(OBJS) bzip2.o libbz2.so.1.0.1 libbz2.so.1.0 bzip2-shared + rm -f $(OBJS) bzip2.o libbz2.so.1.0.2 libbz2.so.1.0 bzip2-shared blocksort.o: blocksort.c $(CC) $(CFLAGS) -c blocksort.c diff --git a/README b/README index 22945a2..07505d8 100644 --- a/README +++ b/README @@ -1,15 +1,15 @@ This is the README for bzip2, a block-sorting file compressor, version -1.0. This version is fully compatible with the previous public -releases, bzip2-0.1pl2, bzip2-0.9.0 and bzip2-0.9.5. +1.0.2. This version is fully compatible with the previous public +releases, versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0 and 1.0.1. -bzip2-1.0 is distributed under a BSD-style license. For details, +bzip2-1.0.2 is distributed under a BSD-style license. For details, see the file LICENSE. -Complete documentation is available in Postscript form (manual.ps) or -html (manual_toc.html). A plain-text version of the manual page is -available as bzip2.txt. A statement about Y2K issues is now included -in the file Y2K_INFO. +Complete documentation is available in Postscript form (manual.ps), +PDF (manual.pdf, amazingly enough) or html (manual_toc.html). A +plain-text version of the manual page is available as bzip2.txt. +A statement about Y2K issues is now included in the file Y2K_INFO. HOW TO BUILD -- UNIX @@ -33,34 +33,41 @@ not actually execute them. HOW TO BUILD -- UNIX, shared library libbz2.so. Do 'make -f Makefile-libbz2_so'. This Makefile seems to work for -Linux-ELF (RedHat 5.2 on an x86 box), with gcc. I make no claims +Linux-ELF (RedHat 7.2 on an x86 box), with gcc. I make no claims that it works for any other platform, though I suspect it probably will work for most platforms employing both ELF and gcc. -bzip2-shared, a client of the shared library, is also build, but -not self-tested. So I suggest you also build using the normal -Makefile, since that conducts a self-test. +bzip2-shared, a client of the shared library, is also built, but not +self-tested. So I suggest you also build using the normal Makefile, +since that conducts a self-test. A second reason to prefer the +version statically linked to the library is that, on x86 platforms, +building shared objects makes a valuable register (%ebx) unavailable +to gcc, resulting in a slowdown of 10%-20%, at least for bzip2. -Important note for people upgrading .so's from 0.9.0/0.9.5 to -version 1.0. All the functions in the library have been renamed, -from (eg) bzCompress to BZ2_bzCompress, to avoid namespace pollution. +Important note for people upgrading .so's from 0.9.0/0.9.5 to version +1.0.X. All the functions in the library have been renamed, from (eg) +bzCompress to BZ2_bzCompress, to avoid namespace pollution. Unfortunately this means that the libbz2.so created by -Makefile-libbz2_so will not work with any program which used an -older version of the library. Sorry. I do encourage library -clients to make the effort to upgrade to use version 1.0, since -it is both faster and more robust than previous versions. +Makefile-libbz2_so will not work with any program which used an older +version of the library. Sorry. I do encourage library clients to +make the effort to upgrade to use version 1.0, since it is both faster +and more robust than previous versions. HOW TO BUILD -- Windows 95, NT, DOS, Mac, etc. It's difficult for me to support compilation on all these platforms. My approach is to collect binaries for these platforms, and put them -on the master web page (http://sourceware.cygnus.com/bzip2). Look -there. However (FWIW), bzip2-1.0 is very standard ANSI C and should -compile unmodified with MS Visual C. For Win32, there is one -important caveat: in bzip2.c, you must set BZ_UNIX to 0 and -BZ_LCCWIN32 to 1 before building. If you have difficulties building, -you might want to read README.COMPILATION.PROBLEMS. +on the master web page (http://sources.redhat.com/bzip2). Look there. +However (FWIW), bzip2-1.0.X is very standard ANSI C and should compile +unmodified with MS Visual C. If you have difficulties building, you +might want to read README.COMPILATION.PROBLEMS. + +At least using MS Visual C++ 6, you can build from the unmodified +sources by issuing, in a command shell: + nmake -f makefile.msc +(you may need to first run the MSVC-provided script VCVARS32.BAT + so as to set up paths to the MSVC tools correctly). VALIDATION @@ -138,29 +145,37 @@ WHAT'S NEW IN 0.9.5 ? * Many small improvements in file and flag handling. * A Y2K statement. -WHAT'S NEW IN 1.0 +WHAT'S NEW IN 1.0.0 ? See the CHANGES file. +WHAT'S NEW IN 1.0.2 ? + + See the CHANGES file. + + I hope you find bzip2 useful. Feel free to contact me at jseward@acm.org if you have any suggestions or queries. Many people mailed me with comments, suggestions and patches after the releases of bzip-0.15, -bzip-0.21, bzip2-0.1pl2 and bzip2-0.9.0, and the changes in bzip2 are -largely a result of this feedback. I thank you for your comments. +bzip-0.21, and bzip2 versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0 and 1.0.1, +and the changes in bzip2 are largely a result of this feedback. +I thank you for your comments. At least for the time being, bzip2's "home" is (or can be reached via) -http://www.muraroa.demon.co.uk. +http://sources.redhat.com/bzip2. Julian Seward jseward@acm.org -Cambridge, UK -18 July 1996 (version 0.15) -25 August 1996 (version 0.21) - 7 August 1997 (bzip2, version 0.1) -29 August 1997 (bzip2, version 0.1pl2) -23 August 1998 (bzip2, version 0.9.0) - 8 June 1999 (bzip2, version 0.9.5) - 4 Sept 1999 (bzip2, version 0.9.5d) - 5 May 2000 (bzip2, version 1.0pre8) +Cambridge, UK (and what a great town this is!) + +18 July 1996 (version 0.15) +25 August 1996 (version 0.21) + 7 August 1997 (bzip2, version 0.1) +29 August 1997 (bzip2, version 0.1pl2) +23 August 1998 (bzip2, version 0.9.0) + 8 June 1999 (bzip2, version 0.9.5) + 4 Sept 1999 (bzip2, version 0.9.5d) + 5 May 2000 (bzip2, version 1.0pre8) +30 December 2001 (bzip2, version 1.0.2pre1) \ No newline at end of file diff --git a/README.COMPILATION.PROBLEMS b/README.COMPILATION.PROBLEMS index d621ad5..bd1822d 100644 --- a/README.COMPILATION.PROBLEMS +++ b/README.COMPILATION.PROBLEMS @@ -117,11 +117,11 @@ Known problems as of 1.0pre8: All that said: you might be able to get somewhere by finding the line in Makefile-libbz2_so which says - $(CC) -shared -Wl,-soname -Wl,libbz2.so.1.0 -o libbz2.so.1.0.1 $(OBJS) + $(CC) -shared -Wl,-soname -Wl,libbz2.so.1.0 -o libbz2.so.1.0.2 $(OBJS) and replacing with - ($CC) -G -shared -o libbz2.so.1.0.1 -h libbz2.so.1.0 $(OBJS) + $(CC) -G -shared -o libbz2.so.1.0.2 -h libbz2.so.1.0 $(OBJS) If gcc objects to the combination -fpic -fPIC, get rid of the second one, leaving just "-fpic". diff --git a/blocksort.c b/blocksort.c index ec42672..aba3efc 100644 --- a/blocksort.c +++ b/blocksort.c @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-2000 Julian R Seward. All rights reserved. + Copyright (C) 1996-2002 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -981,7 +981,14 @@ void mainSort ( UInt32* ptr, } } - AssertH ( copyStart[ss]-1 == copyEnd[ss], 1007 ); + AssertH ( (copyStart[ss]-1 == copyEnd[ss]) + || + /* Extremely rare case missing in bzip2-1.0.0 and 1.0.1. + Necessity for this case is demonstrated by compressing + a sequence of approximately 48.5 million of character + 251; 1.0.0/1.0.1 will then die here. */ + (copyStart[ss] == 0 && copyEnd[ss] == nblock-1), + 1007 ) for (j = 0; j <= 255; j++) ftab[(j << 8) + ss] |= SETMASK; diff --git a/bzdiff b/bzdiff new file mode 100644 index 0000000..3c2eb85 --- /dev/null +++ b/bzdiff @@ -0,0 +1,76 @@ +#!/bin/sh +# sh is buggy on RS/6000 AIX 3.2. Replace above line with #!/bin/ksh + +# Bzcmp/diff wrapped for bzip2, +# adapted from zdiff by Philippe Troin for Debian GNU/Linux. + +# Bzcmp and bzdiff are used to invoke the cmp or the diff pro- +# gram on compressed files. All options specified are passed +# directly to cmp or diff. If only 1 file is specified, then +# the files compared are file1 and an uncompressed file1.gz. +# If two files are specified, then they are uncompressed (if +# necessary) and fed to cmp or diff. The exit status from cmp +# or diff is preserved. + +PATH="/usr/bin:$PATH"; export PATH +prog=`echo $0 | sed 's|.*/||'` +case "$prog" in + *cmp) comp=${CMP-cmp} ;; + *) comp=${DIFF-diff} ;; +esac + +OPTIONS= +FILES= +for ARG +do + case "$ARG" in + -*) OPTIONS="$OPTIONS $ARG";; + *) if test -f "$ARG"; then + FILES="$FILES $ARG" + else + echo "${prog}: $ARG not found or not a regular file" + exit 1 + fi ;; + esac +done +if test -z "$FILES"; then + echo "Usage: $prog [${comp}_options] file [file]" + exit 1 +fi +tmp=`tempfile -d /tmp -p bz` || { + echo 'cannot create a temporary file' >&2 + exit 1 +} +set $FILES +if test $# -eq 1; then + FILE=`echo "$1" | sed 's/.bz2$//'` + bzip2 -cd "$FILE.bz2" | $comp $OPTIONS - "$FILE" + STAT="$?" + +elif test $# -eq 2; then + case "$1" in + *.bz2) + case "$2" in + *.bz2) + F=`echo "$2" | sed 's|.*/||;s|.bz2$||'` + bzip2 -cdfq "$2" > $tmp + bzip2 -cdfq "$1" | $comp $OPTIONS - $tmp + STAT="$?" + /bin/rm -f $tmp;; + + *) bzip2 -cdfq "$1" | $comp $OPTIONS - "$2" + STAT="$?";; + esac;; + *) case "$2" in + *.bz2) + bzip2 -cdfq "$2" | $comp $OPTIONS "$1" - + STAT="$?";; + *) $comp $OPTIONS "$1" "$2" + STAT="$?";; + esac;; + esac + exit "$STAT" +else + echo "Usage: $prog [${comp}_options] file [file]" + exit 1 +fi diff --git a/bzdiff.1 b/bzdiff.1 new file mode 100644 index 0000000..adb7a8e --- /dev/null +++ b/bzdiff.1 @@ -0,0 +1,47 @@ +\"Shamelessly copied from zmore.1 by Philippe Troin +\"for Debian GNU/Linux +.TH BZDIFF 1 +.SH NAME +bzcmp, bzdiff \- compare bzip2 compressed files +.SH SYNOPSIS +.B bzcmp +[ cmp_options ] file1 +[ file2 ] +.br +.B bzdiff +[ diff_options ] file1 +[ file2 ] +.SH DESCRIPTION +.I Bzcmp +and +.I bzdiff +are used to invoke the +.I cmp +or the +.I diff +program on bzip2 compressed files. All options specified are passed +directly to +.I cmp +or +.IR diff "." +If only 1 file is specified, then the files compared are +.I file1 +and an uncompressed +.IR file1 ".bz2." +If two files are specified, then they are uncompressed if necessary and fed to +.I cmp +or +.IR diff "." +The exit status from +.I cmp +or +.I diff +is preserved. +.SH "SEE ALSO" +cmp(1), diff(1), bzmore(1), bzless(1), bzgrep(1), bzip2(1) +.SH BUGS +Messages from the +.I cmp +or +.I diff +programs refer to temporary filenames instead of those specified. diff --git a/bzgrep b/bzgrep new file mode 100644 index 0000000..dbfc00e --- /dev/null +++ b/bzgrep @@ -0,0 +1,71 @@ +#!/bin/sh + +# Bzgrep wrapped for bzip2, +# adapted from zgrep by Philippe Troin for Debian GNU/Linux. +## zgrep notice: +## zgrep -- a wrapper around a grep program that decompresses files as needed +## Adapted from a version sent by Charles Levert + +PATH="/usr/bin:$PATH"; export PATH + +prog=`echo $0 | sed 's|.*/||'` +case "$prog" in + *egrep) grep=${EGREP-egrep} ;; + *fgrep) grep=${FGREP-fgrep} ;; + *) grep=${GREP-grep} ;; +esac +pat="" +while test $# -ne 0; do + case "$1" in + -e | -f) opt="$opt $1"; shift; pat="$1" + if test "$grep" = grep; then # grep is buggy with -e on SVR4 + grep=egrep + fi;; + -A | -B) opt="$opt $1 $2"; shift;; + -*) opt="$opt $1";; + *) if test -z "$pat"; then + pat="$1" + else + break; + fi;; + esac + shift +done + +if test -z "$pat"; then + echo "grep through bzip2 files" + echo "usage: $prog [grep_options] pattern [files]" + exit 1 +fi + +list=0 +silent=0 +op=`echo "$opt" | sed -e 's/ //g' -e 's/-//g'` +case "$op" in + *l*) list=1 +esac +case "$op" in + *h*) silent=1 +esac + +if test $# -eq 0; then + bzip2 -cdfq | $grep $opt "$pat" + exit $? +fi + +res=0 +for i do + if test -f "$i"; then :; else if test -f "$i.bz2"; then i="$i.bz2"; fi; fi + if test $list -eq 1; then + bzip2 -cdfq "$i" | $grep $opt "$pat" 2>&1 > /dev/null && echo $i + r=$? + elif test $# -eq 1 -o $silent -eq 1; then + bzip2 -cdfq "$i" | $grep $opt "$pat" + r=$? + else + bzip2 -cdfq "$i" | $grep $opt "$pat" | sed "s|^|${i}:|" + r=$? + fi + test "$r" -ne 0 && res="$r" +done +exit $res diff --git a/bzgrep.1 b/bzgrep.1 new file mode 100644 index 0000000..930af8c --- /dev/null +++ b/bzgrep.1 @@ -0,0 +1,56 @@ +\"Shamelessly copied from zmore.1 by Philippe Troin +\"for Debian GNU/Linux +.TH BZGREP 1 +.SH NAME +bzgrep, bzfgrep, bzegrep \- search possibly bzip2 compressed files for a regular expression +.SH SYNOPSIS +.B bzgrep +[ grep_options ] +.BI [\ -e\ ] " pattern" +.IR filename ".\|.\|." +.br +.B bzegrep +[ egrep_options ] +.BI [\ -e\ ] " pattern" +.IR filename ".\|.\|." +.br +.B bzfgrep +[ fgrep_options ] +.BI [\ -e\ ] " pattern" +.IR filename ".\|.\|." +.SH DESCRIPTION +.IR Bzgrep +is used to invoke the +.I grep +on bzip2-compressed files. All options specified are passed directly to +.I grep. +If no file is specified, then the standard input is decompressed +if necessary and fed to grep. +Otherwise the given files are uncompressed if necessary and fed to +.I grep. +.PP +If +.I bzgrep +is invoked as +.I bzegrep +or +.I bzfgrep +then +.I egrep +or +.I fgrep +is used instead of +.I grep. +If the GREP environment variable is set, +.I bzgrep +uses it as the +.I grep +program to be invoked. For example: + + for sh: GREP=fgrep bzgrep string files + for csh: (setenv GREP fgrep; bzgrep string files) +.SH AUTHOR +Charles Levert (charles@comm.polymtl.ca). Adapted to bzip2 by Philippe +Troin for Debian GNU/Linux. +.SH "SEE ALSO" +grep(1), egrep(1), fgrep(1), bzdiff(1), bzmore(1), bzless(1), bzip2(1) diff --git a/bzip2.1 b/bzip2.1 index 7de54a0..623435c 100644 --- a/bzip2.1 +++ b/bzip2.1 @@ -1,7 +1,7 @@ .PU .TH bzip2 1 .SH NAME -bzip2, bunzip2 \- a block-sorting file compressor, v1.0 +bzip2, bunzip2 \- a block-sorting file compressor, v1.0.2 .br bzcat \- decompresses files to stdout .br @@ -197,7 +197,7 @@ to decompress. .TP .B \-z --compress The complement to \-d: forces compression, regardless of the -invokation name. +invocation name. .TP .B \-t --test Check integrity of the specified file(s), but don't decompress them. @@ -211,6 +211,10 @@ existing output files. Also forces .I bzip2 to break hard links to files, which it otherwise wouldn't do. + +bzip2 normally declines to decompress files which don't have the +correct magic header bytes. If forced (-f), however, it will pass +such files through unmodified. This is how GNU gzip behaves. .TP .B \-k --keep Keep (don't delete) input files during compression @@ -239,9 +243,13 @@ information which is primarily of interest for diagnostic purposes. .B \-L --license -V --version Display the software version, license terms and conditions. .TP -.B \-1 to \-9 +.B \-1 (or \-\-fast) to \-9 (or \-\-best) Set the block size to 100 k, 200 k .. 900 k when compressing. Has no effect when decompressing. See MEMORY MANAGEMENT below. +The \-\-fast and \-\-best aliases are primarily for GNU gzip +compatibility. In particular, \-\-fast doesn't make things +significantly faster. +And \-\-best merely selects the default behaviour. .TP .B \-- Treats all subsequent arguments as file names, even if they start @@ -352,11 +360,11 @@ undamaged. .I bzip2recover takes a single argument, the name of the damaged file, -and writes a number of files "rec0001file.bz2", -"rec0002file.bz2", etc, containing the extracted blocks. +and writes a number of files "rec00001file.bz2", +"rec00002file.bz2", etc, containing the extracted blocks. The output filenames are designed so that the use of wildcards in subsequent processing -- for example, -"bzip2 -dc rec*file.bz2 > recovered_data" -- lists the files in +"bzip2 -dc rec*file.bz2 > recovered_data" -- processes the files in the correct order. .I bzip2recover @@ -397,27 +405,31 @@ I/O error messages are not as helpful as they could be. tries hard to detect I/O errors and exit cleanly, but the details of what the problem is sometimes seem rather misleading. -This manual page pertains to version 1.0 of +This manual page pertains to version 1.0.2 of .I bzip2. -Compressed -data created by this version is entirely forwards and backwards -compatible with the previous public releases, versions 0.1pl2, 0.9.0 -and 0.9.5, -but with the following exception: 0.9.0 and above can correctly -decompress multiple concatenated compressed files. 0.1pl2 cannot do -this; it will stop after decompressing just the first file in the -stream. +Compressed data created by this version is entirely forwards and +backwards compatible with the previous public releases, versions +0.1pl2, 0.9.0, 0.9.5, 1.0.0 and 1.0.1, but with the following +exception: 0.9.0 and above can correctly decompress multiple +concatenated compressed files. 0.1pl2 cannot do this; it will stop +after decompressing just the first file in the stream. .I bzip2recover -uses 32-bit integers to represent bit positions in -compressed files, so it cannot handle compressed files more than 512 -megabytes long. This could easily be fixed. +versions prior to this one, 1.0.2, used 32-bit integers to represent +bit positions in compressed files, so it could not handle compressed +files more than 512 megabytes long. Version 1.0.2 and above uses +64-bit ints on some platforms which support them (GNU supported +targets, and Windows). To establish whether or not bzip2recover was +built with such a limitation, run it without arguments. In any event +you can build yourself an unlimited version if you can recompile it +with MaybeUInt64 set to be an unsigned 64-bit integer. + + .SH AUTHOR Julian Seward, jseward@acm.org. -http://sourceware.cygnus.com/bzip2 -http://www.muraroa.demon.co.uk +http://sources.redhat.com/bzip2 The ideas embodied in .I bzip2 @@ -434,6 +446,8 @@ indebted for their help, support and advice. See the manual in the source distribution for pointers to sources of documentation. Christian von Roques encouraged me to look for faster sorting algorithms, so as to speed up compression. Bela Lubkin encouraged me to improve the -worst-case compression performance. Many people sent patches, helped +worst-case compression performance. +The bz* scripts are derived from those of GNU gzip. +Many people sent patches, helped with portability problems, lent machines, gave advice and were generally helpful. diff --git a/bzip2.1.preformatted b/bzip2.1.preformatted index 9f18339..0f20cb5 100644 --- a/bzip2.1.preformatted +++ b/bzip2.1.preformatted @@ -1,11 +1,9 @@ - - - bzip2(1) bzip2(1) + NNAAMMEE - bzip2, bunzip2 - a block-sorting file compressor, v1.0 + bzip2, bunzip2 - a block-sorting file compressor, v1.0.2 bzcat - decompresses files to stdout bzip2recover - recovers data from damaged bzip2 files @@ -22,20 +20,20 @@ DDEESSCCRRIIPPTTIIOONN sorting text compression algorithm, and Huffman coding. Compression is generally considerably better than that achieved by more conventional LZ77/LZ78-based compressors, - and approaches the performance of the PPM family of sta- + and approaches the performance of the PPM family of sta tistical compressors. The command-line options are deliberately very similar to those of _G_N_U _g_z_i_p_, but they are not identical. - _b_z_i_p_2 expects a list of file names to accompany the com- + _b_z_i_p_2 expects a list of file names to accompany the com mand-line flags. Each file is replaced by a compressed version of itself, with the name "original_name.bz2". - Each compressed file has the same modification date, per- - missions, and, when possible, ownership as the correspond- + Each compressed file has the same modification date, per + missions, and, when possible, ownership as the correspond ing original, so that these properties can be correctly restored at decompression time. File name handling is - naive in the sense that there is no mechanism for preserv- + naive in the sense that there is no mechanism for preserv ing original file names, permissions, ownerships or dates in filesystems which lack these concepts, or have serious file name length restrictions, such as MS-DOS. @@ -58,18 +56,6 @@ DDEESSCCRRIIPPTTIIOONN filename.bz2 becomes filename filename.bz becomes filename filename.tbz2 becomes filename.tar - - - - 1 - - - - - -bzip2(1) bzip2(1) - - filename.tbz becomes filename.tar anyothername becomes anyothername.out @@ -78,23 +64,23 @@ bzip2(1) bzip2(1) guess the name of the original file, and uses the original name with _._o_u_t appended. - As with compression, supplying no filenames causes decom- + As with compression, supplying no filenames causes decom pression from standard input to standard output. - _b_u_n_z_i_p_2 will correctly decompress a file which is the con- + _b_u_n_z_i_p_2 will correctly decompress a file which is the con catenation of two or more compressed files. The result is the concatenation of the corresponding uncompressed files. Integrity testing (-t) of concatenated compressed files is also supported. You can also compress or decompress files to the standard - output by giving the -c flag. Multiple files may be com- + output by giving the -c flag. Multiple files may be com pressed and decompressed like this. The resulting outputs are fed sequentially to stdout. Compression of multiple - files in this manner generates a stream containing multi- + files in this manner generates a stream containing multi ple compressed file representations. Such a stream can be decompressed correctly only by _b_z_i_p_2 version 0.9.0 or - later. Earlier versions of _b_z_i_p_2 will stop after decom- + later. Earlier versions of _b_z_i_p_2 will stop after decom pressing the first file in the stream. _b_z_c_a_t (or _b_z_i_p_2 _-_d_c_) decompresses all specified files to @@ -115,7 +101,7 @@ bzip2(1) bzip2(1) As a self-check for your protection, _b_z_i_p_2 uses 32-bit CRCs to make sure that the decompressed version of a file - is identical to the original. This guards against corrup- + is identical to the original. This guards against corrup tion of the compressed data, and against undetected bugs in _b_z_i_p_2 (hopefully very unlikely). The chances of data corruption going undetected is microscopic, about one @@ -125,17 +111,6 @@ bzip2(1) bzip2(1) you recover the original uncompressed data. You can use _b_z_i_p_2_r_e_c_o_v_e_r to try to recover data from damaged files. - - - 2 - - - - - -bzip2(1) bzip2(1) - - Return values: 0 for a normal exit, 1 for environmental problems (file not found, invalid flags, I/O errors, &c), 2 to indicate a corrupt compressed file, 3 for an internal @@ -154,8 +129,8 @@ OOPPTTIIOONNSS and forces _b_z_i_p_2 to decompress. --zz ----ccoommpprreessss - The complement to -d: forces compression, regard- - less of the invokation name. + The complement to -d: forces compression, + regardless of the invocation name. --tt ----tteesstt Check integrity of the specified file(s), but don't @@ -168,6 +143,11 @@ OOPPTTIIOONNSS forces _b_z_i_p_2 to break hard links to files, which it otherwise wouldn't do. + bzip2 normally declines to decompress files which + don't have the correct magic header bytes. If + forced (-f), however, it will pass such files + through unmodified. This is how GNU gzip behaves. + --kk ----kkeeeepp Keep (don't delete) input files during compression or decompression. @@ -190,23 +170,11 @@ OOPPTTIIOONNSS --qq ----qquuiieett Suppress non-essential warning messages. Messages pertaining to I/O errors and other critical events - - - - 3 - - - - - -bzip2(1) bzip2(1) - - will not be suppressed. --vv ----vveerrbboossee Verbose mode -- show the compression ratio for each - file processed. Further -v's increase the ver- + file processed. Further -v's increase the ver bosity level, spewing out lots of information which is primarily of interest for diagnostic purposes. @@ -214,20 +182,24 @@ bzip2(1) bzip2(1) Display the software version, license terms and conditions. - --11 ttoo --99 + --11 ((oorr ----ffaasstt)) ttoo --99 ((oorr ----bbeesstt)) Set the block size to 100 k, 200 k .. 900 k when compressing. Has no effect when decompressing. - See MEMORY MANAGEMENT below. + See MEMORY MANAGEMENT below. The --fast and --best + aliases are primarily for GNU gzip compatibility. + In particular, --fast doesn't make things signifi + cantly faster. And --best merely selects the + default behaviour. ---- Treats all subsequent arguments as file names, even - if they start with a dash. This is so you can han- + if they start with a dash. This is so you can han dle files with names beginning with a dash, for example: bzip2 -- -myfilename. ----rreeppeettiittiivvee--ffaasstt ----rreeppeettiittiivvee--bbeesstt These flags are redundant in versions 0.9.5 and above. They provided some coarse control over the - behaviour of the sorting algorithm in earlier ver- + behaviour of the sorting algorithm in earlier ver sions, which was sometimes useful. 0.9.5 and above have an improved algorithm which renders these flags irrelevant. @@ -238,7 +210,7 @@ MMEEMMOORRYY MMAANNAAGGEEMMEENNTT affects both the compression ratio achieved, and the amount of memory needed for compression and decompression. The flags -1 through -9 specify the block size to be - 100,000 bytes through 900,000 bytes (the default) respec- + 100,000 bytes through 900,000 bytes (the default) respec tively. At decompression time, the block size used for compression is read from the header of the compressed file, and _b_u_n_z_i_p_2 then allocates itself just enough memory @@ -256,18 +228,6 @@ MMEEMMOORRYY MMAANNAAGGEEMMEENNTT Larger block sizes give rapidly diminishing marginal returns. Most of the compression comes from the first two - - - - 4 - - - - - -bzip2(1) bzip2(1) - - or three hundred k of block size, a fact worth bearing in mind when using _b_z_i_p_2 on small machines. It is also important to appreciate that the decompression memory @@ -278,13 +238,13 @@ bzip2(1) bzip2(1) _b_u_n_z_i_p_2 will require about 3700 kbytes to decompress. To support decompression of any file on a 4 megabyte machine, _b_u_n_z_i_p_2 has an option to decompress using approximately - half this amount of memory, about 2300 kbytes. Decompres- + half this amount of memory, about 2300 kbytes. Decompres sion speed is also halved, so you should use this option only where necessary. The relevant flag is -s. - In general, try and use the largest block size memory con- + In general, try and use the largest block size memory con straints allow, since that maximises the compression - achieved. Compression and decompression speed are virtu- + achieved. Compression and decompression speed are virtu ally unaffected by block size. Another significant point applies to files which fit in a @@ -300,11 +260,11 @@ bzip2(1) bzip2(1) Here is a table which summarises the maximum memory usage for different block sizes. Also recorded is the total - compressed size for 14 files of the Calgary Text Compres- + compressed size for 14 files of the Calgary Text Compres sion Corpus totalling 3,141,622 bytes. This column gives some feel for how compression varies with block size. These figures tend to understate the advantage of larger - block sizes for larger files, since the Corpus is domi- + block sizes for larger files, since the Corpus is domi nated by smaller files. Compress Decompress Decompress Corpus @@ -321,22 +281,9 @@ bzip2(1) bzip2(1) -9 7600k 3700k 2350k 828642 - - - - - 5 - - - - - -bzip2(1) bzip2(1) - - RREECCOOVVEERRIINNGG DDAATTAA FFRROOMM DDAAMMAAGGEEDD FFIILLEESS _b_z_i_p_2 compresses files in blocks, usually 900kbytes long. - Each block is handled independently. If a media or trans- + Each block is handled independently. If a media or trans mission error causes a multi-block .bz2 file to become damaged, it may be possible to recover data from the undamaged blocks in the file. @@ -353,19 +300,19 @@ RREECCOOVVEERRIINNGG DDAATTAA FFRROOMM DDAAMMAAGGEEDD F the integrity of the resulting files, and decompress those which are undamaged. - _b_z_i_p_2_r_e_c_o_v_e_r takes a single argument, the name of the dam- - aged file, and writes a number of files "rec0001file.bz2", - "rec0002file.bz2", etc, containing the extracted blocks. - The output filenames are designed so that the use of - wildcards in subsequent processing -- for example, "bzip2 - -dc rec*file.bz2 > recovered_data" -- lists the files in - the correct order. + _b_z_i_p_2_r_e_c_o_v_e_r takes a single argument, the name of the dam + aged file, and writes a number of files + "rec00001file.bz2", "rec00002file.bz2", etc, containing + the extracted blocks. The output filenames are + designed so that the use of wildcards in subsequent pro + cessing -- for example, "bzip2 -dc rec*file.bz2 > recov + ered_data" -- processes the files in the correct order. _b_z_i_p_2_r_e_c_o_v_e_r should be of most use dealing with large .bz2 files, as these will contain many blocks. It is clearly futile to use it on damaged single-block files, since a - damaged block cannot be recovered. If you wish to min- - imise any potential data loss through media or transmis- + damaged block cannot be recovered. If you wish to min + imise any potential data loss through media or transmis sion errors, you might consider compressing with a smaller block size. @@ -379,31 +326,19 @@ PPEERRFFOORRMMAANNCCEE NNOOTTEESS better than previous versions in this respect. The ratio between worst-case and average-case compression time is in the region of 10:1. For previous versions, this figure - was more like 100:1. You can use the -vvvv option to mon- + was more like 100:1. You can use the -vvvv option to mon itor progress in great detail, if you want. Decompression speed is unaffected by these phenomena. _b_z_i_p_2 usually allocates several megabytes of memory to - operate in, and then charges all over it in a fairly ran- - dom fashion. This means that performance, both for com- + operate in, and then charges all over it in a fairly ran + dom fashion. This means that performance, both for com pressing and decompressing, is largely determined by the - - - - 6 - - - - - -bzip2(1) bzip2(1) - - speed at which your machine can service cache misses. Because of this, small changes to the code to reduce the miss rate have been observed to give disproportionately - large performance improvements. I imagine _b_z_i_p_2 will per- + large performance improvements. I imagine _b_z_i_p_2 will per form best on machines with very large caches. @@ -413,50 +348,51 @@ CCAAVVEEAATTSS but the details of what the problem is sometimes seem rather misleading. - This manual page pertains to version 1.0 of _b_z_i_p_2_. Com- + This manual page pertains to version 1.0.2 of _b_z_i_p_2_. Com pressed data created by this version is entirely forwards and backwards compatible with the previous public - releases, versions 0.1pl2, 0.9.0 and 0.9.5, but with the - following exception: 0.9.0 and above can correctly decom- - press multiple concatenated compressed files. 0.1pl2 can- - not do this; it will stop after decompressing just the - first file in the stream. + releases, versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0 and 1.0.1, + but with the following exception: 0.9.0 and above can cor + rectly decompress multiple concatenated compressed files. + 0.1pl2 cannot do this; it will stop after decompressing + just the first file in the stream. + + _b_z_i_p_2_r_e_c_o_v_e_r versions prior to this one, 1.0.2, used + 32-bit integers to represent bit positions in compressed + files, so it could not handle compressed files more than + 512 megabytes long. Version 1.0.2 and above uses 64-bit + ints on some platforms which support them (GNU supported + targets, and Windows). To establish whether or not + bzip2recover was built with such a limitation, run it + without arguments. In any event you can build yourself an + unlimited version if you can recompile it with MaybeUInt64 + set to be an unsigned 64-bit integer. + - _b_z_i_p_2_r_e_c_o_v_e_r uses 32-bit integers to represent bit posi- - tions in compressed files, so it cannot handle compressed - files more than 512 megabytes long. This could easily be - fixed. AAUUTTHHOORR Julian Seward, jseward@acm.org. - http://sourceware.cygnus.com/bzip2 - http://www.muraroa.demon.co.uk + http://sources.redhat.com/bzip2 - The ideas embodied in _b_z_i_p_2 are due to (at least) the fol- - lowing people: Michael Burrows and David Wheeler (for the - block sorting transformation), David Wheeler (again, for - the Huffman coder), Peter Fenwick (for the structured cod- + The ideas embodied in _b_z_i_p_2 are due to (at least) the fol + lowing people: Michael Burrows and David Wheeler (for the + block sorting transformation), David Wheeler (again, for + the Huffman coder), Peter Fenwick (for the structured cod ing model in the original _b_z_i_p_, and many refinements), and - Alistair Moffat, Radford Neal and Ian Witten (for the + Alistair Moffat, Radford Neal and Ian Witten (for the arithmetic coder in the original _b_z_i_p_)_. I am much - indebted for their help, support and advice. See the man- - ual in the source distribution for pointers to sources of + indebted for their help, support and advice. See the man + ual in the source distribution for pointers to sources of documentation. Christian von Roques encouraged me to look - for faster sorting algorithms, so as to speed up compres- + for faster sorting algorithms, so as to speed up compres sion. Bela Lubkin encouraged me to improve the worst-case - compression performance. Many people sent patches, helped - with portability problems, lent machines, gave advice and - were generally helpful. - - - - - - - + compression performance. The bz* scripts are derived from + those of GNU gzip. Many people sent patches, helped with + portability problems, lent machines, gave advice and were + generally helpful. - 7 + bzip2(1) diff --git a/bzip2.c b/bzip2.c index 56adfdc..807f420 100644 --- a/bzip2.c +++ b/bzip2.c @@ -7,7 +7,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-2000 Julian R Seward. All rights reserved. + Copyright (C) 1996-2002 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -113,13 +113,16 @@ /*-- Generic 32-bit Unix. Also works on 64-bit Unix boxes. + This is the default. --*/ #define BZ_UNIX 1 /*-- Win32, as seen by Jacob Navia's excellent port of (Chris Fraser & David Hanson)'s excellent - lcc compiler. + lcc compiler. Or with MS Visual C. + This is selected automatically if compiled by a compiler which + defines _WIN32, not including the Cygwin GCC. --*/ #define BZ_LCCWIN32 0 @@ -156,6 +159,7 @@ --*/ #if BZ_UNIX +# include # include # include # include @@ -164,8 +168,9 @@ # define PATH_SEP '/' # define MY_LSTAT lstat -# define MY_S_IFREG S_ISREG # define MY_STAT stat +# define MY_S_ISREG S_ISREG +# define MY_S_ISDIR S_ISDIR # define APPEND_FILESPEC(root, name) \ root=snocString((root), (name)) @@ -180,19 +185,23 @@ # else # define NORETURN /**/ # endif + # ifdef __DJGPP__ # include # include # undef MY_LSTAT +# undef MY_STAT # define MY_LSTAT stat +# define MY_STAT stat # undef SET_BINARY_MODE # define SET_BINARY_MODE(fd) \ do { \ int retVal = setmode ( fileno ( fd ), \ - O_BINARY ); \ + O_BINARY ); \ ERROR_IF_MINUS_ONE ( retVal ); \ } while ( 0 ) # endif + # ifdef __CYGWIN__ # include # include @@ -200,11 +209,11 @@ # define SET_BINARY_MODE(fd) \ do { \ int retVal = setmode ( fileno ( fd ), \ - O_BINARY ); \ + O_BINARY ); \ ERROR_IF_MINUS_ONE ( retVal ); \ } while ( 0 ) # endif -#endif +#endif /* BZ_UNIX */ @@ -217,46 +226,23 @@ # define PATH_SEP '\\' # define MY_LSTAT _stat # define MY_STAT _stat -# define MY_S_IFREG(x) ((x) & _S_IFREG) +# define MY_S_ISREG(x) ((x) & _S_IFREG) +# define MY_S_ISDIR(x) ((x) & _S_IFDIR) # define APPEND_FLAG(root, name) \ root=snocString((root), (name)) -# if 0 - /*-- lcc-win32 seems to expand wildcards itself --*/ -# define APPEND_FILESPEC(root, spec) \ - do { \ - if ((spec)[0] == '-') { \ - root = snocString((root), (spec)); \ - } else { \ - struct _finddata_t c_file; \ - long hFile; \ - hFile = _findfirst((spec), &c_file); \ - if ( hFile == -1L ) { \ - root = snocString ((root), (spec)); \ - } else { \ - int anInt = 0; \ - while ( anInt == 0 ) { \ - root = snocString((root), \ - &c_file.name[0]); \ - anInt = _findnext(hFile, &c_file); \ - } \ - } \ - } \ - } while ( 0 ) -# else # define APPEND_FILESPEC(root, name) \ root = snocString ((root), (name)) -# endif # define SET_BINARY_MODE(fd) \ do { \ int retVal = setmode ( fileno ( fd ), \ - O_BINARY ); \ + O_BINARY ); \ ERROR_IF_MINUS_ONE ( retVal ); \ } while ( 0 ) -#endif +#endif /* BZ_LCCWIN32 */ /*---------------------------------------------*/ @@ -338,6 +324,7 @@ typedef struct { UChar b[8]; } UInt64; + static void uInt64_from_UInt32s ( UInt64* n, UInt32 lo32, UInt32 hi32 ) { @@ -351,6 +338,7 @@ void uInt64_from_UInt32s ( UInt64* n, UInt32 lo32, UInt32 hi32 ) n->b[0] = (UChar) (lo32 & 0xFF); } + static double uInt64_to_double ( UInt64* n ) { @@ -364,77 +352,6 @@ double uInt64_to_double ( UInt64* n ) return sum; } -static -void uInt64_add ( UInt64* src, UInt64* dst ) -{ - Int32 i; - Int32 carry = 0; - for (i = 0; i < 8; i++) { - carry += ( ((Int32)src->b[i]) + ((Int32)dst->b[i]) ); - dst->b[i] = (UChar)(carry & 0xFF); - carry >>= 8; - } -} - -static -void uInt64_sub ( UInt64* src, UInt64* dst ) -{ - Int32 t, i; - Int32 borrow = 0; - for (i = 0; i < 8; i++) { - t = ((Int32)dst->b[i]) - ((Int32)src->b[i]) - borrow; - if (t < 0) { - dst->b[i] = (UChar)(t + 256); - borrow = 1; - } else { - dst->b[i] = (UChar)t; - borrow = 0; - } - } -} - -static -void uInt64_mul ( UInt64* a, UInt64* b, UInt64* r_hi, UInt64* r_lo ) -{ - UChar sum[16]; - Int32 ia, ib, carry; - for (ia = 0; ia < 16; ia++) sum[ia] = 0; - for (ia = 0; ia < 8; ia++) { - carry = 0; - for (ib = 0; ib < 8; ib++) { - carry += ( ((Int32)sum[ia+ib]) - + ((Int32)a->b[ia]) * ((Int32)b->b[ib]) ); - sum[ia+ib] = (UChar)(carry & 0xFF); - carry >>= 8; - } - sum[ia+8] = (UChar)(carry & 0xFF); - if ((carry >>= 8) != 0) panic ( "uInt64_mul" ); - } - - for (ia = 0; ia < 8; ia++) r_hi->b[ia] = sum[ia+8]; - for (ia = 0; ia < 8; ia++) r_lo->b[ia] = sum[ia]; -} - - -static -void uInt64_shr1 ( UInt64* n ) -{ - Int32 i; - for (i = 0; i < 8; i++) { - n->b[i] >>= 1; - if (i < 7 && (n->b[i+1] & 1)) n->b[i] |= 0x80; - } -} - -static -void uInt64_shl1 ( UInt64* n ) -{ - Int32 i; - for (i = 7; i >= 0; i--) { - n->b[i] <<= 1; - if (i > 0 && (n->b[i-1] & 0x80)) n->b[i]++; - } -} static Bool uInt64_isZero ( UInt64* n ) @@ -445,49 +362,23 @@ Bool uInt64_isZero ( UInt64* n ) return 1; } -static + +/* Divide *n by 10, and return the remainder. */ +static Int32 uInt64_qrm10 ( UInt64* n ) { - /* Divide *n by 10, and return the remainder. Long division - is difficult, so we cheat and instead multiply by - 0xCCCC CCCC CCCC CCCD, which is 0.8 (viz, 0.1 << 3). - */ + UInt32 rem, tmp; Int32 i; - UInt64 tmp1, tmp2, n_orig, zero_point_eight; - - zero_point_eight.b[1] = zero_point_eight.b[2] = - zero_point_eight.b[3] = zero_point_eight.b[4] = - zero_point_eight.b[5] = zero_point_eight.b[6] = - zero_point_eight.b[7] = 0xCC; - zero_point_eight.b[0] = 0xCD; - - n_orig = *n; - - /* divide n by 10, - by multiplying by 0.8 and then shifting right 3 times */ - uInt64_mul ( n, &zero_point_eight, &tmp1, &tmp2 ); - uInt64_shr1(&tmp1); uInt64_shr1(&tmp1); uInt64_shr1(&tmp1); - *n = tmp1; - - /* tmp1 = 8*n, tmp2 = 2*n */ - uInt64_shl1(&tmp1); uInt64_shl1(&tmp1); uInt64_shl1(&tmp1); - tmp2 = *n; uInt64_shl1(&tmp2); - - /* tmp1 = 10*n */ - uInt64_add ( &tmp2, &tmp1 ); - - /* n_orig = n_orig - 10*n */ - uInt64_sub ( &tmp1, &n_orig ); - - /* n_orig should now hold quotient, in range 0 .. 9 */ - for (i = 7; i >= 1; i--) - if (n_orig.b[i] != 0) panic ( "uInt64_qrm10(1)" ); - if (n_orig.b[0] > 9) - panic ( "uInt64_qrm10(2)" ); - - return (int)n_orig.b[0]; + rem = 0; + for (i = 7; i >= 0; i--) { + tmp = rem * 256 + n->b[i]; + n->b[i] = tmp / 10; + rem = tmp % 10; + } + return rem; } + /* ... and the Whole Entire Point of all this UInt64 stuff is so that we can supply the following function. */ @@ -504,7 +395,8 @@ void uInt64_toAscii ( char* outbuf, UInt64* n ) nBuf++; } while (!uInt64_isZero(&n_copy)); outbuf[nBuf] = 0; - for (i = 0; i < nBuf; i++) outbuf[i] = buf[nBuf-i-1]; + for (i = 0; i < nBuf; i++) + outbuf[i] = buf[nBuf-i-1]; } @@ -566,35 +458,38 @@ void compressStream ( FILE *stream, FILE *zStream ) if (ret == EOF) goto errhandler_io; if (zStream != stdout) { ret = fclose ( zStream ); + outputHandleJustInCase = NULL; if (ret == EOF) goto errhandler_io; } + outputHandleJustInCase = NULL; if (ferror(stream)) goto errhandler_io; ret = fclose ( stream ); if (ret == EOF) goto errhandler_io; - if (nbytes_in_lo32 == 0 && nbytes_in_hi32 == 0) - nbytes_in_lo32 = 1; - if (verbosity >= 1) { - Char buf_nin[32], buf_nout[32]; - UInt64 nbytes_in, nbytes_out; - double nbytes_in_d, nbytes_out_d; - uInt64_from_UInt32s ( &nbytes_in, - nbytes_in_lo32, nbytes_in_hi32 ); - uInt64_from_UInt32s ( &nbytes_out, - nbytes_out_lo32, nbytes_out_hi32 ); - nbytes_in_d = uInt64_to_double ( &nbytes_in ); - nbytes_out_d = uInt64_to_double ( &nbytes_out ); - uInt64_toAscii ( buf_nin, &nbytes_in ); - uInt64_toAscii ( buf_nout, &nbytes_out ); - fprintf ( stderr, "%6.3f:1, %6.3f bits/byte, " - "%5.2f%% saved, %s in, %s out.\n", - nbytes_in_d / nbytes_out_d, - (8.0 * nbytes_out_d) / nbytes_in_d, - 100.0 * (1.0 - nbytes_out_d / nbytes_in_d), - buf_nin, - buf_nout - ); + if (nbytes_in_lo32 == 0 && nbytes_in_hi32 == 0) { + fprintf ( stderr, " no data compressed.\n"); + } else { + Char buf_nin[32], buf_nout[32]; + UInt64 nbytes_in, nbytes_out; + double nbytes_in_d, nbytes_out_d; + uInt64_from_UInt32s ( &nbytes_in, + nbytes_in_lo32, nbytes_in_hi32 ); + uInt64_from_UInt32s ( &nbytes_out, + nbytes_out_lo32, nbytes_out_hi32 ); + nbytes_in_d = uInt64_to_double ( &nbytes_in ); + nbytes_out_d = uInt64_to_double ( &nbytes_out ); + uInt64_toAscii ( buf_nin, &nbytes_in ); + uInt64_toAscii ( buf_nout, &nbytes_out ); + fprintf ( stderr, "%6.3f:1, %6.3f bits/byte, " + "%5.2f%% saved, %s in, %s out.\n", + nbytes_in_d / nbytes_out_d, + (8.0 * nbytes_out_d) / nbytes_in_d, + 100.0 * (1.0 - nbytes_out_d / nbytes_in_d), + buf_nin, + buf_nout + ); + } } return; @@ -652,7 +547,7 @@ Bool uncompressStream ( FILE *zStream, FILE *stream ) while (bzerr == BZ_OK) { nread = BZ2_bzRead ( &bzerr, bzf, obuf, 5000 ); - if (bzerr == BZ_DATA_ERROR_MAGIC) goto errhandler; + if (bzerr == BZ_DATA_ERROR_MAGIC) goto trycat; if ((bzerr == BZ_OK || bzerr == BZ_STREAM_END) && nread > 0) fwrite ( obuf, sizeof(UChar), nread, stream ); if (ferror(stream)) goto errhandler_io; @@ -668,9 +563,9 @@ Bool uncompressStream ( FILE *zStream, FILE *stream ) if (bzerr != BZ_OK) panic ( "decompress:bzReadGetUnused" ); if (nUnused == 0 && myfeof(zStream)) break; - } + closeok: if (ferror(zStream)) goto errhandler_io; ret = fclose ( zStream ); if (ret == EOF) goto errhandler_io; @@ -680,11 +575,26 @@ Bool uncompressStream ( FILE *zStream, FILE *stream ) if (ret != 0) goto errhandler_io; if (stream != stdout) { ret = fclose ( stream ); + outputHandleJustInCase = NULL; if (ret == EOF) goto errhandler_io; } + outputHandleJustInCase = NULL; if (verbosity >= 2) fprintf ( stderr, "\n " ); return True; + trycat: + if (forceOverwrite) { + rewind(zStream); + while (True) { + if (myfeof(zStream)) break; + nread = fread ( obuf, sizeof(UChar), 5000, zStream ); + if (ferror(zStream)) goto errhandler_io; + if (nread > 0) fwrite ( obuf, sizeof(UChar), nread, stream ); + if (ferror(stream)) goto errhandler_io; + } + goto closeok; + } + errhandler: BZ2_bzReadClose ( &bzerr_dummy, bzf ); switch (bzerr) { @@ -832,7 +742,7 @@ void cadvise ( void ) stderr, "\nIt is possible that the compressed file(s) have become corrupted.\n" "You can use the -tvv option to test integrity of such files.\n\n" - "You can use the `bzip2recover' program to *attempt* to recover\n" + "You can use the `bzip2recover' program to attempt to recover\n" "data from undamaged sections of corrupted files.\n\n" ); } @@ -855,28 +765,55 @@ void showFileNames ( void ) static void cleanUpAndFail ( Int32 ec ) { - IntNative retVal; + IntNative retVal; + struct MY_STAT statBuf; if ( srcMode == SM_F2F && opMode != OM_TEST && deleteOutputOnInterrupt ) { - if (noisy) - fprintf ( stderr, "%s: Deleting output file %s, if it exists.\n", - progName, outName ); - if (outputHandleJustInCase != NULL) - fclose ( outputHandleJustInCase ); - retVal = remove ( outName ); - if (retVal != 0) + + /* Check whether input file still exists. Delete output file + only if input exists to avoid loss of data. Joerg Prante, 5 + January 2002. (JRS 06-Jan-2002: other changes in 1.0.2 mean + this is less likely to happen. But to be ultra-paranoid, we + do the check anyway.) */ + retVal = MY_STAT ( inName, &statBuf ); + if (retVal == 0) { + if (noisy) + fprintf ( stderr, + "%s: Deleting output file %s, if it exists.\n", + progName, outName ); + if (outputHandleJustInCase != NULL) + fclose ( outputHandleJustInCase ); + retVal = remove ( outName ); + if (retVal != 0) + fprintf ( stderr, + "%s: WARNING: deletion of output file " + "(apparently) failed.\n", + progName ); + } else { fprintf ( stderr, - "%s: WARNING: deletion of output file (apparently) failed.\n", + "%s: WARNING: deletion of output file suppressed\n", + progName ); + fprintf ( stderr, + "%s: since input file no longer exists. Output file\n", progName ); + fprintf ( stderr, + "%s: `%s' may be incomplete.\n", + progName, outName ); + fprintf ( stderr, + "%s: I suggest doing an integrity test (bzip2 -tv)" + " of it.\n", + progName ); + } } + if (noisy && numFileNames > 0 && numFilesProcessed < numFileNames) { fprintf ( stderr, "%s: WARNING: some files have not been processed:\n" - "\t%d specified on command line, %d not processed yet.\n\n", - progName, numFileNames, - numFileNames - numFilesProcessed ); + "%s: %d specified on command line, %d not processed yet.\n\n", + progName, progName, + numFileNames, numFileNames - numFilesProcessed ); } setExit(ec); exit(exitValue); @@ -915,14 +852,16 @@ void crcError ( void ) static void compressedStreamEOF ( void ) { - fprintf ( stderr, - "\n%s: Compressed file ends unexpectedly;\n\t" - "perhaps it is corrupted? *Possible* reason follows.\n", - progName ); - perror ( progName ); - showFileNames(); - cadvise(); - cleanUpAndFail( 2 ); + if (noisy) { + fprintf ( stderr, + "\n%s: Compressed file ends unexpectedly;\n\t" + "perhaps it is corrupted? *Possible* reason follows.\n", + progName ); + perror ( progName ); + showFileNames(); + cadvise(); + } + cleanUpAndFail( 2 ); } @@ -1038,6 +977,11 @@ void configError ( void ) /*--- The main driver machinery ---*/ /*---------------------------------------------------*/ +/* All rather crufty. The main problem is that input files + are stat()d multiple times before use. This should be + cleaned up. +*/ + /*---------------------------------------------*/ static void pad ( Char *s ) @@ -1081,6 +1025,32 @@ Bool fileExists ( Char* name ) } +/*---------------------------------------------*/ +/* Open an output file safely with O_EXCL and good permissions. + This avoids a race condition in versions < 1.0.2, in which + the file was first opened and then had its interim permissions + set safely. We instead use open() to create the file with + the interim permissions required. (--- --- rw-). + + For non-Unix platforms, if we are not worrying about + security issues, simple this simply behaves like fopen. +*/ +FILE* fopen_output_safely ( Char* name, const char* mode ) +{ +# if BZ_UNIX + FILE* fp; + IntNative fh; + fh = open(name, O_WRONLY|O_CREAT|O_EXCL, S_IWUSR|S_IRUSR); + if (fh == -1) return NULL; + fp = fdopen(fh, mode); + if (fp == NULL) close(fh); + return fp; +# else + return fopen(name, mode); +# endif +} + + /*---------------------------------------------*/ /*-- if in doubt, return True @@ -1093,7 +1063,7 @@ Bool notAStandardFile ( Char* name ) i = MY_LSTAT ( name, &statBuf ); if (i != 0) return True; - if (MY_S_IFREG(statBuf.st_mode)) return False; + if (MY_S_ISREG(statBuf.st_mode)) return False; return True; } @@ -1115,42 +1085,66 @@ Int32 countHardLinks ( Char* name ) /*---------------------------------------------*/ +/* Copy modification date, access date, permissions and owner from the + source to destination file. We have to copy this meta-info off + into fileMetaInfo before starting to compress / decompress it, + because doing it afterwards means we get the wrong access time. + + To complicate matters, in compress() and decompress() below, the + sequence of tests preceding the call to saveInputFileMetaInfo() + involves calling fileExists(), which in turn establishes its result + by attempting to fopen() the file, and if successful, immediately + fclose()ing it again. So we have to assume that the fopen() call + does not cause the access time field to be updated. + + Reading of the man page for stat() (man 2 stat) on RedHat 7.2 seems + to imply that merely doing open() will not affect the access time. + Therefore we merely need to hope that the C library only does + open() as a result of fopen(), and not any kind of read()-ahead + cleverness. + + It sounds pretty fragile to me. Whether this carries across + robustly to arbitrary Unix-like platforms (or even works robustly + on this one, RedHat 7.2) is unknown to me. Nevertheless ... +*/ +#if BZ_UNIX +static +struct MY_STAT fileMetaInfo; +#endif + static -void copyDatePermissionsAndOwner ( Char *srcName, Char *dstName ) +void saveInputFileMetaInfo ( Char *srcName ) { -#if BZ_UNIX +# if BZ_UNIX + IntNative retVal; + /* Note use of stat here, not lstat. */ + retVal = MY_STAT( srcName, &fileMetaInfo ); + ERROR_IF_NOT_ZERO ( retVal ); +# endif +} + + +static +void applySavedMetaInfoToOutputFile ( Char *dstName ) +{ +# if BZ_UNIX IntNative retVal; - struct MY_STAT statBuf; struct utimbuf uTimBuf; - retVal = MY_LSTAT ( srcName, &statBuf ); - ERROR_IF_NOT_ZERO ( retVal ); - uTimBuf.actime = statBuf.st_atime; - uTimBuf.modtime = statBuf.st_mtime; + uTimBuf.actime = fileMetaInfo.st_atime; + uTimBuf.modtime = fileMetaInfo.st_mtime; - retVal = chmod ( dstName, statBuf.st_mode ); + retVal = chmod ( dstName, fileMetaInfo.st_mode ); ERROR_IF_NOT_ZERO ( retVal ); retVal = utime ( dstName, &uTimBuf ); ERROR_IF_NOT_ZERO ( retVal ); - retVal = chown ( dstName, statBuf.st_uid, statBuf.st_gid ); + retVal = chown ( dstName, fileMetaInfo.st_uid, fileMetaInfo.st_gid ); /* chown() will in many cases return with EPERM, which can be safely ignored. */ -#endif -} - - -/*---------------------------------------------*/ -static -void setInterimPermissions ( Char *dstName ) -{ -#if BZ_UNIX - IntNative retVal; - retVal = chmod ( dstName, S_IRUSR | S_IWUSR ); - ERROR_IF_NOT_ZERO ( retVal ); -#endif +# endif } @@ -1158,10 +1152,19 @@ void setInterimPermissions ( Char *dstName ) static Bool containsDubiousChars ( Char* name ) { - Bool cdc = False; +# if BZ_UNIX + /* On unix, files can contain any characters and the file expansion + * is performed by the shell. + */ + return False; +# else /* ! BZ_UNIX */ + /* On non-unix (Win* platforms), wildcard characters are not allowed in + * filenames. + */ for (; *name != '\0'; name++) - if (*name == '?' || *name == '*') cdc = True; - return cdc; + if (*name == '?' || *name == '*') return True; + return False; +# endif /* BZ_UNIX */ } @@ -1201,6 +1204,7 @@ void compress ( Char *name ) FILE *inStr; FILE *outStr; Int32 n, i; + struct MY_STAT statBuf; deleteOutputOnInterrupt = False; @@ -1246,6 +1250,16 @@ void compress ( Char *name ) return; } } + if ( srcMode == SM_F2F || srcMode == SM_F2O ) { + MY_STAT(inName, &statBuf); + if ( MY_S_ISDIR(statBuf.st_mode) ) { + fprintf( stderr, + "%s: Input file %s is a directory.\n", + progName,inName); + setExit(1); + return; + } + } if ( srcMode == SM_F2F && !forceOverwrite && notAStandardFile ( inName )) { if (noisy) fprintf ( stderr, "%s: Input file %s is not a normal file.\n", @@ -1253,11 +1267,15 @@ void compress ( Char *name ) setExit(1); return; } - if ( srcMode == SM_F2F && !forceOverwrite && fileExists ( outName ) ) { - fprintf ( stderr, "%s: Output file %s already exists.\n", - progName, outName ); - setExit(1); - return; + if ( srcMode == SM_F2F && fileExists ( outName ) ) { + if (forceOverwrite) { + remove(outName); + } else { + fprintf ( stderr, "%s: Output file %s already exists.\n", + progName, outName ); + setExit(1); + return; + } } if ( srcMode == SM_F2F && !forceOverwrite && (n=countHardLinks ( inName )) > 0) { @@ -1267,6 +1285,12 @@ void compress ( Char *name ) return; } + if ( srcMode == SM_F2F ) { + /* Save the file's meta-info before we open it. Doing it later + means we mess up the access times. */ + saveInputFileMetaInfo ( inName ); + } + switch ( srcMode ) { case SM_I2O: @@ -1306,7 +1330,7 @@ void compress ( Char *name ) case SM_F2F: inStr = fopen ( inName, "rb" ); - outStr = fopen ( outName, "wb" ); + outStr = fopen_output_safely ( outName, "wb" ); if ( outStr == NULL) { fprintf ( stderr, "%s: Can't create output file %s: %s.\n", progName, outName, strerror(errno) ); @@ -1321,7 +1345,6 @@ void compress ( Char *name ) setExit(1); return; }; - setInterimPermissions ( outName ); break; default: @@ -1343,7 +1366,7 @@ void compress ( Char *name ) /*--- If there was an I/O error, we won't get here. ---*/ if ( srcMode == SM_F2F ) { - copyDatePermissionsAndOwner ( inName, outName ); + applySavedMetaInfoToOutputFile ( outName ); deleteOutputOnInterrupt = False; if ( !keepInputFiles ) { IntNative retVal = remove ( inName ); @@ -1364,6 +1387,7 @@ void uncompress ( Char *name ) Int32 n, i; Bool magicNumberOK; Bool cantGuess; + struct MY_STAT statBuf; deleteOutputOnInterrupt = False; @@ -1405,6 +1429,16 @@ void uncompress ( Char *name ) setExit(1); return; } + if ( srcMode == SM_F2F || srcMode == SM_F2O ) { + MY_STAT(inName, &statBuf); + if ( MY_S_ISDIR(statBuf.st_mode) ) { + fprintf( stderr, + "%s: Input file %s is a directory.\n", + progName,inName); + setExit(1); + return; + } + } if ( srcMode == SM_F2F && !forceOverwrite && notAStandardFile ( inName )) { if (noisy) fprintf ( stderr, "%s: Input file %s is not a normal file.\n", @@ -1419,11 +1453,15 @@ void uncompress ( Char *name ) progName, inName, outName ); /* just a warning, no return */ } - if ( srcMode == SM_F2F && !forceOverwrite && fileExists ( outName ) ) { - fprintf ( stderr, "%s: Output file %s already exists.\n", - progName, outName ); - setExit(1); - return; + if ( srcMode == SM_F2F && fileExists ( outName ) ) { + if (forceOverwrite) { + remove(outName); + } else { + fprintf ( stderr, "%s: Output file %s already exists.\n", + progName, outName ); + setExit(1); + return; + } } if ( srcMode == SM_F2F && !forceOverwrite && (n=countHardLinks ( inName ) ) > 0) { @@ -1433,6 +1471,12 @@ void uncompress ( Char *name ) return; } + if ( srcMode == SM_F2F ) { + /* Save the file's meta-info before we open it. Doing it later + means we mess up the access times. */ + saveInputFileMetaInfo ( inName ); + } + switch ( srcMode ) { case SM_I2O: @@ -1463,7 +1507,7 @@ void uncompress ( Char *name ) case SM_F2F: inStr = fopen ( inName, "rb" ); - outStr = fopen ( outName, "wb" ); + outStr = fopen_output_safely ( outName, "wb" ); if ( outStr == NULL) { fprintf ( stderr, "%s: Can't create output file %s: %s.\n", progName, outName, strerror(errno) ); @@ -1478,7 +1522,6 @@ void uncompress ( Char *name ) setExit(1); return; }; - setInterimPermissions ( outName ); break; default: @@ -1501,7 +1544,7 @@ void uncompress ( Char *name ) /*--- If there was an I/O error, we won't get here. ---*/ if ( magicNumberOK ) { if ( srcMode == SM_F2F ) { - copyDatePermissionsAndOwner ( inName, outName ); + applySavedMetaInfoToOutputFile ( outName ); deleteOutputOnInterrupt = False; if ( !keepInputFiles ) { IntNative retVal = remove ( inName ); @@ -1539,6 +1582,7 @@ void testf ( Char *name ) { FILE *inStr; Bool allOK; + struct MY_STAT statBuf; deleteOutputOnInterrupt = False; @@ -1565,6 +1609,16 @@ void testf ( Char *name ) setExit(1); return; } + if ( srcMode != SM_I2O ) { + MY_STAT(inName, &statBuf); + if ( MY_S_ISDIR(statBuf.st_mode) ) { + fprintf( stderr, + "%s: Input file %s is a directory.\n", + progName,inName); + setExit(1); + return; + } + } switch ( srcMode ) { @@ -1603,6 +1657,7 @@ void testf ( Char *name ) } /*--- Now the input handle is sane. Do the Biz. ---*/ + outputHandleJustInCase = NULL; allOK = testStream ( inStr ); if (allOK && verbosity >= 1) fprintf ( stderr, "ok\n" ); @@ -1619,7 +1674,7 @@ void license ( void ) "bzip2, a block-sorting file compressor. " "Version %s.\n" " \n" - " Copyright (C) 1996-2000 by Julian Seward.\n" + " Copyright (C) 1996-2002 by Julian Seward.\n" " \n" " This program is free software; you can redistribute it and/or modify\n" " it under the terms set out in the LICENSE file, which is included\n" @@ -1658,6 +1713,8 @@ void usage ( Char *fullProgName ) " -V --version display software version & license\n" " -s --small use less memory (at most 2500k)\n" " -1 .. -9 set block size to 100k .. 900k\n" + " --fast alias for -1\n" + " --best alias for -9\n" "\n" " If invoked as `bzip2', default action is to compress.\n" " as `bunzip2', default action is to decompress.\n" @@ -1666,9 +1723,9 @@ void usage ( Char *fullProgName ) " If no file names are given, bzip2 compresses or decompresses\n" " from standard input to standard output. You can combine\n" " short flags, so `-v -4' means the same as -v4 or -4v, &c.\n" -#if BZ_UNIX +# if BZ_UNIX "\n" -#endif +# endif , BZ2_bzlibVersion(), @@ -1818,11 +1875,11 @@ IntNative main ( IntNative argc, Char *argv[] ) /*-- Set up signal handlers for mem access errors --*/ signal (SIGSEGV, mySIGSEGVorSIGBUScatcher); -#if BZ_UNIX -#ifndef __DJGPP__ +# if BZ_UNIX +# ifndef __DJGPP__ signal (SIGBUS, mySIGSEGVorSIGBUScatcher); -#endif -#endif +# endif +# endif copyFileName ( inName, "(none)" ); copyFileName ( outName, "(none)" ); @@ -1933,6 +1990,8 @@ IntNative main ( IntNative argc, Char *argv[] ) if (ISFLAG("--exponential")) workFactor = 1; else if (ISFLAG("--repetitive-best")) redundant(aa->name); else if (ISFLAG("--repetitive-fast")) redundant(aa->name); else + if (ISFLAG("--fast")) blockSize100k = 1; else + if (ISFLAG("--best")) blockSize100k = 9; else if (ISFLAG("--verbose")) verbosity++; else if (ISFLAG("--help")) { usage ( progName ); exit ( 0 ); } else diff --git a/bzip2.txt b/bzip2.txt index 4f1ae86..6afe358 100644 --- a/bzip2.txt +++ b/bzip2.txt @@ -1,7 +1,6 @@ - NAME - bzip2, bunzip2 - a block-sorting file compressor, v1.0 + bzip2, bunzip2 - a block-sorting file compressor, v1.0.2 bzcat - decompresses files to stdout bzip2recover - recovers data from damaged bzip2 files @@ -18,20 +17,20 @@ DESCRIPTION sorting text compression algorithm, and Huffman coding. Compression is generally considerably better than that achieved by more conventional LZ77/LZ78-based compressors, - and approaches the performance of the PPM family of sta- + and approaches the performance of the PPM family of sta tistical compressors. The command-line options are deliberately very similar to those of GNU gzip, but they are not identical. - bzip2 expects a list of file names to accompany the com- + bzip2 expects a list of file names to accompany the com mand-line flags. Each file is replaced by a compressed version of itself, with the name "original_name.bz2". - Each compressed file has the same modification date, per- - missions, and, when possible, ownership as the correspond- + Each compressed file has the same modification date, per + missions, and, when possible, ownership as the correspond ing original, so that these properties can be correctly restored at decompression time. File name handling is - naive in the sense that there is no mechanism for preserv- + naive in the sense that there is no mechanism for preserv ing original file names, permissions, ownerships or dates in filesystems which lack these concepts, or have serious file name length restrictions, such as MS-DOS. @@ -62,23 +61,23 @@ DESCRIPTION guess the name of the original file, and uses the original name with .out appended. - As with compression, supplying no filenames causes decom- + As with compression, supplying no filenames causes decom pression from standard input to standard output. - bunzip2 will correctly decompress a file which is the con- + bunzip2 will correctly decompress a file which is the con catenation of two or more compressed files. The result is the concatenation of the corresponding uncompressed files. Integrity testing (-t) of concatenated compressed files is also supported. You can also compress or decompress files to the standard - output by giving the -c flag. Multiple files may be com- + output by giving the -c flag. Multiple files may be com pressed and decompressed like this. The resulting outputs are fed sequentially to stdout. Compression of multiple - files in this manner generates a stream containing multi- + files in this manner generates a stream containing multi ple compressed file representations. Such a stream can be decompressed correctly only by bzip2 version 0.9.0 or - later. Earlier versions of bzip2 will stop after decom- + later. Earlier versions of bzip2 will stop after decom pressing the first file in the stream. bzcat (or bzip2 -dc) decompresses all specified files to @@ -99,7 +98,7 @@ DESCRIPTION As a self-check for your protection, bzip2 uses 32-bit CRCs to make sure that the decompressed version of a file - is identical to the original. This guards against corrup- + is identical to the original. This guards against corrup tion of the compressed data, and against undetected bugs in bzip2 (hopefully very unlikely). The chances of data corruption going undetected is microscopic, about one @@ -127,8 +126,8 @@ OPTIONS and forces bzip2 to decompress. -z --compress - The complement to -d: forces compression, regard- - less of the invokation name. + The complement to -d: forces compression, + regardless of the invocation name. -t --test Check integrity of the specified file(s), but don't @@ -141,6 +140,11 @@ OPTIONS forces bzip2 to break hard links to files, which it otherwise wouldn't do. + bzip2 normally declines to decompress files which + don't have the correct magic header bytes. If + forced (-f), however, it will pass such files + through unmodified. This is how GNU gzip behaves. + -k --keep Keep (don't delete) input files during compression or decompression. @@ -167,7 +171,7 @@ OPTIONS -v --verbose Verbose mode -- show the compression ratio for each - file processed. Further -v's increase the ver- + file processed. Further -v's increase the ver bosity level, spewing out lots of information which is primarily of interest for diagnostic purposes. @@ -175,20 +179,24 @@ OPTIONS Display the software version, license terms and conditions. - -1 to -9 + -1 (or --fast) to -9 (or --best) Set the block size to 100 k, 200 k .. 900 k when compressing. Has no effect when decompressing. - See MEMORY MANAGEMENT below. + See MEMORY MANAGEMENT below. The --fast and --best + aliases are primarily for GNU gzip compatibility. + In particular, --fast doesn't make things signifi + cantly faster. And --best merely selects the + default behaviour. -- Treats all subsequent arguments as file names, even - if they start with a dash. This is so you can han- + if they start with a dash. This is so you can han dle files with names beginning with a dash, for example: bzip2 -- -myfilename. --repetitive-fast --repetitive-best These flags are redundant in versions 0.9.5 and above. They provided some coarse control over the - behaviour of the sorting algorithm in earlier ver- + behaviour of the sorting algorithm in earlier ver sions, which was sometimes useful. 0.9.5 and above have an improved algorithm which renders these flags irrelevant. @@ -199,7 +207,7 @@ MEMORY MANAGEMENT affects both the compression ratio achieved, and the amount of memory needed for compression and decompression. The flags -1 through -9 specify the block size to be - 100,000 bytes through 900,000 bytes (the default) respec- + 100,000 bytes through 900,000 bytes (the default) respec tively. At decompression time, the block size used for compression is read from the header of the compressed file, and bunzip2 then allocates itself just enough memory @@ -227,13 +235,13 @@ MEMORY MANAGEMENT bunzip2 will require about 3700 kbytes to decompress. To support decompression of any file on a 4 megabyte machine, bunzip2 has an option to decompress using approximately - half this amount of memory, about 2300 kbytes. Decompres- + half this amount of memory, about 2300 kbytes. Decompres sion speed is also halved, so you should use this option only where necessary. The relevant flag is -s. - In general, try and use the largest block size memory con- + In general, try and use the largest block size memory con straints allow, since that maximises the compression - achieved. Compression and decompression speed are virtu- + achieved. Compression and decompression speed are virtu ally unaffected by block size. Another significant point applies to files which fit in a @@ -249,11 +257,11 @@ MEMORY MANAGEMENT Here is a table which summarises the maximum memory usage for different block sizes. Also recorded is the total - compressed size for 14 files of the Calgary Text Compres- + compressed size for 14 files of the Calgary Text Compres sion Corpus totalling 3,141,622 bytes. This column gives some feel for how compression varies with block size. These figures tend to understate the advantage of larger - block sizes for larger files, since the Corpus is domi- + block sizes for larger files, since the Corpus is domi nated by smaller files. Compress Decompress Decompress Corpus @@ -272,7 +280,7 @@ MEMORY MANAGEMENT RECOVERING DATA FROM DAMAGED FILES bzip2 compresses files in blocks, usually 900kbytes long. - Each block is handled independently. If a media or trans- + Each block is handled independently. If a media or trans mission error causes a multi-block .bz2 file to become damaged, it may be possible to recover data from the undamaged blocks in the file. @@ -289,19 +297,19 @@ RECOVERING DATA FROM DAMAGED FILES the integrity of the resulting files, and decompress those which are undamaged. - bzip2recover takes a single argument, the name of the dam- - aged file, and writes a number of files "rec0001file.bz2", - "rec0002file.bz2", etc, containing the extracted blocks. - The output filenames are designed so that the use of - wildcards in subsequent processing -- for example, "bzip2 - -dc rec*file.bz2 > recovered_data" -- lists the files in - the correct order. + bzip2recover takes a single argument, the name of the dam + aged file, and writes a number of files + "rec00001file.bz2", "rec00002file.bz2", etc, containing + the extracted blocks. The output filenames are + designed so that the use of wildcards in subsequent pro + cessing -- for example, "bzip2 -dc rec*file.bz2 > recov + ered_data" -- processes the files in the correct order. bzip2recover should be of most use dealing with large .bz2 files, as these will contain many blocks. It is clearly futile to use it on damaged single-block files, since a - damaged block cannot be recovered. If you wish to min- - imise any potential data loss through media or transmis- + damaged block cannot be recovered. If you wish to min + imise any potential data loss through media or transmis sion errors, you might consider compressing with a smaller block size. @@ -315,19 +323,19 @@ PERFORMANCE NOTES better than previous versions in this respect. The ratio between worst-case and average-case compression time is in the region of 10:1. For previous versions, this figure - was more like 100:1. You can use the -vvvv option to mon- + was more like 100:1. You can use the -vvvv option to mon itor progress in great detail, if you want. Decompression speed is unaffected by these phenomena. bzip2 usually allocates several megabytes of memory to - operate in, and then charges all over it in a fairly ran- - dom fashion. This means that performance, both for com- + operate in, and then charges all over it in a fairly ran + dom fashion. This means that performance, both for com pressing and decompressing, is largely determined by the speed at which your machine can service cache misses. Because of this, small changes to the code to reduce the miss rate have been observed to give disproportionately - large performance improvements. I imagine bzip2 will per- + large performance improvements. I imagine bzip2 will per form best on machines with very large caches. @@ -337,40 +345,46 @@ CAVEATS but the details of what the problem is sometimes seem rather misleading. - This manual page pertains to version 1.0 of bzip2. Com- + This manual page pertains to version 1.0.2 of bzip2. Com pressed data created by this version is entirely forwards and backwards compatible with the previous public - releases, versions 0.1pl2, 0.9.0 and 0.9.5, but with the - following exception: 0.9.0 and above can correctly decom- - press multiple concatenated compressed files. 0.1pl2 can- - not do this; it will stop after decompressing just the - first file in the stream. - - bzip2recover uses 32-bit integers to represent bit posi- - tions in compressed files, so it cannot handle compressed - files more than 512 megabytes long. This could easily be - fixed. + releases, versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0 and 1.0.1, + but with the following exception: 0.9.0 and above can cor + rectly decompress multiple concatenated compressed files. + 0.1pl2 cannot do this; it will stop after decompressing + just the first file in the stream. + + bzip2recover versions prior to this one, 1.0.2, used + 32-bit integers to represent bit positions in compressed + files, so it could not handle compressed files more than + 512 megabytes long. Version 1.0.2 and above uses 64-bit + ints on some platforms which support them (GNU supported + targets, and Windows). To establish whether or not + bzip2recover was built with such a limitation, run it + without arguments. In any event you can build yourself an + unlimited version if you can recompile it with MaybeUInt64 + set to be an unsigned 64-bit integer. AUTHOR Julian Seward, jseward@acm.org. - http://sourceware.cygnus.com/bzip2 - http://www.muraroa.demon.co.uk + http://sources.redhat.com/bzip2 - The ideas embodied in bzip2 are due to (at least) the fol- + The ideas embodied in bzip2 are due to (at least) the fol lowing people: Michael Burrows and David Wheeler (for the block sorting transformation), David Wheeler (again, for - the Huffman coder), Peter Fenwick (for the structured cod- + the Huffman coder), Peter Fenwick (for the structured cod ing model in the original bzip, and many refinements), and Alistair Moffat, Radford Neal and Ian Witten (for the arithmetic coder in the original bzip). I am much - indebted for their help, support and advice. See the man- + indebted for their help, support and advice. See the man ual in the source distribution for pointers to sources of documentation. Christian von Roques encouraged me to look - for faster sorting algorithms, so as to speed up compres- + for faster sorting algorithms, so as to speed up compres sion. Bela Lubkin encouraged me to improve the worst-case - compression performance. Many people sent patches, helped - with portability problems, lent machines, gave advice and - were generally helpful. + compression performance. The bz* scripts are derived from + those of GNU gzip. Many people sent patches, helped with + portability problems, lent machines, gave advice and were + generally helpful. diff --git a/bzip2recover.c b/bzip2recover.c index ba3d175..286873b 100644 --- a/bzip2recover.c +++ b/bzip2recover.c @@ -9,7 +9,7 @@ salvage from damaged files created by the accompanying bzip2-1.0 program. - Copyright (C) 1996-2000 Julian R Seward. All rights reserved. + Copyright (C) 1996-2002 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -57,6 +57,29 @@ #include #include + +/* This program records bit locations in the file to be recovered. + That means that if 64-bit ints are not supported, we will not + be able to recover .bz2 files over 512MB (2^32 bits) long. + On GNU supported platforms, we take advantage of the 64-bit + int support to circumvent this problem. Ditto MSVC. + + This change occurred in version 1.0.2; all prior versions have + the 512MB limitation. +*/ +#ifdef __GNUC__ + typedef unsigned long long int MaybeUInt64; +# define MaybeUInt64_FMT "%Lu" +#else +#ifdef _MSC_VER + typedef unsigned __int64 MaybeUInt64; +# define MaybeUInt64_FMT "%I64u" +#else + typedef unsigned int MaybeUInt64; +# define MaybeUInt64_FMT "%u" +#endif +#endif + typedef unsigned int UInt32; typedef int Int32; typedef unsigned char UChar; @@ -66,13 +89,25 @@ typedef unsigned char Bool; #define False ((Bool)0) -Char inFileName[2000]; -Char outFileName[2000]; -Char progName[2000]; +#define BZ_MAX_FILENAME 2000 + +Char inFileName[BZ_MAX_FILENAME]; +Char outFileName[BZ_MAX_FILENAME]; +Char progName[BZ_MAX_FILENAME]; + +MaybeUInt64 bytesOut = 0; +MaybeUInt64 bytesIn = 0; -UInt32 bytesOut = 0; -UInt32 bytesIn = 0; +/*---------------------------------------------------*/ +/*--- Header bytes ---*/ +/*---------------------------------------------------*/ + +#define BZ_HDR_B 0x42 /* 'B' */ +#define BZ_HDR_Z 0x5a /* 'Z' */ +#define BZ_HDR_h 0x68 /* 'h' */ +#define BZ_HDR_0 0x30 /* '0' */ + /*---------------------------------------------------*/ /*--- I/O errors ---*/ @@ -116,6 +151,23 @@ void mallocFail ( Int32 n ) } +/*---------------------------------------------*/ +void tooManyBlocks ( Int32 max_handled_blocks ) +{ + fprintf ( stderr, + "%s: `%s' appears to contain more than %d blocks\n", + progName, inFileName, max_handled_blocks ); + fprintf ( stderr, + "%s: and cannot be handled. To fix, increase\n", + progName ); + fprintf ( stderr, + "%s: BZ_MAX_HANDLED_BLOCKS in bzip2recover.c, and recompile.\n", + progName ); + exit ( 1 ); +} + + + /*---------------------------------------------------*/ /*--- Bit stream I/O ---*/ /*---------------------------------------------------*/ @@ -254,27 +306,37 @@ Bool endsInBz2 ( Char* name ) /*--- ---*/ /*---------------------------------------------------*/ +/* This logic isn't really right when it comes to Cygwin. */ +#ifdef _WIN32 +# define BZ_SPLIT_SYM '\\' /* path splitter on Windows platform */ +#else +# define BZ_SPLIT_SYM '/' /* path splitter on Unix platform */ +#endif + #define BLOCK_HEADER_HI 0x00003141UL #define BLOCK_HEADER_LO 0x59265359UL #define BLOCK_ENDMARK_HI 0x00001772UL #define BLOCK_ENDMARK_LO 0x45385090UL +/* Increase if necessary. However, a .bz2 file with > 50000 blocks + would have an uncompressed size of at least 40GB, so the chances + are low you'll need to up this. +*/ +#define BZ_MAX_HANDLED_BLOCKS 50000 -UInt32 bStart[20000]; -UInt32 bEnd[20000]; -UInt32 rbStart[20000]; -UInt32 rbEnd[20000]; +MaybeUInt64 bStart [BZ_MAX_HANDLED_BLOCKS]; +MaybeUInt64 bEnd [BZ_MAX_HANDLED_BLOCKS]; +MaybeUInt64 rbStart[BZ_MAX_HANDLED_BLOCKS]; +MaybeUInt64 rbEnd [BZ_MAX_HANDLED_BLOCKS]; Int32 main ( Int32 argc, Char** argv ) { FILE* inFile; FILE* outFile; BitStream* bsIn, *bsWr; - Int32 currBlock, b, wrBlock; - UInt32 bitsRead; - Int32 rbCtr; - + Int32 b, wrBlock, currBlock, rbCtr; + MaybeUInt64 bitsRead; UInt32 buffHi, buffLo, blockCRC; Char* p; @@ -282,11 +344,37 @@ Int32 main ( Int32 argc, Char** argv ) strcpy ( progName, argv[0] ); inFileName[0] = outFileName[0] = 0; - fprintf ( stderr, "bzip2recover 1.0: extracts blocks from damaged .bz2 files.\n" ); + fprintf ( stderr, + "bzip2recover 1.0.2: extracts blocks from damaged .bz2 files.\n" ); if (argc != 2) { fprintf ( stderr, "%s: usage is `%s damaged_file_name'.\n", progName, progName ); + switch (sizeof(MaybeUInt64)) { + case 8: + fprintf(stderr, + "\trestrictions on size of recovered file: None\n"); + break; + case 4: + fprintf(stderr, + "\trestrictions on size of recovered file: 512 MB\n"); + fprintf(stderr, + "\tto circumvent, recompile with MaybeUInt64 as an\n" + "\tunsigned 64-bit int.\n"); + break; + default: + fprintf(stderr, + "\tsizeof(MaybeUInt64) is not 4 or 8 -- " + "configuration error.\n"); + break; + } + exit(1); + } + + if (strlen(argv[1]) >= BZ_MAX_FILENAME-20) { + fprintf ( stderr, + "%s: supplied filename is suspiciously (>= %d chars) long. Bye!\n", + progName, strlen(argv[1]) ); exit(1); } @@ -316,7 +404,8 @@ Int32 main ( Int32 argc, Char** argv ) (bitsRead - bStart[currBlock]) >= 40) { bEnd[currBlock] = bitsRead-1; if (currBlock > 0) - fprintf ( stderr, " block %d runs from %d to %d (incomplete)\n", + fprintf ( stderr, " block %d runs from " MaybeUInt64_FMT + " to " MaybeUInt64_FMT " (incomplete)\n", currBlock, bStart[currBlock], bEnd[currBlock] ); } else currBlock--; @@ -330,17 +419,22 @@ Int32 main ( Int32 argc, Char** argv ) ( (buffHi & 0x0000ffff) == BLOCK_ENDMARK_HI && buffLo == BLOCK_ENDMARK_LO) ) { - if (bitsRead > 49) - bEnd[currBlock] = bitsRead-49; else + if (bitsRead > 49) { + bEnd[currBlock] = bitsRead-49; + } else { bEnd[currBlock] = 0; + } if (currBlock > 0 && (bEnd[currBlock] - bStart[currBlock]) >= 130) { - fprintf ( stderr, " block %d runs from %d to %d\n", + fprintf ( stderr, " block %d runs from " MaybeUInt64_FMT + " to " MaybeUInt64_FMT "\n", rbCtr+1, bStart[currBlock], bEnd[currBlock] ); rbStart[rbCtr] = bStart[currBlock]; rbEnd[rbCtr] = bEnd[currBlock]; rbCtr++; } + if (currBlock >= BZ_MAX_HANDLED_BLOCKS) + tooManyBlocks(BZ_MAX_HANDLED_BLOCKS); currBlock++; bStart[currBlock] = bitsRead; @@ -400,10 +494,25 @@ Int32 main ( Int32 argc, Char** argv ) wrBlock++; } else if (bitsRead == rbStart[wrBlock]) { - outFileName[0] = 0; - sprintf ( outFileName, "rec%4d", wrBlock+1 ); - for (p = outFileName; *p != 0; p++) if (*p == ' ') *p = '0'; - strcat ( outFileName, inFileName ); + /* Create the output file name, correctly handling leading paths. + (31.10.2001 by Sergey E. Kusikov) */ + Char* split; + Int32 ofs, k; + for (k = 0; k < BZ_MAX_FILENAME; k++) + outFileName[k] = 0; + strcpy (outFileName, inFileName); + split = strrchr (outFileName, BZ_SPLIT_SYM); + if (split == NULL) { + split = outFileName; + } else { + ++split; + } + /* Now split points to the start of the basename. */ + ofs = split - outFileName; + sprintf (split, "rec%5d", wrBlock+1); + for (p = split; *p != 0; p++) if (*p == ' ') *p = '0'; + strcat (outFileName, inFileName + ofs); + if ( !endsInBz2(outFileName)) strcat ( outFileName, ".bz2" ); fprintf ( stderr, " writing block %d to `%s' ...\n", @@ -416,8 +525,10 @@ Int32 main ( Int32 argc, Char** argv ) exit(1); } bsWr = bsOpenWriteStream ( outFile ); - bsPutUChar ( bsWr, 'B' ); bsPutUChar ( bsWr, 'Z' ); - bsPutUChar ( bsWr, 'h' ); bsPutUChar ( bsWr, '9' ); + bsPutUChar ( bsWr, BZ_HDR_B ); + bsPutUChar ( bsWr, BZ_HDR_Z ); + bsPutUChar ( bsWr, BZ_HDR_h ); + bsPutUChar ( bsWr, BZ_HDR_0 + 9 ); bsPutUChar ( bsWr, 0x31 ); bsPutUChar ( bsWr, 0x41 ); bsPutUChar ( bsWr, 0x59 ); bsPutUChar ( bsWr, 0x26 ); bsPutUChar ( bsWr, 0x53 ); bsPutUChar ( bsWr, 0x59 ); diff --git a/bzlib.c b/bzlib.c index 4a06d9f..7d1cb27 100644 --- a/bzlib.c +++ b/bzlib.c @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-2000 Julian R Seward. All rights reserved. + Copyright (C) 1996-2002 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -93,10 +93,39 @@ void BZ2_bz__AssertH__fail ( int errcode ) "component, you should also report this bug to the author(s)\n" "of that program. Please make an effort to report this bug;\n" "timely and accurate bug reports eventually lead to higher\n" - "quality software. Thanks. Julian Seward, 21 March 2000.\n\n", + "quality software. Thanks. Julian Seward, 30 December 2001.\n\n", errcode, BZ2_bzlibVersion() ); + + if (errcode == 1007) { + fprintf(stderr, + "\n*** A special note about internal error number 1007 ***\n" + "\n" + "Experience suggests that a common cause of i.e. 1007\n" + "is unreliable memory or other hardware. The 1007 assertion\n" + "just happens to cross-check the results of huge numbers of\n" + "memory reads/writes, and so acts (unintendedly) as a stress\n" + "test of your memory system.\n" + "\n" + "I suggest the following: try compressing the file again,\n" + "possibly monitoring progress in detail with the -vv flag.\n" + "\n" + "* If the error cannot be reproduced, and/or happens at different\n" + " points in compression, you may have a flaky memory system.\n" + " Try a memory-test program. I have used Memtest86\n" + " (www.memtest86.com). At the time of writing it is free (GPLd).\n" + " Memtest86 tests memory much more thorougly than your BIOSs\n" + " power-on test, and may find failures that the BIOS doesn't.\n" + "\n" + "* If the error can be repeatably reproduced, this is a bug in\n" + " bzip2, and I would very much like to hear about it. Please\n" + " let me know, and, ideally, save a copy of the file causing the\n" + " problem -- without which I will be unable to investigate it.\n" + "\n" + ); + } + exit(3); } #endif @@ -1402,7 +1431,7 @@ BZFILE * bzopen_or_bzdopen smallMode = 1; break; default: if (isdigit((int)(*mode))) { - blockSize100k = *mode-'0'; + blockSize100k = *mode-BZ_HDR_0; } } mode++; diff --git a/bzlib.h b/bzlib.h index c9447a2..9ac43a1 100644 --- a/bzlib.h +++ b/bzlib.h @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-2000 Julian R Seward. All rights reserved. + Copyright (C) 1996-2002 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -110,8 +110,10 @@ typedef #define BZ_EXPORT #endif +/* Need a definitition for FILE */ +#include + #ifdef _WIN32 -# include # include # ifdef small /* windows.h define small to char */ diff --git a/bzlib_private.h b/bzlib_private.h index fb51c7a..ff973c3 100644 --- a/bzlib_private.h +++ b/bzlib_private.h @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-2000 Julian R Seward. All rights reserved. + Copyright (C) 1996-2002 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -76,7 +76,7 @@ /*-- General stuff. --*/ -#define BZ_VERSION "1.0.1, 23-June-2000" +#define BZ_VERSION "1.0.2, 30-Dec-2001" typedef char Char; typedef unsigned char Bool; @@ -137,6 +137,13 @@ extern void bz_internal_error ( int errcode ); #define BZFREE(ppp) (strm->bzfree)(strm->opaque,(ppp)) +/*-- Header bytes. --*/ + +#define BZ_HDR_B 0x42 /* 'B' */ +#define BZ_HDR_Z 0x5a /* 'Z' */ +#define BZ_HDR_h 0x68 /* 'h' */ +#define BZ_HDR_0 0x30 /* '0' */ + /*-- Constants for the back end. --*/ #define BZ_MAX_ALPHA_SIZE 258 diff --git a/bzmore b/bzmore new file mode 100644 index 0000000..d314043 --- /dev/null +++ b/bzmore @@ -0,0 +1,61 @@ +#!/bin/sh + +# Bzmore wrapped for bzip2, +# adapted from zmore by Philippe Troin for Debian GNU/Linux. + +PATH="/usr/bin:$PATH"; export PATH + +prog=`echo $0 | sed 's|.*/||'` +case "$prog" in + *less) more=less ;; + *) more=more ;; +esac + +if test "`echo -n a`" = "-n a"; then + # looks like a SysV system: + n1=''; n2='\c' +else + n1='-n'; n2='' +fi +oldtty=`stty -g 2>/dev/null` +if stty -cbreak 2>/dev/null; then + cb='cbreak'; ncb='-cbreak' +else + # 'stty min 1' resets eof to ^a on both SunOS and SysV! + cb='min 1 -icanon'; ncb='icanon eof ^d' +fi +if test $? -eq 0 -a -n "$oldtty"; then + trap 'stty $oldtty 2>/dev/null; exit' 0 2 3 5 10 13 15 +else + trap 'stty $ncb echo 2>/dev/null; exit' 0 2 3 5 10 13 15 +fi + +if test $# = 0; then + if test -t 0; then + echo usage: $prog files... + else + bzip2 -cdfq | eval $more + fi +else + FIRST=1 + for FILE + do + if test $FIRST -eq 0; then + echo $n1 "--More--(Next file: $FILE)$n2" + stty $cb -echo 2>/dev/null + ANS=`dd bs=1 count=1 2>/dev/null` + stty $ncb echo 2>/dev/null + echo " " + if test "$ANS" = 'e' -o "$ANS" = 'q'; then + exit + fi + fi + if test "$ANS" != 's'; then + echo "------> $FILE <------" + bzip2 -cdfq "$FILE" | eval $more + fi + if test -t; then + FIRST=0 + fi + done +fi diff --git a/bzmore.1 b/bzmore.1 new file mode 100644 index 0000000..b437d3b --- /dev/null +++ b/bzmore.1 @@ -0,0 +1,152 @@ +.\"Shamelessly copied from zmore.1 by Philippe Troin +.\"for Debian GNU/Linux +.TH BZMORE 1 +.SH NAME +bzmore, bzless \- file perusal filter for crt viewing of bzip2 compressed text +.SH SYNOPSIS +.B bzmore +[ name ... ] +.br +.B bzless +[ name ... ] +.SH NOTE +In the following description, +.I bzless +and +.I less +can be used interchangeably with +.I bzmore +and +.I more. +.SH DESCRIPTION +.I Bzmore +is a filter which allows examination of compressed or plain text files +one screenful at a time on a soft-copy terminal. +.I bzmore +works on files compressed with +.I bzip2 +and also on uncompressed files. +If a file does not exist, +.I bzmore +looks for a file of the same name with the addition of a .bz2 suffix. +.PP +.I Bzmore +normally pauses after each screenful, printing --More-- +at the bottom of the screen. +If the user then types a carriage return, one more line is displayed. +If the user hits a space, +another screenful is displayed. Other possibilities are enumerated later. +.PP +.I Bzmore +looks in the file +.I /etc/termcap +to determine terminal characteristics, +and to determine the default window size. +On a terminal capable of displaying 24 lines, +the default window size is 22 lines. +Other sequences which may be typed when +.I bzmore +pauses, and their effects, are as follows (\fIi\fP is an optional integer +argument, defaulting to 1) : +.PP +.IP \fIi\|\fP +display +.I i +more lines, (or another screenful if no argument is given) +.PP +.IP ^D +display 11 more lines (a ``scroll''). +If +.I i +is given, then the scroll size is set to \fIi\|\fP. +.PP +.IP d +same as ^D (control-D) +.PP +.IP \fIi\|\fPz +same as typing a space except that \fIi\|\fP, if present, becomes the new +window size. Note that the window size reverts back to the default at the +end of the current file. +.PP +.IP \fIi\|\fPs +skip \fIi\|\fP lines and print a screenful of lines +.PP +.IP \fIi\|\fPf +skip \fIi\fP screenfuls and print a screenful of lines +.PP +.IP "q or Q" +quit reading the current file; go on to the next (if any) +.PP +.IP "e or q" +When the prompt --More--(Next file: +.IR file ) +is printed, this command causes bzmore to exit. +.PP +.IP s +When the prompt --More--(Next file: +.IR file ) +is printed, this command causes bzmore to skip the next file and continue. +.PP +.IP = +Display the current line number. +.PP +.IP \fIi\|\fP/expr +search for the \fIi\|\fP-th occurrence of the regular expression \fIexpr.\fP +If the pattern is not found, +.I bzmore +goes on to the next file (if any). +Otherwise, a screenful is displayed, starting two lines before the place +where the expression was found. +The user's erase and kill characters may be used to edit the regular +expression. +Erasing back past the first column cancels the search command. +.PP +.IP \fIi\|\fPn +search for the \fIi\|\fP-th occurrence of the last regular expression entered. +.PP +.IP !command +invoke a shell with \fIcommand\|\fP. +The character `!' in "command" are replaced with the +previous shell command. The sequence "\\!" is replaced by "!". +.PP +.IP ":q or :Q" +quit reading the current file; go on to the next (if any) +(same as q or Q). +.PP +.IP . +(dot) repeat the previous command. +.PP +The commands take effect immediately, i.e., it is not necessary to +type a carriage return. +Up to the time when the command character itself is given, +the user may hit the line kill character to cancel the numerical +argument being formed. +In addition, the user may hit the erase character to redisplay the +--More-- message. +.PP +At any time when output is being sent to the terminal, the user can +hit the quit key (normally control\-\\). +.I Bzmore +will stop sending output, and will display the usual --More-- +prompt. +The user may then enter one of the above commands in the normal manner. +Unfortunately, some output is lost when this is done, due to the +fact that any characters waiting in the terminal's output queue +are flushed when the quit signal occurs. +.PP +The terminal is set to +.I noecho +mode by this program so that the output can be continuous. +What you type will thus not show on your terminal, except for the / and ! +commands. +.PP +If the standard output is not a teletype, then +.I bzmore +acts just like +.I bzcat, +except that a header is printed before each file. +.SH FILES +.DT +/etc/termcap Terminal data base +.SH "SEE ALSO" +more(1), less(1), bzip2(1), bzdiff(1), bzgrep(1) diff --git a/compress.c b/compress.c index cc5e31d..56501c1 100644 --- a/compress.c +++ b/compress.c @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-2000 Julian R Seward. All rights reserved. + Copyright (C) 1996-2002 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -663,10 +663,10 @@ void BZ2_compressBlock ( EState* s, Bool is_last_block ) /*-- If this is the first block, create the stream header. --*/ if (s->blockNo == 1) { BZ2_bsInitWrite ( s ); - bsPutUChar ( s, 'B' ); - bsPutUChar ( s, 'Z' ); - bsPutUChar ( s, 'h' ); - bsPutUChar ( s, (UChar)('0' + s->blockSize100k) ); + bsPutUChar ( s, BZ_HDR_B ); + bsPutUChar ( s, BZ_HDR_Z ); + bsPutUChar ( s, BZ_HDR_h ); + bsPutUChar ( s, (UChar)(BZ_HDR_0 + s->blockSize100k) ); } if (s->nblock > 0) { diff --git a/crctable.c b/crctable.c index 61c040c..b16746a 100644 --- a/crctable.c +++ b/crctable.c @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-2000 Julian R Seward. All rights reserved. + Copyright (C) 1996-2002 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions diff --git a/decompress.c b/decompress.c index cdced18..e921347 100644 --- a/decompress.c +++ b/decompress.c @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-2000 Julian R Seward. All rights reserved. + Copyright (C) 1996-2002 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -235,18 +235,18 @@ Int32 BZ2_decompress ( DState* s ) switch (s->state) { GET_UCHAR(BZ_X_MAGIC_1, uc); - if (uc != 'B') RETURN(BZ_DATA_ERROR_MAGIC); + if (uc != BZ_HDR_B) RETURN(BZ_DATA_ERROR_MAGIC); GET_UCHAR(BZ_X_MAGIC_2, uc); - if (uc != 'Z') RETURN(BZ_DATA_ERROR_MAGIC); + if (uc != BZ_HDR_Z) RETURN(BZ_DATA_ERROR_MAGIC); GET_UCHAR(BZ_X_MAGIC_3, uc) - if (uc != 'h') RETURN(BZ_DATA_ERROR_MAGIC); + if (uc != BZ_HDR_h) RETURN(BZ_DATA_ERROR_MAGIC); GET_BITS(BZ_X_MAGIC_4, s->blockSize100k, 8) - if (s->blockSize100k < '1' || - s->blockSize100k > '9') RETURN(BZ_DATA_ERROR_MAGIC); - s->blockSize100k -= '0'; + if (s->blockSize100k < (BZ_HDR_0 + 1) || + s->blockSize100k > (BZ_HDR_0 + 9)) RETURN(BZ_DATA_ERROR_MAGIC); + s->blockSize100k -= BZ_HDR_0; if (s->smallDecompress) { s->ll16 = BZALLOC( s->blockSize100k * 100000 * sizeof(UInt16) ); diff --git a/dlltest.c b/dlltest.c index f79279c..2d7dcca 100644 --- a/dlltest.c +++ b/dlltest.c @@ -19,7 +19,7 @@ #ifdef _WIN32 -#define BZ2_LIBNAME "libbz2-1.0.0.DLL" +#define BZ2_LIBNAME "libbz2-1.0.2.DLL" #include static int BZ2DLLLoaded = 0; @@ -130,8 +130,8 @@ int main(int argc,char *argv[]) }else{ fp_w = stdout; } - if((BZ2fp_r == NULL && (BZ2fp_r = BZ2_bzdopen(fileno(stdin),"rb"))==NULL) - || (BZ2fp_r != NULL && (BZ2fp_r = BZ2_bzopen(fn_r,"rb"))==NULL)){ + if((fn_r == NULL && (BZ2fp_r = BZ2_bzdopen(fileno(stdin),"rb"))==NULL) + || (fn_r != NULL && (BZ2fp_r = BZ2_bzopen(fn_r,"rb"))==NULL)){ printf("can't bz2openstream\n"); exit(1); } diff --git a/huffman.c b/huffman.c index 9b446c4..293095c 100644 --- a/huffman.c +++ b/huffman.c @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-2000 Julian R Seward. All rights reserved. + Copyright (C) 1996-2002 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions diff --git a/makefile.msc b/makefile.msc index 3fe4232..799a18a 100644 --- a/makefile.msc +++ b/makefile.msc @@ -4,7 +4,7 @@ # Fixed up by JRS for bzip2-0.9.5d release. CC=cl -CFLAGS= -DWIN32 -MD -Ox -D_FILE_OFFSET_BITS=64 +CFLAGS= -DWIN32 -MD -Ox -D_FILE_OFFSET_BITS=64 -nologo OBJS= blocksort.obj \ huffman.obj \ diff --git a/manual.texi b/manual.texi index 336776a..5bc27d5 100644 --- a/manual.texi +++ b/manual.texi @@ -2,10 +2,10 @@ @setfilename bzip2.info @ignore -This file documents bzip2 version 1.0, and associated library +This file documents bzip2 version 1.0.2, and associated library libbzip2, written by Julian Seward (jseward@acm.org). -Copyright (C) 1996-2000 Julian R Seward +Copyright (C) 1996-2002 Julian R Seward Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice @@ -30,8 +30,8 @@ END-INFO-DIR-ENTRY @titlepage @title bzip2 and libbzip2 @subtitle a program and library for data compression -@subtitle copyright (C) 1996-2000 Julian Seward -@subtitle version 1.0 of 21 March 2000 +@subtitle copyright (C) 1996-2002 Julian Seward +@subtitle version 1.0.2 of 30 December 2001 @author Julian Seward @end titlepage @@ -40,11 +40,17 @@ END-INFO-DIR-ENTRY @parskip 2mm @end iftex -@node Top, Overview, (dir), (dir) +@node Top,,, (dir) + +The following text is the License for this software. You should +find it identical to that contained in the file LICENSE in the +source distribution. + +@bf{------------------ START OF THE LICENSE ------------------} This program, @code{bzip2}, and associated library @code{libbzip2}, are -Copyright (C) 1996-2000 Julian R Seward. All rights reserved. +Copyright (C) 1996-2002 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -82,13 +88,15 @@ Julian Seward, Cambridge, UK. @code{jseward@@acm.org} -@code{http://sourceware.cygnus.com/bzip2} +@code{bzip2}/@code{libbzip2} version 1.0.2 of 30 December 2001. -@code{http://www.cacheprof.org} +@bf{------------------ END OF THE LICENSE ------------------} -@code{http://www.muraroa.demon.co.uk} +Web sites: -@code{bzip2}/@code{libbzip2} version 1.0 of 21 March 2000. +@code{http://sources.redhat.com/bzip2} + +@code{http://www.cacheprof.org} PATENTS: To the best of my knowledge, @code{bzip2} does not use any patented algorithms. However, I do not have the resources available to carry out @@ -101,7 +109,6 @@ above statement. -@node Overview, Implementation, Top, Top @chapter Introduction @code{bzip2} compresses files using the Burrows-Wheeler @@ -134,7 +141,7 @@ and nothing else. @unnumberedsubsubsec NAME @itemize @item @code{bzip2}, @code{bunzip2} -- a block-sorting file compressor, v1.0 +- a block-sorting file compressor, v1.0.2 @item @code{bzcat} - decompresses files to stdout @item @code{bzip2recover} @@ -264,6 +271,11 @@ This really performs a trial decompression and throws away the result. Force overwrite of output files. Normally, @code{bzip2} will not overwrite existing output files. Also forces @code{bzip2} to break hard links to files, which it otherwise wouldn't do. + +@code{bzip2} normally declines to decompress files which don't have the +correct magic header bytes. If forced (@code{-f}), however, it will +pass such files through unmodified. This is how GNU @code{gzip} +behaves. @item -k --keep Keep (don't delete) input files during compression or decompression. @@ -286,9 +298,13 @@ Further @code{-v}'s increase the verbosity level, spewing out lots of information which is primarily of interest for diagnostic purposes. @item -L --license -V --version Display the software version, license terms and conditions. -@item -1 to -9 +@item -1 (or --fast) to -9 (or --best) Set the block size to 100 k, 200 k .. 900 k when compressing. Has no effect when decompressing. See MEMORY MANAGEMENT below. +The @code{--fast} and @code{--best} aliases are primarily for GNU +@code{gzip} compatibility. In particular, @code{--fast} doesn't make +things significantly faster. And @code{--best} merely selects the +default behaviour. @item -- Treats all subsequent arguments as file names, even if they start with a dash. This is so you can handle files with names beginning @@ -389,21 +405,19 @@ integrity of the resulting files, and decompress those which are undamaged. @code{bzip2recover} -takes a single argument, the name of the damaged file, -and writes a number of files @code{rec0001file.bz2}, - @code{rec0002file.bz2}, etc, containing the extracted blocks. - The output filenames are designed so that the use of - wildcards in subsequent processing -- for example, -@code{bzip2 -dc rec*file.bz2 > recovered_data} -- lists the files in - the correct order. +takes a single argument, the name of the damaged file, and writes a +number of files @code{rec00001file.bz2}, @code{rec00002file.bz2}, etc, +containing the extracted blocks. The output filenames are designed so +that the use of wildcards in subsequent processing -- for example, +@code{bzip2 -dc rec*file.bz2 > recovered_data} -- processes the files in +the correct order. @code{bzip2recover} should be of most use dealing with large @code{.bz2} - files, as these will contain many blocks. It is clearly - futile to use it on damaged single-block files, since a - damaged block cannot be recovered. If you wish to minimise -any potential data loss through media or transmission errors, -you might consider compressing with a smaller - block size. +files, as these will contain many blocks. It is clearly futile to use +it on damaged single-block files, since a damaged block cannot be +recovered. If you wish to minimise any potential data loss through +media or transmission errors, you might consider compressing with a +smaller block size. @unnumberedsubsubsec PERFORMANCE NOTES @@ -435,22 +449,31 @@ I/O error messages are not as helpful as they could be. @code{bzip2} tries hard to detect I/O errors and exit cleanly, but the details of what the problem is sometimes seem rather misleading. -This manual page pertains to version 1.0 of @code{bzip2}. Compressed +This manual page pertains to version 1.0.2 of @code{bzip2}. Compressed data created by this version is entirely forwards and backwards -compatible with the previous public releases, versions 0.1pl2, 0.9.0 and -0.9.5, but with the following exception: 0.9.0 and above can correctly -decompress multiple concatenated compressed files. 0.1pl2 cannot do -this; it will stop after decompressing just the first file in the -stream. +compatible with the previous public releases, versions 0.1pl2, 0.9.0, +0.9.5, 1.0.0 and 1.0.1, but with the following exception: 0.9.0 and +above can correctly decompress multiple concatenated compressed files. +0.1pl2 cannot do this; it will stop after decompressing just the first +file in the stream. + +@code{bzip2recover} versions prior to this one, 1.0.2, used 32-bit +integers to represent bit positions in compressed files, so it could not +handle compressed files more than 512 megabytes long. Version 1.0.2 and +above uses 64-bit ints on some platforms which support them (GNU +supported targets, and Windows). To establish whether or not +@code{bzip2recover} was built with such a limitation, run it without +arguments. In any event you can build yourself an unlimited version if +you can recompile it with @code{MaybeUInt64} set to be an unsigned +64-bit integer. -@code{bzip2recover} uses 32-bit integers to represent bit positions in -compressed files, so it cannot handle compressed files more than 512 -megabytes long. This could easily be fixed. @unnumberedsubsubsec AUTHOR Julian Seward, @code{jseward@@acm.org}. +@code{http://sources.redhat.com/bzip2} + The ideas embodied in @code{bzip2} are due to (at least) the following people: Michael Burrows and David Wheeler (for the block sorting transformation), David Wheeler (again, for the Huffman coder), Peter @@ -461,8 +484,9 @@ indebted for their help, support and advice. See the manual in the source distribution for pointers to sources of documentation. Christian von Roques encouraged me to look for faster sorting algorithms, so as to speed up compression. Bela Lubkin encouraged me to improve the -worst-case compression performance. Many people sent patches, helped -with portability problems, lent machines, gave advice and were generally +worst-case compression performance. The @code{bz*} scripts are derived +from those of GNU @code{gzip}. Many people sent patches, helped with +portability problems, lent machines, gave advice and were generally helpful. @end quotation @@ -1769,16 +1793,20 @@ was compiled with @code{BZ_NO_STDIO} set. For a normal compile, an assertion failure yields the message @example bzip2/libbzip2: internal error number N. - This is a bug in bzip2/libbzip2, 1.0 of 21-Mar-2000. + This is a bug in bzip2/libbzip2, 1.0.2, 30-Dec-2001. Please report it to me at: jseward@@acm.org. If this happened when you were using some program which uses libbzip2 as a component, you should also report this bug to the author(s) of that program. Please make an effort to report this bug; timely and accurate bug reports eventually lead to higher - quality software. Thanks. Julian Seward, 21 March 2000. + quality software. Thanks. Julian Seward, 30 December 2001. @end example -where @code{N} is some error code number. @code{exit(3)} -is then called. +where @code{N} is some error code number. If @code{N == 1007}, it also +prints some extra text advising the reader that unreliable memory is +often associated with internal error 1007. (This is a +frequently-observed-phenomenon with versions 1.0.0/1.0.1). + +@code{exit(3)} is then called. For a @code{stdio}-free library, assertion failures result in a call to a function declared as: @@ -2056,10 +2084,10 @@ Maybe this isn't what you want. If you want a compressor and/or library which is faster, uses less memory but gets pretty good compression, and has minimal latency, consider Jean-loup -Gailly's and Mark Adler's work, @code{zlib-1.1.2} and +Gailly's and Mark Adler's work, @code{zlib-1.1.3} and @code{gzip-1.2.4}. Look for them at -@code{http://www.cdrom.com/pub/infozip/zlib} and +@code{http://www.zlib.org} and @code{http://www.gzip.org} respectively. For something faster and lighter still, you might try Markus F X J diff --git a/mk251.c b/mk251.c new file mode 100644 index 0000000..205778a --- /dev/null +++ b/mk251.c @@ -0,0 +1,16 @@ + +/* Spew out a long sequence of the byte 251. When fed to bzip2 + versions 1.0.0 or 1.0.1, causes it to die with internal error + 1007 in blocksort.c. This assertion misses an extremely rare + case, which is fixed in this version (1.0.2) and above. +*/ + +#include + +int main () +{ + int i; + for (i = 0; i < 48500000 ; i++) + putchar(251); + return 0; +} diff --git a/randtable.c b/randtable.c index 983089d..5c922e9 100644 --- a/randtable.c +++ b/randtable.c @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-2000 Julian R Seward. All rights reserved. + Copyright (C) 1996-2002 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions diff --git a/words3 b/words3 index 8486a84..7a6b462 100644 --- a/words3 +++ b/words3 @@ -15,8 +15,8 @@ not actually execute them. Instructions for use are in the preformatted manual page, in the file bzip2.txt. For more detailed documentation, read the full manual. -It is available in Postscript form (manual.ps) and HTML form -(manual_toc.html). +It is available in Postscript form (manual.ps), PDF form (manual.pdf), +and HTML form (manual_toc.html). You can also do "bzip2 --help" to see some helpful information. "bzip2 -L" displays the software license. -- cgit v1.2.3 From 4d540bfc95a4b0eefc1d1f388ec33534aaeb3a2f Mon Sep 17 00:00:00 2001 From: Julian Seward Date: Tue, 15 Feb 2005 22:13:13 +0100 Subject: bzip2-1.0.3 --- CHANGES | 22 + LICENSE | 7 +- Makefile | 51 +- Makefile-libbz2_so | 12 +- README | 40 +- README.COMPILATION.PROBLEMS | 103 +- README.XML.STUFF | 31 + blocksort.c | 4 +- bz-common.xsl | 39 + bz-fo.xsl | 257 ++++ bz-html.xsl | 20 + bzip.css | 74 ++ bzip2.1 | 17 +- bzip2.1.preformatted | 247 ++-- bzip2.c | 20 +- bzip2.txt | 119 +- bzip2recover.c | 12 +- bzlib.c | 59 +- bzlib.h | 6 +- bzlib_private.h | 6 +- compress.c | 16 +- crctable.c | 4 +- decompress.c | 20 +- entities.xml | 9 + format.pl | 53 + huffman.c | 23 +- manual.texi | 2243 -------------------------------- manual.xml | 2966 +++++++++++++++++++++++++++++++++++++++++++ randtable.c | 4 +- xmlproc.sh | 99 ++ 30 files changed, 3947 insertions(+), 2636 deletions(-) create mode 100644 README.XML.STUFF create mode 100644 bz-common.xsl create mode 100644 bz-fo.xsl create mode 100644 bz-html.xsl create mode 100644 bzip.css create mode 100644 entities.xml create mode 100755 format.pl delete mode 100644 manual.texi create mode 100644 manual.xml create mode 100755 xmlproc.sh diff --git a/CHANGES b/CHANGES index d984395..e31b03a 100644 --- a/CHANGES +++ b/CHANGES @@ -251,3 +251,25 @@ of bzip2: * added --fast and --best aliases for -1 -9 for gzip compatibility. + +1.0.3 (15 Feb 05) +~~~~~~~~~~~~~~~~~ +Fixes some minor bugs since the last version, 1.0.2. + +* Further robustification against corrupted compressed data. + There are currently no known bitstreams which can cause the + decompressor to crash, loop or access memory which does not + belong to it. If you are using bzip2 or the library to + decompress bitstreams from untrusted sources, an upgrade + to 1.0.3 is recommended. + +* The documentation has been converted to XML, from which html + and pdf can be derived. + +* Various minor bugs in the documentation have been fixed. + +* Fixes for various compilation warnings with newer versions of + gcc, and on 64-bit platforms. + +* The BZ_NO_STDIO cpp symbol was not properly observed in 1.0.2. + This has been fixed. diff --git a/LICENSE b/LICENSE index 9d4fa43..e60845b 100644 --- a/LICENSE +++ b/LICENSE @@ -1,6 +1,7 @@ -This program, "bzip2" and associated library "libbzip2", are -copyright (C) 1996-2002 Julian R Seward. All rights reserved. +This program, "bzip2", the associated library "libbzip2", and all +documentation, are copyright (C) 1996-2005 Julian R Seward. All +rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -35,5 +36,5 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. Julian Seward, Cambridge, UK. jseward@acm.org -bzip2/libbzip2 version 1.0.2 of 30 December 2001 +bzip2/libbzip2 version 1.0.3 of 15 February 2005 diff --git a/Makefile b/Makefile index 8305235..eea329a 100644 --- a/Makefile +++ b/Makefile @@ -7,9 +7,8 @@ AR=ar RANLIB=ranlib LDFLAGS= -# Suitably paranoid flags to avoid bugs in gcc-2.7 BIGFILES=-D_FILE_OFFSET_BITS=64 -CFLAGS=-Wall -Winline -O2 -fomit-frame-pointer -fno-strength-reduce $(BIGFILES) +CFLAGS=-Wall -Winline -O -g $(BIGFILES) # Where you want it installed when you do 'make install' PREFIX=/usr @@ -96,7 +95,6 @@ install: bzip2 bzip2recover echo ".so man1/bzmore.1" > $(PREFIX)/man/man1/bzless.1 echo ".so man1/bzdiff.1" > $(PREFIX)/man/man1/bzcmp.1 -distclean: clean clean: rm -f *.o libbz2.a bzip2 bzip2recover \ sample1.rb2 sample2.rb2 sample3.rb2 \ @@ -122,8 +120,12 @@ bzip2.o: bzip2.c bzip2recover.o: bzip2recover.c $(CC) $(CFLAGS) -c bzip2recover.c -DISTNAME=bzip2-1.0.2 -tarfile: + +distclean: clean + rm -f manual.ps manual.html manual.pdf + +DISTNAME=bzip2-1.0.3 +dist: check manual rm -f $(DISTNAME) ln -sf . $(DISTNAME) tar cvf $(DISTNAME).tar \ @@ -139,9 +141,6 @@ tarfile: $(DISTNAME)/bzlib.h \ $(DISTNAME)/bzlib_private.h \ $(DISTNAME)/Makefile \ - $(DISTNAME)/manual.texi \ - $(DISTNAME)/manual.ps \ - $(DISTNAME)/manual.pdf \ $(DISTNAME)/LICENSE \ $(DISTNAME)/bzip2.1 \ $(DISTNAME)/bzip2.1.preformatted \ @@ -157,9 +156,12 @@ tarfile: $(DISTNAME)/sample2.bz2 \ $(DISTNAME)/sample3.bz2 \ $(DISTNAME)/dlltest.c \ - $(DISTNAME)/*.html \ + $(DISTNAME)/manual.html \ + $(DISTNAME)/manual.pdf \ + $(DISTNAME)/manual.ps \ $(DISTNAME)/README \ $(DISTNAME)/README.COMPILATION.PROBLEMS \ + $(DISTNAME)/README.XML.STUFF \ $(DISTNAME)/CHANGES \ $(DISTNAME)/libbz2.def \ $(DISTNAME)/libbz2.dsp \ @@ -175,18 +177,29 @@ tarfile: $(DISTNAME)/bzmore.1 \ $(DISTNAME)/bzgrep \ $(DISTNAME)/bzgrep.1 \ - $(DISTNAME)/Makefile-libbz2_so + $(DISTNAME)/Makefile-libbz2_so \ + $(DISTNAME)/bz-common.xsl \ + $(DISTNAME)/bz-fo.xsl \ + $(DISTNAME)/bz-html.xsl \ + $(DISTNAME)/bzip.css \ + $(DISTNAME)/entities.xml \ + $(DISTNAME)/manual.xml \ + $(DISTNAME)/format.pl \ + $(DISTNAME)/xmlproc.sh gzip -v $(DISTNAME).tar -# For rebuilding the manual from sources on my RedHat 7.2 box -manual: manual.ps manual.pdf manual.html +# For rebuilding the manual from sources on my SuSE 9.1 box + +MANUAL_SRCS= bz-common.xsl bz-fo.xsl bz-html.xsl bzip.css \ + entities.xml manual.xml + +manual: manual.html manual.ps manual.pdf -manual.ps: manual.texi - tex manual.texi - dvips -o manual.ps manual.dvi +manual.ps: $(MANUAL_SRCS) + ./xmlproc.sh -ps manual.xml -manual.pdf: manual.ps - ps2pdf manual.ps +manual.pdf: $(MANUAL_SRCS) + ./xmlproc.sh -pdf manual.xml -manual.html: manual.texi - texi2html -split_chapter manual.texi +manual.html: $(MANUAL_SRCS) + ./xmlproc.sh -html manual.xml diff --git a/Makefile-libbz2_so b/Makefile-libbz2_so index 4986fe2..458c5a1 100644 --- a/Makefile-libbz2_so +++ b/Makefile-libbz2_so @@ -1,6 +1,6 @@ # This Makefile builds a shared version of the library, -# libbz2.so.1.0.2, with soname libbz2.so.1.0, +# libbz2.so.1.0.3, with soname libbz2.so.1.0, # at least on x86-Linux (RedHat 7.2), # with gcc-2.96 20000731 (Red Hat Linux 7.1 2.96-98). # Please see the README file for some @@ -9,7 +9,7 @@ SHELL=/bin/sh CC=gcc BIGFILES=-D_FILE_OFFSET_BITS=64 -CFLAGS=-fpic -fPIC -Wall -Winline -O2 -fomit-frame-pointer -fno-strength-reduce $(BIGFILES) +CFLAGS=-fpic -fPIC -Wall -Winline -O -g OBJS= blocksort.o \ huffman.o \ @@ -20,13 +20,13 @@ OBJS= blocksort.o \ bzlib.o all: $(OBJS) - $(CC) -shared -Wl,-soname -Wl,libbz2.so.1.0 -o libbz2.so.1.0.2 $(OBJS) - $(CC) $(CFLAGS) -o bzip2-shared bzip2.c libbz2.so.1.0.2 + $(CC) -shared -Wl,-soname -Wl,libbz2.so.1.0 -o libbz2.so.1.0.3 $(OBJS) + $(CC) $(CFLAGS) -o bzip2-shared bzip2.c libbz2.so.1.0.3 rm -f libbz2.so.1.0 - ln -s libbz2.so.1.0.2 libbz2.so.1.0 + ln -s libbz2.so.1.0.3 libbz2.so.1.0 clean: - rm -f $(OBJS) bzip2.o libbz2.so.1.0.2 libbz2.so.1.0 bzip2-shared + rm -f $(OBJS) bzip2.o libbz2.so.1.0.3 libbz2.so.1.0 bzip2-shared blocksort.o: blocksort.c $(CC) $(CFLAGS) -c blocksort.c diff --git a/README b/README index 07505d8..1aff448 100644 --- a/README +++ b/README @@ -1,15 +1,15 @@ This is the README for bzip2, a block-sorting file compressor, version -1.0.2. This version is fully compatible with the previous public -releases, versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0 and 1.0.1. +1.0.3. This version is fully compatible with the previous public +releases, versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1 and 1.0.2. -bzip2-1.0.2 is distributed under a BSD-style license. For details, +bzip2-1.0.3 is distributed under a BSD-style license. For details, see the file LICENSE. Complete documentation is available in Postscript form (manual.ps), -PDF (manual.pdf, amazingly enough) or html (manual_toc.html). A -plain-text version of the manual page is available as bzip2.txt. -A statement about Y2K issues is now included in the file Y2K_INFO. +PDF (manual.pdf) or html (manual.html). A plain-text version of the +manual page is available as bzip2.txt. A statement about Y2K issues +is now included in the file Y2K_INFO. HOW TO BUILD -- UNIX @@ -78,8 +78,7 @@ importance. To validate bzip2, I used a modified version of Mark Nelson's churn program. Churn is an automated test driver which recursively traverses a directory structure, using bzip2 to compress and then decompress each file it encounters, and checking that the -decompressed data is the same as the original. There are more details -in Section 4 of the user guide. +decompressed data is the same as the original. @@ -119,9 +118,9 @@ DISCLAIMER: PATENTS: To the best of my knowledge, bzip2 does not use any patented - algorithms. However, I do not have the resources available to - carry out a full patent search. Therefore I cannot give any - guarantee of the above statement. + algorithms. However, I do not have the resources to carry out + a patent search. Therefore I cannot give any guarantee of the + above statement. End of legalities. @@ -153,22 +152,26 @@ WHAT'S NEW IN 1.0.2 ? See the CHANGES file. +WHAT'S NEW IN 1.0.3 ? + + See the CHANGES file. + I hope you find bzip2 useful. Feel free to contact me at - jseward@acm.org + jseward@bzip.org if you have any suggestions or queries. Many people mailed me with comments, suggestions and patches after the releases of bzip-0.15, -bzip-0.21, and bzip2 versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0 and 1.0.1, -and the changes in bzip2 are largely a result of this feedback. +bzip-0.21, and bzip2 versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1 and +1.0.2, and the changes in bzip2 are largely a result of this feedback. I thank you for your comments. At least for the time being, bzip2's "home" is (or can be reached via) -http://sources.redhat.com/bzip2. +http://www.bzip.org Julian Seward -jseward@acm.org +jseward@bzip.org -Cambridge, UK (and what a great town this is!) +Cambridge, UK. 18 July 1996 (version 0.15) 25 August 1996 (version 0.21) @@ -178,4 +181,5 @@ Cambridge, UK (and what a great town this is!) 8 June 1999 (bzip2, version 0.9.5) 4 Sept 1999 (bzip2, version 0.9.5d) 5 May 2000 (bzip2, version 1.0pre8) -30 December 2001 (bzip2, version 1.0.2pre1) \ No newline at end of file +30 December 2001 (bzip2, version 1.0.2pre1) +15 February 2005 (bzip2, version 1.0.3) diff --git a/README.COMPILATION.PROBLEMS b/README.COMPILATION.PROBLEMS index bd1822d..f1bc396 100644 --- a/README.COMPILATION.PROBLEMS +++ b/README.COMPILATION.PROBLEMS @@ -1,11 +1,10 @@ -bzip2-1.0 should compile without problems on the vast majority of +bzip2-1.0.3 should compile without problems on the vast majority of platforms. Using the supplied Makefile, I've built and tested it -myself for x86-linux, sparc-solaris, alpha-linux, x86-cygwin32 and -alpha-tru64unix. With makefile.msc, Visual C++ 6.0 and nmake, you can -build a native Win32 version too. Large file support seems to work -correctly on at least alpha-tru64unix and x86-cygwin32 (on Windows -2000). +myself for x86-linux and x86_64-linux. With makefile.msc, Visual C++ +6.0 and nmake, you can build a native Win32 version too. Large file +support seems to work correctly on at least alpha-tru64unix and +x86-cygwin32 (on Windows 2000). When I say "large file" I mean a file of size 2,147,483,648 (2^31) bytes or above. Many older OSs can't handle files above this size, @@ -22,7 +21,7 @@ The technique of adding -D_FILE_OFFSET_BITS=64 to get large file support is, as far as I know, the Recommended Way to get correct large file support. For more details, see the Large File Support Specification, published by the Large File Summit, at - http://www.sas.com/standard/large.file/ + http://ftp.sas.com/standards/large.file As a general comment, if you get compilation errors which you think are related to large file support, try removing the above define from @@ -38,93 +37,3 @@ You can use the spewG.c program to generate huge files to test bzip2's large file support, if you are feeling paranoid. Be aware though that any compilation problems which affect bzip2 will also affect spewG.c, alas. - - -Known problems as of 1.0pre8: -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -* HP/UX 10.20 and 11.00, using gcc (2.7.2.3 and 2.95.2): A large - number of warnings appear, including the following: - - /usr/include/sys/resource.h: In function `getrlimit': - /usr/include/sys/resource.h:168: - warning: implicit declaration of function `__getrlimit64' - /usr/include/sys/resource.h: In function `setrlimit': - /usr/include/sys/resource.h:170: - warning: implicit declaration of function `__setrlimit64' - - This would appear to be a problem with large file support, header - files and gcc. gcc may or may not give up at this point. If it - fails, you might be able to improve matters by adding - -D__STDC_EXT__=1 - to the BIGFILES variable in the Makefile (ie, change its definition - to - BIGFILES=-D_FILE_OFFSET_BITS=64 -D__STDC_EXT__=1 - - Even if gcc does produce a binary which appears to work (ie passes - its self-tests), you might want to test it to see if it works properly - on large files. - - -* HP/UX 10.20 and 11.00, using HP's cc compiler. - - No specific problems for this combination, except that you'll need to - specify the -Ae flag, and zap the gcc-specific stuff - -Wall -Winline -O2 -fomit-frame-pointer -fno-strength-reduce. - You should retain -D_FILE_OFFSET_BITS=64 in order to get large - file support -- which is reported to work ok for this HP/UX + cc - combination. - - -* SunOS 4.1.X. - - Amazingly, there are still people out there using this venerable old - banger. I shouldn't be too rude -- I started life on SunOS, and - it was a pretty darn good OS, way back then. Anyway: - - SunOS doesn't seem to have strerror(), so you'll have to use - perror(), perhaps by doing adding this (warning: UNTESTED CODE): - - char* strerror ( int errnum ) - { - if (errnum < 0 || errnum >= sys_nerr) - return "Unknown error"; - else - return sys_errlist[errnum]; - } - - Or you could comment out the relevant calls to strerror; they're - not mission-critical. Or you could upgrade to Solaris. Ha ha ha! - (what?? you think I've got Bad Attitude?) - - -* Making a shared library on Solaris. (Not really a compilation - problem, but many people ask ...) - - Firstly, if you have Solaris 8, either you have libbz2.so already - on your system, or you can install it from the Solaris CD. - - Secondly, be aware that there are potential naming conflicts - between the .so file supplied with Solaris 8, and the .so file - which Makefile-libbz2_so will make. Makefile-libbz2_so creates - a .so which has the names which I intend to be "official" as - of version 1.0.0 and onwards. Unfortunately, the .so in - Solaris 8 appeared before I decided on the final names, so - the two libraries are incompatible. We have since communicated - and I hope that the problems will have been solved in the next - version of Solaris, whenever that might appear. - - All that said: you might be able to get somewhere - by finding the line in Makefile-libbz2_so which says - - $(CC) -shared -Wl,-soname -Wl,libbz2.so.1.0 -o libbz2.so.1.0.2 $(OBJS) - - and replacing with - - $(CC) -G -shared -o libbz2.so.1.0.2 -h libbz2.so.1.0 $(OBJS) - - If gcc objects to the combination -fpic -fPIC, get rid of - the second one, leaving just "-fpic". - - -That's the end of the currently known compilation problems. diff --git a/README.XML.STUFF b/README.XML.STUFF new file mode 100644 index 0000000..0ff209f --- /dev/null +++ b/README.XML.STUFF @@ -0,0 +1,31 @@ +The script xmlproc.sh takes an xml file as input, +and processes it to create .pdf, .html or .ps output. +It uses format.pl, a perl script to format 
 blocks nicely,
+ and add CDATA tags so writers do not have to use eg. < 
+
+The file "entities.xml" must be edited to reflect current
+version, year, etc.
+
+
+Usage:
+
+  xmlproc.sh -v manual.xml
+  Validates an xml file to ensure no dtd-compliance errors
+
+  xmlproc.sh -html manual.xml
+  Output: manual.html
+
+  xmlproc.sh -pdf manual.xml
+  Output: manual.pdf
+
+  xmlproc.sh -ps manual.xml
+  Output: manual.ps
+
+
+Notum bene: 
+- pdfxmltex barfs if given a filename with an underscore in it
+
+- xmltex won't work yet - there's a bug in passivetex
+    which we are all waiting for Sebastian to fix.
+  So we are going the xml -> pdf -> ps route for the time being,
+    using pdfxmltex.
diff --git a/blocksort.c b/blocksort.c
index aba3efc..33ec9f5 100644
--- a/blocksort.c
+++ b/blocksort.c
@@ -8,7 +8,7 @@
   This file is a part of bzip2 and/or libbzip2, a program and
   library for lossless, block-sorting data compression.
 
-  Copyright (C) 1996-2002 Julian R Seward.  All rights reserved.
+  Copyright (C) 1996-2005 Julian R Seward.  All rights reserved.
 
   Redistribution and use in source and binary forms, with or without
   modification, are permitted provided that the following conditions
@@ -42,7 +42,7 @@
   SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
   Julian Seward, Cambridge, UK.
-  jseward@acm.org
+  jseward@bzip.org
   bzip2/libbzip2 version 1.0 of 21 March 2000
 
   This program is based on (at least) the work of:
diff --git a/bz-common.xsl b/bz-common.xsl
new file mode 100644
index 0000000..66fcd6f
--- /dev/null
+++ b/bz-common.xsl
@@ -0,0 +1,39 @@
+ 
+
+
+
+ 
+
+
+
+ 
+ 
+   
+    
+      
+     
+  
+
+
+
+
+set       toc,title
+book      toc,title,figure,table,example,equation
+chapter   toc,title
+section   toc
+sect1     toc
+sect2     toc
+sect3     toc
+sect4     nop
+sect5     nop
+qandaset  toc
+qandadiv  nop
+appendix  toc,title
+article/appendix  nop
+article   toc,title
+preface   toc,title
+reference toc,title
+
+
+
diff --git a/bz-fo.xsl b/bz-fo.xsl
new file mode 100644
index 0000000..7f2a767
--- /dev/null
+++ b/bz-fo.xsl
@@ -0,0 +1,257 @@
+ 
+
+
+
+
+
+
+
+
+
+
+
+
+      
+     
+   
+
+
+
+
+ 
+
+
+
+
+
+
+  
+
+
+
+
+  blue
+
+
+
+
+  
+    
+  
+
+
+
+  
+    
+  
+
+
+
+
+  
+  
+  
+    
+      
+    
+  
+  
+    
+      
+        
+          •
+          •
+          •
+        
+      
+    
+    
+          
+    
+  
+  
+    
+      
+        
+      
+    
+    
+      
+        
+      
+    
+  
+
+
+
+
+  
+  
+  
+    
+      
+        
+      
+    
+    
+          
+    
+  
+  
+    
+      
+        
+      
+    
+    
+      
+        
+      
+    
+  
+
+
+
+
+
+  
+    
+  
+    
+  
+  
+    
+      
+    
+  
+
+
+
+
+
+  
+  
+  
+  
+    
+      0pt
+    
+  
+  
+    
+      
+      
+      
+        
+          
+            baseline
+             
+               
+            
+          
+          
+            baseline
+            
+              
+                
+                
+                
+                
+              
+            
+          
+        
+      
+    
+  
+  
+  
+    
+      
+    
+    
+      
+    
+    
+      
+    
+  
+
+
+
+
+
+  
+  
+  
+  
+    
+      0pt
+    
+  
+  
+    
+      
+        
+        
+        
+      
+      
+      
+      
+        
+          
+            baseline
+            
+               
+            
+          
+          
+            baseline
+            
+              
+                
+                
+                
+                
+              
+            
+          
+        
+      
+    
+  
+  
+  
+    
+      
+    
+    
+      
+    
+    
+      
+    
+  
+
+
+
+
diff --git a/bz-html.xsl b/bz-html.xsl
new file mode 100644
index 0000000..1785fff
--- /dev/null
+++ b/bz-html.xsl
@@ -0,0 +1,20 @@
+ 
+ ]>
+
+
+
+
+
+
+
+
+
+
+  
+  
+
+
+
diff --git a/bzip.css b/bzip.css
new file mode 100644
index 0000000..43193d8
--- /dev/null
+++ b/bzip.css
@@ -0,0 +1,74 @@
+/* Colours:
+#74240f  dark brown      h1, h2, h3, h4
+#336699  medium blue     links
+#339999  turquoise       link hover colour
+#202020  almost black    general text
+#761596  purple          md5sum text
+#626262  dark gray       pre border
+#eeeeee  very light gray pre background
+#f2f2f9  very light blue nav table background
+#3366cc  medium blue     nav table border
+*/
+
+a, a:link, a:visited, a:active { color: #336699; }
+a:hover { color: #339999; }
+
+body { font: 80%/126% sans-serif; }
+h1, h2, h3, h4 { color: #74240f; }
+
+dt { color: #336699; font-weight: bold }
+dd { 
+ margin-left: 1.5em; 
+ padding-bottom: 0.8em;
+}
+
+/* -- ruler -- */
+div.hr_blue { 
+  height:  3px; 
+  background:#ffffff url("/images/hr_blue.png") repeat-x; }
+div.hr_blue hr { display:none; }
+
+/* release styles */
+#release p { margin-top: 0.4em; }
+#release .md5sum { color: #761596; }
+
+
+/* ------ styles for docs|manuals|howto ------ */
+/* -- lists -- */
+ul  { 
+ margin:     0px 4px 16px 16px;
+ padding:    0px;
+ list-style: url("/images/li-blue.png"); 
+}
+ul li { 
+ margin-bottom: 10px;
+}
+ul ul	{ 
+ list-style-type:  none; 
+ list-style-image: none; 
+ margin-left:      0px; 
+}
+
+/* header / footer nav tables */
+table.nav {
+ border:     solid 1px #3366cc;
+ background: #f2f2f9;
+ background-color: #f2f2f9;
+ margin-bottom: 0.5em;
+}
+/* don't have underlined links in chunked nav menus */
+table.nav a { text-decoration: none; }
+table.nav a:hover { text-decoration: underline; }
+table.nav td { font-size: 85%; }
+
+code, tt, pre { font-size: 120%; }
+code, tt { color: #761596; }
+
+div.literallayout, pre.programlisting, pre.screen {
+ color:      #000000;
+ padding:    0.5em;
+ background: #eeeeee;
+ border:     1px solid #626262;
+ background-color: #eeeeee;
+ margin: 4px 0px 4px 0px; 
+}
diff --git a/bzip2.1 b/bzip2.1
index 623435c..d2c0661 100644
--- a/bzip2.1
+++ b/bzip2.1
@@ -1,7 +1,7 @@
 .PU
 .TH bzip2 1
 .SH NAME
-bzip2, bunzip2 \- a block-sorting file compressor, v1.0.2
+bzip2, bunzip2 \- a block-sorting file compressor, v1.0.3
 .br
 bzcat \- decompresses files to stdout
 .br
@@ -405,19 +405,19 @@ I/O error messages are not as helpful as they could be.
 tries hard to detect I/O errors and exit cleanly, but the details of
 what the problem is sometimes seem rather misleading.
 
-This manual page pertains to version 1.0.2 of
+This manual page pertains to version 1.0.3 of
 .I bzip2.  
 Compressed data created by this version is entirely forwards and
 backwards compatible with the previous public releases, versions
-0.1pl2, 0.9.0, 0.9.5, 1.0.0 and 1.0.1, but with the following
+0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1 and 1.0.2, but with the following
 exception: 0.9.0 and above can correctly decompress multiple
 concatenated compressed files.  0.1pl2 cannot do this; it will stop
 after decompressing just the first file in the stream.
 
 .I bzip2recover
-versions prior to this one, 1.0.2, used 32-bit integers to represent
-bit positions in compressed files, so it could not handle compressed
-files more than 512 megabytes long.  Version 1.0.2 and above uses
+versions prior to 1.0.2 used 32-bit integers to represent
+bit positions in compressed files, so they could not handle compressed
+files more than 512 megabytes long.  Versions 1.0.2 and above use
 64-bit ints on some platforms which support them (GNU supported
 targets, and Windows).  To establish whether or not bzip2recover was
 built with such a limitation, run it without arguments.  In any event
@@ -427,9 +427,9 @@ with MaybeUInt64 set to be an unsigned 64-bit integer.
 
 
 .SH AUTHOR
-Julian Seward, jseward@acm.org.
+Julian Seward, jsewardbzip.org.
 
-http://sources.redhat.com/bzip2
+http://www.bzip.org
 
 The ideas embodied in
 .I bzip2
@@ -447,6 +447,7 @@ source distribution for pointers to sources of documentation.  Christian
 von Roques encouraged me to look for faster sorting algorithms, so as to
 speed up compression.  Bela Lubkin encouraged me to improve the
 worst-case compression performance.  
+Donna Robinson XMLised the documentation.
 The bz* scripts are derived from those of GNU gzip.
 Many people sent patches, helped
 with portability problems, lent machines, gave advice and were generally
diff --git a/bzip2.1.preformatted b/bzip2.1.preformatted
index 0f20cb5..129ca83 100644
--- a/bzip2.1.preformatted
+++ b/bzip2.1.preformatted
@@ -3,43 +3,43 @@ bzip2(1)                                                 bzip2(1)
 
 
 NNAAMMEE
-       bzip2, bunzip2 - a block-sorting file compressor, v1.0.2
-       bzcat - decompresses files to stdout
-       bzip2recover - recovers data from damaged bzip2 files
+       bzip2, bunzip2 − a block‐sorting file compressor, v1.0.3
+       bzcat − decompresses files to stdout
+       bzip2recover − recovers data from damaged bzip2 files
 
 
 SSYYNNOOPPSSIISS
-       bbzziipp22 [ --ccddffkkqqssttvvzzVVLL112233445566778899 ] [ _f_i_l_e_n_a_m_e_s _._._.  ]
-       bbuunnzziipp22 [ --ffkkvvssVVLL ] [ _f_i_l_e_n_a_m_e_s _._._.  ]
-       bbzzccaatt [ --ss ] [ _f_i_l_e_n_a_m_e_s _._._.  ]
+       bbzziipp22 [ −−ccddffkkqqssttvvzzVVLL112233445566778899 ] [ _f_i_l_e_n_a_m_e_s _._._.  ]
+       bbuunnzziipp22 [ −−ffkkvvssVVLL ] [ _f_i_l_e_n_a_m_e_s _._._.  ]
+       bbzzccaatt [ −−ss ] [ _f_i_l_e_n_a_m_e_s _._._.  ]
        bbzziipp22rreeccoovveerr _f_i_l_e_n_a_m_e
 
 
 DDEESSCCRRIIPPTTIIOONN
-       _b_z_i_p_2  compresses  files  using  the Burrows-Wheeler block
+       _b_z_i_p_2  compresses  files  using  the Burrows‐Wheeler block
        sorting text compression algorithm,  and  Huffman  coding.
        Compression  is  generally  considerably  better than that
-       achieved by more conventional LZ77/LZ78-based compressors,
-       and  approaches  the performance of the PPM family of sta
+       achieved by more conventional LZ77/LZ78‐based compressors,
+       and  approaches  the performance of the PPM family of sta­
        tistical compressors.
 
-       The command-line options are deliberately very similar  to
+       The command‐line options are deliberately very similar  to
        those of _G_N_U _g_z_i_p_, but they are not identical.
 
-       _b_z_i_p_2  expects  a list of file names to accompany the com
-       mand-line flags.  Each file is replaced  by  a  compressed
+       _b_z_i_p_2  expects  a list of file names to accompany the com­
+       mand‐line flags.  Each file is replaced  by  a  compressed
        version  of  itself,  with  the  name "original_name.bz2".
-       Each compressed file has the same modification date,  per
-       missions, and, when possible, ownership as the correspond
+       Each compressed file has the same modification date,  per­
+       missions, and, when possible, ownership as the correspond­
        ing original, so that these properties  can  be  correctly
        restored  at  decompression  time.   File name handling is
-       naive in the sense that there is no mechanism for preserv
+       naive in the sense that there is no mechanism for preserv­
        ing  original file names, permissions, ownerships or dates
        in filesystems which lack these concepts, or have  serious
-       file name length restrictions, such as MS-DOS.
+       file name length restrictions, such as MS‐DOS.
 
        _b_z_i_p_2  and  _b_u_n_z_i_p_2 will by default not overwrite existing
-       files.  If you want this to happen, specify the -f flag.
+       files.  If you want this to happen, specify the −f flag.
 
        If no file names  are  specified,  _b_z_i_p_2  compresses  from
        standard  input  to  standard output.  In this case, _b_z_i_p_2
@@ -47,7 +47,7 @@ DDEESSCCRRIIPPTTIIOONN
        this  would  be  entirely  incomprehensible  and therefore
        pointless.
 
-       _b_u_n_z_i_p_2 (or _b_z_i_p_2 _-_d_) decompresses  all  specified  files.
+       _b_u_n_z_i_p_2 (or _b_z_i_p_2 _−_d_) decompresses  all  specified  files.
        Files which were not created by _b_z_i_p_2 will be detected and
        ignored, and a warning issued.  _b_z_i_p_2  attempts  to  guess
        the  filename  for  the decompressed file from that of the
@@ -64,26 +64,26 @@ DDEESSCCRRIIPPTTIIOONN
        guess the name of the original file, and uses the original
        name with _._o_u_t appended.
 
-       As  with compression, supplying no filenames causes decom
+       As  with compression, supplying no filenames causes decom­
        pression from standard input to standard output.
 
-       _b_u_n_z_i_p_2 will correctly decompress a file which is the con
+       _b_u_n_z_i_p_2 will correctly decompress a file which is the con­
        catenation of two or more compressed files.  The result is
        the concatenation of the corresponding uncompressed files.
-       Integrity testing (-t) of concatenated compressed files is
+       Integrity testing (−t) of concatenated compressed files is
        also supported.
 
        You can also compress or decompress files to the  standard
-       output  by giving the -c flag.  Multiple files may be com
+       output  by giving the −c flag.  Multiple files may be com­
        pressed and decompressed like this.  The resulting outputs
        are  fed  sequentially to stdout.  Compression of multiple
-       files in this manner generates a stream containing  multi
+       files in this manner generates a stream containing  multi­
        ple compressed file representations.  Such a stream can be
        decompressed correctly only  by  _b_z_i_p_2  version  0.9.0  or
-       later.   Earlier  versions of _b_z_i_p_2 will stop after decom
+       later.   Earlier  versions of _b_z_i_p_2 will stop after decom­
        pressing the first file in the stream.
 
-       _b_z_c_a_t (or _b_z_i_p_2 _-_d_c_) decompresses all specified  files  to
+       _b_z_c_a_t (or _b_z_i_p_2 _‐_d_c_) decompresses all specified  files  to
        the standard output.
 
        _b_z_i_p_2  will  read arguments from the environment variables
@@ -99,15 +99,15 @@ DDEESSCCRRIIPPTTIIOONN
        most  file  compressors)  is  coded at about 8.05 bits per
        byte, giving an expansion of around 0.5%.
 
-       As a self-check for your  protection,  _b_z_i_p_2  uses  32-bit
+       As a self‐check for your  protection,  _b_z_i_p_2  uses  32‐bit
        CRCs  to make sure that the decompressed version of a file
-       is identical to the original.  This guards against corrup
+       is identical to the original.  This guards against corrup­
        tion  of  the compressed data, and against undetected bugs
        in _b_z_i_p_2 (hopefully very unlikely).  The chances  of  data
        corruption  going  undetected  is  microscopic,  about one
        chance in four billion for each file processed.  Be aware,
        though,  that  the  check occurs upon decompression, so it
-       can only tell you that something is wrong.  It can't  help
+       can only tell you that something is wrong.  It can’t  help
        you  recover  the original uncompressed data.  You can use
        _b_z_i_p_2_r_e_c_o_v_e_r to try to recover data from damaged files.
 
@@ -118,41 +118,41 @@ DDEESSCCRRIIPPTTIIOONN
 
 
 OOPPTTIIOONNSS
-       --cc ----ssttddoouutt
+       −−cc ‐‐‐‐ssttddoouutt
               Compress or decompress to standard output.
 
-       --dd ----ddeeccoommpprreessss
+       −−dd ‐‐‐‐ddeeccoommpprreessss
               Force  decompression.  _b_z_i_p_2_, _b_u_n_z_i_p_2 and _b_z_c_a_t are
               really the same program,  and  the  decision  about
               what  actions to take is done on the basis of which
               name is used.  This flag overrides that  mechanism,
               and forces _b_z_i_p_2 to decompress.
 
-       --zz ----ccoommpprreessss
-              The   complement   to   -d:   forces   compression,
+       −−zz ‐‐‐‐ccoommpprreessss
+              The   complement   to   −d:   forces   compression,
               regardless of the invocation name.
 
-       --tt ----tteesstt
-              Check integrity of the specified file(s), but don't
+       −−tt ‐‐‐‐tteesstt
+              Check integrity of the specified file(s), but don’t
               decompress  them.   This  really  performs  a trial
               decompression and throws away the result.
 
-       --ff ----ffoorrccee
+       −−ff ‐‐‐‐ffoorrccee
               Force overwrite of output files.   Normally,  _b_z_i_p_2
               will  not  overwrite  existing  output files.  Also
               forces _b_z_i_p_2 to break hard links to files, which it
-              otherwise wouldn't do.
+              otherwise wouldn’t do.
 
               bzip2  normally  declines to decompress files which
-              don't have the  correct  magic  header  bytes.   If
-              forced  (-f),  however,  it  will  pass  such files
+              don’t have the  correct  magic  header  bytes.   If
+              forced  (‐f),  however,  it  will  pass  such files
               through unmodified.  This is how GNU gzip  behaves.
 
-       --kk ----kkeeeepp
-              Keep  (don't delete) input files during compression
+       −−kk ‐‐‐‐kkeeeepp
+              Keep  (don’t delete) input files during compression
               or decompression.
 
-       --ss ----ssmmaallll
+       −−ss ‐‐‐‐ssmmaallll
               Reduce memory usage, for compression, decompression
               and  testing.   Files  are  decompressed and tested
               using a modified algorithm which only requires  2.5
@@ -160,46 +160,46 @@ OOPPTTIIOONNSS
               decompressed in 2300k of memory,  albeit  at  about
               half the normal speed.
 
-              During  compression,  -s  selects  a  block size of
+              During  compression,  −s  selects  a  block size of
               200k, which limits memory use to  around  the  same
               figure,  at  the expense of your compression ratio.
               In short, if your  machine  is  low  on  memory  (8
-              megabytes  or  less),  use  -s for everything.  See
+              megabytes  or  less),  use  −s for everything.  See
               MEMORY MANAGEMENT below.
 
-       --qq ----qquuiieett
-              Suppress non-essential warning messages.   Messages
+       −−qq ‐‐‐‐qquuiieett
+              Suppress non‐essential warning messages.   Messages
               pertaining  to I/O errors and other critical events
               will not be suppressed.
 
-       --vv ----vveerrbboossee
-              Verbose mode -- show the compression ratio for each
-              file  processed.   Further  -v's  increase the ver
+       −−vv ‐‐‐‐vveerrbboossee
+              Verbose mode ‐‐ show the compression ratio for each
+              file  processed.   Further  −v’s  increase the ver­
               bosity level, spewing out lots of information which
               is primarily of interest for diagnostic purposes.
 
-       --LL ----lliicceennssee --VV ----vveerrssiioonn
+       −−LL ‐‐‐‐lliicceennssee ‐‐VV ‐‐‐‐vveerrssiioonn
               Display  the  software  version,  license terms and
               conditions.
 
-       --11 ((oorr ----ffaasstt)) ttoo --99 ((oorr ----bbeesstt))
+       −−11 ((oorr −−−−ffaasstt)) ttoo −−99 ((oorr −−−−bbeesstt))
               Set the block size to 100 k, 200 k ..  900  k  when
               compressing.   Has  no  effect  when decompressing.
-              See MEMORY MANAGEMENT below.  The --fast and --best
+              See MEMORY MANAGEMENT below.  The −−fast and −−best
               aliases  are  primarily for GNU gzip compatibility.
-              In particular, --fast doesn't make things  signifi
-              cantly  faster.   And  --best  merely  selects  the
+              In particular, −−fast doesn’t make things  signifi­
+              cantly  faster.   And  −−best  merely  selects  the
               default behaviour.
 
-       ----     Treats all subsequent arguments as file names, even
-              if they start with a dash.  This is so you can han
+       −−‐‐     Treats all subsequent arguments as file names, even
+              if they start with a dash.  This is so you can han­
               dle files with names beginning  with  a  dash,  for
-              example: bzip2 -- -myfilename.
+              example: bzip2 −‐ −myfilename.
 
-       ----rreeppeettiittiivvee--ffaasstt ----rreeppeettiittiivvee--bbeesstt
+       −−‐‐rreeppeettiittiivvee‐‐ffaasstt ‐‐‐‐rreeppeettiittiivvee‐‐bbeesstt
               These  flags  are  redundant  in versions 0.9.5 and
               above.  They provided some coarse control over  the
-              behaviour  of the sorting algorithm in earlier ver
+              behaviour  of the sorting algorithm in earlier ver­
               sions, which was sometimes useful.  0.9.5 and above
               have  an  improved  algorithm  which  renders these
               flags irrelevant.
@@ -209,13 +209,13 @@ MMEEMMOORRYY MMAANNAAGGEEMMEENNTT
        _b_z_i_p_2 compresses large files in blocks.   The  block  size
        affects  both  the  compression  ratio  achieved,  and the
        amount of memory needed for compression and decompression.
-       The  flags  -1  through  -9  specify  the block size to be
-       100,000 bytes through 900,000 bytes (the default)  respec
+       The  flags  −1  through  −9  specify  the block size to be
+       100,000 bytes through 900,000 bytes (the default)  respec­
        tively.   At  decompression  time, the block size used for
        compression is read from  the  header  of  the  compressed
        file, and _b_u_n_z_i_p_2 then allocates itself just enough memory
        to decompress the file.  Since block sizes are  stored  in
-       compressed  files,  it follows that the flags -1 to -9 are
+       compressed  files,  it follows that the flags −1 to −9 are
        irrelevant to and so ignored during decompression.
 
        Compression and decompression requirements, in bytes,  can
@@ -238,21 +238,21 @@ MMEEMMOORRYY MMAANNAAGGEEMMEENNTT
        _b_u_n_z_i_p_2 will require about 3700 kbytes to decompress.   To
        support decompression of any file on a 4 megabyte machine,
        _b_u_n_z_i_p_2 has an option to  decompress  using  approximately
-       half this amount of memory, about 2300 kbytes.  Decompres
+       half this amount of memory, about 2300 kbytes.  Decompres­
        sion speed is also halved, so you should use  this  option
-       only where necessary.  The relevant flag is -s.
+       only where necessary.  The relevant flag is ‐s.
 
-       In general, try and use the largest block size memory con
+       In general, try and use the largest block size memory con­
        straints  allow,  since  that  maximises  the  compression
-       achieved.   Compression and decompression speed are virtu
+       achieved.   Compression and decompression speed are virtu­
        ally unaffected by block size.
 
        Another significant point applies to files which fit in  a
-       single  block  --  that  means  most files you'd encounter
+       single  block  ‐‐  that  means  most files you’d encounter
        using a large block  size.   The  amount  of  real  memory
        touched is proportional to the size of the file, since the
        file is smaller than a block.  For example, compressing  a
-       file  20,000  bytes  long  with the flag -9 will cause the
+       file  20,000  bytes  long  with the flag ‐9 will cause the
        compressor to allocate around 7600k of  memory,  but  only
        touch 400k + 20000 * 8 = 560 kbytes of it.  Similarly, the
        decompressor will allocate 3700k but  only  touch  100k  +
@@ -260,59 +260,59 @@ MMEEMMOORRYY MMAANNAAGGEEMMEENNTT
 
        Here  is a table which summarises the maximum memory usage
        for different block sizes.  Also  recorded  is  the  total
-       compressed  size for 14 files of the Calgary Text Compres
+       compressed  size for 14 files of the Calgary Text Compres­
        sion Corpus totalling 3,141,622 bytes.  This column  gives
        some  feel  for  how  compression  varies with block size.
        These figures tend to understate the advantage  of  larger
-       block  sizes  for  larger files, since the Corpus is domi
+       block  sizes  for  larger files, since the Corpus is domi­
        nated by smaller files.
 
                   Compress   Decompress   Decompress   Corpus
-           Flag     usage      usage       -s usage     Size
+           Flag     usage      usage       ‐s usage     Size
 
-            -1      1200k       500k         350k      914704
-            -2      2000k       900k         600k      877703
-            -3      2800k      1300k         850k      860338
-            -4      3600k      1700k        1100k      846899
-            -5      4400k      2100k        1350k      845160
-            -6      5200k      2500k        1600k      838626
-            -7      6100k      2900k        1850k      834096
-            -8      6800k      3300k        2100k      828642
-            -9      7600k      3700k        2350k      828642
+            ‐1      1200k       500k         350k      914704
+            ‐2      2000k       900k         600k      877703
+            ‐3      2800k      1300k         850k      860338
+            ‐4      3600k      1700k        1100k      846899
+            ‐5      4400k      2100k        1350k      845160
+            ‐6      5200k      2500k        1600k      838626
+            ‐7      6100k      2900k        1850k      834096
+            ‐8      6800k      3300k        2100k      828642
+            ‐9      7600k      3700k        2350k      828642
 
 
 RREECCOOVVEERRIINNGG DDAATTAA FFRROOMM DDAAMMAAGGEEDD FFIILLEESS
        _b_z_i_p_2 compresses files in blocks, usually 900kbytes  long.
-       Each block is handled independently.  If a media or trans
-       mission error causes a multi-block  .bz2  file  to  become
+       Each block is handled independently.  If a media or trans­
+       mission error causes a multi‐block  .bz2  file  to  become
        damaged,  it  may  be  possible  to  recover data from the
        undamaged blocks in the file.
 
        The compressed representation of each block  is  delimited
-       by  a  48-bit pattern, which makes it possible to find the
+       by  a  48‐bit pattern, which makes it possible to find the
        block boundaries with reasonable  certainty.   Each  block
-       also  carries its own 32-bit CRC, so damaged blocks can be
+       also  carries its own 32‐bit CRC, so damaged blocks can be
        distinguished from undamaged ones.
 
        _b_z_i_p_2_r_e_c_o_v_e_r is a  simple  program  whose  purpose  is  to
        search  for blocks in .bz2 files, and write each block out
-       into its own .bz2 file.  You can then use _b_z_i_p_2 -t to test
+       into its own .bz2 file.  You can then use _b_z_i_p_2 −t to test
        the integrity of the resulting files, and decompress those
        which are undamaged.
 
-       _b_z_i_p_2_r_e_c_o_v_e_r takes a single argument, the name of the dam
+       _b_z_i_p_2_r_e_c_o_v_e_r takes a single argument, the name of the dam­
        aged    file,    and    writes    a    number   of   files
        "rec00001file.bz2",  "rec00002file.bz2",  etc,  containing
        the   extracted   blocks.   The   output   filenames   are
-       designed  so  that the use of wildcards in subsequent pro
-       cessing  -- for example, "bzip2 -dc  rec*file.bz2 > recov
-       ered_data" -- processes the files in the correct order.
+       designed  so  that the use of wildcards in subsequent pro­
+       cessing  ‐‐ for example, "bzip2 ‐dc  rec*file.bz2 > recov­
+       ered_data" ‐‐ processes the files in the correct order.
 
        _b_z_i_p_2_r_e_c_o_v_e_r should be of most use dealing with large .bz2
        files,  as  these will contain many blocks.  It is clearly
-       futile to use it on damaged single-block  files,  since  a
-       damaged  block  cannot  be recovered.  If you wish to min
-       imise any potential data loss through media  or  transmis
+       futile to use it on damaged single‐block  files,  since  a
+       damaged  block  cannot  be recovered.  If you wish to min­
+       imise any potential data loss through media  or  transmis­
        sion errors, you might consider compressing with a smaller
        block size.
 
@@ -324,21 +324,21 @@ PPEERRFFOORRMMAANNCCEE NNOOTTEESS
        ..."   (repeated  several hundred times) may compress more
        slowly than normal.  Versions 0.9.5 and  above  fare  much
        better  than previous versions in this respect.  The ratio
-       between worst-case and average-case compression time is in
+       between worst‐case and average‐case compression time is in
        the  region  of  10:1.  For previous versions, this figure
-       was more like 100:1.  You can use the -vvvv option to mon
+       was more like 100:1.  You can use the −vvvv option to mon­
        itor progress in great detail, if you want.
 
        Decompression speed is unaffected by these phenomena.
 
        _b_z_i_p_2  usually  allocates  several  megabytes of memory to
-       operate in, and then charges all over it in a fairly  ran
-       dom  fashion.   This means that performance, both for com
+       operate in, and then charges all over it in a fairly  ran­
+       dom  fashion.   This means that performance, both for com­
        pressing and decompressing, is largely determined  by  the
        speed  at  which  your  machine  can service cache misses.
        Because of this, small changes to the code to  reduce  the
        miss  rate  have  been observed to give disproportionately
-       large performance improvements.  I imagine _b_z_i_p_2 will per
+       large performance improvements.  I imagine _b_z_i_p_2 will per­
        form best on machines with very large caches.
 
 
@@ -348,50 +348,51 @@ CCAAVVEEAATTSS
        but  the  details  of  what  the problem is sometimes seem
        rather misleading.
 
-       This manual page pertains to version 1.0.2 of _b_z_i_p_2_.  Com
+       This manual page pertains to version 1.0.3 of _b_z_i_p_2_.  Com­
        pressed  data created by this version is entirely forwards
        and  backwards  compatible  with   the   previous   public
-       releases,  versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0 and 1.0.1,
-       but with the following exception: 0.9.0 and above can cor
-       rectly  decompress multiple concatenated compressed files.
-       0.1pl2 cannot do this; it will  stop  after  decompressing
-       just the first file in the stream.
+       releases,  versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1 and
+       1.0.2, but with the following exception: 0.9.0  and  above
+       can  correctly decompress multiple concatenated compressed
+       files.  0.1pl2 cannot do this; it will stop  after  decom­
+       pressing just the first file in the stream.
 
-       _b_z_i_p_2_r_e_c_o_v_e_r  versions  prior  to  this  one,  1.0.2, used
-       32-bit integers to represent bit positions  in  compressed
-       files,  so  it could not handle compressed files more than
-       512 megabytes long.  Version 1.0.2 and above  uses  64-bit
-       ints  on  some platforms which support them (GNU supported
-       targets,  and  Windows).   To  establish  whether  or  not
-       bzip2recover  was  built  with  such  a limitation, run it
-       without arguments.  In any event you can build yourself an
-       unlimited version if you can recompile it with MaybeUInt64
-       set to be an unsigned 64-bit integer.
+       _b_z_i_p_2_r_e_c_o_v_e_r  versions prior to 1.0.2 used 32‐bit integers
+       to represent bit positions in compressed  files,  so  they
+       could  not handle compressed files more than 512 megabytes
+       long.  Versions 1.0.2 and above use 64‐bit  ints  on  some
+       platforms  which  support them (GNU supported targets, and
+       Windows).  To establish whether or  not  bzip2recover  was
+       built  with  such  a limitation, run it without arguments.
+       In any event you can build yourself an  unlimited  version
+       if  you  can  recompile  it  with MaybeUInt64 set to be an
+       unsigned 64‐bit integer.
 
 
 
 
 AAUUTTHHOORR
-       Julian Seward, jseward@acm.org.
+       Julian Seward, jsewardbzip.org.
 
-       http://sources.redhat.com/bzip2
+       http://www.bzip.org
 
-       The ideas embodied in _b_z_i_p_2 are due to (at least) the fol
+       The ideas embodied in _b_z_i_p_2 are due to (at least) the fol­
        lowing  people: Michael Burrows and David Wheeler (for the
        block sorting transformation), David Wheeler  (again,  for
-       the Huffman coder), Peter Fenwick (for the structured cod
+       the Huffman coder), Peter Fenwick (for the structured cod­
        ing model in the original _b_z_i_p_, and many refinements), and
        Alistair  Moffat,  Radford  Neal  and  Ian Witten (for the
        arithmetic  coder  in  the  original  _b_z_i_p_)_.   I  am  much
-       indebted for their help, support and advice.  See the man
+       indebted for their help, support and advice.  See the man­
        ual in the source distribution for pointers to sources  of
        documentation.  Christian von Roques encouraged me to look
-       for faster sorting algorithms, so as to speed up  compres
-       sion.  Bela Lubkin encouraged me to improve the worst-case
-       compression performance.  The bz* scripts are derived from
-       those  of GNU gzip.  Many people sent patches, helped with
-       portability problems, lent machines, gave advice and  were
-       generally helpful.
+       for faster sorting algorithms, so as to speed up  compres­
+       sion.  Bela Lubkin encouraged me to improve the worst‐case
+       compression performance.  Donna Robinson XMLised the docu­
+       mentation.   The bz* scripts are derived from those of GNU
+       gzip.  Many people sent patches, helped  with  portability
+       problems,  lent  machines,  gave advice and were generally
+       helpful.
 
 
 
diff --git a/bzip2.c b/bzip2.c
index 807f420..79f87a5 100644
--- a/bzip2.c
+++ b/bzip2.c
@@ -7,7 +7,7 @@
   This file is a part of bzip2 and/or libbzip2, a program and
   library for lossless, block-sorting data compression.
 
-  Copyright (C) 1996-2002 Julian R Seward.  All rights reserved.
+  Copyright (C) 1996-2005 Julian R Seward.  All rights reserved.
 
   Redistribution and use in source and binary forms, with or without
   modification, are permitted provided that the following conditions
@@ -41,7 +41,7 @@
   SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
   Julian Seward, Cambridge, UK.
-  jseward@acm.org
+  jseward@bzip.org
   bzip2/libbzip2 version 1.0 of 21 March 2000
 
   This program is based on (at least) the work of:
@@ -525,6 +525,7 @@ Bool uncompressStream ( FILE *zStream, FILE *stream )
    UChar   obuf[5000];
    UChar   unused[BZ_MAX_UNUSED];
    Int32   nUnused;
+   void*   unusedTmpV;
    UChar*  unusedTmp;
 
    nUnused = 0;
@@ -554,9 +555,10 @@ Bool uncompressStream ( FILE *zStream, FILE *stream )
       }
       if (bzerr != BZ_STREAM_END) goto errhandler;
 
-      BZ2_bzReadGetUnused ( &bzerr, bzf, (void**)(&unusedTmp), &nUnused );
+      BZ2_bzReadGetUnused ( &bzerr, bzf, &unusedTmpV, &nUnused );
       if (bzerr != BZ_OK) panic ( "decompress:bzReadGetUnused" );
 
+      unusedTmp = (UChar*)unusedTmpV;
       for (i = 0; i < nUnused; i++) unused[i] = unusedTmp[i];
 
       BZ2_bzReadClose ( &bzerr, bzf );
@@ -639,6 +641,7 @@ Bool testStream ( FILE *zStream )
    UChar   obuf[5000];
    UChar   unused[BZ_MAX_UNUSED];
    Int32   nUnused;
+   void*   unusedTmpV;
    UChar*  unusedTmp;
 
    nUnused = 0;
@@ -662,9 +665,10 @@ Bool testStream ( FILE *zStream )
       }
       if (bzerr != BZ_STREAM_END) goto errhandler;
 
-      BZ2_bzReadGetUnused ( &bzerr, bzf, (void**)(&unusedTmp), &nUnused );
+      BZ2_bzReadGetUnused ( &bzerr, bzf, &unusedTmpV, &nUnused );
       if (bzerr != BZ_OK) panic ( "test:bzReadGetUnused" );
 
+      unusedTmp = (UChar*)unusedTmpV;
       for (i = 0; i < nUnused; i++) unused[i] = unusedTmp[i];
 
       BZ2_bzReadClose ( &bzerr, bzf );
@@ -828,7 +832,7 @@ void panic ( Char* s )
              "\n%s: PANIC -- internal consistency error:\n"
              "\t%s\n"
              "\tThis is a BUG.  Please report it to me at:\n"
-             "\tjseward@acm.org\n",
+             "\tjseward@bzip.org\n",
              progName, s );
    showFileNames();
    cleanUpAndFail( 3 );
@@ -908,7 +912,7 @@ void mySIGSEGVorSIGBUScatcher ( IntNative n )
       "   The user's manual, Section 4.3, has more info on (1) and (2).\n"
       "   \n"
       "   If you suspect this is a bug in bzip2, or are unsure about (1)\n"
-      "   or (2), feel free to report it to me at: jseward@acm.org.\n"
+      "   or (2), feel free to report it to me at: jseward@bzip.org.\n"
       "   Section 4.3 of the user's manual describes the info a useful\n"
       "   bug report should have.  If the manual is available on your\n"
       "   system, please try and read it before mailing me.  If you don't\n"
@@ -931,7 +935,7 @@ void mySIGSEGVorSIGBUScatcher ( IntNative n )
       "   The user's manual, Section 4.3, has more info on (2) and (3).\n"
       "   \n"
       "   If you suspect this is a bug in bzip2, or are unsure about (2)\n"
-      "   or (3), feel free to report it to me at: jseward@acm.org.\n"
+      "   or (3), feel free to report it to me at: jseward@bzip.org.\n"
       "   Section 4.3 of the user's manual describes the info a useful\n"
       "   bug report should have.  If the manual is available on your\n"
       "   system, please try and read it before mailing me.  If you don't\n"
@@ -1674,7 +1678,7 @@ void license ( void )
     "bzip2, a block-sorting file compressor.  "
     "Version %s.\n"
     "   \n"
-    "   Copyright (C) 1996-2002 by Julian Seward.\n"
+    "   Copyright (C) 1996-2005 by Julian Seward.\n"
     "   \n"
     "   This program is free software; you can redistribute it and/or modify\n"
     "   it under the terms set out in the LICENSE file, which is included\n"
diff --git a/bzip2.txt b/bzip2.txt
index 6afe358..bf895b6 100644
--- a/bzip2.txt
+++ b/bzip2.txt
@@ -1,6 +1,6 @@
 
 NAME
-       bzip2, bunzip2 - a block-sorting file compressor, v1.0.2
+       bzip2, bunzip2 - a block-sorting file compressor, v1.0.3
        bzcat - decompresses files to stdout
        bzip2recover - recovers data from damaged bzip2 files
 
@@ -17,20 +17,20 @@ DESCRIPTION
        sorting text compression algorithm,  and  Huffman  coding.
        Compression  is  generally  considerably  better than that
        achieved by more conventional LZ77/LZ78-based compressors,
-       and  approaches  the performance of the PPM family of sta
+       and  approaches  the performance of the PPM family of sta-
        tistical compressors.
 
        The command-line options are deliberately very similar  to
        those of GNU gzip, but they are not identical.
 
-       bzip2  expects  a list of file names to accompany the com
+       bzip2  expects  a list of file names to accompany the com-
        mand-line flags.  Each file is replaced  by  a  compressed
        version  of  itself,  with  the  name "original_name.bz2".
-       Each compressed file has the same modification date,  per
-       missions, and, when possible, ownership as the correspond
+       Each compressed file has the same modification date,  per-
+       missions, and, when possible, ownership as the correspond-
        ing original, so that these properties  can  be  correctly
        restored  at  decompression  time.   File name handling is
-       naive in the sense that there is no mechanism for preserv
+       naive in the sense that there is no mechanism for preserv-
        ing  original file names, permissions, ownerships or dates
        in filesystems which lack these concepts, or have  serious
        file name length restrictions, such as MS-DOS.
@@ -61,23 +61,23 @@ DESCRIPTION
        guess the name of the original file, and uses the original
        name with .out appended.
 
-       As  with compression, supplying no filenames causes decom
+       As  with compression, supplying no filenames causes decom-
        pression from standard input to standard output.
 
-       bunzip2 will correctly decompress a file which is the con
+       bunzip2 will correctly decompress a file which is the con-
        catenation of two or more compressed files.  The result is
        the concatenation of the corresponding uncompressed files.
        Integrity testing (-t) of concatenated compressed files is
        also supported.
 
        You can also compress or decompress files to the  standard
-       output  by giving the -c flag.  Multiple files may be com
+       output  by giving the -c flag.  Multiple files may be com-
        pressed and decompressed like this.  The resulting outputs
        are  fed  sequentially to stdout.  Compression of multiple
-       files in this manner generates a stream containing  multi
+       files in this manner generates a stream containing  multi-
        ple compressed file representations.  Such a stream can be
        decompressed correctly only  by  bzip2  version  0.9.0  or
-       later.   Earlier  versions of bzip2 will stop after decom
+       later.   Earlier  versions of bzip2 will stop after decom-
        pressing the first file in the stream.
 
        bzcat (or bzip2 -dc) decompresses all specified  files  to
@@ -98,7 +98,7 @@ DESCRIPTION
 
        As a self-check for your  protection,  bzip2  uses  32-bit
        CRCs  to make sure that the decompressed version of a file
-       is identical to the original.  This guards against corrup
+       is identical to the original.  This guards against corrup-
        tion  of  the compressed data, and against undetected bugs
        in bzip2 (hopefully very unlikely).  The chances  of  data
        corruption  going  undetected  is  microscopic,  about one
@@ -171,7 +171,7 @@ OPTIONS
 
        -v --verbose
               Verbose mode -- show the compression ratio for each
-              file  processed.   Further  -v's  increase the ver
+              file  processed.   Further  -v's  increase the ver-
               bosity level, spewing out lots of information which
               is primarily of interest for diagnostic purposes.
 
@@ -184,19 +184,19 @@ OPTIONS
               compressing.   Has  no  effect  when decompressing.
               See MEMORY MANAGEMENT below.  The --fast and --best
               aliases  are  primarily for GNU gzip compatibility.
-              In particular, --fast doesn't make things  signifi
+              In particular, --fast doesn't make things  signifi-
               cantly  faster.   And  --best  merely  selects  the
               default behaviour.
 
        --     Treats all subsequent arguments as file names, even
-              if they start with a dash.  This is so you can han
+              if they start with a dash.  This is so you can han-
               dle files with names beginning  with  a  dash,  for
               example: bzip2 -- -myfilename.
 
        --repetitive-fast --repetitive-best
               These  flags  are  redundant  in versions 0.9.5 and
               above.  They provided some coarse control over  the
-              behaviour  of the sorting algorithm in earlier ver
+              behaviour  of the sorting algorithm in earlier ver-
               sions, which was sometimes useful.  0.9.5 and above
               have  an  improved  algorithm  which  renders these
               flags irrelevant.
@@ -207,7 +207,7 @@ MEMORY MANAGEMENT
        affects  both  the  compression  ratio  achieved,  and the
        amount of memory needed for compression and decompression.
        The  flags  -1  through  -9  specify  the block size to be
-       100,000 bytes through 900,000 bytes (the default)  respec
+       100,000 bytes through 900,000 bytes (the default)  respec-
        tively.   At  decompression  time, the block size used for
        compression is read from  the  header  of  the  compressed
        file, and bunzip2 then allocates itself just enough memory
@@ -235,13 +235,13 @@ MEMORY MANAGEMENT
        bunzip2 will require about 3700 kbytes to decompress.   To
        support decompression of any file on a 4 megabyte machine,
        bunzip2 has an option to  decompress  using  approximately
-       half this amount of memory, about 2300 kbytes.  Decompres
+       half this amount of memory, about 2300 kbytes.  Decompres-
        sion speed is also halved, so you should use  this  option
        only where necessary.  The relevant flag is -s.
 
-       In general, try and use the largest block size memory con
+       In general, try and use the largest block size memory con-
        straints  allow,  since  that  maximises  the  compression
-       achieved.   Compression and decompression speed are virtu
+       achieved.   Compression and decompression speed are virtu-
        ally unaffected by block size.
 
        Another significant point applies to files which fit in  a
@@ -257,11 +257,11 @@ MEMORY MANAGEMENT
 
        Here  is a table which summarises the maximum memory usage
        for different block sizes.  Also  recorded  is  the  total
-       compressed  size for 14 files of the Calgary Text Compres
+       compressed  size for 14 files of the Calgary Text Compres-
        sion Corpus totalling 3,141,622 bytes.  This column  gives
        some  feel  for  how  compression  varies with block size.
        These figures tend to understate the advantage  of  larger
-       block  sizes  for  larger files, since the Corpus is domi
+       block  sizes  for  larger files, since the Corpus is domi-
        nated by smaller files.
 
                   Compress   Decompress   Decompress   Corpus
@@ -280,7 +280,7 @@ MEMORY MANAGEMENT
 
 RECOVERING DATA FROM DAMAGED FILES
        bzip2 compresses files in blocks, usually 900kbytes  long.
-       Each block is handled independently.  If a media or trans
+       Each block is handled independently.  If a media or trans-
        mission error causes a multi-block  .bz2  file  to  become
        damaged,  it  may  be  possible  to  recover data from the
        undamaged blocks in the file.
@@ -297,19 +297,19 @@ RECOVERING DATA FROM DAMAGED FILES
        the integrity of the resulting files, and decompress those
        which are undamaged.
 
-       bzip2recover takes a single argument, the name of the dam
+       bzip2recover takes a single argument, the name of the dam-
        aged    file,    and    writes    a    number   of   files
        "rec00001file.bz2",  "rec00002file.bz2",  etc,  containing
        the   extracted   blocks.   The   output   filenames   are
-       designed  so  that the use of wildcards in subsequent pro
-       cessing  -- for example, "bzip2 -dc  rec*file.bz2 > recov
+       designed  so  that the use of wildcards in subsequent pro-
+       cessing  -- for example, "bzip2 -dc  rec*file.bz2 > recov-
        ered_data" -- processes the files in the correct order.
 
        bzip2recover should be of most use dealing with large .bz2
        files,  as  these will contain many blocks.  It is clearly
        futile to use it on damaged single-block  files,  since  a
-       damaged  block  cannot  be recovered.  If you wish to min
-       imise any potential data loss through media  or  transmis
+       damaged  block  cannot  be recovered.  If you wish to min-
+       imise any potential data loss through media  or  transmis-
        sion errors, you might consider compressing with a smaller
        block size.
 
@@ -323,19 +323,19 @@ PERFORMANCE NOTES
        better  than previous versions in this respect.  The ratio
        between worst-case and average-case compression time is in
        the  region  of  10:1.  For previous versions, this figure
-       was more like 100:1.  You can use the -vvvv option to mon
+       was more like 100:1.  You can use the -vvvv option to mon-
        itor progress in great detail, if you want.
 
        Decompression speed is unaffected by these phenomena.
 
        bzip2  usually  allocates  several  megabytes of memory to
-       operate in, and then charges all over it in a fairly  ran
-       dom  fashion.   This means that performance, both for com
+       operate in, and then charges all over it in a fairly  ran-
+       dom  fashion.   This means that performance, both for com-
        pressing and decompressing, is largely determined  by  the
        speed  at  which  your  machine  can service cache misses.
        Because of this, small changes to the code to  reduce  the
        miss  rate  have  been observed to give disproportionately
-       large performance improvements.  I imagine bzip2 will per
+       large performance improvements.  I imagine bzip2 will per-
        form best on machines with very large caches.
 
 
@@ -345,46 +345,47 @@ CAVEATS
        but  the  details  of  what  the problem is sometimes seem
        rather misleading.
 
-       This manual page pertains to version 1.0.2 of bzip2.  Com
+       This manual page pertains to version 1.0.3 of bzip2.  Com-
        pressed  data created by this version is entirely forwards
        and  backwards  compatible  with   the   previous   public
-       releases,  versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0 and 1.0.1,
-       but with the following exception: 0.9.0 and above can cor
-       rectly  decompress multiple concatenated compressed files.
-       0.1pl2 cannot do this; it will  stop  after  decompressing
-       just the first file in the stream.
-
-       bzip2recover  versions  prior  to  this  one,  1.0.2, used
-       32-bit integers to represent bit positions  in  compressed
-       files,  so  it could not handle compressed files more than
-       512 megabytes long.  Version 1.0.2 and above  uses  64-bit
-       ints  on  some platforms which support them (GNU supported
-       targets,  and  Windows).   To  establish  whether  or  not
-       bzip2recover  was  built  with  such  a limitation, run it
-       without arguments.  In any event you can build yourself an
-       unlimited version if you can recompile it with MaybeUInt64
-       set to be an unsigned 64-bit integer.
+       releases,  versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1 and
+       1.0.2, but with the following exception: 0.9.0  and  above
+       can  correctly decompress multiple concatenated compressed
+       files.  0.1pl2 cannot do this; it will stop  after  decom-
+       pressing just the first file in the stream.
+
+       bzip2recover  versions prior to 1.0.2 used 32-bit integers
+       to represent bit positions in compressed  files,  so  they
+       could  not handle compressed files more than 512 megabytes
+       long.  Versions 1.0.2 and above use 64-bit  ints  on  some
+       platforms  which  support them (GNU supported targets, and
+       Windows).  To establish whether or  not  bzip2recover  was
+       built  with  such  a limitation, run it without arguments.
+       In any event you can build yourself an  unlimited  version
+       if  you  can  recompile  it  with MaybeUInt64 set to be an
+       unsigned 64-bit integer.
 
 
 AUTHOR
-       Julian Seward, jseward@acm.org.
+       Julian Seward, jsewardbzip.org.
 
-       http://sources.redhat.com/bzip2
+       http://www.bzip.org
 
-       The ideas embodied in bzip2 are due to (at least) the fol
+       The ideas embodied in bzip2 are due to (at least) the fol-
        lowing  people: Michael Burrows and David Wheeler (for the
        block sorting transformation), David Wheeler  (again,  for
-       the Huffman coder), Peter Fenwick (for the structured cod
+       the Huffman coder), Peter Fenwick (for the structured cod-
        ing model in the original bzip, and many refinements), and
        Alistair  Moffat,  Radford  Neal  and  Ian Witten (for the
        arithmetic  coder  in  the  original  bzip).   I  am  much
-       indebted for their help, support and advice.  See the man
+       indebted for their help, support and advice.  See the man-
        ual in the source distribution for pointers to sources  of
        documentation.  Christian von Roques encouraged me to look
-       for faster sorting algorithms, so as to speed up  compres
+       for faster sorting algorithms, so as to speed up  compres-
        sion.  Bela Lubkin encouraged me to improve the worst-case
-       compression performance.  The bz* scripts are derived from
-       those  of GNU gzip.  Many people sent patches, helped with
-       portability problems, lent machines, gave advice and  were
-       generally helpful.
+       compression performance.  Donna Robinson XMLised the docu-
+       mentation.   The bz* scripts are derived from those of GNU
+       gzip.  Many people sent patches, helped  with  portability
+       problems,  lent  machines,  gave advice and were generally
+       helpful.
 
diff --git a/bzip2recover.c b/bzip2recover.c
index 286873b..5cd405d 100644
--- a/bzip2recover.c
+++ b/bzip2recover.c
@@ -7,9 +7,9 @@
 /*--
   This program is bzip2recover, a program to attempt data 
   salvage from damaged files created by the accompanying
-  bzip2-1.0 program.
+  bzip2-1.0.3 program.
 
-  Copyright (C) 1996-2002 Julian R Seward.  All rights reserved.
+  Copyright (C) 1996-2005 Julian R Seward.  All rights reserved.
 
   Redistribution and use in source and binary forms, with or without
   modification, are permitted provided that the following conditions
@@ -43,8 +43,8 @@
   SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
   Julian Seward, Cambridge, UK.
-  jseward@acm.org
-  bzip2/libbzip2 version 1.0 of 21 March 2000
+  jseward@bzip.org
+  bzip2/libbzip2 version 1.0.3 of 15 February 2005
 --*/
 
 /*--
@@ -345,7 +345,7 @@ Int32 main ( Int32 argc, Char** argv )
    inFileName[0] = outFileName[0] = 0;
 
    fprintf ( stderr, 
-             "bzip2recover 1.0.2: extracts blocks from damaged .bz2 files.\n" );
+             "bzip2recover 1.0.3: extracts blocks from damaged .bz2 files.\n" );
 
    if (argc != 2) {
       fprintf ( stderr, "%s: usage is `%s damaged_file_name'.\n",
@@ -374,7 +374,7 @@ Int32 main ( Int32 argc, Char** argv )
    if (strlen(argv[1]) >= BZ_MAX_FILENAME-20) {
       fprintf ( stderr, 
                 "%s: supplied filename is suspiciously (>= %d chars) long.  Bye!\n",
-                progName, strlen(argv[1]) );
+                progName, (int)strlen(argv[1]) );
       exit(1);
    }
 
diff --git a/bzlib.c b/bzlib.c
index 7d1cb27..e9c1e87 100644
--- a/bzlib.c
+++ b/bzlib.c
@@ -8,7 +8,7 @@
   This file is a part of bzip2 and/or libbzip2, a program and
   library for lossless, block-sorting data compression.
 
-  Copyright (C) 1996-2002 Julian R Seward.  All rights reserved.
+  Copyright (C) 1996-2005 Julian R Seward.  All rights reserved.
 
   Redistribution and use in source and binary forms, with or without
   modification, are permitted provided that the following conditions
@@ -42,7 +42,7 @@
   SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
   Julian Seward, Cambridge, UK.
-  jseward@acm.org
+  jseward@bzip.org
   bzip2/libbzip2 version 1.0 of 21 March 2000
 
   This program is based on (at least) the work of:
@@ -88,12 +88,12 @@ void BZ2_bz__AssertH__fail ( int errcode )
    fprintf(stderr, 
       "\n\nbzip2/libbzip2: internal error number %d.\n"
       "This is a bug in bzip2/libbzip2, %s.\n"
-      "Please report it to me at: jseward@acm.org.  If this happened\n"
+      "Please report it to me at: jseward@bzip.org.  If this happened\n"
       "when you were using some program which uses libbzip2 as a\n"
       "component, you should also report this bug to the author(s)\n"
       "of that program.  Please make an effort to report this bug;\n"
       "timely and accurate bug reports eventually lead to higher\n"
-      "quality software.  Thanks.  Julian Seward, 30 December 2001.\n\n",
+      "quality software.  Thanks.  Julian Seward, 15 February 2005.\n\n",
       errcode,
       BZ2_bzlibVersion()
    );
@@ -574,8 +574,11 @@ int BZ_API(BZ2_bzDecompressInit)
 
 
 /*---------------------------------------------------*/
+/* Return  True iff data corruption is discovered.
+   Returns False if there is no problem.
+*/
 static
-void unRLE_obuf_to_output_FAST ( DState* s )
+Bool unRLE_obuf_to_output_FAST ( DState* s )
 {
    UChar k1;
 
@@ -584,7 +587,7 @@ void unRLE_obuf_to_output_FAST ( DState* s )
       while (True) {
          /* try to finish existing run */
          while (True) {
-            if (s->strm->avail_out == 0) return;
+            if (s->strm->avail_out == 0) return False;
             if (s->state_out_len == 0) break;
             *( (UChar*)(s->strm->next_out) ) = s->state_out_ch;
             BZ_UPDATE_CRC ( s->calculatedBlockCRC, s->state_out_ch );
@@ -594,10 +597,13 @@ void unRLE_obuf_to_output_FAST ( DState* s )
             s->strm->total_out_lo32++;
             if (s->strm->total_out_lo32 == 0) s->strm->total_out_hi32++;
          }
-   
+
          /* can a new run be started? */
-         if (s->nblock_used == s->save_nblock+1) return;
+         if (s->nblock_used == s->save_nblock+1) return False;
                
+         /* Only caused by corrupt data stream? */
+         if (s->nblock_used > s->save_nblock+1)
+            return True;
    
          s->state_out_len = 1;
          s->state_out_ch = s->k0;
@@ -667,6 +673,10 @@ void unRLE_obuf_to_output_FAST ( DState* s )
                cs_avail_out--;
             }
          }   
+         /* Only caused by corrupt data stream? */
+         if (c_nblock_used > s_save_nblockPP)
+            return True;
+
          /* can a new run be started? */
          if (c_nblock_used == s_save_nblockPP) {
             c_state_out_len = 0; goto return_notr;
@@ -712,6 +722,7 @@ void unRLE_obuf_to_output_FAST ( DState* s )
       s->strm->avail_out    = cs_avail_out;
       /* end save */
    }
+   return False;
 }
 
 
@@ -732,8 +743,11 @@ __inline__ Int32 BZ2_indexIntoF ( Int32 indx, Int32 *cftab )
 
 
 /*---------------------------------------------------*/
+/* Return  True iff data corruption is discovered.
+   Returns False if there is no problem.
+*/
 static
-void unRLE_obuf_to_output_SMALL ( DState* s )
+Bool unRLE_obuf_to_output_SMALL ( DState* s )
 {
    UChar k1;
 
@@ -742,7 +756,7 @@ void unRLE_obuf_to_output_SMALL ( DState* s )
       while (True) {
          /* try to finish existing run */
          while (True) {
-            if (s->strm->avail_out == 0) return;
+            if (s->strm->avail_out == 0) return False;
             if (s->state_out_len == 0) break;
             *( (UChar*)(s->strm->next_out) ) = s->state_out_ch;
             BZ_UPDATE_CRC ( s->calculatedBlockCRC, s->state_out_ch );
@@ -754,8 +768,11 @@ void unRLE_obuf_to_output_SMALL ( DState* s )
          }
    
          /* can a new run be started? */
-         if (s->nblock_used == s->save_nblock+1) return;
-               
+         if (s->nblock_used == s->save_nblock+1) return False;
+
+         /* Only caused by corrupt data stream? */
+         if (s->nblock_used > s->save_nblock+1)
+            return True;
    
          s->state_out_len = 1;
          s->state_out_ch = s->k0;
@@ -788,7 +805,7 @@ void unRLE_obuf_to_output_SMALL ( DState* s )
       while (True) {
          /* try to finish existing run */
          while (True) {
-            if (s->strm->avail_out == 0) return;
+            if (s->strm->avail_out == 0) return False;
             if (s->state_out_len == 0) break;
             *( (UChar*)(s->strm->next_out) ) = s->state_out_ch;
             BZ_UPDATE_CRC ( s->calculatedBlockCRC, s->state_out_ch );
@@ -800,7 +817,11 @@ void unRLE_obuf_to_output_SMALL ( DState* s )
          }
    
          /* can a new run be started? */
-         if (s->nblock_used == s->save_nblock+1) return;
+         if (s->nblock_used == s->save_nblock+1) return False;
+
+         /* Only caused by corrupt data stream? */
+         if (s->nblock_used > s->save_nblock+1)
+            return True;
    
          s->state_out_len = 1;
          s->state_out_ch = s->k0;
@@ -830,6 +851,7 @@ void unRLE_obuf_to_output_SMALL ( DState* s )
 /*---------------------------------------------------*/
 int BZ_API(BZ2_bzDecompress) ( bz_stream *strm )
 {
+   Bool    corrupt;
    DState* s;
    if (strm == NULL) return BZ_PARAM_ERROR;
    s = strm->state;
@@ -840,12 +862,13 @@ int BZ_API(BZ2_bzDecompress) ( bz_stream *strm )
       if (s->state == BZ_X_IDLE) return BZ_SEQUENCE_ERROR;
       if (s->state == BZ_X_OUTPUT) {
          if (s->smallDecompress)
-            unRLE_obuf_to_output_SMALL ( s ); else
-            unRLE_obuf_to_output_FAST  ( s );
+            corrupt = unRLE_obuf_to_output_SMALL ( s ); else
+            corrupt = unRLE_obuf_to_output_FAST  ( s );
+         if (corrupt) return BZ_DATA_ERROR;
          if (s->nblock_used == s->save_nblock+1 && s->state_out_len == 0) {
             BZ_FINALISE_CRC ( s->calculatedBlockCRC );
             if (s->verbosity >= 3) 
-               VPrintf2 ( " {0x%x, 0x%x}", s->storedBlockCRC, 
+               VPrintf2 ( " {0x%08x, 0x%08x}", s->storedBlockCRC, 
                           s->calculatedBlockCRC );
             if (s->verbosity >= 2) VPrintf0 ( "]" );
             if (s->calculatedBlockCRC != s->storedBlockCRC)
@@ -863,7 +886,7 @@ int BZ_API(BZ2_bzDecompress) ( bz_stream *strm )
          Int32 r = BZ2_decompress ( s );
          if (r == BZ_STREAM_END) {
             if (s->verbosity >= 3)
-               VPrintf2 ( "\n    combined CRCs: stored = 0x%x, computed = 0x%x", 
+               VPrintf2 ( "\n    combined CRCs: stored = 0x%08x, computed = 0x%08x", 
                           s->storedCombinedCRC, s->calculatedCombinedCRC );
             if (s->calculatedCombinedCRC != s->storedCombinedCRC)
                return BZ_DATA_ERROR;
diff --git a/bzlib.h b/bzlib.h
index 9ac43a1..3237243 100644
--- a/bzlib.h
+++ b/bzlib.h
@@ -8,7 +8,7 @@
   This file is a part of bzip2 and/or libbzip2, a program and
   library for lossless, block-sorting data compression.
 
-  Copyright (C) 1996-2002 Julian R Seward.  All rights reserved.
+  Copyright (C) 1996-2005 Julian R Seward.  All rights reserved.
 
   Redistribution and use in source and binary forms, with or without
   modification, are permitted provided that the following conditions
@@ -42,7 +42,7 @@
   SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
   Julian Seward, Cambridge, UK.
-  jseward@acm.org
+  jseward@bzip.org
   bzip2/libbzip2 version 1.0 of 21 March 2000
 
   This program is based on (at least) the work of:
@@ -110,8 +110,10 @@ typedef
 #define BZ_EXPORT
 #endif
 
+#ifndef BZ_NO_STDIO
 /* Need a definitition for FILE */
 #include 
+#endif
 
 #ifdef _WIN32
 #   include 
diff --git a/bzlib_private.h b/bzlib_private.h
index ff973c3..ca76fe6 100644
--- a/bzlib_private.h
+++ b/bzlib_private.h
@@ -8,7 +8,7 @@
   This file is a part of bzip2 and/or libbzip2, a program and
   library for lossless, block-sorting data compression.
 
-  Copyright (C) 1996-2002 Julian R Seward.  All rights reserved.
+  Copyright (C) 1996-2005 Julian R Seward.  All rights reserved.
 
   Redistribution and use in source and binary forms, with or without
   modification, are permitted provided that the following conditions
@@ -42,7 +42,7 @@
   SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
   Julian Seward, Cambridge, UK.
-  jseward@acm.org
+  jseward@bzip.org
   bzip2/libbzip2 version 1.0 of 21 March 2000
 
   This program is based on (at least) the work of:
@@ -76,7 +76,7 @@
 
 /*-- General stuff. --*/
 
-#define BZ_VERSION  "1.0.2, 30-Dec-2001"
+#define BZ_VERSION  "1.0.3, 15-Feb-2005"
 
 typedef char            Char;
 typedef unsigned char   Bool;
diff --git a/compress.c b/compress.c
index 56501c1..7e0c291 100644
--- a/compress.c
+++ b/compress.c
@@ -8,7 +8,7 @@
   This file is a part of bzip2 and/or libbzip2, a program and
   library for lossless, block-sorting data compression.
 
-  Copyright (C) 1996-2002 Julian R Seward.  All rights reserved.
+  Copyright (C) 1996-2005 Julian R Seward.  All rights reserved.
 
   Redistribution and use in source and binary forms, with or without
   modification, are permitted provided that the following conditions
@@ -42,7 +42,7 @@
   SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
   Julian Seward, Cambridge, UK.
-  jseward@acm.org
+  jseward@bzip.org
   bzip2/libbzip2 version 1.0 of 21 March 2000
 
   This program is based on (at least) the work of:
@@ -488,9 +488,11 @@ void sendMTFValues ( EState* s )
       /*--
         Recompute the tables based on the accumulated frequencies.
       --*/
+      /* maxLen was changed from 20 to 17 in bzip2-1.0.3.  See 
+         comment in huffman.c for details. */
       for (t = 0; t < nGroups; t++)
          BZ2_hbMakeCodeLengths ( &(s->len[t][0]), &(s->rfreq[t][0]), 
-                                 alphaSize, 20 );
+                                 alphaSize, 17 /*20*/ );
    }
 
 
@@ -527,7 +529,7 @@ void sendMTFValues ( EState* s )
          if (s->len[t][i] > maxLen) maxLen = s->len[t][i];
          if (s->len[t][i] < minLen) minLen = s->len[t][i];
       }
-      AssertH ( !(maxLen > 20), 3004 );
+      AssertH ( !(maxLen > 17 /*20*/ ), 3004 );
       AssertH ( !(minLen < 1),  3005 );
       BZ2_hbAssignCodes ( &(s->code[t][0]), &(s->len[t][0]), 
                           minLen, maxLen, alphaSize );
@@ -651,8 +653,8 @@ void BZ2_compressBlock ( EState* s, Bool is_last_block )
       if (s->blockNo > 1) s->numZ = 0;
 
       if (s->verbosity >= 2)
-         VPrintf4( "    block %d: crc = 0x%8x, "
-                   "combined CRC = 0x%8x, size = %d\n",
+         VPrintf4( "    block %d: crc = 0x%08x, "
+                   "combined CRC = 0x%08x, size = %d\n",
                    s->blockNo, s->blockCRC, s->combinedCRC, s->nblock );
 
       BZ2_blockSort ( s );
@@ -703,7 +705,7 @@ void BZ2_compressBlock ( EState* s, Bool is_last_block )
       bsPutUChar ( s, 0x50 ); bsPutUChar ( s, 0x90 );
       bsPutUInt32 ( s, s->combinedCRC );
       if (s->verbosity >= 2)
-         VPrintf1( "    final combined CRC = 0x%x\n   ", s->combinedCRC );
+         VPrintf1( "    final combined CRC = 0x%08x\n   ", s->combinedCRC );
       bsFinishWrite ( s );
    }
 }
diff --git a/crctable.c b/crctable.c
index b16746a..b6dadfc 100644
--- a/crctable.c
+++ b/crctable.c
@@ -8,7 +8,7 @@
   This file is a part of bzip2 and/or libbzip2, a program and
   library for lossless, block-sorting data compression.
 
-  Copyright (C) 1996-2002 Julian R Seward.  All rights reserved.
+  Copyright (C) 1996-2005 Julian R Seward.  All rights reserved.
 
   Redistribution and use in source and binary forms, with or without
   modification, are permitted provided that the following conditions
@@ -42,7 +42,7 @@
   SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
   Julian Seward, Cambridge, UK.
-  jseward@acm.org
+  jseward@bzip.org
   bzip2/libbzip2 version 1.0 of 21 March 2000
 
   This program is based on (at least) the work of:
diff --git a/decompress.c b/decompress.c
index e921347..81c3d2c 100644
--- a/decompress.c
+++ b/decompress.c
@@ -8,7 +8,7 @@
   This file is a part of bzip2 and/or libbzip2, a program and
   library for lossless, block-sorting data compression.
 
-  Copyright (C) 1996-2002 Julian R Seward.  All rights reserved.
+  Copyright (C) 1996-2005 Julian R Seward.  All rights reserved.
 
   Redistribution and use in source and binary forms, with or without
   modification, are permitted provided that the following conditions
@@ -42,7 +42,7 @@
   SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
   Julian Seward, Cambridge, UK.
-  jseward@acm.org
+  jseward@bzip.org
   bzip2/libbzip2 version 1.0 of 21 March 2000
 
   This program is based on (at least) the work of:
@@ -524,17 +524,23 @@ Int32 BZ2_decompress ( DState* s )
       if (s->origPtr < 0 || s->origPtr >= nblock)
          RETURN(BZ_DATA_ERROR);
 
+      /*-- Set up cftab to facilitate generation of T^(-1) --*/
+      s->cftab[0] = 0;
+      for (i = 1; i <= 256; i++) s->cftab[i] = s->unzftab[i-1];
+      for (i = 1; i <= 256; i++) s->cftab[i] += s->cftab[i-1];
+      for (i = 0; i <= 256; i++) {
+         if (s->cftab[i] < 0 || s->cftab[i] > nblock) {
+            /* s->cftab[i] can legitimately be == nblock */
+            RETURN(BZ_DATA_ERROR);
+         }
+      }
+
       s->state_out_len = 0;
       s->state_out_ch  = 0;
       BZ_INITIALISE_CRC ( s->calculatedBlockCRC );
       s->state = BZ_X_OUTPUT;
       if (s->verbosity >= 2) VPrintf0 ( "rt+rld" );
 
-      /*-- Set up cftab to facilitate generation of T^(-1) --*/
-      s->cftab[0] = 0;
-      for (i = 1; i <= 256; i++) s->cftab[i] = s->unzftab[i-1];
-      for (i = 1; i <= 256; i++) s->cftab[i] += s->cftab[i-1];
-
       if (s->smallDecompress) {
 
          /*-- Make a copy of cftab, used in generation of T --*/
diff --git a/entities.xml b/entities.xml
new file mode 100644
index 0000000..6d0975f
--- /dev/null
+++ b/entities.xml
@@ -0,0 +1,9 @@
+
+
+
+
+
+
+
+
+
diff --git a/format.pl b/format.pl
new file mode 100755
index 0000000..8ab47ac
--- /dev/null
+++ b/format.pl
@@ -0,0 +1,53 @@
+#!/usr/bin/perl -w
+use strict;
+
+# get command line values:
+if ( $#ARGV !=1 ) {
+    die "Usage:  $0 xml_infile xml_outfile\n";
+}
+
+my $infile = shift;
+# check infile exists
+die "Can't find file \"$infile\""
+  unless -f $infile;
+# check we can read infile
+if (! -r $infile) {
+    die "Can't read input $infile\n";
+}
+# check we can open infile
+open( INFILE,"<$infile" ) or 
+    die "Can't input $infile $!";
+
+#my $outfile = 'fmt-manual.xml';
+my $outfile = shift;
+#print "Infile: $infile, Outfile: $outfile\n";
+# check we can write to outfile
+open( OUTFILE,">$outfile" ) or 
+    die "Can't output $outfile $! for writing";
+
+my ($prev, $curr, $str);
+$prev = ''; $curr = '';
+while (  ) {
+
+		print OUTFILE $prev;
+    $prev = $curr;
+    $curr = $_;
+    $str = '';
+
+    if ( $prev =~ /$|$/ ) {
+        chomp $prev;
+        $curr = join( '', $prev, "|<\/screen>/ ) {
+        chomp $prev;
+        $curr = join( '', $prev, "]]>", $curr );
+				$prev = '';
+        next;
+    }
+}
+print OUTFILE $curr;
+close INFILE;
+close OUTFILE;
+exit;
diff --git a/huffman.c b/huffman.c
index 293095c..5bf190b 100644
--- a/huffman.c
+++ b/huffman.c
@@ -8,7 +8,7 @@
   This file is a part of bzip2 and/or libbzip2, a program and
   library for lossless, block-sorting data compression.
 
-  Copyright (C) 1996-2002 Julian R Seward.  All rights reserved.
+  Copyright (C) 1996-2005 Julian R Seward.  All rights reserved.
 
   Redistribution and use in source and binary forms, with or without
   modification, are permitted provided that the following conditions
@@ -42,7 +42,7 @@
   SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
   Julian Seward, Cambridge, UK.
-  jseward@acm.org
+  jseward@bzip.org
   bzip2/libbzip2 version 1.0 of 21 March 2000
 
   This program is based on (at least) the work of:
@@ -162,7 +162,24 @@ void BZ2_hbMakeCodeLengths ( UChar *len,
       
       if (! tooLong) break;
 
-      for (i = 1; i < alphaSize; i++) {
+      /* 17 Oct 04: keep-going condition for the following loop used
+         to be 'i < alphaSize', which missed the last element,
+         theoretically leading to the possibility of the compressor
+         looping.  However, this count-scaling step is only needed if
+         one of the generated Huffman code words is longer than
+         maxLen, which up to and including version 1.0.2 was 20 bits,
+         which is extremely unlikely.  In version 1.0.3 maxLen was
+         changed to 17 bits, which has minimal effect on compression
+         ratio, but does mean this scaling step is used from time to
+         time, enough to verify that it works.
+
+         This means that bzip2-1.0.3 and later will only produce
+         Huffman codes with a maximum length of 17 bits.  However, in
+         order to preserve backwards compatibility with bitstreams
+         produced by versions pre-1.0.3, the decompressor must still
+         handle lengths of up to 20. */
+
+      for (i = 1; i <= alphaSize; i++) {
          j = weight[i] >> 8;
          j = 1 + (j / 2);
          weight[i] = j << 8;
diff --git a/manual.texi b/manual.texi
deleted file mode 100644
index 5bc27d5..0000000
--- a/manual.texi
+++ /dev/null
@@ -1,2243 +0,0 @@
-\input texinfo  @c                                  -*- Texinfo -*-
-@setfilename bzip2.info
-
-@ignore
-This file documents bzip2 version 1.0.2, and associated library
-libbzip2, written by Julian Seward (jseward@acm.org).
-
-Copyright (C) 1996-2002 Julian R Seward
-
-Permission is granted to make and distribute verbatim copies of
-this manual provided the copyright notice and this permission notice
-are preserved on all copies.
-
-Permission is granted to copy and distribute translations of this manual
-into another language, under the above conditions for verbatim copies.
-@end ignore
-
-@ifinfo
-@format
-START-INFO-DIR-ENTRY
-* Bzip2: (bzip2).		A program and library for data compression.
-END-INFO-DIR-ENTRY
-@end format
-
-@end ifinfo
-
-@iftex
-@c @finalout
-@settitle bzip2 and libbzip2
-@titlepage
-@title bzip2 and libbzip2
-@subtitle a program and library for data compression
-@subtitle copyright (C) 1996-2002 Julian Seward
-@subtitle version 1.0.2 of 30 December 2001
-@author Julian Seward
-
-@end titlepage
-
-@parindent 0mm
-@parskip 2mm
-
-@end iftex
-@node Top,,, (dir)
-
-The following text is the License for this software.  You should
-find it identical to that contained in the file LICENSE in the 
-source distribution.
-
-@bf{------------------ START OF THE LICENSE ------------------}
-
-This program, @code{bzip2}, 
-and associated library @code{libbzip2}, are
-Copyright (C) 1996-2002 Julian R Seward.  All rights reserved.
-
-Redistribution and use in source and binary forms, with or without
-modification, are permitted provided that the following conditions
-are met:
-@itemize @bullet
-@item
-   Redistributions of source code must retain the above copyright
-   notice, this list of conditions and the following disclaimer.
-@item
-   The origin of this software must not be misrepresented; you must 
-   not claim that you wrote the original software.  If you use this 
-   software in a product, an acknowledgment in the product 
-   documentation would be appreciated but is not required.
-@item
-   Altered source versions must be plainly marked as such, and must
-   not be misrepresented as being the original software.
-@item
-   The name of the author may not be used to endorse or promote 
-   products derived from this software without specific prior written 
-   permission.
-@end itemize
-THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
-OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
-DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
-DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
-GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
-WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
-NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
-SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-Julian Seward, Cambridge, UK.
-
-@code{jseward@@acm.org}
-
-@code{bzip2}/@code{libbzip2} version 1.0.2 of 30 December 2001.
-
-@bf{------------------ END OF THE LICENSE ------------------}
-
-Web sites:
-
-@code{http://sources.redhat.com/bzip2}
-
-@code{http://www.cacheprof.org}
-
-PATENTS: To the best of my knowledge, @code{bzip2} does not use any patented
-algorithms.  However, I do not have the resources available to carry out
-a full patent search.  Therefore I cannot give any guarantee of the
-above statement.
-
-
-
-
-
-
-
-@chapter Introduction
-
-@code{bzip2}  compresses  files  using the Burrows-Wheeler 
-block-sorting text compression algorithm,  and  Huffman  coding.
-Compression  is  generally  considerably  better than that
-achieved by more conventional LZ77/LZ78-based compressors,
-and  approaches  the performance of the PPM family of statistical compressors.
-
-@code{bzip2} is built on top of @code{libbzip2}, a flexible library
-for handling compressed data in the @code{bzip2} format.  This manual
-describes both how to use the program and 
-how to work with the library interface.  Most of the
-manual is devoted to this library, not the program, 
-which is good news if your interest is only in the program.
-
-Chapter 2 describes how to use @code{bzip2}; this is the only part 
-you need to read if you just want to know how to operate the program.
-Chapter 3 describes the programming interfaces in detail, and
-Chapter 4 records some miscellaneous notes which I thought
-ought to be recorded somewhere.
-
-
-@chapter How to use @code{bzip2}
-
-This chapter contains a copy of the @code{bzip2} man page,
-and nothing else.
-
-@quotation
-
-@unnumberedsubsubsec NAME
-@itemize
-@item @code{bzip2}, @code{bunzip2}
-- a block-sorting file compressor, v1.0.2
-@item @code{bzcat} 
-- decompresses files to stdout
-@item @code{bzip2recover}
-- recovers data from damaged bzip2 files
-@end itemize
-
-@unnumberedsubsubsec SYNOPSIS
-@itemize
-@item @code{bzip2} [ -cdfkqstvzVL123456789 ] [ filenames ...  ]
-@item @code{bunzip2} [ -fkvsVL ] [ filenames ...  ]
-@item @code{bzcat} [ -s ] [ filenames ...  ]
-@item @code{bzip2recover} filename
-@end itemize
-
-@unnumberedsubsubsec DESCRIPTION
-
-@code{bzip2} compresses files using the Burrows-Wheeler block sorting
-text compression algorithm, and Huffman coding.  Compression is
-generally considerably better than that achieved by more conventional
-LZ77/LZ78-based compressors, and approaches the performance of the PPM
-family of statistical compressors.
-
-The command-line options are deliberately very similar to those of GNU
-@code{gzip}, but they are not identical.
-
-@code{bzip2} expects a list of file names to accompany the command-line
-flags.  Each file is replaced by a compressed version of itself, with
-the name @code{original_name.bz2}.  Each compressed file has the same
-modification date, permissions, and, when possible, ownership as the
-corresponding original, so that these properties can be correctly
-restored at decompression time.  File name handling is naive in the
-sense that there is no mechanism for preserving original file names,
-permissions, ownerships or dates in filesystems which lack these
-concepts, or have serious file name length restrictions, such as MS-DOS.
-
-@code{bzip2} and @code{bunzip2} will by default not overwrite existing
-files.  If you want this to happen, specify the @code{-f} flag.
-
-If no file names are specified, @code{bzip2} compresses from standard
-input to standard output.  In this case, @code{bzip2} will decline to
-write compressed output to a terminal, as this would be entirely
-incomprehensible and therefore pointless.
-
-@code{bunzip2} (or @code{bzip2 -d}) decompresses all
-specified files.  Files which were not created by @code{bzip2}
-will be detected and ignored, and a warning issued.  
-@code{bzip2} attempts to guess the filename for the decompressed file 
-from that of the compressed file as follows:
-@itemize
-@item @code{filename.bz2 } becomes @code{filename}
-@item @code{filename.bz  } becomes @code{filename}
-@item @code{filename.tbz2} becomes @code{filename.tar}
-@item @code{filename.tbz } becomes @code{filename.tar}
-@item @code{anyothername } becomes @code{anyothername.out}
-@end itemize
-If the file does not end in one of the recognised endings, 
-@code{.bz2}, @code{.bz}, 
-@code{.tbz2} or @code{.tbz}, @code{bzip2} complains that it cannot
-guess the name of the original file, and uses the original name
-with @code{.out} appended.
-
-As with compression, supplying no
-filenames causes decompression from standard input to standard output.
-
-@code{bunzip2} will correctly decompress a file which is the
-concatenation of two or more compressed files.  The result is the
-concatenation of the corresponding uncompressed files.  Integrity
-testing (@code{-t}) of concatenated compressed files is also supported.
-
-You can also compress or decompress files to the standard output by
-giving the @code{-c} flag.  Multiple files may be compressed and
-decompressed like this.  The resulting outputs are fed sequentially to
-stdout.  Compression of multiple files in this manner generates a stream
-containing multiple compressed file representations.  Such a stream
-can be decompressed correctly only by @code{bzip2} version 0.9.0 or
-later.  Earlier versions of @code{bzip2} will stop after decompressing
-the first file in the stream.
-
-@code{bzcat} (or @code{bzip2 -dc}) decompresses all specified files to
-the standard output.
-
-@code{bzip2} will read arguments from the environment variables
-@code{BZIP2} and @code{BZIP}, in that order, and will process them
-before any arguments read from the command line.  This gives a 
-convenient way to supply default arguments.
-
-Compression is always performed, even if the compressed file is slightly
-larger than the original.  Files of less than about one hundred bytes
-tend to get larger, since the compression mechanism has a constant
-overhead in the region of 50 bytes.  Random data (including the output
-of most file compressors) is coded at about 8.05 bits per byte, giving
-an expansion of around 0.5%.
-
-As a self-check for your protection, @code{bzip2} uses 32-bit CRCs to
-make sure that the decompressed version of a file is identical to the
-original.  This guards against corruption of the compressed data, and
-against undetected bugs in @code{bzip2} (hopefully very unlikely).  The
-chances of data corruption going undetected is microscopic, about one
-chance in four billion for each file processed.  Be aware, though, that
-the check occurs upon decompression, so it can only tell you that
-something is wrong.  It can't help you recover the original uncompressed
-data.  You can use @code{bzip2recover} to try to recover data from
-damaged files.
-
-Return values: 0 for a normal exit, 1 for environmental problems (file
-not found, invalid flags, I/O errors, &c), 2 to indicate a corrupt
-compressed file, 3 for an internal consistency error (eg, bug) which
-caused @code{bzip2} to panic.
-
-
-@unnumberedsubsubsec OPTIONS
-@table @code
-@item -c  --stdout
-Compress or decompress to standard output.
-@item -d  --decompress
-Force decompression.  @code{bzip2}, @code{bunzip2} and @code{bzcat} are
-really the same program, and the decision about what actions to take is
-done on the basis of which name is used.  This flag overrides that
-mechanism, and forces bzip2 to decompress.
-@item -z --compress
-The complement to @code{-d}: forces compression, regardless of the
-invokation name.
-@item -t --test
-Check integrity of the specified file(s), but don't decompress them.
-This really performs a trial decompression and throws away the result.
-@item -f --force
-Force overwrite of output files.  Normally, @code{bzip2} will not overwrite
-existing output files.  Also forces @code{bzip2} to break hard links
-to files, which it otherwise wouldn't do.
-
-@code{bzip2} normally declines to decompress files which don't have the
-correct magic header bytes.  If forced (@code{-f}), however, it will
-pass such files through unmodified.  This is how GNU @code{gzip}
-behaves.
-@item -k --keep
-Keep (don't delete) input files during compression
-or decompression.
-@item -s --small
-Reduce memory usage, for compression, decompression and testing.  Files
-are decompressed and tested using a modified algorithm which only
-requires 2.5 bytes per block byte.  This means any file can be
-decompressed in 2300k of memory, albeit at about half the normal speed.
-
-During compression, @code{-s} selects a block size of 200k, which limits
-memory use to around the same figure, at the expense of your compression
-ratio.  In short, if your machine is low on memory (8 megabytes or
-less), use -s for everything.  See MEMORY MANAGEMENT below.
-@item -q --quiet
-Suppress non-essential warning messages.  Messages pertaining to
-I/O errors and other critical events will not be suppressed.
-@item -v --verbose
-Verbose mode -- show the compression ratio for each file processed.
-Further @code{-v}'s increase the verbosity level, spewing out lots of
-information which is primarily of interest for diagnostic purposes.
-@item -L --license -V --version
-Display the software version, license terms and conditions.
-@item -1 (or --fast) to -9 (or --best)
-Set the block size to 100 k, 200 k ..  900 k when compressing.  Has no
-effect when decompressing.  See MEMORY MANAGEMENT below.
-The @code{--fast} and @code{--best} aliases are primarily for GNU
-@code{gzip} compatibility.  In particular, @code{--fast} doesn't make
-things significantly faster.  And @code{--best} merely selects the
-default behaviour.
-@item --
-Treats all subsequent arguments as file names, even if they start
-with a dash.  This is so you can handle files with names beginning
-with a dash, for example: @code{bzip2 -- -myfilename}.
-@item --repetitive-fast 
-@item --repetitive-best
-These flags are redundant in versions 0.9.5 and above.  They provided
-some coarse control over the behaviour of the sorting algorithm in
-earlier versions, which was sometimes useful.  0.9.5 and above have an
-improved algorithm which renders these flags irrelevant.
-@end table
-
-
-@unnumberedsubsubsec MEMORY MANAGEMENT
-
-@code{bzip2} compresses large files in blocks.  The block size affects
-both the compression ratio achieved, and the amount of memory needed for
-compression and decompression.  The flags @code{-1} through @code{-9}
-specify the block size to be 100,000 bytes through 900,000 bytes (the
-default) respectively.  At decompression time, the block size used for
-compression is read from the header of the compressed file, and
-@code{bunzip2} then allocates itself just enough memory to decompress
-the file.  Since block sizes are stored in compressed files, it follows
-that the flags @code{-1} to @code{-9} are irrelevant to and so ignored
-during decompression.
-
-Compression and decompression requirements, in bytes, can be estimated
-as:
-@example
-     Compression:   400k + ( 8 x block size )
-
-     Decompression: 100k + ( 4 x block size ), or
-                    100k + ( 2.5 x block size )
-@end example
-Larger block sizes give rapidly diminishing marginal returns.  Most of
-the compression comes from the first two or three hundred k of block
-size, a fact worth bearing in mind when using @code{bzip2} on small machines.
-It is also important to appreciate that the decompression memory
-requirement is set at compression time by the choice of block size.
-
-For files compressed with the default 900k block size, @code{bunzip2}
-will require about 3700 kbytes to decompress.  To support decompression
-of any file on a 4 megabyte machine, @code{bunzip2} has an option to
-decompress using approximately half this amount of memory, about 2300
-kbytes.  Decompression speed is also halved, so you should use this
-option only where necessary.  The relevant flag is @code{-s}.
-
-In general, try and use the largest block size memory constraints allow,
-since that maximises the compression achieved.  Compression and
-decompression speed are virtually unaffected by block size.
-
-Another significant point applies to files which fit in a single block
--- that means most files you'd encounter using a large block size.  The
-amount of real memory touched is proportional to the size of the file,
-since the file is smaller than a block.  For example, compressing a file
-20,000 bytes long with the flag @code{-9} will cause the compressor to
-allocate around 7600k of memory, but only touch 400k + 20000 * 8 = 560
-kbytes of it.  Similarly, the decompressor will allocate 3700k but only
-touch 100k + 20000 * 4 = 180 kbytes.
-
-Here is a table which summarises the maximum memory usage for different
-block sizes.  Also recorded is the total compressed size for 14 files of
-the Calgary Text Compression Corpus totalling 3,141,622 bytes.  This
-column gives some feel for how compression varies with block size.
-These figures tend to understate the advantage of larger block sizes for
-larger files, since the Corpus is dominated by smaller files.
-@example
-          Compress   Decompress   Decompress   Corpus
-   Flag     usage      usage       -s usage     Size
-
-    -1      1200k       500k         350k      914704
-    -2      2000k       900k         600k      877703
-    -3      2800k      1300k         850k      860338
-    -4      3600k      1700k        1100k      846899
-    -5      4400k      2100k        1350k      845160
-    -6      5200k      2500k        1600k      838626
-    -7      6100k      2900k        1850k      834096
-    -8      6800k      3300k        2100k      828642
-    -9      7600k      3700k        2350k      828642
-@end example
-
-@unnumberedsubsubsec RECOVERING DATA FROM DAMAGED FILES
-
-@code{bzip2} compresses files in blocks, usually 900kbytes long.  Each
-block is handled independently.  If a media or transmission error causes
-a multi-block @code{.bz2} file to become damaged, it may be possible to
-recover data from the undamaged blocks in the file.
-
-The compressed representation of each block is delimited by a 48-bit
-pattern, which makes it possible to find the block boundaries with
-reasonable certainty.  Each block also carries its own 32-bit CRC, so
-damaged blocks can be distinguished from undamaged ones.
-
-@code{bzip2recover} is a simple program whose purpose is to search for
-blocks in @code{.bz2} files, and write each block out into its own
-@code{.bz2} file.  You can then use @code{bzip2 -t} to test the
-integrity of the resulting files, and decompress those which are
-undamaged.
-
-@code{bzip2recover} 
-takes a single argument, the name of the damaged file, and writes a
-number of files @code{rec00001file.bz2}, @code{rec00002file.bz2}, etc,
-containing the extracted blocks.  The output filenames are designed so
-that the use of wildcards in subsequent processing -- for example,
-@code{bzip2 -dc rec*file.bz2 > recovered_data} -- processes the files in
-the correct order.
-
-@code{bzip2recover} should be of most use dealing with large @code{.bz2}
-files, as these will contain many blocks.  It is clearly futile to use
-it on damaged single-block files, since a damaged block cannot be
-recovered.  If you wish to minimise any potential data loss through
-media or transmission errors, you might consider compressing with a
-smaller block size.
-
-
-@unnumberedsubsubsec PERFORMANCE NOTES
-
-The sorting phase of compression gathers together similar strings in the
-file.  Because of this, files containing very long runs of repeated
-symbols, like "aabaabaabaab ..."  (repeated several hundred times) may
-compress more slowly than normal.  Versions 0.9.5 and above fare much
-better than previous versions in this respect.  The ratio between
-worst-case and average-case compression time is in the region of 10:1.
-For previous versions, this figure was more like 100:1.  You can use the
-@code{-vvvv} option to monitor progress in great detail, if you want.
-
-Decompression speed is unaffected by these phenomena.
-
-@code{bzip2} usually allocates several megabytes of memory to operate
-in, and then charges all over it in a fairly random fashion.  This means
-that performance, both for compressing and decompressing, is largely
-determined by the speed at which your machine can service cache misses.
-Because of this, small changes to the code to reduce the miss rate have
-been observed to give disproportionately large performance improvements.
-I imagine @code{bzip2} will perform best on machines with very large
-caches.
-
-
-@unnumberedsubsubsec CAVEATS
-
-I/O error messages are not as helpful as they could be.  @code{bzip2}
-tries hard to detect I/O errors and exit cleanly, but the details of
-what the problem is sometimes seem rather misleading.
-
-This manual page pertains to version 1.0.2 of @code{bzip2}.  Compressed
-data created by this version is entirely forwards and backwards
-compatible with the previous public releases, versions 0.1pl2, 0.9.0,
-0.9.5, 1.0.0 and 1.0.1, but with the following exception: 0.9.0 and
-above can correctly decompress multiple concatenated compressed files.
-0.1pl2 cannot do this; it will stop after decompressing just the first
-file in the stream.
-
-@code{bzip2recover} versions prior to this one, 1.0.2, used 32-bit
-integers to represent bit positions in compressed files, so it could not
-handle compressed files more than 512 megabytes long.  Version 1.0.2 and
-above uses 64-bit ints on some platforms which support them (GNU
-supported targets, and Windows).  To establish whether or not
-@code{bzip2recover} was built with such a limitation, run it without
-arguments.  In any event you can build yourself an unlimited version if
-you can recompile it with @code{MaybeUInt64} set to be an unsigned
-64-bit integer.
-
-
-
-@unnumberedsubsubsec AUTHOR
-Julian Seward, @code{jseward@@acm.org}.
-
-@code{http://sources.redhat.com/bzip2}
-
-The ideas embodied in @code{bzip2} are due to (at least) the following
-people: Michael Burrows and David Wheeler (for the block sorting
-transformation), David Wheeler (again, for the Huffman coder), Peter
-Fenwick (for the structured coding model in the original @code{bzip},
-and many refinements), and Alistair Moffat, Radford Neal and Ian Witten
-(for the arithmetic coder in the original @code{bzip}).  I am much
-indebted for their help, support and advice.  See the manual in the
-source distribution for pointers to sources of documentation.  Christian
-von Roques encouraged me to look for faster sorting algorithms, so as to
-speed up compression.  Bela Lubkin encouraged me to improve the
-worst-case compression performance.  The @code{bz*} scripts are derived
-from those of GNU @code{gzip}.  Many people sent patches, helped with
-portability problems, lent machines, gave advice and were generally
-helpful.
-
-@end quotation
-
-
-
-
-@chapter Programming with @code{libbzip2}
-
-This chapter describes the programming interface to @code{libbzip2}.
-
-For general background information, particularly about memory
-use and performance aspects, you'd be well advised to read Chapter 2
-as well.
-
-@section Top-level structure
-
-@code{libbzip2} is a flexible library for compressing and decompressing
-data in the @code{bzip2} data format.  Although packaged as a single
-entity, it helps to regard the library as three separate parts: the low
-level interface, and the high level interface, and some utility
-functions.
-
-The structure of @code{libbzip2}'s interfaces is similar to
-that of Jean-loup Gailly's and Mark Adler's excellent @code{zlib} 
-library.
-
-All externally visible symbols have names beginning @code{BZ2_}.
-This is new in version 1.0.  The intention is to minimise pollution
-of the namespaces of library clients.
-
-@subsection Low-level summary
-
-This interface provides services for compressing and decompressing
-data in memory.  There's no provision for dealing with files, streams
-or any other I/O mechanisms, just straight memory-to-memory work.
-In fact, this part of the library can be compiled without inclusion
-of @code{stdio.h}, which may be helpful for embedded applications.
-
-The low-level part of the library has no global variables and
-is therefore thread-safe.
-
-Six routines make up the low level interface: 
-@code{BZ2_bzCompressInit}, @code{BZ2_bzCompress}, and @* @code{BZ2_bzCompressEnd}
-for compression,
-and a corresponding trio @code{BZ2_bzDecompressInit}, @* @code{BZ2_bzDecompress}
-and @code{BZ2_bzDecompressEnd} for decompression.  
-The @code{*Init} functions allocate
-memory for compression/decompression and do other
-initialisations, whilst the @code{*End} functions close down operations
-and release memory.
-
-The real work is done by @code{BZ2_bzCompress} and @code{BZ2_bzDecompress}.  
-These compress and decompress data from a user-supplied input buffer
-to a user-supplied output buffer.  These buffers can be any size;
-arbitrary quantities of data are handled by making repeated calls
-to these functions.  This is a flexible mechanism allowing a 
-consumer-pull style of activity, or producer-push, or a mixture of
-both.
-
-
-
-@subsection High-level summary
-
-This interface provides some handy wrappers around the low-level
-interface to facilitate reading and writing @code{bzip2} format
-files (@code{.bz2} files).  The routines provide hooks to facilitate
-reading files in which the @code{bzip2} data stream is embedded 
-within some larger-scale file structure, or where there are
-multiple @code{bzip2} data streams concatenated end-to-end.
-
-For reading files, @code{BZ2_bzReadOpen}, @code{BZ2_bzRead},
-@code{BZ2_bzReadClose} and @* @code{BZ2_bzReadGetUnused} are supplied.  For
-writing files, @code{BZ2_bzWriteOpen}, @code{BZ2_bzWrite} and
-@code{BZ2_bzWriteFinish} are available.
-
-As with the low-level library, no global variables are used
-so the library is per se thread-safe.  However, if I/O errors
-occur whilst reading or writing the underlying compressed files,
-you may have to consult @code{errno} to determine the cause of
-the error.  In that case, you'd need a C library which correctly
-supports @code{errno} in a multithreaded environment.
-
-To make the library a little simpler and more portable,
-@code{BZ2_bzReadOpen} and @code{BZ2_bzWriteOpen} require you to pass them file
-handles (@code{FILE*}s) which have previously been opened for reading or
-writing respectively.  That avoids portability problems associated with
-file operations and file attributes, whilst not being much of an
-imposition on the programmer.
-
-
-
-@subsection Utility functions summary
-For very simple needs, @code{BZ2_bzBuffToBuffCompress} and
-@code{BZ2_bzBuffToBuffDecompress} are provided.  These compress
-data in memory from one buffer to another buffer in a single
-function call.  You should assess whether these functions
-fulfill your memory-to-memory compression/decompression
-requirements before investing effort in understanding the more
-general but more complex low-level interface.
-
-Yoshioka Tsuneo (@code{QWF00133@@niftyserve.or.jp} /
-@code{tsuneo-y@@is.aist-nara.ac.jp}) has contributed some functions to
-give better @code{zlib} compatibility.  These functions are
-@code{BZ2_bzopen}, @code{BZ2_bzread}, @code{BZ2_bzwrite}, @code{BZ2_bzflush},
-@code{BZ2_bzclose},
-@code{BZ2_bzerror} and @code{BZ2_bzlibVersion}.  You may find these functions
-more convenient for simple file reading and writing, than those in the
-high-level interface.  These functions are not (yet) officially part of
-the library, and are minimally documented here.  If they break, you
-get to keep all the pieces.  I hope to document them properly when time
-permits.
-
-Yoshioka also contributed modifications to allow the library to be
-built as a Windows DLL.
-
-
-@section Error handling
-
-The library is designed to recover cleanly in all situations, including
-the worst-case situation of decompressing random data.  I'm not 
-100% sure that it can always do this, so you might want to add
-a signal handler to catch segmentation violations during decompression
-if you are feeling especially paranoid.  I would be interested in
-hearing more about the robustness of the library to corrupted
-compressed data.
-
-Version 1.0 is much more robust in this respect than
-0.9.0 or 0.9.5.  Investigations with Checker (a tool for 
-detecting problems with memory management, similar to Purify)
-indicate that, at least for the few files I tested, all single-bit
-errors in the decompressed data are caught properly, with no
-segmentation faults, no reads of uninitialised data and no 
-out of range reads or writes.  So it's certainly much improved,
-although I wouldn't claim it to be totally bombproof.
-
-The file @code{bzlib.h} contains all definitions needed to use
-the library.  In particular, you should definitely not include
-@code{bzlib_private.h}.
-
-In @code{bzlib.h}, the various return values are defined.  The following
-list is not intended as an exhaustive description of the circumstances 
-in which a given value may be returned -- those descriptions are given
-later.  Rather, it is intended to convey the rough meaning of each
-return value.  The first five actions are normal and not intended to 
-denote an error situation.
-@table @code
-@item BZ_OK
-The requested action was completed successfully.
-@item BZ_RUN_OK
-@itemx BZ_FLUSH_OK
-@itemx BZ_FINISH_OK
-In @code{BZ2_bzCompress}, the requested flush/finish/nothing-special action
-was completed successfully.
-@item BZ_STREAM_END
-Compression of data was completed, or the logical stream end was
-detected during decompression.
-@end table
-
-The following return values indicate an error of some kind.
-@table @code
-@item BZ_CONFIG_ERROR
-Indicates that the library has been improperly compiled on your
-platform -- a major configuration error.  Specifically, it means
-that @code{sizeof(char)}, @code{sizeof(short)} and @code{sizeof(int)}
-are not 1, 2 and 4 respectively, as they should be.  Note that the 
-library should still work properly on 64-bit platforms which follow
-the LP64 programming model -- that is, where @code{sizeof(long)}
-and @code{sizeof(void*)} are 8.  Under LP64, @code{sizeof(int)} is
-still 4, so @code{libbzip2}, which doesn't use the @code{long} type,
-is OK.
-@item BZ_SEQUENCE_ERROR
-When using the library, it is important to call the functions in the
-correct sequence and with data structures (buffers etc) in the correct
-states.  @code{libbzip2} checks as much as it can to ensure this is
-happening, and returns @code{BZ_SEQUENCE_ERROR} if not.  Code which
-complies precisely with the function semantics, as detailed below,
-should never receive this value; such an event denotes buggy code
-which you should investigate.
-@item BZ_PARAM_ERROR
-Returned when a parameter to a function call is out of range 
-or otherwise manifestly incorrect.  As with @code{BZ_SEQUENCE_ERROR},
-this denotes a bug in the client code.  The distinction between
-@code{BZ_PARAM_ERROR} and @code{BZ_SEQUENCE_ERROR} is a bit hazy, but still worth
-making.
-@item BZ_MEM_ERROR
-Returned when a request to allocate memory failed.  Note that the
-quantity of memory needed to decompress a stream cannot be determined
-until the stream's header has been read.  So @code{BZ2_bzDecompress} and
-@code{BZ2_bzRead} may return @code{BZ_MEM_ERROR} even though some of
-the compressed data has been read.  The same is not true for
-compression; once @code{BZ2_bzCompressInit} or @code{BZ2_bzWriteOpen} have
-successfully completed, @code{BZ_MEM_ERROR} cannot occur.
-@item BZ_DATA_ERROR
-Returned when a data integrity error is detected during decompression.
-Most importantly, this means when stored and computed CRCs for the
-data do not match.  This value is also returned upon detection of any
-other anomaly in the compressed data.
-@item BZ_DATA_ERROR_MAGIC
-As a special case of @code{BZ_DATA_ERROR}, it is sometimes useful to
-know when the compressed stream does not start with the correct
-magic bytes (@code{'B' 'Z' 'h'}).  
-@item BZ_IO_ERROR
-Returned by @code{BZ2_bzRead} and @code{BZ2_bzWrite} when there is an error
-reading or writing in the compressed file, and by @code{BZ2_bzReadOpen}
-and @code{BZ2_bzWriteOpen} for attempts to use a file for which the
-error indicator (viz, @code{ferror(f)}) is set.
-On receipt of @code{BZ_IO_ERROR}, the caller should consult
-@code{errno} and/or @code{perror} to acquire operating-system
-specific information about the problem.
-@item BZ_UNEXPECTED_EOF
-Returned by @code{BZ2_bzRead} when the compressed file finishes
-before the logical end of stream is detected.
-@item BZ_OUTBUFF_FULL
-Returned by @code{BZ2_bzBuffToBuffCompress} and
-@code{BZ2_bzBuffToBuffDecompress} to indicate that the output data
-will not fit into the output buffer provided.
-@end table
-
-
-
-@section Low-level interface
-
-@subsection @code{BZ2_bzCompressInit}
-@example
-typedef 
-   struct @{
-      char *next_in;
-      unsigned int avail_in;
-      unsigned int total_in_lo32;
-      unsigned int total_in_hi32;
-
-      char *next_out;
-      unsigned int avail_out;
-      unsigned int total_out_lo32;
-      unsigned int total_out_hi32;
-
-      void *state;
-
-      void *(*bzalloc)(void *,int,int);
-      void (*bzfree)(void *,void *);
-      void *opaque;
-   @} 
-   bz_stream;
-
-int BZ2_bzCompressInit ( bz_stream *strm, 
-                         int blockSize100k, 
-                         int verbosity,
-                         int workFactor );
-
-@end example
-
-Prepares for compression.  The @code{bz_stream} structure
-holds all data pertaining to the compression activity.  
-A @code{bz_stream} structure should be allocated and initialised
-prior to the call.
-The fields of @code{bz_stream}
-comprise the entirety of the user-visible data.  @code{state}
-is a pointer to the private data structures required for compression.
-
-Custom memory allocators are supported, via fields @code{bzalloc}, 
-@code{bzfree},
-and @code{opaque}.  The value 
-@code{opaque} is passed to as the first argument to
-all calls to @code{bzalloc} and @code{bzfree}, but is 
-otherwise ignored by the library.
-The call @code{bzalloc ( opaque, n, m )} is expected to return a 
-pointer @code{p} to
-@code{n * m} bytes of memory, and @code{bzfree ( opaque, p )} 
-should free
-that memory.
-
-If you don't want to use a custom memory allocator, set @code{bzalloc}, 
-@code{bzfree} and
-@code{opaque} to @code{NULL}, 
-and the library will then use the standard @code{malloc}/@code{free}
-routines.
-
-Before calling @code{BZ2_bzCompressInit}, fields @code{bzalloc}, 
-@code{bzfree} and @code{opaque} should
-be filled appropriately, as just described.  Upon return, the internal
-state will have been allocated and initialised, and @code{total_in_lo32}, 
-@code{total_in_hi32}, @code{total_out_lo32} and 
-@code{total_out_hi32} will have been set to zero.  
-These four fields are used by the library
-to inform the caller of the total amount of data passed into and out of
-the library, respectively.  You should not try to change them.
-As of version 1.0, 64-bit counts are maintained, even on 32-bit
-platforms, using the @code{_hi32} fields to store the upper 32 bits
-of the count.  So, for example, the total amount of data in
-is @code{(total_in_hi32 << 32) + total_in_lo32}.
-
-Parameter @code{blockSize100k} specifies the block size to be used for
-compression.  It should be a value between 1 and 9 inclusive, and the
-actual block size used is 100000 x this figure.  9 gives the best
-compression but takes most memory.
-
-Parameter @code{verbosity} should be set to a number between 0 and 4
-inclusive.  0 is silent, and greater numbers give increasingly verbose
-monitoring/debugging output.  If the library has been compiled with
-@code{-DBZ_NO_STDIO}, no such output will appear for any verbosity
-setting.
-
-Parameter @code{workFactor} controls how the compression phase behaves
-when presented with worst case, highly repetitive, input data.  If
-compression runs into difficulties caused by repetitive data, the
-library switches from the standard sorting algorithm to a fallback
-algorithm.  The fallback is slower than the standard algorithm by
-perhaps a factor of three, but always behaves reasonably, no matter how
-bad the input.
-
-Lower values of @code{workFactor} reduce the amount of effort the
-standard algorithm will expend before resorting to the fallback.  You
-should set this parameter carefully; too low, and many inputs will be
-handled by the fallback algorithm and so compress rather slowly, too
-high, and your average-to-worst case compression times can become very
-large.  The default value of 30 gives reasonable behaviour over a wide
-range of circumstances.
-
-Allowable values range from 0 to 250 inclusive.  0 is a special case,
-equivalent to using the default value of 30.
-
-Note that the compressed output generated is the same regardless of
-whether or not the fallback algorithm is used.
-
-Be aware also that this parameter may disappear entirely in future
-versions of the library.  In principle it should be possible to devise a
-good way to automatically choose which algorithm to use.  Such a
-mechanism would render the parameter obsolete.
-
-Possible return values:
-@display
-      @code{BZ_CONFIG_ERROR}
-         if the library has been mis-compiled
-      @code{BZ_PARAM_ERROR} 
-         if @code{strm} is @code{NULL} 
-         or @code{blockSize} < 1 or @code{blockSize} > 9
-         or @code{verbosity} < 0 or @code{verbosity} > 4
-         or @code{workFactor} < 0 or @code{workFactor} > 250
-      @code{BZ_MEM_ERROR} 
-         if not enough memory is available
-      @code{BZ_OK} 
-         otherwise
-@end display
-Allowable next actions:
-@display
-      @code{BZ2_bzCompress} 
-         if @code{BZ_OK} is returned
-      no specific action needed in case of error
-@end display
-
-@subsection @code{BZ2_bzCompress}
-@example
-   int BZ2_bzCompress ( bz_stream *strm, int action );
-@end example
-Provides more input and/or output buffer space for the library.  The
-caller maintains input and output buffers, and calls @code{BZ2_bzCompress} to
-transfer data between them.
-
-Before each call to @code{BZ2_bzCompress}, @code{next_in} should point at
-the data to be compressed, and @code{avail_in} should indicate how many
-bytes the library may read.  @code{BZ2_bzCompress} updates @code{next_in},
-@code{avail_in} and @code{total_in} to reflect the number of bytes it
-has read.
-
-Similarly, @code{next_out} should point to a buffer in which the
-compressed data is to be placed, with @code{avail_out} indicating how
-much output space is available.  @code{BZ2_bzCompress} updates
-@code{next_out}, @code{avail_out} and @code{total_out} to reflect the
-number of bytes output.
-
-You may provide and remove as little or as much data as you like on each
-call of @code{BZ2_bzCompress}.  In the limit, it is acceptable to supply and
-remove data one byte at a time, although this would be terribly
-inefficient.  You should always ensure that at least one byte of output
-space is available at each call.
-
-A second purpose of @code{BZ2_bzCompress} is to request a change of mode of the
-compressed stream.  
-
-Conceptually, a compressed stream can be in one of four states: IDLE,
-RUNNING, FLUSHING and FINISHING.  Before initialisation
-(@code{BZ2_bzCompressInit}) and after termination (@code{BZ2_bzCompressEnd}), a
-stream is regarded as IDLE.
-
-Upon initialisation (@code{BZ2_bzCompressInit}), the stream is placed in the
-RUNNING state.  Subsequent calls to @code{BZ2_bzCompress} should pass
-@code{BZ_RUN} as the requested action; other actions are illegal and
-will result in @code{BZ_SEQUENCE_ERROR}.
-
-At some point, the calling program will have provided all the input data
-it wants to.  It will then want to finish up -- in effect, asking the
-library to process any data it might have buffered internally.  In this
-state, @code{BZ2_bzCompress} will no longer attempt to read data from
-@code{next_in}, but it will want to write data to @code{next_out}.
-Because the output buffer supplied by the user can be arbitrarily small,
-the finishing-up operation cannot necessarily be done with a single call
-of @code{BZ2_bzCompress}.
-
-Instead, the calling program passes @code{BZ_FINISH} as an action to
-@code{BZ2_bzCompress}.  This changes the stream's state to FINISHING.  Any
-remaining input (ie, @code{next_in[0 .. avail_in-1]}) is compressed and
-transferred to the output buffer.  To do this, @code{BZ2_bzCompress} must be
-called repeatedly until all the output has been consumed.  At that
-point, @code{BZ2_bzCompress} returns @code{BZ_STREAM_END}, and the stream's
-state is set back to IDLE.  @code{BZ2_bzCompressEnd} should then be
-called.
-
-Just to make sure the calling program does not cheat, the library makes
-a note of @code{avail_in} at the time of the first call to
-@code{BZ2_bzCompress} which has @code{BZ_FINISH} as an action (ie, at the
-time the program has announced its intention to not supply any more
-input).  By comparing this value with that of @code{avail_in} over
-subsequent calls to @code{BZ2_bzCompress}, the library can detect any
-attempts to slip in more data to compress.  Any calls for which this is
-detected will return @code{BZ_SEQUENCE_ERROR}.  This indicates a
-programming mistake which should be corrected.
-
-Instead of asking to finish, the calling program may ask
-@code{BZ2_bzCompress} to take all the remaining input, compress it and
-terminate the current (Burrows-Wheeler) compression block.  This could
-be useful for error control purposes.  The mechanism is analogous to
-that for finishing: call @code{BZ2_bzCompress} with an action of
-@code{BZ_FLUSH}, remove output data, and persist with the
-@code{BZ_FLUSH} action until the value @code{BZ_RUN} is returned.  As
-with finishing, @code{BZ2_bzCompress} detects any attempt to provide more
-input data once the flush has begun.
-
-Once the flush is complete, the stream returns to the normal RUNNING
-state.
-
-This all sounds pretty complex, but isn't really.  Here's a table
-which shows which actions are allowable in each state, what action
-will be taken, what the next state is, and what the non-error return
-values are.  Note that you can't explicitly ask what state the
-stream is in, but nor do you need to -- it can be inferred from the
-values returned by @code{BZ2_bzCompress}.
-@display
-IDLE/@code{any}           
-      Illegal.  IDLE state only exists after @code{BZ2_bzCompressEnd} or
-      before @code{BZ2_bzCompressInit}.
-      Return value = @code{BZ_SEQUENCE_ERROR}
-
-RUNNING/@code{BZ_RUN}     
-      Compress from @code{next_in} to @code{next_out} as much as possible.
-      Next state = RUNNING
-      Return value = @code{BZ_RUN_OK}
-
-RUNNING/@code{BZ_FLUSH}   
-      Remember current value of @code{next_in}.  Compress from @code{next_in}
-      to @code{next_out} as much as possible, but do not accept any more input.  
-      Next state = FLUSHING
-      Return value = @code{BZ_FLUSH_OK}
-
-RUNNING/@code{BZ_FINISH}  
-      Remember current value of @code{next_in}.  Compress from @code{next_in}
-      to @code{next_out} as much as possible, but do not accept any more input.
-      Next state = FINISHING
-      Return value = @code{BZ_FINISH_OK}
-
-FLUSHING/@code{BZ_FLUSH}  
-      Compress from @code{next_in} to @code{next_out} as much as possible, 
-      but do not accept any more input.  
-      If all the existing input has been used up and all compressed
-      output has been removed
-         Next state = RUNNING; Return value = @code{BZ_RUN_OK}
-      else
-         Next state = FLUSHING; Return value = @code{BZ_FLUSH_OK}
-
-FLUSHING/other     
-      Illegal.
-      Return value = @code{BZ_SEQUENCE_ERROR}
-
-FINISHING/@code{BZ_FINISH}  
-      Compress from @code{next_in} to @code{next_out} as much as possible,
-      but to not accept any more input.  
-      If all the existing input has been used up and all compressed
-      output has been removed
-         Next state = IDLE; Return value = @code{BZ_STREAM_END}
-      else
-         Next state = FINISHING; Return value = @code{BZ_FINISHING}
-
-FINISHING/other
-      Illegal.
-      Return value = @code{BZ_SEQUENCE_ERROR}
-@end display
-
-That still looks complicated?  Well, fair enough.  The usual sequence
-of calls for compressing a load of data is:
-@itemize @bullet
-@item Get started with @code{BZ2_bzCompressInit}.
-@item Shovel data in and shlurp out its compressed form using zero or more
-calls of @code{BZ2_bzCompress} with action = @code{BZ_RUN}.
-@item Finish up.  
-Repeatedly call @code{BZ2_bzCompress} with action = @code{BZ_FINISH}, 
-copying out the compressed output, until @code{BZ_STREAM_END} is returned.
-@item Close up and go home.  Call @code{BZ2_bzCompressEnd}.
-@end itemize
-If the data you want to compress fits into your input buffer all
-at once, you can skip the calls of @code{BZ2_bzCompress ( ..., BZ_RUN )} and 
-just do the @code{BZ2_bzCompress ( ..., BZ_FINISH )} calls.
-
-All required memory is allocated by @code{BZ2_bzCompressInit}.  The
-compression library can accept any data at all (obviously).  So you
-shouldn't get any error return values from the @code{BZ2_bzCompress} calls.
-If you do, they will be @code{BZ_SEQUENCE_ERROR}, and indicate a bug in
-your programming.
-
-Trivial other possible return values:
-@display
-      @code{BZ_PARAM_ERROR}   
-         if @code{strm} is @code{NULL}, or @code{strm->s} is @code{NULL}
-@end display
-
-@subsection @code{BZ2_bzCompressEnd}
-@example
-int BZ2_bzCompressEnd ( bz_stream *strm );
-@end example
-Releases all memory associated with a compression stream.
-
-Possible return values:
-@display
-   @code{BZ_PARAM_ERROR}    if @code{strm} is @code{NULL} or @code{strm->s} is @code{NULL}
-   @code{BZ_OK}    otherwise
-@end display
-
-
-@subsection @code{BZ2_bzDecompressInit}
-@example
-int BZ2_bzDecompressInit ( bz_stream *strm, int verbosity, int small );
-@end example
-Prepares for decompression.  As with @code{BZ2_bzCompressInit}, a
-@code{bz_stream} record should be allocated and initialised before the
-call.  Fields @code{bzalloc}, @code{bzfree} and @code{opaque} should be
-set if a custom memory allocator is required, or made @code{NULL} for
-the normal @code{malloc}/@code{free} routines.  Upon return, the internal
-state will have been initialised, and @code{total_in} and
-@code{total_out} will be zero.
-
-For the meaning of parameter @code{verbosity}, see @code{BZ2_bzCompressInit}.
-
-If @code{small} is nonzero, the library will use an alternative
-decompression algorithm which uses less memory but at the cost of
-decompressing more slowly (roughly speaking, half the speed, but the
-maximum memory requirement drops to around 2300k).  See Chapter 2 for
-more information on memory management.
-
-Note that the amount of memory needed to decompress
-a stream cannot be determined until the stream's header has been read,
-so even if @code{BZ2_bzDecompressInit} succeeds, a subsequent
-@code{BZ2_bzDecompress} could fail with @code{BZ_MEM_ERROR}.
-
-Possible return values:
-@display
-      @code{BZ_CONFIG_ERROR}
-         if the library has been mis-compiled
-      @code{BZ_PARAM_ERROR}
-         if @code{(small != 0 && small != 1)}
-         or @code{(verbosity < 0 || verbosity > 4)}
-      @code{BZ_MEM_ERROR}
-         if insufficient memory is available
-@end display
-
-Allowable next actions:
-@display
-      @code{BZ2_bzDecompress}
-         if @code{BZ_OK} was returned
-      no specific action required in case of error
-@end display
-
- 
-
-@subsection @code{BZ2_bzDecompress}
-@example
-int BZ2_bzDecompress ( bz_stream *strm );
-@end example
-Provides more input and/out output buffer space for the library.  The
-caller maintains input and output buffers, and uses @code{BZ2_bzDecompress}
-to transfer data between them.
-
-Before each call to @code{BZ2_bzDecompress}, @code{next_in} 
-should point at the compressed data,
-and @code{avail_in} should indicate how many bytes the library
-may read.  @code{BZ2_bzDecompress} updates @code{next_in}, @code{avail_in} 
-and @code{total_in}
-to reflect the number of bytes it has read.
-
-Similarly, @code{next_out} should point to a buffer in which the uncompressed
-output is to be placed, with @code{avail_out} indicating how much output space
-is available.  @code{BZ2_bzCompress} updates @code{next_out},
-@code{avail_out} and @code{total_out} to reflect
-the number of bytes output.
-
-You may provide and remove as little or as much data as you like on
-each call of @code{BZ2_bzDecompress}.  
-In the limit, it is acceptable to
-supply and remove data one byte at a time, although this would be
-terribly inefficient.  You should always ensure that at least one
-byte of output space is available at each call.
-
-Use of @code{BZ2_bzDecompress} is simpler than @code{BZ2_bzCompress}.
-
-You should provide input and remove output as described above, and
-repeatedly call @code{BZ2_bzDecompress} until @code{BZ_STREAM_END} is
-returned.  Appearance of @code{BZ_STREAM_END} denotes that
-@code{BZ2_bzDecompress} has detected the logical end of the compressed
-stream.  @code{BZ2_bzDecompress} will not produce @code{BZ_STREAM_END} until
-all output data has been placed into the output buffer, so once
-@code{BZ_STREAM_END} appears, you are guaranteed to have available all
-the decompressed output, and @code{BZ2_bzDecompressEnd} can safely be
-called.
-
-If case of an error return value, you should call @code{BZ2_bzDecompressEnd}
-to clean up and release memory.
-
-Possible return values:
-@display
-      @code{BZ_PARAM_ERROR}
-         if @code{strm} is @code{NULL} or @code{strm->s} is @code{NULL}
-         or @code{strm->avail_out < 1}
-      @code{BZ_DATA_ERROR}
-         if a data integrity error is detected in the compressed stream
-      @code{BZ_DATA_ERROR_MAGIC}
-         if the compressed stream doesn't begin with the right magic bytes
-      @code{BZ_MEM_ERROR}
-         if there wasn't enough memory available
-      @code{BZ_STREAM_END}
-         if the logical end of the data stream was detected and all
-         output in has been consumed, eg @code{s->avail_out > 0}
-      @code{BZ_OK}
-         otherwise
-@end display
-Allowable next actions:
-@display
-      @code{BZ2_bzDecompress}
-         if @code{BZ_OK} was returned
-      @code{BZ2_bzDecompressEnd}
-         otherwise
-@end display
-
-
-@subsection @code{BZ2_bzDecompressEnd}
-@example
-int BZ2_bzDecompressEnd ( bz_stream *strm );
-@end example
-Releases all memory associated with a decompression stream.
-
-Possible return values:
-@display
-      @code{BZ_PARAM_ERROR}
-         if @code{strm} is @code{NULL} or @code{strm->s} is @code{NULL}
-      @code{BZ_OK}
-         otherwise
-@end display
-
-Allowable next actions:
-@display
-      None.
-@end display
-
-
-@section High-level interface
-
-This interface provides functions for reading and writing 
-@code{bzip2} format files.  First, some general points.
-
-@itemize @bullet
-@item All of the functions take an @code{int*} first argument,
-  @code{bzerror}.
-  After each call, @code{bzerror} should be consulted first to determine
-  the outcome of the call.  If @code{bzerror} is @code{BZ_OK}, 
-  the call completed
-  successfully, and only then should the return value of the function
-  (if any) be consulted.  If @code{bzerror} is @code{BZ_IO_ERROR}, 
-  there was an error
-  reading/writing the underlying compressed file, and you should
-  then consult @code{errno}/@code{perror} to determine the 
-  cause of the difficulty.
-  @code{bzerror} may also be set to various other values; precise details are
-  given on a per-function basis below.
-@item If @code{bzerror} indicates an error 
-  (ie, anything except @code{BZ_OK} and @code{BZ_STREAM_END}),
-  you should immediately call @code{BZ2_bzReadClose} (or @code{BZ2_bzWriteClose},
-  depending on whether you are attempting to read or to write)
-  to free up all resources associated
-  with the stream.  Once an error has been indicated, behaviour of all calls
-  except @code{BZ2_bzReadClose} (@code{BZ2_bzWriteClose}) is undefined.  
-  The implication is that (1) @code{bzerror} should
-  be checked after each call, and (2) if @code{bzerror} indicates an error, 
-  @code{BZ2_bzReadClose} (@code{BZ2_bzWriteClose}) should then be called to clean up.
-@item The @code{FILE*} arguments passed to
-   @code{BZ2_bzReadOpen}/@code{BZ2_bzWriteOpen}  
-  should be set to binary mode.
-  Most Unix systems will do this by default, but other platforms,
-  including Windows and Mac, will not.  If you omit this, you may
-  encounter problems when moving code to new platforms.
-@item Memory allocation requests are handled by
-  @code{malloc}/@code{free}.  
-  At present
-  there is no facility for user-defined memory allocators in the file I/O
-  functions (could easily be added, though).
-@end itemize
-
-
-
-@subsection @code{BZ2_bzReadOpen}
-@example
-   typedef void BZFILE;
-
-   BZFILE *BZ2_bzReadOpen ( int *bzerror, FILE *f, 
-                            int small, int verbosity,
-                            void *unused, int nUnused );
-@end example
-Prepare to read compressed data from file handle @code{f}.  @code{f}
-should refer to a file which has been opened for reading, and for which
-the error indicator (@code{ferror(f)})is not set.  If @code{small} is 1,
-the library will try to decompress using less memory, at the expense of
-speed.
-
-For reasons explained below, @code{BZ2_bzRead} will decompress the
-@code{nUnused} bytes starting at @code{unused}, before starting to read
-from the file @code{f}.  At most @code{BZ_MAX_UNUSED} bytes may be
-supplied like this.  If this facility is not required, you should pass
-@code{NULL} and @code{0} for @code{unused} and n@code{Unused}
-respectively.
-
-For the meaning of parameters @code{small} and @code{verbosity},
-see @code{BZ2_bzDecompressInit}.
-
-The amount of memory needed to decompress a file cannot be determined
-until the file's header has been read.  So it is possible that
-@code{BZ2_bzReadOpen} returns @code{BZ_OK} but a subsequent call of
-@code{BZ2_bzRead} will return @code{BZ_MEM_ERROR}.
-
-Possible assignments to @code{bzerror}:
-@display
-      @code{BZ_CONFIG_ERROR}
-         if the library has been mis-compiled
-      @code{BZ_PARAM_ERROR}
-         if @code{f} is @code{NULL} 
-         or @code{small} is neither @code{0} nor @code{1}                 
-         or @code{(unused == NULL && nUnused != 0)}
-         or @code{(unused != NULL && !(0 <= nUnused <= BZ_MAX_UNUSED))}
-      @code{BZ_IO_ERROR}    
-         if @code{ferror(f)} is nonzero
-      @code{BZ_MEM_ERROR}   
-         if insufficient memory is available
-      @code{BZ_OK}
-         otherwise.
-@end display
-
-Possible return values:
-@display
-      Pointer to an abstract @code{BZFILE}        
-         if @code{bzerror} is @code{BZ_OK}   
-      @code{NULL}
-         otherwise
-@end display
-
-Allowable next actions:
-@display
-      @code{BZ2_bzRead}
-         if @code{bzerror} is @code{BZ_OK}   
-      @code{BZ2_bzClose} 
-         otherwise
-@end display
-
-
-@subsection @code{BZ2_bzRead}
-@example
-   int BZ2_bzRead ( int *bzerror, BZFILE *b, void *buf, int len );
-@end example
-Reads up to @code{len} (uncompressed) bytes from the compressed file 
-@code{b} into
-the buffer @code{buf}.  If the read was successful, 
-@code{bzerror} is set to @code{BZ_OK}
-and the number of bytes read is returned.  If the logical end-of-stream
-was detected, @code{bzerror} will be set to @code{BZ_STREAM_END}, 
-and the number
-of bytes read is returned.  All other @code{bzerror} values denote an error.
-
-@code{BZ2_bzRead} will supply @code{len} bytes,
-unless the logical stream end is detected
-or an error occurs.  Because of this, it is possible to detect the 
-stream end by observing when the number of bytes returned is 
-less than the number
-requested.  Nevertheless, this is regarded as inadvisable; you should
-instead check @code{bzerror} after every call and watch out for
-@code{BZ_STREAM_END}.
-
-Internally, @code{BZ2_bzRead} copies data from the compressed file in chunks
-of size @code{BZ_MAX_UNUSED} bytes
-before decompressing it.  If the file contains more bytes than strictly
-needed to reach the logical end-of-stream, @code{BZ2_bzRead} will almost certainly
-read some of the trailing data before signalling @code{BZ_SEQUENCE_END}.
-To collect the read but unused data once @code{BZ_SEQUENCE_END} has 
-appeared, call @code{BZ2_bzReadGetUnused} immediately before @code{BZ2_bzReadClose}.
-
-Possible assignments to @code{bzerror}:
-@display
-      @code{BZ_PARAM_ERROR}
-         if @code{b} is @code{NULL} or @code{buf} is @code{NULL} or @code{len < 0}
-      @code{BZ_SEQUENCE_ERROR} 
-         if @code{b} was opened with @code{BZ2_bzWriteOpen}
-      @code{BZ_IO_ERROR} 
-         if there is an error reading from the compressed file
-      @code{BZ_UNEXPECTED_EOF} 
-         if the compressed file ended before the logical end-of-stream was detected
-      @code{BZ_DATA_ERROR} 
-         if a data integrity error was detected in the compressed stream
-      @code{BZ_DATA_ERROR_MAGIC}
-         if the stream does not begin with the requisite header bytes (ie, is not 
-         a @code{bzip2} data file).  This is really a special case of @code{BZ_DATA_ERROR}.
-      @code{BZ_MEM_ERROR} 
-         if insufficient memory was available
-      @code{BZ_STREAM_END} 
-         if the logical end of stream was detected.
-      @code{BZ_OK}
-         otherwise.
-@end display
-
-Possible return values:
-@display
-      number of bytes read
-         if @code{bzerror} is @code{BZ_OK} or @code{BZ_STREAM_END}
-      undefined
-         otherwise
-@end display
-
-Allowable next actions:
-@display
-      collect data from @code{buf}, then @code{BZ2_bzRead} or @code{BZ2_bzReadClose}
-         if @code{bzerror} is @code{BZ_OK} 
-      collect data from @code{buf}, then @code{BZ2_bzReadClose} or @code{BZ2_bzReadGetUnused} 
-         if @code{bzerror} is @code{BZ_SEQUENCE_END}   
-      @code{BZ2_bzReadClose} 
-         otherwise
-@end display
-
-
-
-@subsection @code{BZ2_bzReadGetUnused}
-@example
-   void BZ2_bzReadGetUnused ( int* bzerror, BZFILE *b, 
-                              void** unused, int* nUnused );
-@end example
-Returns data which was read from the compressed file but was not needed
-to get to the logical end-of-stream.  @code{*unused} is set to the address
-of the data, and @code{*nUnused} to the number of bytes.  @code{*nUnused} will
-be set to a value between @code{0} and @code{BZ_MAX_UNUSED} inclusive.
-
-This function may only be called once @code{BZ2_bzRead} has signalled 
-@code{BZ_STREAM_END} but before @code{BZ2_bzReadClose}.
-
-Possible assignments to @code{bzerror}:
-@display
-      @code{BZ_PARAM_ERROR} 
-         if @code{b} is @code{NULL} 
-         or @code{unused} is @code{NULL} or @code{nUnused} is @code{NULL}
-      @code{BZ_SEQUENCE_ERROR} 
-         if @code{BZ_STREAM_END} has not been signalled
-         or if @code{b} was opened with @code{BZ2_bzWriteOpen}
-     @code{BZ_OK}
-         otherwise
-@end display
-
-Allowable next actions:
-@display 
-      @code{BZ2_bzReadClose}
-@end display
-
-
-@subsection @code{BZ2_bzReadClose}
-@example
-   void BZ2_bzReadClose ( int *bzerror, BZFILE *b );
-@end example
-Releases all memory pertaining to the compressed file @code{b}.  
-@code{BZ2_bzReadClose} does not call @code{fclose} on the underlying file
-handle, so you should do that yourself if appropriate.
-@code{BZ2_bzReadClose} should be called to clean up after all error
-situations.
-
-Possible assignments to @code{bzerror}:
-@display
-      @code{BZ_SEQUENCE_ERROR} 
-         if @code{b} was opened with @code{BZ2_bzOpenWrite} 
-      @code{BZ_OK} 
-         otherwise
-@end display
-
-Allowable next actions:
-@display
-      none
-@end display
-
-
-
-@subsection @code{BZ2_bzWriteOpen}
-@example
-   BZFILE *BZ2_bzWriteOpen ( int *bzerror, FILE *f, 
-                             int blockSize100k, int verbosity,
-                             int workFactor );
-@end example
-Prepare to write compressed data to file handle @code{f}.  
-@code{f} should refer to
-a file which has been opened for writing, and for which the error
-indicator (@code{ferror(f)})is not set.  
-
-For the meaning of parameters @code{blockSize100k},
-@code{verbosity} and @code{workFactor}, see
-@* @code{BZ2_bzCompressInit}.
-
-All required memory is allocated at this stage, so if the call
-completes successfully, @code{BZ_MEM_ERROR} cannot be signalled by a
-subsequent call to @code{BZ2_bzWrite}.
-
-Possible assignments to @code{bzerror}:
-@display 
-      @code{BZ_CONFIG_ERROR}
-         if the library has been mis-compiled
-      @code{BZ_PARAM_ERROR} 
-         if @code{f} is @code{NULL} 
-         or @code{blockSize100k < 1} or @code{blockSize100k > 9}
-      @code{BZ_IO_ERROR} 
-         if @code{ferror(f)} is nonzero
-      @code{BZ_MEM_ERROR} 
-         if insufficient memory is available
-      @code{BZ_OK} 
-         otherwise
-@end display
-
-Possible return values:
-@display
-      Pointer to an abstract @code{BZFILE}  
-         if @code{bzerror} is @code{BZ_OK}   
-      @code{NULL} 
-         otherwise
-@end display
-
-Allowable next actions:
-@display
-      @code{BZ2_bzWrite} 
-         if @code{bzerror} is @code{BZ_OK} 
-         (you could go directly to @code{BZ2_bzWriteClose}, but this would be pretty pointless)
-      @code{BZ2_bzWriteClose} 
-         otherwise
-@end display
-
-
-
-@subsection @code{BZ2_bzWrite}
-@example
-   void BZ2_bzWrite ( int *bzerror, BZFILE *b, void *buf, int len );
-@end example
-Absorbs @code{len} bytes from the buffer @code{buf}, eventually to be
-compressed and written to the file.
-
-Possible assignments to @code{bzerror}:
-@display
-      @code{BZ_PARAM_ERROR} 
-         if @code{b} is @code{NULL} or @code{buf} is @code{NULL} or @code{len < 0}
-      @code{BZ_SEQUENCE_ERROR} 
-         if b was opened with @code{BZ2_bzReadOpen}
-      @code{BZ_IO_ERROR} 
-         if there is an error writing the compressed file.
-      @code{BZ_OK} 
-         otherwise
-@end display
-
-
-
-
-@subsection @code{BZ2_bzWriteClose}
-@example
-   void BZ2_bzWriteClose ( int *bzerror, BZFILE* f,
-                           int abandon,
-                           unsigned int* nbytes_in,
-                           unsigned int* nbytes_out );
-
-   void BZ2_bzWriteClose64 ( int *bzerror, BZFILE* f,
-                             int abandon,
-                             unsigned int* nbytes_in_lo32,
-                             unsigned int* nbytes_in_hi32,
-                             unsigned int* nbytes_out_lo32,
-                             unsigned int* nbytes_out_hi32 );
-@end example
-
-Compresses and flushes to the compressed file all data so far supplied
-by @code{BZ2_bzWrite}.  The logical end-of-stream markers are also written, so
-subsequent calls to @code{BZ2_bzWrite} are illegal.  All memory associated 
-with the compressed file @code{b} is released.  
-@code{fflush} is called on the
-compressed file, but it is not @code{fclose}'d.
-
-If @code{BZ2_bzWriteClose} is called to clean up after an error, the only
-action is to release the memory.  The library records the error codes
-issued by previous calls, so this situation will be detected
-automatically.  There is no attempt to complete the compression
-operation, nor to @code{fflush} the compressed file.  You can force this
-behaviour to happen even in the case of no error, by passing a nonzero
-value to @code{abandon}.
-
-If @code{nbytes_in} is non-null, @code{*nbytes_in} will be set to be the
-total volume of uncompressed data handled.  Similarly, @code{nbytes_out}
-will be set to the total volume of compressed data written.  For 
-compatibility with older versions of the library, @code{BZ2_bzWriteClose}
-only yields the lower 32 bits of these counts.  Use
-@code{BZ2_bzWriteClose64} if you want the full 64 bit counts.  These
-two functions are otherwise absolutely identical.
-
-
-Possible assignments to @code{bzerror}:
-@display
-      @code{BZ_SEQUENCE_ERROR} 
-         if @code{b} was opened with @code{BZ2_bzReadOpen}
-      @code{BZ_IO_ERROR} 
-         if there is an error writing the compressed file
-      @code{BZ_OK} 
-         otherwise
-@end display
-
-@subsection Handling embedded compressed data streams
-
-The high-level library facilitates use of
-@code{bzip2} data streams which form some part of a surrounding, larger
-data stream.
-@itemize @bullet
-@item For writing, the library takes an open file handle, writes
-compressed data to it, @code{fflush}es it but does not @code{fclose} it.
-The calling application can write its own data before and after the
-compressed data stream, using that same file handle.
-@item Reading is more complex, and the facilities are not as general
-as they could be since generality is hard to reconcile with efficiency.
-@code{BZ2_bzRead} reads from the compressed file in blocks of size
-@code{BZ_MAX_UNUSED} bytes, and in doing so probably will overshoot
-the logical end of compressed stream.
-To recover this data once decompression has
-ended, call @code{BZ2_bzReadGetUnused} after the last call of @code{BZ2_bzRead}
-(the one returning @code{BZ_STREAM_END}) but before calling
-@code{BZ2_bzReadClose}.
-@end itemize
-
-This mechanism makes it easy to decompress multiple @code{bzip2}
-streams placed end-to-end.  As the end of one stream, when @code{BZ2_bzRead}
-returns @code{BZ_STREAM_END}, call @code{BZ2_bzReadGetUnused} to collect the
-unused data (copy it into your own buffer somewhere).  
-That data forms the start of the next compressed stream.
-To start uncompressing that next stream, call @code{BZ2_bzReadOpen} again,
-feeding in the unused data via the @code{unused}/@code{nUnused}
-parameters.
-Keep doing this until @code{BZ_STREAM_END} return coincides with the
-physical end of file (@code{feof(f)}).  In this situation
-@code{BZ2_bzReadGetUnused}
-will of course return no data.
-
-This should give some feel for how the high-level interface can be used.
-If you require extra flexibility, you'll have to bite the bullet and get
-to grips with the low-level interface.
-
-@subsection Standard file-reading/writing code
-Here's how you'd write data to a compressed file:
-@example @code
-FILE*   f;
-BZFILE* b;
-int     nBuf;
-char    buf[ /* whatever size you like */ ];
-int     bzerror;
-int     nWritten;
-
-f = fopen ( "myfile.bz2", "w" );
-if (!f) @{
-   /* handle error */
-@}
-b = BZ2_bzWriteOpen ( &bzerror, f, 9 );
-if (bzerror != BZ_OK) @{
-   BZ2_bzWriteClose ( b );
-   /* handle error */
-@}
-
-while ( /* condition */ ) @{
-   /* get data to write into buf, and set nBuf appropriately */
-   nWritten = BZ2_bzWrite ( &bzerror, b, buf, nBuf );
-   if (bzerror == BZ_IO_ERROR) @{ 
-      BZ2_bzWriteClose ( &bzerror, b );
-      /* handle error */
-   @}
-@}
-
-BZ2_bzWriteClose ( &bzerror, b );
-if (bzerror == BZ_IO_ERROR) @{
-   /* handle error */
-@}
-@end example
-And to read from a compressed file:
-@example
-FILE*   f;
-BZFILE* b;
-int     nBuf;
-char    buf[ /* whatever size you like */ ];
-int     bzerror;
-int     nWritten;
-
-f = fopen ( "myfile.bz2", "r" );
-if (!f) @{
-   /* handle error */
-@}
-b = BZ2_bzReadOpen ( &bzerror, f, 0, NULL, 0 );
-if (bzerror != BZ_OK) @{
-   BZ2_bzReadClose ( &bzerror, b );
-   /* handle error */
-@}
-
-bzerror = BZ_OK;
-while (bzerror == BZ_OK && /* arbitrary other conditions */) @{
-   nBuf = BZ2_bzRead ( &bzerror, b, buf, /* size of buf */ );
-   if (bzerror == BZ_OK) @{
-      /* do something with buf[0 .. nBuf-1] */
-   @}
-@}
-if (bzerror != BZ_STREAM_END) @{
-   BZ2_bzReadClose ( &bzerror, b );
-   /* handle error */
-@} else @{
-   BZ2_bzReadClose ( &bzerror );
-@}
-@end example
-
-
-
-@section Utility functions
-@subsection @code{BZ2_bzBuffToBuffCompress}
-@example
-   int BZ2_bzBuffToBuffCompress( char*         dest,
-                                 unsigned int* destLen,
-                                 char*         source,
-                                 unsigned int  sourceLen,
-                                 int           blockSize100k,
-                                 int           verbosity,
-                                 int           workFactor );
-@end example
-Attempts to compress the data in @code{source[0 .. sourceLen-1]}
-into the destination buffer, @code{dest[0 .. *destLen-1]}.
-If the destination buffer is big enough, @code{*destLen} is
-set to the size of the compressed data, and @code{BZ_OK} is
-returned.  If the compressed data won't fit, @code{*destLen}
-is unchanged, and @code{BZ_OUTBUFF_FULL} is returned.
-
-Compression in this manner is a one-shot event, done with a single call
-to this function.  The resulting compressed data is a complete
-@code{bzip2} format data stream.  There is no mechanism for making
-additional calls to provide extra input data.  If you want that kind of
-mechanism, use the low-level interface.
-
-For the meaning of parameters @code{blockSize100k}, @code{verbosity}
-and @code{workFactor}, @* see @code{BZ2_bzCompressInit}.
-
-To guarantee that the compressed data will fit in its buffer, allocate
-an output buffer of size 1% larger than the uncompressed data, plus
-six hundred extra bytes.
-
-@code{BZ2_bzBuffToBuffDecompress} will not write data at or
-beyond @code{dest[*destLen]}, even in case of buffer overflow.
-
-Possible return values:
-@display
-      @code{BZ_CONFIG_ERROR}
-         if the library has been mis-compiled
-      @code{BZ_PARAM_ERROR} 
-         if @code{dest} is @code{NULL} or @code{destLen} is @code{NULL}
-         or @code{blockSize100k < 1} or @code{blockSize100k > 9}
-         or @code{verbosity < 0} or @code{verbosity > 4} 
-         or @code{workFactor < 0} or @code{workFactor > 250}
-      @code{BZ_MEM_ERROR}
-         if insufficient memory is available 
-      @code{BZ_OUTBUFF_FULL}
-         if the size of the compressed data exceeds @code{*destLen}
-      @code{BZ_OK} 
-         otherwise
-@end display
-
-
-
-@subsection @code{BZ2_bzBuffToBuffDecompress}
-@example
-   int BZ2_bzBuffToBuffDecompress ( char*         dest,
-                                    unsigned int* destLen,
-                                    char*         source,
-                                    unsigned int  sourceLen,
-                                    int           small,
-                                    int           verbosity );
-@end example
-Attempts to decompress the data in @code{source[0 .. sourceLen-1]}
-into the destination buffer, @code{dest[0 .. *destLen-1]}.
-If the destination buffer is big enough, @code{*destLen} is
-set to the size of the uncompressed data, and @code{BZ_OK} is
-returned.  If the compressed data won't fit, @code{*destLen}
-is unchanged, and @code{BZ_OUTBUFF_FULL} is returned.
-
-@code{source} is assumed to hold a complete @code{bzip2} format
-data stream.  @* @code{BZ2_bzBuffToBuffDecompress} tries to decompress
-the entirety of the stream into the output buffer.
-
-For the meaning of parameters @code{small} and @code{verbosity},
-see @code{BZ2_bzDecompressInit}.
-
-Because the compression ratio of the compressed data cannot be known in
-advance, there is no easy way to guarantee that the output buffer will
-be big enough.  You may of course make arrangements in your code to
-record the size of the uncompressed data, but such a mechanism is beyond
-the scope of this library.
-
-@code{BZ2_bzBuffToBuffDecompress} will not write data at or
-beyond @code{dest[*destLen]}, even in case of buffer overflow.
-
-Possible return values:
-@display
-      @code{BZ_CONFIG_ERROR}
-         if the library has been mis-compiled
-      @code{BZ_PARAM_ERROR} 
-         if @code{dest} is @code{NULL} or @code{destLen} is @code{NULL}
-         or @code{small != 0 && small != 1}
-         or @code{verbosity < 0} or @code{verbosity > 4} 
-      @code{BZ_MEM_ERROR}
-         if insufficient memory is available 
-      @code{BZ_OUTBUFF_FULL}
-         if the size of the compressed data exceeds @code{*destLen}
-      @code{BZ_DATA_ERROR}
-         if a data integrity error was detected in the compressed data
-      @code{BZ_DATA_ERROR_MAGIC}
-         if the compressed data doesn't begin with the right magic bytes
-      @code{BZ_UNEXPECTED_EOF}
-         if the compressed data ends unexpectedly
-      @code{BZ_OK} 
-         otherwise
-@end display
-
-
-
-@section @code{zlib} compatibility functions
-Yoshioka Tsuneo has contributed some functions to
-give better @code{zlib} compatibility.  These functions are
-@code{BZ2_bzopen}, @code{BZ2_bzread}, @code{BZ2_bzwrite}, @code{BZ2_bzflush},
-@code{BZ2_bzclose},
-@code{BZ2_bzerror} and @code{BZ2_bzlibVersion}.
-These functions are not (yet) officially part of
-the library.  If they break, you get to keep all the pieces.
-Nevertheless, I think they work ok.
-@example
-typedef void BZFILE;
-
-const char * BZ2_bzlibVersion ( void );
-@end example
-Returns a string indicating the library version.
-@example
-BZFILE * BZ2_bzopen  ( const char *path, const char *mode );
-BZFILE * BZ2_bzdopen ( int        fd,    const char *mode );
-@end example
-Opens a @code{.bz2} file for reading or writing, using either its name
-or a pre-existing file descriptor. 
-Analogous to @code{fopen} and @code{fdopen}.
-@example         
-int BZ2_bzread  ( BZFILE* b, void* buf, int len );
-int BZ2_bzwrite ( BZFILE* b, void* buf, int len );
-@end example
-Reads/writes data from/to a previously opened @code{BZFILE}.
-Analogous to @code{fread} and @code{fwrite}.
-@example
-int  BZ2_bzflush ( BZFILE* b );
-void BZ2_bzclose ( BZFILE* b );
-@end example
-Flushes/closes a @code{BZFILE}.  @code{BZ2_bzflush} doesn't actually do
-anything.  Analogous to @code{fflush} and @code{fclose}.
-
-@example 
-const char * BZ2_bzerror ( BZFILE *b, int *errnum )
-@end example
-Returns a string describing the more recent error status of
-@code{b}, and also sets @code{*errnum} to its numerical value.
-
-
-@section Using the library in a @code{stdio}-free environment
-
-@subsection Getting rid of @code{stdio}
-
-In a deeply embedded application, you might want to use just
-the memory-to-memory functions.  You can do this conveniently
-by compiling the library with preprocessor symbol @code{BZ_NO_STDIO}
-defined.  Doing this gives you a library containing only the following
-eight functions:
-
-@code{BZ2_bzCompressInit}, @code{BZ2_bzCompress}, @code{BZ2_bzCompressEnd} @*
-@code{BZ2_bzDecompressInit}, @code{BZ2_bzDecompress}, @code{BZ2_bzDecompressEnd} @*
-@code{BZ2_bzBuffToBuffCompress}, @code{BZ2_bzBuffToBuffDecompress}
-
-When compiled like this, all functions will ignore @code{verbosity}
-settings.
-
-@subsection Critical error handling
-@code{libbzip2} contains a number of internal assertion checks which
-should, needless to say, never be activated.  Nevertheless, if an
-assertion should fail, behaviour depends on whether or not the library
-was compiled with @code{BZ_NO_STDIO} set.
-
-For a normal compile, an assertion failure yields the message
-@example
-   bzip2/libbzip2: internal error number N.
-   This is a bug in bzip2/libbzip2, 1.0.2, 30-Dec-2001.
-   Please report it to me at: jseward@@acm.org.  If this happened
-   when you were using some program which uses libbzip2 as a
-   component, you should also report this bug to the author(s)
-   of that program.  Please make an effort to report this bug;
-   timely and accurate bug reports eventually lead to higher
-   quality software.  Thanks.  Julian Seward, 30 December 2001.
-@end example
-where @code{N} is some error code number.  If @code{N == 1007}, it also
-prints some extra text advising the reader that unreliable memory is
-often associated with internal error 1007.  (This is a
-frequently-observed-phenomenon with versions 1.0.0/1.0.1).
-
-@code{exit(3)} is then called.
-
-For a @code{stdio}-free library, assertion failures result
-in a call to a function declared as:
-@example
-   extern void bz_internal_error ( int errcode );
-@end example
-The relevant code is passed as a parameter.  You should supply
-such a function.
-
-In either case, once an assertion failure has occurred, any 
-@code{bz_stream} records involved can be regarded as invalid.
-You should not attempt to resume normal operation with them.
-
-You may, of course, change critical error handling to suit
-your needs.  As I said above, critical errors indicate bugs
-in the library and should not occur.  All "normal" error
-situations are indicated via error return codes from functions,
-and can be recovered from.
-
-
-@section Making a Windows DLL
-Everything related to Windows has been contributed by Yoshioka Tsuneo
-@* (@code{QWF00133@@niftyserve.or.jp} /
-@code{tsuneo-y@@is.aist-nara.ac.jp}), so you should send your queries to
-him (but perhaps Cc: me, @code{jseward@@acm.org}).
-
-My vague understanding of what to do is: using Visual C++ 5.0,
-open the project file @code{libbz2.dsp}, and build.  That's all.
-
-If you can't
-open the project file for some reason, make a new one, naming these files:
-@code{blocksort.c}, @code{bzlib.c}, @code{compress.c}, 
-@code{crctable.c}, @code{decompress.c}, @code{huffman.c}, @*
-@code{randtable.c} and @code{libbz2.def}.  You will also need
-to name the header files @code{bzlib.h} and @code{bzlib_private.h}.
-
-If you don't use VC++, you may need to define the proprocessor symbol
-@code{_WIN32}. 
-
-Finally, @code{dlltest.c} is a sample program using the DLL.  It has a
-project file, @code{dlltest.dsp}.
-
-If you just want a makefile for Visual C, have a look at
-@code{makefile.msc}.
-
-Be aware that if you compile @code{bzip2} itself on Win32, you must set
-@code{BZ_UNIX} to 0 and @code{BZ_LCCWIN32} to 1, in the file
-@code{bzip2.c}, before compiling.  Otherwise the resulting binary won't
-work correctly.
-
-I haven't tried any of this stuff myself, but it all looks plausible.
-
-
-
-@chapter Miscellanea
-
-These are just some random thoughts of mine.  Your mileage may
-vary.
-
-@section Limitations of the compressed file format
-@code{bzip2-1.0}, @code{0.9.5} and @code{0.9.0}
-use exactly the same file format as the previous
-version, @code{bzip2-0.1}.  This decision was made in the interests of
-stability.  Creating yet another incompatible compressed file format
-would create further confusion and disruption for users.
-
-Nevertheless, this is not a painless decision.  Development
-work since the release of @code{bzip2-0.1} in August 1997
-has shown complexities in the file format which slow down
-decompression and, in retrospect, are unnecessary.  These are:
-@itemize @bullet
-@item The run-length encoder, which is the first of the 
-      compression transformations, is entirely irrelevant.
-      The original purpose was to protect the sorting algorithm
-      from the very worst case input: a string of repeated
-      symbols.  But algorithm steps Q6a and Q6b in the original
-      Burrows-Wheeler technical report (SRC-124) show how
-      repeats can be handled without difficulty in block
-      sorting.
-@item The randomisation mechanism doesn't really need to be
-      there.  Udi Manber and Gene Myers published a suffix
-      array construction algorithm a few years back, which
-      can be employed to sort any block, no matter how 
-      repetitive, in O(N log N) time.  Subsequent work by
-      Kunihiko Sadakane has produced a derivative O(N (log N)^2) 
-      algorithm which usually outperforms the Manber-Myers
-      algorithm.
-
-      I could have changed to Sadakane's algorithm, but I find
-      it to be slower than @code{bzip2}'s existing algorithm for
-      most inputs, and the randomisation mechanism protects
-      adequately against bad cases.  I didn't think it was
-      a good tradeoff to make.  Partly this is due to the fact
-      that I was not flooded with email complaints about
-      @code{bzip2-0.1}'s performance on repetitive data, so
-      perhaps it isn't a problem for real inputs.
-
-      Probably the best long-term solution,
-      and the one I have incorporated into 0.9.5 and above,
-      is to use the existing sorting
-      algorithm initially, and fall back to a O(N (log N)^2)
-      algorithm if the standard algorithm gets into difficulties.
-@item The compressed file format was never designed to be
-      handled by a library, and I have had to jump though
-      some hoops to produce an efficient implementation of
-      decompression.  It's a bit hairy.  Try passing
-      @code{decompress.c} through the C preprocessor 
-      and you'll see what I mean.  Much of this complexity
-      could have been avoided if the compressed size of
-      each block of data was recorded in the data stream.
-@item An Adler-32 checksum, rather than a CRC32 checksum,
-      would be faster to compute.
-@end itemize
-It would be fair to say that the @code{bzip2} format was frozen
-before I properly and fully understood the performance
-consequences of doing so.
-
-Improvements which I was able to incorporate into
-0.9.0, despite using the same file format, are:
-@itemize @bullet
-@item Single array implementation of the inverse BWT.  This
-      significantly speeds up decompression, presumably
-      because it reduces the number of cache misses.
-@item Faster inverse MTF transform for large MTF values.  The
-      new implementation is based on the notion of sliding blocks
-      of values.
-@item @code{bzip2-0.9.0} now reads and writes files with @code{fread}
-      and @code{fwrite}; version 0.1 used @code{putc} and @code{getc}.
-      Duh!  Well, you live and learn.
-
-@end itemize
-Further ahead, it would be nice 
-to be able to do random access into files.  This will 
-require some careful design of compressed file formats.
-
-
-
-@section Portability issues
-After some consideration, I have decided not to use
-GNU @code{autoconf} to configure 0.9.5 or 1.0.
-
-@code{autoconf}, admirable and wonderful though it is, 
-mainly assists with portability problems between Unix-like
-platforms.  But @code{bzip2} doesn't have much in the way
-of portability problems on Unix; most of the difficulties appear
-when porting to the Mac, or to Microsoft's operating systems.
-@code{autoconf} doesn't help in those cases, and brings in a 
-whole load of new complexity.
-
-Most people should be able to compile the library and program
-under Unix straight out-of-the-box, so to speak, especially 
-if you have a version of GNU C available.
-
-There are a couple of @code{__inline__} directives in the code.  GNU C
-(@code{gcc}) should be able to handle them.  If you're not using
-GNU C, your C compiler shouldn't see them at all.
-If your compiler does, for some reason, see them and doesn't
-like them, just @code{#define} @code{__inline__} to be @code{/* */}.  One
-easy way to do this is to compile with the flag @code{-D__inline__=}, 
-which should be understood by most Unix compilers.
-
-If you still have difficulties, try compiling with the macro
-@code{BZ_STRICT_ANSI} defined.  This should enable you to build the
-library in a strictly ANSI compliant environment.  Building the program
-itself like this is dangerous and not supported, since you remove
-@code{bzip2}'s checks against compressing directories, symbolic links,
-devices, and other not-really-a-file entities.  This could cause
-filesystem corruption!
-
-One other thing: if you create a @code{bzip2} binary for public
-distribution, please try and link it statically (@code{gcc -s}).  This
-avoids all sorts of library-version issues that others may encounter
-later on.
-
-If you build @code{bzip2} on Win32, you must set @code{BZ_UNIX} to 0 and
-@code{BZ_LCCWIN32} to 1, in the file @code{bzip2.c}, before compiling.
-Otherwise the resulting binary won't work correctly.
-
-
-
-@section Reporting bugs
-I tried pretty hard to make sure @code{bzip2} is
-bug free, both by design and by testing.  Hopefully
-you'll never need to read this section for real.
-
-Nevertheless, if @code{bzip2} dies with a segmentation
-fault, a bus error or an internal assertion failure, it
-will ask you to email me a bug report.  Experience with
-version 0.1 shows that almost all these problems can
-be traced to either compiler bugs or hardware problems.
-@itemize @bullet
-@item
-Recompile the program with no optimisation, and see if it
-works.  And/or try a different compiler.
-I heard all sorts of stories about various flavours
-of GNU C (and other compilers) generating bad code for
-@code{bzip2}, and I've run across two such examples myself.
-
-2.7.X versions of GNU C are known to generate bad code from
-time to time, at high optimisation levels.  
-If you get problems, try using the flags
-@code{-O2} @code{-fomit-frame-pointer} @code{-fno-strength-reduce}.
-You should specifically @emph{not} use @code{-funroll-loops}.
-
-You may notice that the Makefile runs six tests as part of
-the build process.  If the program passes all of these, it's
-a pretty good (but not 100%) indication that the compiler has
-done its job correctly.
-@item
-If @code{bzip2} crashes randomly, and the crashes are not
-repeatable, you may have a flaky memory subsystem.  @code{bzip2}
-really hammers your memory hierarchy, and if it's a bit marginal,
-you may get these problems.  Ditto if your disk or I/O subsystem
-is slowly failing.  Yup, this really does happen.
-
-Try using a different machine of the same type, and see if
-you can repeat the problem.
-@item This isn't really a bug, but ... If @code{bzip2} tells
-you your file is corrupted on decompression, and you
-obtained the file via FTP, there is a possibility that you
-forgot to tell FTP to do a binary mode transfer.  That absolutely
-will cause the file to be non-decompressible.  You'll have to transfer
-it again.
-@end itemize
-
-If you've incorporated @code{libbzip2} into your own program
-and are getting problems, please, please, please, check that the 
-parameters you are passing in calls to the library, are
-correct, and in accordance with what the documentation says
-is allowable.  I have tried to make the library robust against
-such problems, but I'm sure I haven't succeeded.
-
-Finally, if the above comments don't help, you'll have to send
-me a bug report.  Now, it's just amazing how many people will 
-send me a bug report saying something like
-@display
-   bzip2 crashed with segmentation fault on my machine
-@end display
-and absolutely nothing else.  Needless to say, a such a report
-is @emph{totally, utterly, completely and comprehensively 100% useless; 
-a waste of your time, my time, and net bandwidth}.
-With no details at all, there's no way I can possibly begin
-to figure out what the problem is.
-
-The rules of the game are: facts, facts, facts.  Don't omit
-them because "oh, they won't be relevant".  At the bare 
-minimum:
-@display
-   Machine type.  Operating system version.  
-   Exact version of @code{bzip2} (do @code{bzip2 -V}).  
-   Exact version of the compiler used.  
-   Flags passed to the compiler.
-@end display
-However, the most important single thing that will help me is
-the file that you were trying to compress or decompress at the
-time the problem happened.  Without that, my ability to do anything
-more than speculate about the cause, is limited.
-
-Please remember that I connect to the Internet with a modem, so
-you should contact me before mailing me huge files.
-
-
-@section Did you get the right package?
-
-@code{bzip2} is a resource hog.  It soaks up large amounts of CPU cycles
-and memory.  Also, it gives very large latencies.  In the worst case, you
-can feed many megabytes of uncompressed data into the library before
-getting any compressed output, so this probably rules out applications
-requiring interactive behaviour.
-
-These aren't faults of my implementation, I hope, but more
-an intrinsic property of the Burrows-Wheeler transform (unfortunately).  
-Maybe this isn't what you want.
-
-If you want a compressor and/or library which is faster, uses less
-memory but gets pretty good compression, and has minimal latency,
-consider Jean-loup
-Gailly's and Mark Adler's work, @code{zlib-1.1.3} and
-@code{gzip-1.2.4}.  Look for them at
-
-@code{http://www.zlib.org} and
-@code{http://www.gzip.org} respectively.
-
-For something faster and lighter still, you might try Markus F X J
-Oberhumer's @code{LZO} real-time compression/decompression library, at
-@* @code{http://wildsau.idv.uni-linz.ac.at/mfx/lzo.html}.
-
-If you want to use the @code{bzip2} algorithms to compress small blocks
-of data, 64k bytes or smaller, for example on an on-the-fly disk
-compressor, you'd be well advised not to use this library.  Instead,
-I've made a special library tuned for that kind of use.  It's part of
-@code{e2compr-0.40}, an on-the-fly disk compressor for the Linux
-@code{ext2} filesystem.  Look at
-@code{http://www.netspace.net.au/~reiter/e2compr}.
-
-
-
-@section Testing
-
-A record of the tests I've done.
-
-First, some data sets:
-@itemize @bullet
-@item B: a directory containing 6001 files, one for every length in the
-      range 0 to 6000 bytes.  The files contain random lowercase
-      letters.  18.7 megabytes.
-@item H: my home directory tree.  Documents, source code, mail files,
-      compressed data.  H contains B, and also a directory of 
-      files designed as boundary cases for the sorting; mostly very
-      repetitive, nasty files.  565 megabytes.
-@item A: directory tree holding various applications built from source:
-      @code{egcs}, @code{gcc-2.8.1}, KDE, GTK, Octave, etc.
-      2200 megabytes.
-@end itemize
-The tests conducted are as follows.  Each test means compressing 
-(a copy of) each file in the data set, decompressing it and
-comparing it against the original.
-
-First, a bunch of tests with block sizes and internal buffer
-sizes set very small, 
-to detect any problems with the
-blocking and buffering mechanisms.  
-This required modifying the source code so as to try to 
-break it.
-@enumerate
-@item Data set H, with
-      buffer size of 1 byte, and block size of 23 bytes.
-@item Data set B, buffer sizes 1 byte, block size 1 byte.
-@item As (2) but small-mode decompression.
-@item As (2) with block size 2 bytes.
-@item As (2) with block size 3 bytes.
-@item As (2) with block size 4 bytes.
-@item As (2) with block size 5 bytes.
-@item As (2) with block size 6 bytes and small-mode decompression.
-@item H with buffer size of 1 byte, but normal block
-      size (up to 900000 bytes).
-@end enumerate
-Then some tests with unmodified source code.
-@enumerate
-@item H, all settings normal.
-@item As (1), with small-mode decompress.
-@item H, compress with flag @code{-1}.
-@item H, compress with flag @code{-s}, decompress with flag @code{-s}.
-@item Forwards compatibility: H, @code{bzip2-0.1pl2} compressing,
-      @code{bzip2-0.9.5} decompressing, all settings normal.
-@item Backwards compatibility:  H, @code{bzip2-0.9.5} compressing,
-      @code{bzip2-0.1pl2} decompressing, all settings normal.
-@item Bigger tests: A, all settings normal.
-@item As (7), using the fallback (Sadakane-like) sorting algorithm.
-@item As (8), compress with flag @code{-1}, decompress with flag
-      @code{-s}.
-@item H, using the fallback sorting algorithm.
-@item Forwards compatibility: A, @code{bzip2-0.1pl2} compressing,
-      @code{bzip2-0.9.5} decompressing, all settings normal.
-@item Backwards compatibility:  A, @code{bzip2-0.9.5} compressing,
-      @code{bzip2-0.1pl2} decompressing, all settings normal.
-@item Misc test: about 400 megabytes of @code{.tar} files with
-      @code{bzip2} compiled with Checker (a memory access error
-       detector, like Purify).
-@item Misc tests to make sure it builds and runs ok on non-Linux/x86
-      platforms.
-@end enumerate
-These tests were conducted on a 225 MHz IDT WinChip machine, running
-Linux 2.0.36.  They represent nearly a week of continuous computation.
-All tests completed successfully.
-
-
-@section Further reading
-@code{bzip2} is not research work, in the sense that it doesn't present
-any new ideas.  Rather, it's an engineering exercise based on existing
-ideas.
-
-Four documents describe essentially all the ideas behind @code{bzip2}:
-@example
-Michael Burrows and D. J. Wheeler:
-  "A block-sorting lossless data compression algorithm"
-   10th May 1994. 
-   Digital SRC Research Report 124.
-   ftp://ftp.digital.com/pub/DEC/SRC/research-reports/SRC-124.ps.gz
-   If you have trouble finding it, try searching at the
-   New Zealand Digital Library, http://www.nzdl.org.
-
-Daniel S. Hirschberg and Debra A. LeLewer
-  "Efficient Decoding of Prefix Codes"
-   Communications of the ACM, April 1990, Vol 33, Number 4.
-   You might be able to get an electronic copy of this
-      from the ACM Digital Library.
-
-David J. Wheeler
-   Program bred3.c and accompanying document bred3.ps.
-   This contains the idea behind the multi-table Huffman
-   coding scheme.
-   ftp://ftp.cl.cam.ac.uk/users/djw3/
-
-Jon L. Bentley and Robert Sedgewick
-  "Fast Algorithms for Sorting and Searching Strings"
-   Available from Sedgewick's web page,
-   www.cs.princeton.edu/~rs
-@end example
-The following paper gives valuable additional insights into the
-algorithm, but is not immediately the basis of any code
-used in bzip2.
-@example
-Peter Fenwick:
-   Block Sorting Text Compression
-   Proceedings of the 19th Australasian Computer Science Conference,
-     Melbourne, Australia.  Jan 31 - Feb 2, 1996.
-   ftp://ftp.cs.auckland.ac.nz/pub/peter-f/ACSC96paper.ps
-@end example
-Kunihiko Sadakane's sorting algorithm, mentioned above,
-is available from:
-@example
-http://naomi.is.s.u-tokyo.ac.jp/~sada/papers/Sada98b.ps.gz
-@end example
-The Manber-Myers suffix array construction
-algorithm is described in a paper
-available from:
-@example
-http://www.cs.arizona.edu/people/gene/PAPERS/suffix.ps
-@end example
-Finally, the following paper documents some recent investigations
-I made into the performance of sorting algorithms:
-@example
-Julian Seward:
-   On the Performance of BWT Sorting Algorithms
-   Proceedings of the IEEE Data Compression Conference 2000
-     Snowbird, Utah.  28-30 March 2000.
-@end example
-
-
-@contents
-
-@bye
-
diff --git a/manual.xml b/manual.xml
new file mode 100644
index 0000000..1ab5bd7
--- /dev/null
+++ b/manual.xml
@@ -0,0 +1,2966 @@
+ 
+
+ %common-ents;
+]>
+
+
+
+ 
+  bzip2 and libbzip2, version 1.0.3
+  A program and library for data compression
+  
+   &bz-lifespan;
+   Julian Seward
+  
+  Version &bz-version; of &bz-date;
+
+  
+   
+    Julian
+    Seward
+    
+     &bz-url;
+    
+   
+  
+
+  
+
+  This program, bzip2, the
+  associated library libbzip2, and
+  all documentation, are copyright © &bz-lifespan; Julian Seward.
+  All rights reserved.
+
+  Redistribution and use in source and binary forms, with
+  or without modification, are permitted provided that the
+  following conditions are met:
+
+  
+
+   Redistributions of source code must retain the
+   above copyright notice, this list of conditions and the
+   following disclaimer.
+
+   The origin of this software must not be
+   misrepresented; you must not claim that you wrote the original
+   software.  If you use this software in a product, an
+   acknowledgment in the product documentation would be
+   appreciated but is not required.
+
+   Altered source versions must be plainly marked
+   as such, and must not be misrepresented as being the original
+   software.
+
+   The name of the author may not be used to
+   endorse or promote products derived from this software without
+   specific prior written permission.
+
+  
+
+  THIS SOFTWARE IS PROVIDED BY THE AUTHOR "AS IS" AND ANY
+  EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+  THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+  PARTICULAR PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE
+  AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+  EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
+  TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+  DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+  ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+  LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
+  IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+  THE POSSIBILITY OF SUCH DAMAGE.
+
+ PATENTS: To the best of my knowledge,
+ bzip2 and
+ libbzip2 do not use any patented
+ algorithms.  However, I do not have the resources to carry
+ out a patent search.  Therefore I cannot give any guarantee of
+ the above statement.
+ 
+
+
+
+
+
+
+
+
+Introduction
+
+bzip2 compresses files
+using the Burrows-Wheeler block-sorting text compression
+algorithm, and Huffman coding.  Compression is generally
+considerably better than that achieved by more conventional
+LZ77/LZ78-based compressors, and approaches the performance of
+the PPM family of statistical compressors.
+
+bzip2 is built on top of
+libbzip2, a flexible library for
+handling compressed data in the
+bzip2 format.  This manual
+describes both how to use the program and how to work with the
+library interface.  Most of the manual is devoted to this
+library, not the program, which is good news if your interest is
+only in the program.
+
+
+
+  describes how to use
+ bzip2; this is the only part
+ you need to read if you just want to know how to operate the
+ program.
+
+  describes the
+ programming interfaces in detail, and
+
+  records some
+ miscellaneous notes which I thought ought to be recorded
+ somewhere.
+
+
+
+
+
+
+
+How to use bzip2
+
+This chapter contains a copy of the
+bzip2 man page, and nothing
+else.
+
+
+NAME
+
+
+
+ bzip2,
+  bunzip2 - a block-sorting file
+  compressor, v1.0.3
+
+ bzcat -
+   decompresses files to stdout
+
+ bzip2recover -
+   recovers data from damaged bzip2 files
+
+
+
+
+
+
+
+SYNOPSIS
+
+
+
+ bzip2 [
+  -cdfkqstvzVL123456789 ] [ filenames ...  ]
+
+ bunzip2 [
+  -fkvsVL ] [ filenames ...  ]
+
+ bzcat [ -s ] [
+  filenames ...  ]
+
+ bzip2recover
+  filename
+
+
+
+
+
+
+
+DESCRIPTION
+
+bzip2 compresses files
+using the Burrows-Wheeler block sorting text compression
+algorithm, and Huffman coding.  Compression is generally
+considerably better than that achieved by more conventional
+LZ77/LZ78-based compressors, and approaches the performance of
+the PPM family of statistical compressors.
+
+The command-line options are deliberately very similar to
+those of GNU gzip, but they are
+not identical.
+
+bzip2 expects a list of
+file names to accompany the command-line flags.  Each file is
+replaced by a compressed version of itself, with the name
+original_name.bz2.  Each
+compressed file has the same modification date, permissions, and,
+when possible, ownership as the corresponding original, so that
+these properties can be correctly restored at decompression time.
+File name handling is naive in the sense that there is no
+mechanism for preserving original file names, permissions,
+ownerships or dates in filesystems which lack these concepts, or
+have serious file name length restrictions, such as
+MS-DOS.
+
+bzip2 and
+bunzip2 will by default not
+overwrite existing files.  If you want this to happen, specify
+the -f flag.
+
+If no file names are specified,
+bzip2 compresses from standard
+input to standard output.  In this case,
+bzip2 will decline to write
+compressed output to a terminal, as this would be entirely
+incomprehensible and therefore pointless.
+
+bunzip2 (or
+bzip2 -d) decompresses all
+specified files.  Files which were not created by
+bzip2 will be detected and
+ignored, and a warning issued.
+bzip2 attempts to guess the
+filename for the decompressed file from that of the compressed
+file as follows:
+
+
+
+ filename.bz2 
+  becomes
+  filename
+
+ filename.bz 
+  becomes
+  filename
+
+ filename.tbz2
+  becomes
+  filename.tar
+
+ filename.tbz 
+  becomes
+  filename.tar
+
+ anyothername 
+  becomes
+  anyothername.out
+
+
+
+If the file does not end in one of the recognised endings,
+.bz2,
+.bz,
+.tbz2 or
+.tbz,
+bzip2 complains that it cannot
+guess the name of the original file, and uses the original name
+with .out appended.
+
+As with compression, supplying no filenames causes
+decompression from standard input to standard output.
+
+bunzip2 will correctly
+decompress a file which is the concatenation of two or more
+compressed files.  The result is the concatenation of the
+corresponding uncompressed files.  Integrity testing
+(-t) of concatenated compressed
+files is also supported.
+
+You can also compress or decompress files to the standard
+output by giving the -c flag.
+Multiple files may be compressed and decompressed like this.  The
+resulting outputs are fed sequentially to stdout.  Compression of
+multiple files in this manner generates a stream containing
+multiple compressed file representations.  Such a stream can be
+decompressed correctly only by
+bzip2 version 0.9.0 or later.
+Earlier versions of bzip2 will
+stop after decompressing the first file in the stream.
+
+bzcat (or
+bzip2 -dc) decompresses all
+specified files to the standard output.
+
+bzip2 will read arguments
+from the environment variables
+BZIP2 and
+BZIP, in that order, and will
+process them before any arguments read from the command line.
+This gives a convenient way to supply default arguments.
+
+Compression is always performed, even if the compressed
+file is slightly larger than the original.  Files of less than
+about one hundred bytes tend to get larger, since the compression
+mechanism has a constant overhead in the region of 50 bytes.
+Random data (including the output of most file compressors) is
+coded at about 8.05 bits per byte, giving an expansion of around
+0.5%.
+
+As a self-check for your protection,
+bzip2 uses 32-bit CRCs to make
+sure that the decompressed version of a file is identical to the
+original.  This guards against corruption of the compressed data,
+and against undetected bugs in
+bzip2 (hopefully very unlikely).
+The chances of data corruption going undetected is microscopic,
+about one chance in four billion for each file processed.  Be
+aware, though, that the check occurs upon decompression, so it
+can only tell you that something is wrong.  It can't help you
+recover the original uncompressed data.  You can use
+bzip2recover to try to recover
+data from damaged files.
+
+Return values: 0 for a normal exit, 1 for environmental
+problems (file not found, invalid flags, I/O errors, etc.), 2
+to indicate a corrupt compressed file, 3 for an internal
+consistency error (eg, bug) which caused
+bzip2 to panic.
+
+
+
+
+
+OPTIONS
+
+
+
+ 
+ -c --stdout
+ Compress or decompress to standard
+  output.
+ 
+
+ 
+ -d --decompress
+ Force decompression.
+  bzip2,
+  bunzip2 and
+  bzcat are really the same
+  program, and the decision about what actions to take is done on
+  the basis of which name is used.  This flag overrides that
+  mechanism, and forces bzip2 to decompress.
+ 
+
+ 
+ -z --compress
+ The complement to
+  -d: forces compression,
+  regardless of the invokation name.
+ 
+
+ 
+ -t --test
+ Check integrity of the specified file(s), but
+  don't decompress them.  This really performs a trial
+  decompression and throws away the result.
+ 
+
+ 
+ -f --force
+ Force overwrite of output files.  Normally,
+  bzip2 will not overwrite
+  existing output files.  Also forces
+  bzip2 to break hard links to
+  files, which it otherwise wouldn't do.
+  bzip2 normally declines
+  to decompress files which don't have the correct magic header
+  bytes. If forced (-f),
+  however, it will pass such files through unmodified. This is
+  how GNU gzip behaves.
+ 
+ 
+
+ 
+ -k --keep
+ Keep (don't delete) input files during
+  compression or decompression.
+ 
+
+ 
+ -s --small
+ Reduce memory usage, for compression,
+  decompression and testing.  Files are decompressed and tested
+  using a modified algorithm which only requires 2.5 bytes per
+  block byte.  This means any file can be decompressed in 2300k
+  of memory, albeit at about half the normal speed.
+  During compression, -s
+  selects a block size of 200k, which limits memory use to around
+  the same figure, at the expense of your compression ratio.  In
+  short, if your machine is low on memory (8 megabytes or less),
+  use -s for everything.  See
+   below.
+ 
+
+ 
+ -q --quiet
+ Suppress non-essential warning messages.
+  Messages pertaining to I/O errors and other critical events
+  will not be suppressed.
+ 
+
+ 
+ -v --verbose
+ Verbose mode -- show the compression ratio for
+  each file processed.  Further
+  -v's increase the verbosity
+  level, spewing out lots of information which is primarily of
+  interest for diagnostic purposes.
+ 
+
+ 
+ -L --license -V --version
+ Display the software version, license terms and
+  conditions.
+ 
+
+ 
+ -1 (or
+ --fast) to
+ -9 (or
+ -best)
+ Set the block size to 100 k, 200 k ...  900 k
+  when compressing.  Has no effect when decompressing.  See  below.  The
+  --fast and
+  --best aliases are primarily
+  for GNU gzip compatibility.
+  In particular, --fast doesn't
+  make things significantly faster.  And
+  --best merely selects the
+  default behaviour.
+ 
+
+ 
+ --
+ Treats all subsequent arguments as file names,
+  even if they start with a dash.  This is so you can handle
+  files with names beginning with a dash, for example:
+  bzip2 --
+  -myfilename.
+ 
+
+ 
+ --repetitive-fast
+ --repetitive-best
+ These flags are redundant in versions 0.9.5 and
+  above.  They provided some coarse control over the behaviour of
+  the sorting algorithm in earlier versions, which was sometimes
+  useful.  0.9.5 and above have an improved algorithm which
+  renders these flags irrelevant.
+ 
+
+
+
+
+
+
+
+MEMORY MANAGEMENT
+
+bzip2 compresses large
+files in blocks.  The block size affects both the compression
+ratio achieved, and the amount of memory needed for compression
+and decompression.  The flags -1
+through -9 specify the block
+size to be 100,000 bytes through 900,000 bytes (the default)
+respectively.  At decompression time, the block size used for
+compression is read from the header of the compressed file, and
+bunzip2 then allocates itself
+just enough memory to decompress the file.  Since block sizes are
+stored in compressed files, it follows that the flags
+-1 to
+-9 are irrelevant to and so
+ignored during decompression.
+
+Compression and decompression requirements, in bytes, can be
+estimated as:
+
+Compression:   400k + ( 8 x block size )
+
+Decompression: 100k + ( 4 x block size ), or
+               100k + ( 2.5 x block size )
+
+
+Larger block sizes give rapidly diminishing marginal
+returns.  Most of the compression comes from the first two or
+three hundred k of block size, a fact worth bearing in mind when
+using bzip2 on small machines.
+It is also important to appreciate that the decompression memory
+requirement is set at compression time by the choice of block
+size.
+
+For files compressed with the default 900k block size,
+bunzip2 will require about 3700
+kbytes to decompress.  To support decompression of any file on a
+4 megabyte machine, bunzip2 has
+an option to decompress using approximately half this amount of
+memory, about 2300 kbytes.  Decompression speed is also halved,
+so you should use this option only where necessary.  The relevant
+flag is -s.
+
+In general, try and use the largest block size memory
+constraints allow, since that maximises the compression achieved.
+Compression and decompression speed are virtually unaffected by
+block size.
+
+Another significant point applies to files which fit in a
+single block -- that means most files you'd encounter using a
+large block size.  The amount of real memory touched is
+proportional to the size of the file, since the file is smaller
+than a block.  For example, compressing a file 20,000 bytes long
+with the flag -9 will cause the
+compressor to allocate around 7600k of memory, but only touch
+400k + 20000 * 8 = 560 kbytes of it.  Similarly, the decompressor
+will allocate 3700k but only touch 100k + 20000 * 4 = 180
+kbytes.
+
+Here is a table which summarises the maximum memory usage
+for different block sizes.  Also recorded is the total compressed
+size for 14 files of the Calgary Text Compression Corpus
+totalling 3,141,622 bytes.  This column gives some feel for how
+compression varies with block size.  These figures tend to
+understate the advantage of larger block sizes for larger files,
+since the Corpus is dominated by smaller files.
+
+
+        Compress   Decompress   Decompress   Corpus
+Flag     usage      usage       -s usage     Size
+
+ -1      1200k       500k         350k      914704
+ -2      2000k       900k         600k      877703
+ -3      2800k      1300k         850k      860338
+ -4      3600k      1700k        1100k      846899
+ -5      4400k      2100k        1350k      845160
+ -6      5200k      2500k        1600k      838626
+ -7      6100k      2900k        1850k      834096
+ -8      6800k      3300k        2100k      828642
+ -9      7600k      3700k        2350k      828642
+
+
+
+
+
+
+RECOVERING DATA FROM DAMAGED FILES
+
+bzip2 compresses files in
+blocks, usually 900kbytes long.  Each block is handled
+independently.  If a media or transmission error causes a
+multi-block .bz2 file to become
+damaged, it may be possible to recover data from the undamaged
+blocks in the file.
+
+The compressed representation of each block is delimited by
+a 48-bit pattern, which makes it possible to find the block
+boundaries with reasonable certainty.  Each block also carries
+its own 32-bit CRC, so damaged blocks can be distinguished from
+undamaged ones.
+
+bzip2recover is a simple
+program whose purpose is to search for blocks in
+.bz2 files, and write each block
+out into its own .bz2 file.  You
+can then use bzip2 -t to test
+the integrity of the resulting files, and decompress those which
+are undamaged.
+
+bzip2recover takes a
+single argument, the name of the damaged file, and writes a
+number of files rec0001file.bz2,
+rec0002file.bz2, etc, containing
+the extracted blocks.  The output filenames are designed so that
+the use of wildcards in subsequent processing -- for example,
+bzip2 -dc rec*file.bz2 >
+recovered_data -- lists the files in the correct
+order.
+
+bzip2recover should be of
+most use dealing with large .bz2
+files, as these will contain many blocks.  It is clearly futile
+to use it on damaged single-block files, since a damaged block
+cannot be recovered.  If you wish to minimise any potential data
+loss through media or transmission errors, you might consider
+compressing with a smaller block size.
+
+
+
+
+
+PERFORMANCE NOTES
+
+The sorting phase of compression gathers together similar
+strings in the file.  Because of this, files containing very long
+runs of repeated symbols, like "aabaabaabaab ..."  (repeated
+several hundred times) may compress more slowly than normal.
+Versions 0.9.5 and above fare much better than previous versions
+in this respect.  The ratio between worst-case and average-case
+compression time is in the region of 10:1.  For previous
+versions, this figure was more like 100:1.  You can use the
+-vvvv option to monitor progress
+in great detail, if you want.
+
+Decompression speed is unaffected by these
+phenomena.
+
+bzip2 usually allocates
+several megabytes of memory to operate in, and then charges all
+over it in a fairly random fashion.  This means that performance,
+both for compressing and decompressing, is largely determined by
+the speed at which your machine can service cache misses.
+Because of this, small changes to the code to reduce the miss
+rate have been observed to give disproportionately large
+performance improvements.  I imagine
+bzip2 will perform best on
+machines with very large caches.
+
+
+
+
+
+
+CAVEATS
+
+I/O error messages are not as helpful as they could be.
+bzip2 tries hard to detect I/O
+errors and exit cleanly, but the details of what the problem is
+sometimes seem rather misleading.
+
+This manual page pertains to version &bz-version; of
+bzip2.  Compressed data created
+by this version is entirely forwards and backwards compatible
+with the previous public releases, versions 0.1pl2, 0.9.0 and
+0.9.5, 1.0.0, 1.0.1 and 1.0.2, but with the following exception: 0.9.0
+and above can correctly decompress multiple concatenated
+compressed files.  0.1pl2 cannot do this; it will stop after
+decompressing just the first file in the stream.
+
+bzip2recover versions
+prior to 1.0.2 used 32-bit integers to represent bit positions in
+compressed files, so it could not handle compressed files more
+than 512 megabytes long.  Versions 1.0.2 and above use 64-bit ints
+on some platforms which support them (GNU supported targets, and
+Windows). To establish whether or not
+bzip2recover was built with such
+a limitation, run it without arguments. In any event you can
+build yourself an unlimited version if you can recompile it with
+MaybeUInt64 set to be an
+unsigned 64-bit integer.
+
+
+
+
+
+
+AUTHOR
+
+Julian Seward,
+&bz-email;
+
+The ideas embodied in
+bzip2 are due to (at least) the
+following people: Michael Burrows and David Wheeler (for the
+block sorting transformation), David Wheeler (again, for the
+Huffman coder), Peter Fenwick (for the structured coding model in
+the original bzip, and many
+refinements), and Alistair Moffat, Radford Neal and Ian Witten
+(for the arithmetic coder in the original
+bzip).  I am much indebted for
+their help, support and advice.  See the manual in the source
+distribution for pointers to sources of documentation.  Christian
+von Roques encouraged me to look for faster sorting algorithms,
+so as to speed up compression.  Bela Lubkin encouraged me to
+improve the worst-case compression performance.  
+Donna Robinson XMLised the documentation.
+Many people sent
+patches, helped with portability problems, lent machines, gave
+advice and were generally helpful.
+
+
+
+
+
+
+
+
+
+Programming with <computeroutput>libbzip2</computeroutput>
+
+
+This chapter describes the programming interface to
+libbzip2.
+
+For general background information, particularly about
+memory use and performance aspects, you'd be well advised to read
+ as well.
+
+
+
+Top-level structure
+
+libbzip2 is a flexible
+library for compressing and decompressing data in the
+bzip2 data format.  Although
+packaged as a single entity, it helps to regard the library as
+three separate parts: the low level interface, and the high level
+interface, and some utility functions.
+
+The structure of
+libbzip2's interfaces is similar
+to that of Jean-loup Gailly's and Mark Adler's excellent
+zlib library.
+
+All externally visible symbols have names beginning
+BZ2_.  This is new in version
+1.0.  The intention is to minimise pollution of the namespaces of
+library clients.
+
+To use any part of the library, you need to
+#include <bzlib.h>
+into your sources.
+
+
+
+
+Low-level summary
+
+This interface provides services for compressing and
+decompressing data in memory.  There's no provision for dealing
+with files, streams or any other I/O mechanisms, just straight
+memory-to-memory work.  In fact, this part of the library can be
+compiled without inclusion of
+stdio.h, which may be helpful
+for embedded applications.
+
+The low-level part of the library has no global variables
+and is therefore thread-safe.
+
+Six routines make up the low level interface:
+BZ2_bzCompressInit,
+BZ2_bzCompress, and
+BZ2_bzCompressEnd for
+compression, and a corresponding trio
+BZ2_bzDecompressInit,
+BZ2_bzDecompress and
+BZ2_bzDecompressEnd for
+decompression.  The *Init
+functions allocate memory for compression/decompression and do
+other initialisations, whilst the
+*End functions close down
+operations and release memory.
+
+The real work is done by
+BZ2_bzCompress and
+BZ2_bzDecompress.  These
+compress and decompress data from a user-supplied input buffer to
+a user-supplied output buffer.  These buffers can be any size;
+arbitrary quantities of data are handled by making repeated calls
+to these functions.  This is a flexible mechanism allowing a
+consumer-pull style of activity, or producer-push, or a mixture
+of both.
+
+
+
+
+
+High-level summary
+
+This interface provides some handy wrappers around the
+low-level interface to facilitate reading and writing
+bzip2 format files
+(.bz2 files).  The routines
+provide hooks to facilitate reading files in which the
+bzip2 data stream is embedded
+within some larger-scale file structure, or where there are
+multiple bzip2 data streams
+concatenated end-to-end.
+
+For reading files,
+BZ2_bzReadOpen,
+BZ2_bzRead,
+BZ2_bzReadClose and 
+BZ2_bzReadGetUnused are
+supplied.  For writing files,
+BZ2_bzWriteOpen,
+BZ2_bzWrite and
+BZ2_bzWriteFinish are
+available.
+
+As with the low-level library, no global variables are used
+so the library is per se thread-safe.  However, if I/O errors
+occur whilst reading or writing the underlying compressed files,
+you may have to consult errno to
+determine the cause of the error.  In that case, you'd need a C
+library which correctly supports
+errno in a multithreaded
+environment.
+
+To make the library a little simpler and more portable,
+BZ2_bzReadOpen and
+BZ2_bzWriteOpen require you to
+pass them file handles (FILE*s)
+which have previously been opened for reading or writing
+respectively.  That avoids portability problems associated with
+file operations and file attributes, whilst not being much of an
+imposition on the programmer.
+
+
+
+
+
+Utility functions summary
+
+For very simple needs,
+BZ2_bzBuffToBuffCompress and
+BZ2_bzBuffToBuffDecompress are
+provided.  These compress data in memory from one buffer to
+another buffer in a single function call.  You should assess
+whether these functions fulfill your memory-to-memory
+compression/decompression requirements before investing effort in
+understanding the more general but more complex low-level
+interface.
+
+Yoshioka Tsuneo
+(QWF00133@niftyserve.or.jp /
+tsuneo-y@is.aist-nara.ac.jp) has
+contributed some functions to give better
+zlib compatibility.  These
+functions are BZ2_bzopen,
+BZ2_bzread,
+BZ2_bzwrite,
+BZ2_bzflush,
+BZ2_bzclose,
+BZ2_bzerror and
+BZ2_bzlibVersion.  You may find
+these functions more convenient for simple file reading and
+writing, than those in the high-level interface.  These functions
+are not (yet) officially part of the library, and are minimally
+documented here.  If they break, you get to keep all the pieces.
+I hope to document them properly when time permits.
+
+Yoshioka also contributed modifications to allow the
+library to be built as a Windows DLL.
+
+
+
+
+
+
+
+Error handling
+
+The library is designed to recover cleanly in all
+situations, including the worst-case situation of decompressing
+random data.  I'm not 100% sure that it can always do this, so
+you might want to add a signal handler to catch segmentation
+violations during decompression if you are feeling especially
+paranoid.  I would be interested in hearing more about the
+robustness of the library to corrupted compressed data.
+
+Version 1.0.3 more robust in this respect than any
+previous version.  Investigations with Valgrind (a tool for detecting
+problems with memory management) indicate
+that, at least for the few files I tested, all single-bit errors
+in the decompressed data are caught properly, with no
+segmentation faults, no uses of uninitialised data, no out of
+range reads or writes, and no infinite looping in the decompressor.
+So it's certainly pretty robust, although
+I wouldn't claim it to be totally bombproof.
+
+The file bzlib.h contains
+all definitions needed to use the library.  In particular, you
+should definitely not include
+bzlib_private.h.
+
+In bzlib.h, the various
+return values are defined.  The following list is not intended as
+an exhaustive description of the circumstances in which a given
+value may be returned -- those descriptions are given later.
+Rather, it is intended to convey the rough meaning of each return
+value.  The first five actions are normal and not intended to
+denote an error situation.
+
+
+
+ 
+  BZ_OK
+  The requested action was completed
+   successfully.
+ 
+
+ 
+  BZ_RUN_OK, BZ_FLUSH_OK,
+    BZ_FINISH_OK
+  In 
+   BZ2_bzCompress, the requested
+   flush/finish/nothing-special action was completed
+   successfully.
+ 
+
+ 
+  BZ_STREAM_END
+  Compression of data was completed, or the
+   logical stream end was detected during
+   decompression.
+ 
+
+
+
+The following return values indicate an error of some
+kind.
+
+
+
+ 
+  BZ_CONFIG_ERROR
+  Indicates that the library has been improperly
+   compiled on your platform -- a major configuration error.
+   Specifically, it means that
+   sizeof(char),
+   sizeof(short) and
+   sizeof(int) are not 1, 2 and
+   4 respectively, as they should be.  Note that the library
+   should still work properly on 64-bit platforms which follow
+   the LP64 programming model -- that is, where
+   sizeof(long) and
+   sizeof(void*) are 8.  Under
+   LP64, sizeof(int) is still 4,
+   so libbzip2, which doesn't
+   use the long type, is
+   OK.
+ 
+
+ 
+  BZ_SEQUENCE_ERROR
+  When using the library, it is important to call
+   the functions in the correct sequence and with data structures
+   (buffers etc) in the correct states.
+   libbzip2 checks as much as it
+   can to ensure this is happening, and returns
+   BZ_SEQUENCE_ERROR if not.
+   Code which complies precisely with the function semantics, as
+   detailed below, should never receive this value; such an event
+   denotes buggy code which you should
+   investigate.
+ 
+
+ 
+  BZ_PARAM_ERROR
+  Returned when a parameter to a function call is
+   out of range or otherwise manifestly incorrect.  As with
+   BZ_SEQUENCE_ERROR, this
+   denotes a bug in the client code.  The distinction between
+   BZ_PARAM_ERROR and
+   BZ_SEQUENCE_ERROR is a bit
+   hazy, but still worth making.
+ 
+
+ 
+  BZ_MEM_ERROR
+  Returned when a request to allocate memory
+   failed.  Note that the quantity of memory needed to decompress
+   a stream cannot be determined until the stream's header has
+   been read.  So
+   BZ2_bzDecompress and
+   BZ2_bzRead may return
+   BZ_MEM_ERROR even though some
+   of the compressed data has been read.  The same is not true
+   for compression; once
+   BZ2_bzCompressInit or
+   BZ2_bzWriteOpen have
+   successfully completed,
+   BZ_MEM_ERROR cannot
+   occur.
+ 
+
+ 
+  BZ_DATA_ERROR
+  Returned when a data integrity error is
+   detected during decompression.  Most importantly, this means
+   when stored and computed CRCs for the data do not match.  This
+   value is also returned upon detection of any other anomaly in
+   the compressed data.
+ 
+
+ 
+  BZ_DATA_ERROR_MAGIC
+  As a special case of
+   BZ_DATA_ERROR, it is
+   sometimes useful to know when the compressed stream does not
+   start with the correct magic bytes ('B' 'Z'
+   'h').
+ 
+
+ 
+  BZ_IO_ERROR
+  Returned by
+   BZ2_bzRead and
+   BZ2_bzWrite when there is an
+   error reading or writing in the compressed file, and by
+   BZ2_bzReadOpen and
+   BZ2_bzWriteOpen for attempts
+   to use a file for which the error indicator (viz,
+   ferror(f)) is set.  On
+   receipt of BZ_IO_ERROR, the
+   caller should consult errno
+   and/or perror to acquire
+   operating-system specific information about the
+   problem.
+ 
+
+ 
+  BZ_UNEXPECTED_EOF
+  Returned by
+   BZ2_bzRead when the
+   compressed file finishes before the logical end of stream is
+   detected.
+ 
+
+ 
+  BZ_OUTBUFF_FULL
+  Returned by
+   BZ2_bzBuffToBuffCompress and
+   BZ2_bzBuffToBuffDecompress to
+   indicate that the output data will not fit into the output
+   buffer provided.
+ 
+
+
+
+
+
+
+
+
+Low-level interface
+
+
+
+<computeroutput>BZ2_bzCompressInit</computeroutput>
+
+
+typedef struct {
+  char *next_in;
+  unsigned int avail_in;
+  unsigned int total_in_lo32;
+  unsigned int total_in_hi32;
+
+  char *next_out;
+  unsigned int avail_out;
+  unsigned int total_out_lo32;
+  unsigned int total_out_hi32;
+
+  void *state;
+
+  void *(*bzalloc)(void *,int,int);
+  void (*bzfree)(void *,void *);
+  void *opaque;
+} bz_stream;
+
+int BZ2_bzCompressInit ( bz_stream *strm, 
+                         int blockSize100k, 
+                         int verbosity,
+                         int workFactor );
+
+
+Prepares for compression.  The
+bz_stream structure holds all
+data pertaining to the compression activity.  A
+bz_stream structure should be
+allocated and initialised prior to the call.  The fields of
+bz_stream comprise the entirety
+of the user-visible data.  state
+is a pointer to the private data structures required for
+compression.
+
+Custom memory allocators are supported, via fields
+bzalloc,
+bzfree, and
+opaque.  The value
+opaque is passed to as the first
+argument to all calls to bzalloc
+and bzfree, but is otherwise
+ignored by the library.  The call bzalloc (
+opaque, n, m ) is expected to return a pointer
+p to n *
+m bytes of memory, and bzfree (
+opaque, p ) should free that memory.
+
+If you don't want to use a custom memory allocator, set
+bzalloc,
+bzfree and
+opaque to
+NULL, and the library will then
+use the standard malloc /
+free routines.
+
+Before calling
+BZ2_bzCompressInit, fields
+bzalloc,
+bzfree and
+opaque should be filled
+appropriately, as just described.  Upon return, the internal
+state will have been allocated and initialised, and
+total_in_lo32,
+total_in_hi32,
+total_out_lo32 and
+total_out_hi32 will have been
+set to zero.  These four fields are used by the library to inform
+the caller of the total amount of data passed into and out of the
+library, respectively.  You should not try to change them.  As of
+version 1.0, 64-bit counts are maintained, even on 32-bit
+platforms, using the _hi32
+fields to store the upper 32 bits of the count.  So, for example,
+the total amount of data in is (total_in_hi32
+<< 32) + total_in_lo32.
+
+Parameter blockSize100k
+specifies the block size to be used for compression.  It should
+be a value between 1 and 9 inclusive, and the actual block size
+used is 100000 x this figure.  9 gives the best compression but
+takes most memory.
+
+Parameter verbosity should
+be set to a number between 0 and 4 inclusive.  0 is silent, and
+greater numbers give increasingly verbose monitoring/debugging
+output.  If the library has been compiled with
+-DBZ_NO_STDIO, no such output
+will appear for any verbosity setting.
+
+Parameter workFactor
+controls how the compression phase behaves when presented with
+worst case, highly repetitive, input data.  If compression runs
+into difficulties caused by repetitive data, the library switches
+from the standard sorting algorithm to a fallback algorithm.  The
+fallback is slower than the standard algorithm by perhaps a
+factor of three, but always behaves reasonably, no matter how bad
+the input.
+
+Lower values of workFactor
+reduce the amount of effort the standard algorithm will expend
+before resorting to the fallback.  You should set this parameter
+carefully; too low, and many inputs will be handled by the
+fallback algorithm and so compress rather slowly, too high, and
+your average-to-worst case compression times can become very
+large.  The default value of 30 gives reasonable behaviour over a
+wide range of circumstances.
+
+Allowable values range from 0 to 250 inclusive.  0 is a
+special case, equivalent to using the default value of 30.
+
+Note that the compressed output generated is the same
+regardless of whether or not the fallback algorithm is
+used.
+
+Be aware also that this parameter may disappear entirely in
+future versions of the library.  In principle it should be
+possible to devise a good way to automatically choose which
+algorithm to use.  Such a mechanism would render the parameter
+obsolete.
+
+Possible return values:
+
+
+BZ_CONFIG_ERROR
+  if the library has been mis-compiled
+BZ_PARAM_ERROR
+  if strm is NULL 
+  or blockSize < 1 or blockSize > 9
+  or verbosity < 0 or verbosity > 4
+  or workFactor < 0 or workFactor > 250
+BZ_MEM_ERROR 
+  if not enough memory is available
+BZ_OK 
+  otherwise
+
+
+Allowable next actions:
+
+
+BZ2_bzCompress
+  if BZ_OK is returned
+  no specific action needed in case of error
+
+
+
+
+
+
+<computeroutput>BZ2_bzCompress</computeroutput>
+
+
+int BZ2_bzCompress ( bz_stream *strm, int action );
+
+
+Provides more input and/or output buffer space for the
+library.  The caller maintains input and output buffers, and
+calls BZ2_bzCompress to transfer
+data between them.
+
+Before each call to
+BZ2_bzCompress,
+next_in should point at the data
+to be compressed, and avail_in
+should indicate how many bytes the library may read.
+BZ2_bzCompress updates
+next_in,
+avail_in and
+total_in to reflect the number
+of bytes it has read.
+
+Similarly, next_out should
+point to a buffer in which the compressed data is to be placed,
+with avail_out indicating how
+much output space is available.
+BZ2_bzCompress updates
+next_out,
+avail_out and
+total_out to reflect the number
+of bytes output.
+
+You may provide and remove as little or as much data as you
+like on each call of
+BZ2_bzCompress.  In the limit,
+it is acceptable to supply and remove data one byte at a time,
+although this would be terribly inefficient.  You should always
+ensure that at least one byte of output space is available at
+each call.
+
+A second purpose of
+BZ2_bzCompress is to request a
+change of mode of the compressed stream.
+
+Conceptually, a compressed stream can be in one of four
+states: IDLE, RUNNING, FLUSHING and FINISHING.  Before
+initialisation
+(BZ2_bzCompressInit) and after
+termination (BZ2_bzCompressEnd),
+a stream is regarded as IDLE.
+
+Upon initialisation
+(BZ2_bzCompressInit), the stream
+is placed in the RUNNING state.  Subsequent calls to
+BZ2_bzCompress should pass
+BZ_RUN as the requested action;
+other actions are illegal and will result in
+BZ_SEQUENCE_ERROR.
+
+At some point, the calling program will have provided all
+the input data it wants to.  It will then want to finish up -- in
+effect, asking the library to process any data it might have
+buffered internally.  In this state,
+BZ2_bzCompress will no longer
+attempt to read data from
+next_in, but it will want to
+write data to next_out.  Because
+the output buffer supplied by the user can be arbitrarily small,
+the finishing-up operation cannot necessarily be done with a
+single call of
+BZ2_bzCompress.
+
+Instead, the calling program passes
+BZ_FINISH as an action to
+BZ2_bzCompress.  This changes
+the stream's state to FINISHING.  Any remaining input (ie,
+next_in[0 .. avail_in-1]) is
+compressed and transferred to the output buffer.  To do this,
+BZ2_bzCompress must be called
+repeatedly until all the output has been consumed.  At that
+point, BZ2_bzCompress returns
+BZ_STREAM_END, and the stream's
+state is set back to IDLE.
+BZ2_bzCompressEnd should then be
+called.
+
+Just to make sure the calling program does not cheat, the
+library makes a note of avail_in
+at the time of the first call to
+BZ2_bzCompress which has
+BZ_FINISH as an action (ie, at
+the time the program has announced its intention to not supply
+any more input).  By comparing this value with that of
+avail_in over subsequent calls
+to BZ2_bzCompress, the library
+can detect any attempts to slip in more data to compress.  Any
+calls for which this is detected will return
+BZ_SEQUENCE_ERROR.  This
+indicates a programming mistake which should be corrected.
+
+Instead of asking to finish, the calling program may ask
+BZ2_bzCompress to take all the
+remaining input, compress it and terminate the current
+(Burrows-Wheeler) compression block.  This could be useful for
+error control purposes.  The mechanism is analogous to that for
+finishing: call BZ2_bzCompress
+with an action of BZ_FLUSH,
+remove output data, and persist with the
+BZ_FLUSH action until the value
+BZ_RUN is returned.  As with
+finishing, BZ2_bzCompress
+detects any attempt to provide more input data once the flush has
+begun.
+
+Once the flush is complete, the stream returns to the
+normal RUNNING state.
+
+This all sounds pretty complex, but isn't really.  Here's a
+table which shows which actions are allowable in each state, what
+action will be taken, what the next state is, and what the
+non-error return values are.  Note that you can't explicitly ask
+what state the stream is in, but nor do you need to -- it can be
+inferred from the values returned by
+BZ2_bzCompress.
+
+
+IDLE/any
+  Illegal.  IDLE state only exists after BZ2_bzCompressEnd or
+  before BZ2_bzCompressInit.
+  Return value = BZ_SEQUENCE_ERROR
+
+RUNNING/BZ_RUN
+  Compress from next_in to next_out as much as possible.
+  Next state = RUNNING
+  Return value = BZ_RUN_OK
+
+RUNNING/BZ_FLUSH
+  Remember current value of next_in. Compress from next_in
+  to next_out as much as possible, but do not accept any more input.
+  Next state = FLUSHING
+  Return value = BZ_FLUSH_OK
+
+RUNNING/BZ_FINISH
+  Remember current value of next_in. Compress from next_in
+  to next_out as much as possible, but do not accept any more input.
+  Next state = FINISHING
+  Return value = BZ_FINISH_OK
+
+FLUSHING/BZ_FLUSH
+  Compress from next_in to next_out as much as possible, 
+  but do not accept any more input.
+  If all the existing input has been used up and all compressed
+  output has been removed
+    Next state = RUNNING; Return value = BZ_RUN_OK
+  else
+    Next state = FLUSHING; Return value = BZ_FLUSH_OK
+
+FLUSHING/other     
+  Illegal.
+  Return value = BZ_SEQUENCE_ERROR
+
+FINISHING/BZ_FINISH
+  Compress from next_in to next_out as much as possible,
+  but to not accept any more input.  
+  If all the existing input has been used up and all compressed
+  output has been removed
+    Next state = IDLE; Return value = BZ_STREAM_END
+  else
+    Next state = FINISHING; Return value = BZ_FINISHING
+
+FINISHING/other
+  Illegal.
+  Return value = BZ_SEQUENCE_ERROR
+
+
+
+That still looks complicated?  Well, fair enough.  The
+usual sequence of calls for compressing a load of data is:
+
+
+
+ Get started with
+  BZ2_bzCompressInit.
+
+ Shovel data in and shlurp out its compressed form
+  using zero or more calls of
+  BZ2_bzCompress with action =
+  BZ_RUN.
+
+ Finish up. Repeatedly call
+  BZ2_bzCompress with action =
+  BZ_FINISH, copying out the
+  compressed output, until
+  BZ_STREAM_END is
+  returned. Close up and go home.  Call
+  BZ2_bzCompressEnd.
+
+
+
+If the data you want to compress fits into your input
+buffer all at once, you can skip the calls of
+BZ2_bzCompress ( ..., BZ_RUN )
+and just do the BZ2_bzCompress ( ..., BZ_FINISH
+) calls.
+
+All required memory is allocated by
+BZ2_bzCompressInit.  The
+compression library can accept any data at all (obviously).  So
+you shouldn't get any error return values from the
+BZ2_bzCompress calls.  If you
+do, they will be
+BZ_SEQUENCE_ERROR, and indicate
+a bug in your programming.
+
+Trivial other possible return values:
+
+
+BZ_PARAM_ERROR
+  if strm is NULL, or strm->s is NULL
+
+
+
+
+
+
+<computeroutput>BZ2_bzCompressEnd</computeroutput>
+
+
+int BZ2_bzCompressEnd ( bz_stream *strm );
+
+
+Releases all memory associated with a compression
+stream.
+
+Possible return values:
+
+
+BZ_PARAM_ERROR  if strm is NULL or strm->s is NULL
+BZ_OK           otherwise
+
+
+
+
+
+
+<computeroutput>BZ2_bzDecompressInit</computeroutput>
+
+
+int BZ2_bzDecompressInit ( bz_stream *strm, int verbosity, int small );
+
+
+Prepares for decompression.  As with
+BZ2_bzCompressInit, a
+bz_stream record should be
+allocated and initialised before the call.  Fields
+bzalloc,
+bzfree and
+opaque should be set if a custom
+memory allocator is required, or made
+NULL for the normal
+malloc /
+free routines.  Upon return, the
+internal state will have been initialised, and
+total_in and
+total_out will be zero.
+
+For the meaning of parameter
+verbosity, see
+BZ2_bzCompressInit.
+
+If small is nonzero, the
+library will use an alternative decompression algorithm which
+uses less memory but at the cost of decompressing more slowly
+(roughly speaking, half the speed, but the maximum memory
+requirement drops to around 2300k).  See 
+for more information on memory management.
+
+Note that the amount of memory needed to decompress a
+stream cannot be determined until the stream's header has been
+read, so even if
+BZ2_bzDecompressInit succeeds, a
+subsequent BZ2_bzDecompress
+could fail with
+BZ_MEM_ERROR.
+
+Possible return values:
+
+
+BZ_CONFIG_ERROR
+  if the library has been mis-compiled
+BZ_PARAM_ERROR
+  if ( small != 0 && small != 1 )
+  or (verbosity <; 0 || verbosity > 4)
+BZ_MEM_ERROR
+  if insufficient memory is available
+
+
+Allowable next actions:
+
+
+BZ2_bzDecompress
+  if BZ_OK was returned
+  no specific action required in case of error
+
+
+
+
+
+
+<computeroutput>BZ2_bzDecompress</computeroutput>
+
+
+int BZ2_bzDecompress ( bz_stream *strm );
+
+
+Provides more input and/out output buffer space for the
+library.  The caller maintains input and output buffers, and uses
+BZ2_bzDecompress to transfer
+data between them.
+
+Before each call to
+BZ2_bzDecompress,
+next_in should point at the
+compressed data, and avail_in
+should indicate how many bytes the library may read.
+BZ2_bzDecompress updates
+next_in,
+avail_in and
+total_in to reflect the number
+of bytes it has read.
+
+Similarly, next_out should
+point to a buffer in which the uncompressed output is to be
+placed, with avail_out
+indicating how much output space is available.
+BZ2_bzCompress updates
+next_out,
+avail_out and
+total_out to reflect the number
+of bytes output.
+
+You may provide and remove as little or as much data as you
+like on each call of
+BZ2_bzDecompress.  In the limit,
+it is acceptable to supply and remove data one byte at a time,
+although this would be terribly inefficient.  You should always
+ensure that at least one byte of output space is available at
+each call.
+
+Use of BZ2_bzDecompress is
+simpler than
+BZ2_bzCompress.
+
+You should provide input and remove output as described
+above, and repeatedly call
+BZ2_bzDecompress until
+BZ_STREAM_END is returned.
+Appearance of BZ_STREAM_END
+denotes that BZ2_bzDecompress
+has detected the logical end of the compressed stream.
+BZ2_bzDecompress will not
+produce BZ_STREAM_END until all
+output data has been placed into the output buffer, so once
+BZ_STREAM_END appears, you are
+guaranteed to have available all the decompressed output, and
+BZ2_bzDecompressEnd can safely
+be called.
+
+If case of an error return value, you should call
+BZ2_bzDecompressEnd to clean up
+and release memory.
+
+Possible return values:
+
+
+BZ_PARAM_ERROR
+  if strm is NULL or strm->s is NULL
+  or strm->avail_out < 1
+BZ_DATA_ERROR
+  if a data integrity error is detected in the compressed stream
+BZ_DATA_ERROR_MAGIC
+  if the compressed stream doesn't begin with the right magic bytes
+BZ_MEM_ERROR
+  if there wasn't enough memory available
+BZ_STREAM_END
+  if the logical end of the data stream was detected and all
+  output in has been consumed, eg s-->avail_out > 0
+BZ_OK
+  otherwise
+
+
+Allowable next actions:
+
+
+BZ2_bzDecompress
+  if BZ_OK was returned
+BZ2_bzDecompressEnd
+  otherwise
+
+
+
+
+
+
+<computeroutput>BZ2_bzDecompressEnd</computeroutput>
+
+
+int BZ2_bzDecompressEnd ( bz_stream *strm );
+
+
+Releases all memory associated with a decompression
+stream.
+
+Possible return values:
+
+
+BZ_PARAM_ERROR
+  if strm is NULL or strm->s is NULL
+BZ_OK
+  otherwise
+
+
+Allowable next actions:
+
+
+  None.
+
+
+
+
+
+
+
+
+High-level interface
+
+This interface provides functions for reading and writing
+bzip2 format files.  First, some
+general points.
+
+
+
+ All of the functions take an
+  int* first argument,
+  bzerror.  After each call,
+  bzerror should be consulted
+  first to determine the outcome of the call.  If
+  bzerror is
+  BZ_OK, the call completed
+  successfully, and only then should the return value of the
+  function (if any) be consulted.  If
+  bzerror is
+  BZ_IO_ERROR, there was an
+  error reading/writing the underlying compressed file, and you
+  should then consult errno /
+  perror to determine the cause
+  of the difficulty.  bzerror
+  may also be set to various other values; precise details are
+  given on a per-function basis below.
+
+ If bzerror indicates
+  an error (ie, anything except
+  BZ_OK and
+  BZ_STREAM_END), you should
+  immediately call
+  BZ2_bzReadClose (or
+  BZ2_bzWriteClose, depending on
+  whether you are attempting to read or to write) to free up all
+  resources associated with the stream.  Once an error has been
+  indicated, behaviour of all calls except
+  BZ2_bzReadClose
+  (BZ2_bzWriteClose) is
+  undefined.  The implication is that (1)
+  bzerror should be checked
+  after each call, and (2) if
+  bzerror indicates an error,
+  BZ2_bzReadClose
+  (BZ2_bzWriteClose) should then
+  be called to clean up.
+
+ The FILE* arguments
+  passed to BZ2_bzReadOpen /
+  BZ2_bzWriteOpen should be set
+  to binary mode.  Most Unix systems will do this by default, but
+  other platforms, including Windows and Mac, will not.  If you
+  omit this, you may encounter problems when moving code to new
+  platforms.
+
+ Memory allocation requests are handled by
+  malloc /
+  free.  At present there is no
+  facility for user-defined memory allocators in the file I/O
+  functions (could easily be added, though).
+
+
+
+
+
+
+<computeroutput>BZ2_bzReadOpen</computeroutput>
+
+
+typedef void BZFILE;
+
+BZFILE *BZ2_bzReadOpen( int *bzerror, FILE *f, 
+                        int verbosity, int small,
+                        void *unused, int nUnused );
+
+
+Prepare to read compressed data from file handle
+f.
+f should refer to a file which
+has been opened for reading, and for which the error indicator
+(ferror(f))is not set.  If
+small is 1, the library will try
+to decompress using less memory, at the expense of speed.
+
+For reasons explained below,
+BZ2_bzRead will decompress the
+nUnused bytes starting at
+unused, before starting to read
+from the file f.  At most
+BZ_MAX_UNUSED bytes may be
+supplied like this.  If this facility is not required, you should
+pass NULL and
+0 for
+unused and
+nUnused respectively.
+
+For the meaning of parameters
+small and
+verbosity, see
+BZ2_bzDecompressInit.
+
+The amount of memory needed to decompress a file cannot be
+determined until the file's header has been read.  So it is
+possible that BZ2_bzReadOpen
+returns BZ_OK but a subsequent
+call of BZ2_bzRead will return
+BZ_MEM_ERROR.
+
+Possible assignments to
+bzerror:
+
+
+BZ_CONFIG_ERROR
+  if the library has been mis-compiled
+BZ_PARAM_ERROR
+  if f is NULL
+  or small is neither 0 nor 1
+  or ( unused == NULL && nUnused != 0 )
+  or ( unused != NULL && !(0 <= nUnused <= BZ_MAX_UNUSED) )
+BZ_IO_ERROR
+  if ferror(f) is nonzero
+BZ_MEM_ERROR
+  if insufficient memory is available
+BZ_OK
+  otherwise.
+
+
+Possible return values:
+
+
+Pointer to an abstract BZFILE
+  if bzerror is BZ_OK
+NULL
+  otherwise
+
+
+Allowable next actions:
+
+
+BZ2_bzRead
+  if bzerror is BZ_OK
+BZ2_bzClose
+  otherwise
+
+
+
+
+
+
+<computeroutput>BZ2_bzRead</computeroutput>
+
+
+int BZ2_bzRead ( int *bzerror, BZFILE *b, void *buf, int len );
+
+
+Reads up to len
+(uncompressed) bytes from the compressed file
+b into the buffer
+buf.  If the read was
+successful, bzerror is set to
+BZ_OK and the number of bytes
+read is returned.  If the logical end-of-stream was detected,
+bzerror will be set to
+BZ_STREAM_END, and the number of
+bytes read is returned.  All other
+bzerror values denote an
+error.
+
+BZ2_bzRead will supply
+len bytes, unless the logical
+stream end is detected or an error occurs.  Because of this, it
+is possible to detect the stream end by observing when the number
+of bytes returned is less than the number requested.
+Nevertheless, this is regarded as inadvisable; you should instead
+check bzerror after every call
+and watch out for
+BZ_STREAM_END.
+
+Internally, BZ2_bzRead
+copies data from the compressed file in chunks of size
+BZ_MAX_UNUSED bytes before
+decompressing it.  If the file contains more bytes than strictly
+needed to reach the logical end-of-stream,
+BZ2_bzRead will almost certainly
+read some of the trailing data before signalling
+BZ_SEQUENCE_END.  To collect the
+read but unused data once
+BZ_SEQUENCE_END has appeared,
+call BZ2_bzReadGetUnused
+immediately before
+BZ2_bzReadClose.
+
+Possible assignments to
+bzerror:
+
+
+BZ_PARAM_ERROR
+  if b is NULL or buf is NULL or len < 0
+BZ_SEQUENCE_ERROR
+  if b was opened with BZ2_bzWriteOpen
+BZ_IO_ERROR
+  if there is an error reading from the compressed file
+BZ_UNEXPECTED_EOF
+  if the compressed file ended before 
+  the logical end-of-stream was detected
+BZ_DATA_ERROR
+  if a data integrity error was detected in the compressed stream
+BZ_DATA_ERROR_MAGIC
+  if the stream does not begin with the requisite header bytes 
+  (ie, is not a bzip2 data file).  This is really 
+  a special case of BZ_DATA_ERROR.
+BZ_MEM_ERROR
+  if insufficient memory was available
+BZ_STREAM_END
+  if the logical end of stream was detected.
+BZ_OK
+  otherwise.
+
+
+Possible return values:
+
+
+number of bytes read
+  if bzerror is BZ_OK or BZ_STREAM_END
+undefined
+  otherwise
+
+
+Allowable next actions:
+
+
+collect data from buf, then BZ2_bzRead or BZ2_bzReadClose
+  if bzerror is BZ_OK
+collect data from buf, then BZ2_bzReadClose or BZ2_bzReadGetUnused
+  if bzerror is BZ_SEQUENCE_END
+BZ2_bzReadClose
+  otherwise
+
+
+
+
+
+
+<computeroutput>BZ2_bzReadGetUnused</computeroutput>
+
+
+void BZ2_bzReadGetUnused( int* bzerror, BZFILE *b, 
+                          void** unused, int* nUnused );
+
+
+Returns data which was read from the compressed file but
+was not needed to get to the logical end-of-stream.
+*unused is set to the address of
+the data, and *nUnused to the
+number of bytes.  *nUnused will
+be set to a value between 0 and
+BZ_MAX_UNUSED inclusive.
+
+This function may only be called once
+BZ2_bzRead has signalled
+BZ_STREAM_END but before
+BZ2_bzReadClose.
+
+Possible assignments to
+bzerror:
+
+
+BZ_PARAM_ERROR
+  if b is NULL
+  or unused is NULL or nUnused is NULL
+BZ_SEQUENCE_ERROR
+  if BZ_STREAM_END has not been signalled
+  or if b was opened with BZ2_bzWriteOpen
+BZ_OK
+  otherwise
+
+
+Allowable next actions:
+
+
+BZ2_bzReadClose
+
+
+
+
+
+
+<computeroutput>BZ2_bzReadClose</computeroutput>
+
+
+void BZ2_bzReadClose ( int *bzerror, BZFILE *b );
+
+
+Releases all memory pertaining to the compressed file
+b.
+BZ2_bzReadClose does not call
+fclose on the underlying file
+handle, so you should do that yourself if appropriate.
+BZ2_bzReadClose should be called
+to clean up after all error situations.
+
+Possible assignments to
+bzerror:
+
+
+BZ_SEQUENCE_ERROR
+  if b was opened with BZ2_bzOpenWrite
+BZ_OK
+  otherwise
+
+
+Allowable next actions:
+
+
+none
+
+
+
+
+
+
+<computeroutput>BZ2_bzWriteOpen</computeroutput>
+
+
+BZFILE *BZ2_bzWriteOpen( int *bzerror, FILE *f, 
+                         int blockSize100k, int verbosity,
+                         int workFactor );
+
+
+Prepare to write compressed data to file handle
+f.
+f should refer to a file which
+has been opened for writing, and for which the error indicator
+(ferror(f))is not set.
+
+For the meaning of parameters
+blockSize100k,
+verbosity and
+workFactor, see
+BZ2_bzCompressInit.
+
+All required memory is allocated at this stage, so if the
+call completes successfully,
+BZ_MEM_ERROR cannot be signalled
+by a subsequent call to
+BZ2_bzWrite.
+
+Possible assignments to
+bzerror:
+
+
+BZ_CONFIG_ERROR
+  if the library has been mis-compiled
+BZ_PARAM_ERROR
+  if f is NULL
+  or blockSize100k < 1 or blockSize100k > 9
+BZ_IO_ERROR
+  if ferror(f) is nonzero
+BZ_MEM_ERROR
+  if insufficient memory is available
+BZ_OK
+  otherwise
+
+
+Possible return values:
+
+
+Pointer to an abstract BZFILE
+  if bzerror is BZ_OK
+NULL
+  otherwise
+
+
+Allowable next actions:
+
+
+BZ2_bzWrite
+  if bzerror is BZ_OK
+  (you could go directly to BZ2_bzWriteClose, but this would be pretty pointless)
+BZ2_bzWriteClose
+  otherwise
+
+
+
+
+
+
+<computeroutput>BZ2_bzWrite</computeroutput>
+
+
+void BZ2_bzWrite ( int *bzerror, BZFILE *b, void *buf, int len );
+
+
+Absorbs len bytes from the
+buffer buf, eventually to be
+compressed and written to the file.
+
+Possible assignments to
+bzerror:
+
+
+BZ_PARAM_ERROR
+  if b is NULL or buf is NULL or len < 0
+BZ_SEQUENCE_ERROR
+  if b was opened with BZ2_bzReadOpen
+BZ_IO_ERROR
+  if there is an error writing the compressed file.
+BZ_OK
+  otherwise
+
+
+
+
+
+
+<computeroutput>BZ2_bzWriteClose</computeroutput>
+
+
+void BZ2_bzWriteClose( int *bzerror, BZFILE* f,
+                       int abandon,
+                       unsigned int* nbytes_in,
+                       unsigned int* nbytes_out );
+
+void BZ2_bzWriteClose64( int *bzerror, BZFILE* f,
+                         int abandon,
+                         unsigned int* nbytes_in_lo32,
+                         unsigned int* nbytes_in_hi32,
+                         unsigned int* nbytes_out_lo32,
+                         unsigned int* nbytes_out_hi32 );
+
+
+Compresses and flushes to the compressed file all data so
+far supplied by BZ2_bzWrite.
+The logical end-of-stream markers are also written, so subsequent
+calls to BZ2_bzWrite are
+illegal.  All memory associated with the compressed file
+b is released.
+fflush is called on the
+compressed file, but it is not
+fclose'd.
+
+If BZ2_bzWriteClose is
+called to clean up after an error, the only action is to release
+the memory.  The library records the error codes issued by
+previous calls, so this situation will be detected automatically.
+There is no attempt to complete the compression operation, nor to
+fflush the compressed file.  You
+can force this behaviour to happen even in the case of no error,
+by passing a nonzero value to
+abandon.
+
+If nbytes_in is non-null,
+*nbytes_in will be set to be the
+total volume of uncompressed data handled.  Similarly,
+nbytes_out will be set to the
+total volume of compressed data written.  For compatibility with
+older versions of the library,
+BZ2_bzWriteClose only yields the
+lower 32 bits of these counts.  Use
+BZ2_bzWriteClose64 if you want
+the full 64 bit counts.  These two functions are otherwise
+absolutely identical.
+
+Possible assignments to
+bzerror:
+
+
+BZ_SEQUENCE_ERROR
+  if b was opened with BZ2_bzReadOpen
+BZ_IO_ERROR
+  if there is an error writing the compressed file
+BZ_OK
+  otherwise
+
+
+
+
+
+
+Handling embedded compressed data streams
+
+The high-level library facilitates use of
+bzip2 data streams which form
+some part of a surrounding, larger data stream.
+
+
+
+ For writing, the library takes an open file handle,
+  writes compressed data to it,
+  fflushes it but does not
+  fclose it.  The calling
+  application can write its own data before and after the
+  compressed data stream, using that same file handle.
+
+ Reading is more complex, and the facilities are not as
+  general as they could be since generality is hard to reconcile
+  with efficiency.  BZ2_bzRead
+  reads from the compressed file in blocks of size
+  BZ_MAX_UNUSED bytes, and in
+  doing so probably will overshoot the logical end of compressed
+  stream.  To recover this data once decompression has ended,
+  call BZ2_bzReadGetUnused after
+  the last call of BZ2_bzRead
+  (the one returning
+  BZ_STREAM_END) but before
+  calling
+  BZ2_bzReadClose.
+
+
+
+This mechanism makes it easy to decompress multiple
+bzip2 streams placed end-to-end.
+As the end of one stream, when
+BZ2_bzRead returns
+BZ_STREAM_END, call
+BZ2_bzReadGetUnused to collect
+the unused data (copy it into your own buffer somewhere).  That
+data forms the start of the next compressed stream.  To start
+uncompressing that next stream, call
+BZ2_bzReadOpen again, feeding in
+the unused data via the unused /
+nUnused parameters.  Keep doing
+this until BZ_STREAM_END return
+coincides with the physical end of file
+(feof(f)).  In this situation
+BZ2_bzReadGetUnused will of
+course return no data.
+
+This should give some feel for how the high-level interface
+can be used.  If you require extra flexibility, you'll have to
+bite the bullet and get to grips with the low-level
+interface.
+
+
+
+
+
+Standard file-reading/writing code
+
+Here's how you'd write data to a compressed file:
+
+
+FILE*   f;
+BZFILE* b;
+int     nBuf;
+char    buf[ /* whatever size you like */ ];
+int     bzerror;
+int     nWritten;
+
+f = fopen ( "myfile.bz2", "w" );
+if ( !f ) {
+ /* handle error */
+}
+b = BZ2_bzWriteOpen( &bzerror, f, 9 );
+if (bzerror != BZ_OK) {
+ BZ2_bzWriteClose ( b );
+ /* handle error */
+}
+
+while ( /* condition */ ) {
+ /* get data to write into buf, and set nBuf appropriately */
+ nWritten = BZ2_bzWrite ( &bzerror, b, buf, nBuf );
+ if (bzerror == BZ_IO_ERROR) { 
+   BZ2_bzWriteClose ( &bzerror, b );
+   /* handle error */
+ }
+}
+
+BZ2_bzWriteClose( &bzerror, b );
+if (bzerror == BZ_IO_ERROR) {
+ /* handle error */
+}
+
+
+And to read from a compressed file:
+
+
+FILE*   f;
+BZFILE* b;
+int     nBuf;
+char    buf[ /* whatever size you like */ ];
+int     bzerror;
+int     nWritten;
+
+f = fopen ( "myfile.bz2", "r" );
+if ( !f ) {
+  /* handle error */
+}
+b = BZ2_bzReadOpen ( &bzerror, f, 0, NULL, 0 );
+if ( bzerror != BZ_OK ) {
+  BZ2_bzReadClose ( &bzerror, b );
+  /* handle error */
+}
+
+bzerror = BZ_OK;
+while ( bzerror == BZ_OK && /* arbitrary other conditions */) {
+  nBuf = BZ2_bzRead ( &bzerror, b, buf, /* size of buf */ );
+  if ( bzerror == BZ_OK ) {
+    /* do something with buf[0 .. nBuf-1] */
+  }
+}
+if ( bzerror != BZ_STREAM_END ) {
+   BZ2_bzReadClose ( &bzerror, b );
+   /* handle error */
+} else {
+   BZ2_bzReadClose ( &bzerror );
+}
+
+
+
+
+
+
+
+
+Utility functions
+
+
+
+<computeroutput>BZ2_bzBuffToBuffCompress</computeroutput>
+
+
+int BZ2_bzBuffToBuffCompress( char*         dest,
+                              unsigned int* destLen,
+                              char*         source,
+                              unsigned int  sourceLen,
+                              int           blockSize100k,
+                              int           verbosity,
+                              int           workFactor );
+
+
+Attempts to compress the data in source[0
+.. sourceLen-1] into the destination buffer,
+dest[0 .. *destLen-1].  If the
+destination buffer is big enough,
+*destLen is set to the size of
+the compressed data, and BZ_OK
+is returned.  If the compressed data won't fit,
+*destLen is unchanged, and
+BZ_OUTBUFF_FULL is
+returned.
+
+Compression in this manner is a one-shot event, done with a
+single call to this function.  The resulting compressed data is a
+complete bzip2 format data
+stream.  There is no mechanism for making additional calls to
+provide extra input data.  If you want that kind of mechanism,
+use the low-level interface.
+
+For the meaning of parameters
+blockSize100k,
+verbosity and
+workFactor, see
+BZ2_bzCompressInit.
+
+To guarantee that the compressed data will fit in its
+buffer, allocate an output buffer of size 1% larger than the
+uncompressed data, plus six hundred extra bytes.
+
+BZ2_bzBuffToBuffDecompress
+will not write data at or beyond
+dest[*destLen], even in case of
+buffer overflow.
+
+Possible return values:
+
+
+BZ_CONFIG_ERROR
+  if the library has been mis-compiled
+BZ_PARAM_ERROR
+  if dest is NULL or destLen is NULL
+  or blockSize100k < 1 or blockSize100k > 9
+  or verbosity < 0 or verbosity > 4
+  or workFactor < 0 or workFactor > 250
+BZ_MEM_ERROR
+  if insufficient memory is available 
+BZ_OUTBUFF_FULL
+  if the size of the compressed data exceeds *destLen
+BZ_OK
+  otherwise
+
+
+
+
+
+
+<computeroutput>BZ2_bzBuffToBuffDecompress</computeroutput>
+
+
+int BZ2_bzBuffToBuffDecompress( char*         dest,
+                                unsigned int* destLen,
+                                char*         source,
+                                unsigned int  sourceLen,
+                                int           small,
+                                int           verbosity );
+
+
+Attempts to decompress the data in source[0
+.. sourceLen-1] into the destination buffer,
+dest[0 .. *destLen-1].  If the
+destination buffer is big enough,
+*destLen is set to the size of
+the uncompressed data, and BZ_OK
+is returned.  If the compressed data won't fit,
+*destLen is unchanged, and
+BZ_OUTBUFF_FULL is
+returned.
+
+source is assumed to hold
+a complete bzip2 format data
+stream.
+BZ2_bzBuffToBuffDecompress tries
+to decompress the entirety of the stream into the output
+buffer.
+
+For the meaning of parameters
+small and
+verbosity, see
+BZ2_bzDecompressInit.
+
+Because the compression ratio of the compressed data cannot
+be known in advance, there is no easy way to guarantee that the
+output buffer will be big enough.  You may of course make
+arrangements in your code to record the size of the uncompressed
+data, but such a mechanism is beyond the scope of this
+library.
+
+BZ2_bzBuffToBuffDecompress
+will not write data at or beyond
+dest[*destLen], even in case of
+buffer overflow.
+
+Possible return values:
+
+
+BZ_CONFIG_ERROR
+  if the library has been mis-compiled
+BZ_PARAM_ERROR
+  if dest is NULL or destLen is NULL
+  or small != 0 && small != 1
+  or verbosity < 0 or verbosity > 4
+BZ_MEM_ERROR
+  if insufficient memory is available 
+BZ_OUTBUFF_FULL
+  if the size of the compressed data exceeds *destLen
+BZ_DATA_ERROR
+  if a data integrity error was detected in the compressed data
+BZ_DATA_ERROR_MAGIC
+  if the compressed data doesn't begin with the right magic bytes
+BZ_UNEXPECTED_EOF
+  if the compressed data ends unexpectedly
+BZ_OK
+  otherwise
+
+
+
+
+
+
+
+
+<computeroutput>zlib</computeroutput> compatibility functions
+
+Yoshioka Tsuneo has contributed some functions to give
+better zlib compatibility.
+These functions are BZ2_bzopen,
+BZ2_bzread,
+BZ2_bzwrite,
+BZ2_bzflush,
+BZ2_bzclose,
+BZ2_bzerror and
+BZ2_bzlibVersion.  These
+functions are not (yet) officially part of the library.  If they
+break, you get to keep all the pieces.  Nevertheless, I think
+they work ok.
+
+
+typedef void BZFILE;
+
+const char * BZ2_bzlibVersion ( void );
+
+
+Returns a string indicating the library version.
+
+
+BZFILE * BZ2_bzopen  ( const char *path, const char *mode );
+BZFILE * BZ2_bzdopen ( int        fd,    const char *mode );
+
+
+Opens a .bz2 file for
+reading or writing, using either its name or a pre-existing file
+descriptor.  Analogous to fopen
+and fdopen.
+
+
+int BZ2_bzread  ( BZFILE* b, void* buf, int len );
+int BZ2_bzwrite ( BZFILE* b, void* buf, int len );
+
+
+Reads/writes data from/to a previously opened
+BZFILE.  Analogous to
+fread and
+fwrite.
+
+
+int  BZ2_bzflush ( BZFILE* b );
+void BZ2_bzclose ( BZFILE* b );
+
+
+Flushes/closes a BZFILE.
+BZ2_bzflush doesn't actually do
+anything.  Analogous to fflush
+and fclose.
+
+
+const char * BZ2_bzerror ( BZFILE *b, int *errnum )
+
+
+Returns a string describing the more recent error status of
+b, and also sets
+*errnum to its numerical
+value.
+
+
+
+
+
+Using the library in a <computeroutput>stdio</computeroutput>-free environment
+
+
+
+Getting rid of <computeroutput>stdio</computeroutput>
+
+In a deeply embedded application, you might want to use
+just the memory-to-memory functions.  You can do this
+conveniently by compiling the library with preprocessor symbol
+BZ_NO_STDIO defined.  Doing this
+gives you a library containing only the following eight
+functions:
+
+BZ2_bzCompressInit,
+BZ2_bzCompress,
+BZ2_bzCompressEnd
+BZ2_bzDecompressInit,
+BZ2_bzDecompress,
+BZ2_bzDecompressEnd
+BZ2_bzBuffToBuffCompress,
+BZ2_bzBuffToBuffDecompress
+
+When compiled like this, all functions will ignore
+verbosity settings.
+
+
+
+
+
+Critical error handling
+
+libbzip2 contains a number
+of internal assertion checks which should, needless to say, never
+be activated.  Nevertheless, if an assertion should fail,
+behaviour depends on whether or not the library was compiled with
+BZ_NO_STDIO set.
+
+For a normal compile, an assertion failure yields the
+message:
+
+

+bzip2/libbzip2: internal error number N.
+This is a bug in bzip2/libbzip2, &bz-version; of &bz-date;.
+Please report it to me at: &bz-email;.  If this happened
+when you were using some program which uses libbzip2 as a
+component, you should also report this bug to the author(s)
+of that program.  Please make an effort to report this bug;
+timely and accurate bug reports eventually lead to higher
+quality software.  Thanks.  Julian Seward, &bz-date;.
+

+
+where N is some error code
+number.  If N == 1007, it also
+prints some extra text advising the reader that unreliable memory
+is often associated with internal error 1007. (This is a
+frequently-observed-phenomenon with versions 1.0.0/1.0.1).
+
+exit(3) is then
+called.
+
+For a stdio-free library,
+assertion failures result in a call to a function declared
+as:
+
+
+extern void bz_internal_error ( int errcode );
+
+
+The relevant code is passed as a parameter.  You should
+supply such a function.
+
+In either case, once an assertion failure has occurred, any
+bz_stream records involved can
+be regarded as invalid.  You should not attempt to resume normal
+operation with them.
+
+You may, of course, change critical error handling to suit
+your needs.  As I said above, critical errors indicate bugs in
+the library and should not occur.  All "normal" error situations
+are indicated via error return codes from functions, and can be
+recovered from.
+
+
+
+
+
+
+
+Making a Windows DLL
+
+Everything related to Windows has been contributed by
+Yoshioka Tsuneo
+(QWF00133@niftyserve.or.jp /
+tsuneo-y@is.aist-nara.ac.jp), so
+you should send your queries to him (but perhaps Cc: me,
+&bz-email;).
+
+My vague understanding of what to do is: using Visual C++
+5.0, open the project file
+libbz2.dsp, and build.  That's
+all.
+
+If you can't open the project file for some reason, make a
+new one, naming these files:
+blocksort.c,
+bzlib.c,
+compress.c,
+crctable.c,
+decompress.c,
+huffman.c,
+randtable.c and
+libbz2.def.  You will also need
+to name the header files bzlib.h
+and bzlib_private.h.
+
+If you don't use VC++, you may need to define the
+proprocessor symbol
+_WIN32.
+
+Finally, dlltest.c is a
+sample program using the DLL.  It has a project file,
+dlltest.dsp.
+
+If you just want a makefile for Visual C, have a look at
+makefile.msc.
+
+Be aware that if you compile
+bzip2 itself on Win32, you must
+set BZ_UNIX to 0 and
+BZ_LCCWIN32 to 1, in the file
+bzip2.c, before compiling.
+Otherwise the resulting binary won't work correctly.
+
+I haven't tried any of this stuff myself, but it all looks
+plausible.
+
+
+
+
+
+
+
+
+Miscellanea
+
+These are just some random thoughts of mine.  Your mileage
+may vary.
+
+
+
+Limitations of the compressed file format
+
+bzip2-1.0.X,
+0.9.5 and
+0.9.0 use exactly the same file
+format as the original version,
+bzip2-0.1.  This decision was
+made in the interests of stability.  Creating yet another
+incompatible compressed file format would create further
+confusion and disruption for users.
+
+Nevertheless, this is not a painless decision.  Development
+work since the release of
+bzip2-0.1 in August 1997 has
+shown complexities in the file format which slow down
+decompression and, in retrospect, are unnecessary.  These
+are:
+
+
+
+ The run-length encoder, which is the first of the
+   compression transformations, is entirely irrelevant.  The
+   original purpose was to protect the sorting algorithm from the
+   very worst case input: a string of repeated symbols.  But
+   algorithm steps Q6a and Q6b in the original Burrows-Wheeler
+   technical report (SRC-124) show how repeats can be handled
+   without difficulty in block sorting.
+
+ The randomisation mechanism doesn't really need to be
+   there.  Udi Manber and Gene Myers published a suffix array
+   construction algorithm a few years back, which can be employed
+   to sort any block, no matter how repetitive, in O(N log N)
+   time.  Subsequent work by Kunihiko Sadakane has produced a
+   derivative O(N (log N)^2) algorithm which usually outperforms
+   the Manber-Myers algorithm.
+
+   I could have changed to Sadakane's algorithm, but I find
+   it to be slower than bzip2's
+   existing algorithm for most inputs, and the randomisation
+   mechanism protects adequately against bad cases.  I didn't
+   think it was a good tradeoff to make.  Partly this is due to
+   the fact that I was not flooded with email complaints about
+   bzip2-0.1's performance on
+   repetitive data, so perhaps it isn't a problem for real
+   inputs.
+
+   Probably the best long-term solution, and the one I have
+   incorporated into 0.9.5 and above, is to use the existing
+   sorting algorithm initially, and fall back to a O(N (log N)^2)
+   algorithm if the standard algorithm gets into
+   difficulties.
+
+  The compressed file format was never designed to be
+   handled by a library, and I have had to jump though some hoops
+   to produce an efficient implementation of decompression.  It's
+   a bit hairy.  Try passing
+   decompress.c through the C
+   preprocessor and you'll see what I mean.  Much of this
+   complexity could have been avoided if the compressed size of
+   each block of data was recorded in the data stream.
+
+ An Adler-32 checksum, rather than a CRC32 checksum,
+   would be faster to compute.
+
+
+
+It would be fair to say that the
+bzip2 format was frozen before I
+properly and fully understood the performance consequences of
+doing so.
+
+Improvements which I was able to incorporate into 0.9.0,
+despite using the same file format, are:
+
+
+
+ Single array implementation of the inverse BWT.  This
+  significantly speeds up decompression, presumably because it
+  reduces the number of cache misses.
+
+ Faster inverse MTF transform for large MTF values.
+  The new implementation is based on the notion of sliding blocks
+  of values.
+
+ bzip2-0.9.0 now reads
+  and writes files with fread
+  and fwrite; version 0.1 used
+  putc and
+  getc.  Duh!  Well, you live
+  and learn.
+
+
+
+Further ahead, it would be nice to be able to do random
+access into files.  This will require some careful design of
+compressed file formats.
+
+
+
+
+
+Portability issues
+
+After some consideration, I have decided not to use GNU
+autoconf to configure 0.9.5 or
+1.0.
+
+autoconf, admirable and
+wonderful though it is, mainly assists with portability problems
+between Unix-like platforms.  But
+bzip2 doesn't have much in the
+way of portability problems on Unix; most of the difficulties
+appear when porting to the Mac, or to Microsoft's operating
+systems.  autoconf doesn't help
+in those cases, and brings in a whole load of new
+complexity.
+
+Most people should be able to compile the library and
+program under Unix straight out-of-the-box, so to speak,
+especially if you have a version of GNU C available.
+
+There are a couple of
+__inline__ directives in the
+code.  GNU C (gcc) should be
+able to handle them.  If you're not using GNU C, your C compiler
+shouldn't see them at all.  If your compiler does, for some
+reason, see them and doesn't like them, just
+#define
+__inline__ to be
+/* */.  One easy way to do this
+is to compile with the flag
+-D__inline__=, which should be
+understood by most Unix compilers.
+
+If you still have difficulties, try compiling with the
+macro BZ_STRICT_ANSI defined.
+This should enable you to build the library in a strictly ANSI
+compliant environment.  Building the program itself like this is
+dangerous and not supported, since you remove
+bzip2's checks against
+compressing directories, symbolic links, devices, and other
+not-really-a-file entities.  This could cause filesystem
+corruption!
+
+One other thing: if you create a
+bzip2 binary for public distribution,
+please consider linking it statically (gcc
+-static).  This avoids all sorts of library-version
+issues that others may encounter later on.
+
+If you build bzip2 on
+Win32, you must set BZ_UNIX to 0
+and BZ_LCCWIN32 to 1, in the
+file bzip2.c, before compiling.
+Otherwise the resulting binary won't work correctly.
+
+
+
+
+
+Reporting bugs
+
+I tried pretty hard to make sure
+bzip2 is bug free, both by
+design and by testing.  Hopefully you'll never need to read this
+section for real.
+
+Nevertheless, if bzip2 dies
+with a segmentation fault, a bus error or an internal assertion
+failure, it will ask you to email me a bug report.  Experience from
+years of feedback of bzip2 users indicates that almost all these
+problems can be traced to either compiler bugs or hardware
+problems.
+
+
+
+ Recompile the program with no optimisation, and
+  see if it works.  And/or try a different compiler.  I heard all
+  sorts of stories about various flavours of GNU C (and other
+  compilers) generating bad code for
+  bzip2, and I've run across two
+  such examples myself.
+
+  2.7.X versions of GNU C are known to generate bad code
+  from time to time, at high optimisation levels.  If you get
+  problems, try using the flags
+  -O2
+  -fomit-frame-pointer
+  -fno-strength-reduce.  You
+  should specifically not use
+  -funroll-loops.
+
+  You may notice that the Makefile runs six tests as part
+  of the build process.  If the program passes all of these, it's
+  a pretty good (but not 100%) indication that the compiler has
+  done its job correctly.
+
+ If bzip2
+  crashes randomly, and the crashes are not repeatable, you may
+  have a flaky memory subsystem.
+  bzip2 really hammers your
+  memory hierarchy, and if it's a bit marginal, you may get these
+  problems.  Ditto if your disk or I/O subsystem is slowly
+  failing.  Yup, this really does happen.
+
+  Try using a different machine of the same type, and see
+  if you can repeat the problem.
+
+  This isn't really a bug, but ... If
+  bzip2 tells you your file is
+  corrupted on decompression, and you obtained the file via FTP,
+  there is a possibility that you forgot to tell FTP to do a
+  binary mode transfer.  That absolutely will cause the file to
+  be non-decompressible.  You'll have to transfer it
+  again.
+
+
+
+If you've incorporated
+libbzip2 into your own program
+and are getting problems, please, please, please, check that the
+parameters you are passing in calls to the library, are correct,
+and in accordance with what the documentation says is allowable.
+I have tried to make the library robust against such problems,
+but I'm sure I haven't succeeded.
+
+Finally, if the above comments don't help, you'll have to
+send me a bug report.  Now, it's just amazing how many people
+will send me a bug report saying something like:
+
+
+bzip2 crashed with segmentation fault on my machine
+
+
+and absolutely nothing else.  Needless to say, a such a
+report is totally, utterly, completely and
+comprehensively 100% useless; a waste of your time, my time, and
+net bandwidth.  With no details at all, there's no way
+I can possibly begin to figure out what the problem is.
+
+The rules of the game are: facts, facts, facts.  Don't omit
+them because "oh, they won't be relevant".  At the bare
+minimum:
+
+
+Machine type.  Operating system version.  
+Exact version of bzip2 (do bzip2 -V).  
+Exact version of the compiler used.  
+Flags passed to the compiler.
+
+
+However, the most important single thing that will help me
+is the file that you were trying to compress or decompress at the
+time the problem happened.  Without that, my ability to do
+anything more than speculate about the cause, is limited.
+
+
+
+
+
+Did you get the right package?
+
+bzip2 is a resource hog.
+It soaks up large amounts of CPU cycles and memory.  Also, it
+gives very large latencies.  In the worst case, you can feed many
+megabytes of uncompressed data into the library before getting
+any compressed output, so this probably rules out applications
+requiring interactive behaviour.
+
+These aren't faults of my implementation, I hope, but more
+an intrinsic property of the Burrows-Wheeler transform
+(unfortunately).  Maybe this isn't what you want.
+
+If you want a compressor and/or library which is faster,
+uses less memory but gets pretty good compression, and has
+minimal latency, consider Jean-loup Gailly's and Mark Adler's
+work, zlib-1.2.1 and
+gzip-1.2.4.  Look for them at 
+http://www.zlib.org and 
+http://www.gzip.org
+respectively.
+
+For something faster and lighter still, you might try Markus F
+X J Oberhumer's LZO real-time
+compression/decompression library, at 
+http://www.oberhumer.com/opensource.
+
+
+
+
+
+
+Further Reading
+
+bzip2 is not research
+work, in the sense that it doesn't present any new ideas.
+Rather, it's an engineering exercise based on existing
+ideas.
+
+Four documents describe essentially all the ideas behind
+bzip2:
+
+Michael Burrows and D. J. Wheeler:
+  "A block-sorting lossless data compression algorithm"
+   10th May 1994. 
+   Digital SRC Research Report 124.
+   ftp://ftp.digital.com/pub/DEC/SRC/research-reports/SRC-124.ps.gz
+   If you have trouble finding it, try searching at the
+   New Zealand Digital Library, http://www.nzdl.org.
+
+Daniel S. Hirschberg and Debra A. LeLewer
+  "Efficient Decoding of Prefix Codes"
+   Communications of the ACM, April 1990, Vol 33, Number 4.
+   You might be able to get an electronic copy of this
+   from the ACM Digital Library.
+
+David J. Wheeler
+   Program bred3.c and accompanying document bred3.ps.
+   This contains the idea behind the multi-table Huffman coding scheme.
+   ftp://ftp.cl.cam.ac.uk/users/djw3/
+
+Jon L. Bentley and Robert Sedgewick
+  "Fast Algorithms for Sorting and Searching Strings"
+   Available from Sedgewick's web page,
+   www.cs.princeton.edu/~rs
+
+
+The following paper gives valuable additional insights into
+the algorithm, but is not immediately the basis of any code used
+in bzip2.
+
+Peter Fenwick:
+   Block Sorting Text Compression
+   Proceedings of the 19th Australasian Computer Science Conference,
+     Melbourne, Australia.  Jan 31 - Feb 2, 1996.
+   ftp://ftp.cs.auckland.ac.nz/pub/peter-f/ACSC96paper.ps
+
+Kunihiko Sadakane's sorting algorithm, mentioned above, is
+available from:
+
+http://naomi.is.s.u-tokyo.ac.jp/~sada/papers/Sada98b.ps.gz
+
+
+The Manber-Myers suffix array construction algorithm is
+described in a paper available from:
+
+http://www.cs.arizona.edu/people/gene/PAPERS/suffix.ps
+
+
+Finally, the following papers document some
+investigations I made into the performance of sorting
+and decompression algorithms:
+
+Julian Seward
+   On the Performance of BWT Sorting Algorithms
+   Proceedings of the IEEE Data Compression Conference 2000
+     Snowbird, Utah.  28-30 March 2000.
+
+Julian Seward
+   Space-time Tradeoffs in the Inverse B-W Transform
+   Proceedings of the IEEE Data Compression Conference 2001
+     Snowbird, Utah.  27-29 March 2001.
+
+
+
+
+
+
+
diff --git a/randtable.c b/randtable.c
index 5c922e9..940462d 100644
--- a/randtable.c
+++ b/randtable.c
@@ -8,7 +8,7 @@
   This file is a part of bzip2 and/or libbzip2, a program and
   library for lossless, block-sorting data compression.
 
-  Copyright (C) 1996-2002 Julian R Seward.  All rights reserved.
+  Copyright (C) 1996-2005 Julian R Seward.  All rights reserved.
 
   Redistribution and use in source and binary forms, with or without
   modification, are permitted provided that the following conditions
@@ -42,7 +42,7 @@
   SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
   Julian Seward, Cambridge, UK.
-  jseward@acm.org
+  jseward@bzip.org
   bzip2/libbzip2 version 1.0 of 21 March 2000
 
   This program is based on (at least) the work of:
diff --git a/xmlproc.sh b/xmlproc.sh
new file mode 100755
index 0000000..6fe4d57
--- /dev/null
+++ b/xmlproc.sh
@@ -0,0 +1,99 @@
+#!/bin/bash
+# see the README in this directory for usage etc.
+
+usage() {
+  echo '';
+  echo 'Usage: xmlproc.sh -[option] ';
+  echo 'Specify a target from:';
+  echo '-v      verify xml file conforms to dtd';
+  echo '-html   output in html format (single file)';
+  echo '-ps     output in postscript format';
+  echo '-pdf    output in pdf format';
+  exit;
+}
+
+if test $# -ne 2; then
+  usage
+fi
+# assign the variable for the output type
+action=$1; shift
+# assign the output filename
+xmlfile=$1; shift
+# and check user input it correct
+if !(test -f $xmlfile); then
+  echo "No such file: $xmlfile";
+  exit;
+fi
+# some other stuff we will use
+OUT=output
+xsl_fo=bz-fo.xsl
+xsl_html=bz-html.xsl
+
+basename=$xmlfile
+basename=${basename//'.xml'/''}
+
+fofile="${basename}.fo"
+htmlfile="${basename}.html"
+pdffile="${basename}.pdf"
+psfile="${basename}.ps"
+xmlfmtfile="${basename}.fmt"
+
+# first process the xmlfile with CDATA tags
+./format.pl $xmlfile $xmlfmtfile
+# so the shell knows where the catalogs live
+export XML_CATALOG_FILES=/etc/xml/catalog
+
+# post-processing tidy up
+cleanup() {
+  echo "Cleaning up: # $@" 
+  while [ $# != 0 ]
+  do
+    arg=$1; shift;
+    echo "  deleting $arg";
+    rm $arg
+  done
+}
+
+case $action in
+  -v)
+   flags='--noout --xinclude --noblanks --postvalid'
+   dtd='--dtdvalid http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd'
+   xmllint $flags $dtd $xmlfmtfile 2> $OUT 
+   egrep 'error' $OUT 
+   rm $OUT
+  ;;
+
+  -html)
+   echo "Creating $htmlfile ..."
+   xsltproc --nonet --xinclude  -o $htmlfile $xsl_html $xmlfmtfile
+   cleanup $xmlfmtfile
+  ;;
+
+  -pdf)
+   echo "Creating $pdffile ..."
+   xsltproc --nonet --xinclude -o $fofile $xsl_fo $xmlfmtfile
+   pdfxmltex $fofile >$OUT $OUT $OUT $OUT $OUT $OUT $OUT $OUT $OUT 
Date: Wed, 20 Dec 2006 22:13:13 +0100
Subject: bzip2-1.0.4

---
 CHANGES                     |  41 ++++++++-
 LICENSE                     |   9 +-
 Makefile                    |  30 +++++--
 Makefile-libbz2_so          |  31 +++++--
 README                      | 100 ++++++++++++---------
 README.COMPILATION.PROBLEMS |  37 ++++++--
 README.XML.STUFF            |  22 ++++-
 Y2K_INFO                    |  34 --------
 blocksort.c                 |  77 ++++-------------
 bz-fo.xsl                   |  19 ++++
 bzdiff                      |   4 +-
 bzgrep                      |   6 +-
 bzip2.1                     |   6 +-
 bzip2.1.preformatted        |  14 +--
 bzip2.c                     | 205 ++++++++++++++------------------------------
 bzip2.txt                   |  14 +--
 bzip2recover.c              |  88 ++++++-------------
 bzlib.c                     |  95 ++++++--------------
 bzlib.h                     |  69 +++------------
 bzlib_private.h             |  88 ++++++-------------
 compress.c                  |  86 +++++--------------
 crctable.c                  |  60 +++----------
 decompress.c                |  66 +++-----------
 dlltest.c                   |   5 +-
 entities.xml                |   6 +-
 format.pl                   |  15 ++++
 huffman.c                   |  66 +++-----------
 manual.xml                  |  28 +++---
 mk251.c                     |  15 ++++
 randtable.c                 |  60 +++----------
 spewG.c                     |  15 ++++
 unzcrash.c                  |  17 +++-
 words0                      |   6 +-
 words2                      |   2 +-
 words3                      |  13 ++-
 xmlproc.sh                  |  19 +++-
 36 files changed, 593 insertions(+), 875 deletions(-)
 delete mode 100644 Y2K_INFO

diff --git a/CHANGES b/CHANGES
index e31b03a..0959ea0 100644
--- a/CHANGES
+++ b/CHANGES
@@ -1,3 +1,16 @@
+ ------------------------------------------------------------------
+ This file is part of bzip2/libbzip2, a program and library for
+ lossless, block-sorting data compression.
+
+ bzip2/libbzip2 version 1.0.4 of 20 December 2006
+ Copyright (C) 1996-2006 Julian Seward 
+
+ Please read the WARNING, DISCLAIMER and PATENTS sections in the 
+ README file.
+
+ This program is released under the terms of the license contained
+ in the file LICENSE.
+ ------------------------------------------------------------------
 
 
 0.9.0
@@ -261,7 +274,7 @@ Fixes some minor bugs since the last version, 1.0.2.
   decompressor to crash, loop or access memory which does not
   belong to it.  If you are using bzip2 or the library to 
   decompress bitstreams from untrusted sources, an upgrade
-  to 1.0.3 is recommended.
+  to 1.0.3 is recommended.  This fixes CAN-2005-1260.
 
 * The documentation has been converted to XML, from which html
   and pdf can be derived.
@@ -273,3 +286,29 @@ Fixes some minor bugs since the last version, 1.0.2.
 
 * The BZ_NO_STDIO cpp symbol was not properly observed in 1.0.2.
   This has been fixed.
+
+
+1.0.4 (20 Dec 06)
+~~~~~~~~~~~~~~~~~
+Fixes some minor bugs since the last version, 1.0.3.
+
+* Fix file permissions race problem (CAN-2005-0953).
+
+* Avoid possible segfault in BZ2_bzclose.  From Coverity's NetBSD
+  scan.
+
+* 'const'/prototype cleanups in the C code.
+
+* Change default install location to /usr/local, and handle multiple
+  'make install's without error.
+
+* Sanitise file names more carefully in bzgrep.  Fixes CAN-2005-0758
+  to the extent that applies to bzgrep.
+
+* Use 'mktemp' rather than 'tempfile' in bzdiff.
+
+* Tighten up a couple of assertions in blocksort.c following automated
+  analysis.
+
+* Fix minor doc/comment bugs.
+
diff --git a/LICENSE b/LICENSE
index e60845b..4458e35 100644
--- a/LICENSE
+++ b/LICENSE
@@ -1,6 +1,8 @@
 
+--------------------------------------------------------------------------
+
 This program, "bzip2", the associated library "libbzip2", and all
-documentation, are copyright (C) 1996-2005 Julian R Seward.  All
+documentation, are copyright (C) 1996-2006 Julian R Seward.  All
 rights reserved.
 
 Redistribution and use in source and binary forms, with or without
@@ -35,6 +37,7 @@ NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
 Julian Seward, Cambridge, UK.
-jseward@acm.org
-bzip2/libbzip2 version 1.0.3 of 15 February 2005
+jseward@bzip.org
+bzip2/libbzip2 version 1.0.4 of 20 December 2006
 
+--------------------------------------------------------------------------
diff --git a/Makefile b/Makefile
index eea329a..c4cbc04 100644
--- a/Makefile
+++ b/Makefile
@@ -1,3 +1,16 @@
+# ------------------------------------------------------------------
+# This file is part of bzip2/libbzip2, a program and library for
+# lossless, block-sorting data compression.
+#
+# bzip2/libbzip2 version 1.0.4 of 20 December 2006
+# Copyright (C) 1996-2006 Julian Seward 
+#
+# Please read the WARNING, DISCLAIMER and PATENTS sections in the 
+# README file.
+#
+# This program is released under the terms of the license contained
+# in the file LICENSE.
+# ------------------------------------------------------------------
 
 SHELL=/bin/sh
 
@@ -8,10 +21,10 @@ RANLIB=ranlib
 LDFLAGS=
 
 BIGFILES=-D_FILE_OFFSET_BITS=64
-CFLAGS=-Wall -Winline -O -g $(BIGFILES)
+CFLAGS=-Wall -Winline -O2 -g $(BIGFILES)
 
 # Where you want it installed when you do 'make install'
-PREFIX=/usr
+PREFIX=/usr/local
 
 
 OBJS= blocksort.o  \
@@ -77,14 +90,14 @@ install: bzip2 bzip2recover
 	cp -f libbz2.a $(PREFIX)/lib
 	chmod a+r $(PREFIX)/lib/libbz2.a
 	cp -f bzgrep $(PREFIX)/bin/bzgrep
-	ln $(PREFIX)/bin/bzgrep $(PREFIX)/bin/bzegrep
-	ln $(PREFIX)/bin/bzgrep $(PREFIX)/bin/bzfgrep
+	ln -s -f $(PREFIX)/bin/bzgrep $(PREFIX)/bin/bzegrep
+	ln -s -f $(PREFIX)/bin/bzgrep $(PREFIX)/bin/bzfgrep
 	chmod a+x $(PREFIX)/bin/bzgrep
 	cp -f bzmore $(PREFIX)/bin/bzmore
-	ln $(PREFIX)/bin/bzmore $(PREFIX)/bin/bzless
+	ln -s -f $(PREFIX)/bin/bzmore $(PREFIX)/bin/bzless
 	chmod a+x $(PREFIX)/bin/bzmore
 	cp -f bzdiff $(PREFIX)/bin/bzdiff
-	ln $(PREFIX)/bin/bzdiff $(PREFIX)/bin/bzcmp
+	ln -s -f $(PREFIX)/bin/bzdiff $(PREFIX)/bin/bzcmp
 	chmod a+x $(PREFIX)/bin/bzdiff
 	cp -f bzgrep.1 bzmore.1 bzdiff.1 $(PREFIX)/man/man1
 	chmod a+r $(PREFIX)/man/man1/bzgrep.1
@@ -124,10 +137,10 @@ bzip2recover.o: bzip2recover.c
 distclean: clean
 	rm -f manual.ps manual.html manual.pdf
 
-DISTNAME=bzip2-1.0.3
+DISTNAME=bzip2-1.0.4
 dist: check manual
 	rm -f $(DISTNAME)
-	ln -sf . $(DISTNAME)
+	ln -s -f . $(DISTNAME)
 	tar cvf $(DISTNAME).tar \
 	   $(DISTNAME)/blocksort.c \
 	   $(DISTNAME)/huffman.c \
@@ -167,7 +180,6 @@ dist: check manual
 	   $(DISTNAME)/libbz2.dsp \
 	   $(DISTNAME)/dlltest.dsp \
 	   $(DISTNAME)/makefile.msc \
-	   $(DISTNAME)/Y2K_INFO \
 	   $(DISTNAME)/unzcrash.c \
 	   $(DISTNAME)/spewG.c \
 	   $(DISTNAME)/mk251.c \
diff --git a/Makefile-libbz2_so b/Makefile-libbz2_so
index 458c5a1..2b65f3b 100644
--- a/Makefile-libbz2_so
+++ b/Makefile-libbz2_so
@@ -1,15 +1,30 @@
 
 # This Makefile builds a shared version of the library, 
-# libbz2.so.1.0.3, with soname libbz2.so.1.0,
+# libbz2.so.1.0.4, with soname libbz2.so.1.0,
 # at least on x86-Linux (RedHat 7.2), 
 # with gcc-2.96 20000731 (Red Hat Linux 7.1 2.96-98).  
-# Please see the README file for some 
-# important info about building the library like this.
+# Please see the README file for some important info 
+# about building the library like this.
+
+# ------------------------------------------------------------------
+# This file is part of bzip2/libbzip2, a program and library for
+# lossless, block-sorting data compression.
+#
+# bzip2/libbzip2 version 1.0.4 of 20 December 2006
+# Copyright (C) 1996-2006 Julian Seward 
+#
+# Please read the WARNING, DISCLAIMER and PATENTS sections in the 
+# README file.
+#
+# This program is released under the terms of the license contained
+# in the file LICENSE.
+# ------------------------------------------------------------------
+
 
 SHELL=/bin/sh
 CC=gcc
 BIGFILES=-D_FILE_OFFSET_BITS=64
-CFLAGS=-fpic -fPIC -Wall -Winline -O -g
+CFLAGS=-fpic -fPIC -Wall -Winline -O2 -g $(BIGFILES)
 
 OBJS= blocksort.o  \
       huffman.o    \
@@ -20,13 +35,13 @@ OBJS= blocksort.o  \
       bzlib.o
 
 all: $(OBJS)
-	$(CC) -shared -Wl,-soname -Wl,libbz2.so.1.0 -o libbz2.so.1.0.3 $(OBJS)
-	$(CC) $(CFLAGS) -o bzip2-shared bzip2.c libbz2.so.1.0.3
+	$(CC) -shared -Wl,-soname -Wl,libbz2.so.1.0 -o libbz2.so.1.0.4 $(OBJS)
+	$(CC) $(CFLAGS) -o bzip2-shared bzip2.c libbz2.so.1.0.4
 	rm -f libbz2.so.1.0
-	ln -s libbz2.so.1.0.3 libbz2.so.1.0
+	ln -s libbz2.so.1.0.4 libbz2.so.1.0
 
 clean: 
-	rm -f $(OBJS) bzip2.o libbz2.so.1.0.3 libbz2.so.1.0 bzip2-shared
+	rm -f $(OBJS) bzip2.o libbz2.so.1.0.4 libbz2.so.1.0 bzip2-shared
 
 blocksort.o: blocksort.c
 	$(CC) $(CFLAGS) -c blocksort.c
diff --git a/README b/README
index 1aff448..b18c096 100644
--- a/README
+++ b/README
@@ -1,33 +1,48 @@
 
-This is the README for bzip2, a block-sorting file compressor, version
-1.0.3.  This version is fully compatible with the previous public
-releases, versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1 and 1.0.2.
+This is the README for bzip2/libzip2.
+This version is fully compatible with the previous public releases.
 
-bzip2-1.0.3 is distributed under a BSD-style license.  For details,
-see the file LICENSE.
+------------------------------------------------------------------
+This file is part of bzip2/libbzip2, a program and library for
+lossless, block-sorting data compression.
+
+bzip2/libbzip2 version 1.0.4 of 20 December 2006
+Copyright (C) 1996-2006 Julian Seward 
+
+Please read the WARNING, DISCLAIMER and PATENTS sections in this file.
+
+This program is released under the terms of the license contained
+in the file LICENSE.
+------------------------------------------------------------------
 
 Complete documentation is available in Postscript form (manual.ps),
 PDF (manual.pdf) or html (manual.html).  A plain-text version of the
-manual page is available as bzip2.txt.  A statement about Y2K issues
-is now included in the file Y2K_INFO.
+manual page is available as bzip2.txt.
 
 
 HOW TO BUILD -- UNIX
 
-Type `make'.  This builds the library libbz2.a and then the
-programs bzip2 and bzip2recover.  Six self-tests are run.
-If the self-tests complete ok, carry on to installation:
+Type 'make'.  This builds the library libbz2.a and then the programs
+bzip2 and bzip2recover.  Six self-tests are run.  If the self-tests
+complete ok, carry on to installation:
+
+To install in /usr/local/bin, /usr/local/lib, /usr/local/man and
+/usr/local/include, type
 
-To install in /usr/bin, /usr/lib, /usr/man and /usr/include, type
    make install
-To install somewhere else, eg, /xxx/yyy/{bin,lib,man,include}, type 
+
+To install somewhere else, eg, /xxx/yyy/{bin,lib,man,include}, type
+
    make install PREFIX=/xxx/yyy
+
 If you are (justifiably) paranoid and want to see what 'make install'
 is going to do, you can first do
+
    make -n install                      or
    make -n install PREFIX=/xxx/yyy      respectively.
-The -n instructs make to show the commands it would execute, but
-not actually execute them.
+
+The -n instructs make to show the commands it would execute, but not
+actually execute them.
 
 
 HOW TO BUILD -- UNIX, shared library libbz2.so.
@@ -49,23 +64,25 @@ Important note for people upgrading .so's from 0.9.0/0.9.5 to version
 bzCompress to BZ2_bzCompress, to avoid namespace pollution.
 Unfortunately this means that the libbz2.so created by
 Makefile-libbz2_so will not work with any program which used an older
-version of the library.  Sorry.  I do encourage library clients to
-make the effort to upgrade to use version 1.0, since it is both faster
-and more robust than previous versions.
+version of the library.  I do encourage library clients to make the
+effort to upgrade to use version 1.0, since it is both faster and more
+robust than previous versions.
 
 
 HOW TO BUILD -- Windows 95, NT, DOS, Mac, etc.
 
 It's difficult for me to support compilation on all these platforms.
 My approach is to collect binaries for these platforms, and put them
-on the master web page (http://sources.redhat.com/bzip2).  Look there.
-However (FWIW), bzip2-1.0.X is very standard ANSI C and should compile
+on the master web site (http://www.bzip.org).  Look there.  However
+(FWIW), bzip2-1.0.X is very standard ANSI C and should compile
 unmodified with MS Visual C.  If you have difficulties building, you
 might want to read README.COMPILATION.PROBLEMS.
 
 At least using MS Visual C++ 6, you can build from the unmodified
 sources by issuing, in a command shell: 
+
    nmake -f makefile.msc
+
 (you may need to first run the MSVC-provided script VCVARS32.BAT
  so as to set up paths to the MSVC tools correctly).
 
@@ -86,18 +103,19 @@ Please read and be aware of the following:
 
 WARNING:
 
-   This program (attempts to) compress data by performing several
-   non-trivial transformations on it.  Unless you are 100% familiar
-   with *all* the algorithms contained herein, and with the
-   consequences of modifying them, you should NOT meddle with the
-   compression or decompression machinery.  Incorrect changes can and
-   very likely *will* lead to disastrous loss of data.
+   This program and library (attempts to) compress data by 
+   performing several non-trivial transformations on it.  
+   Unless you are 100% familiar with *all* the algorithms 
+   contained herein, and with the consequences of modifying them, 
+   you should NOT meddle with the compression or decompression 
+   machinery.  Incorrect changes can and very likely *will* 
+   lead to disastrous loss of data.
 
 
 DISCLAIMER:
 
    I TAKE NO RESPONSIBILITY FOR ANY LOSS OF DATA ARISING FROM THE
-   USE OF THIS PROGRAM, HOWSOEVER CAUSED.
+   USE OF THIS PROGRAM/LIBRARY, HOWSOEVER CAUSED.
 
    Every compression of a file implies an assumption that the
    compressed file can be decompressed to reproduce the original.
@@ -110,19 +128,18 @@ DISCLAIMER:
    PROGRAM UNLESS YOU ARE PREPARED TO ACCEPT THE POSSIBILITY, HOWEVER
    SMALL, THAT THE DATA WILL NOT BE RECOVERABLE.
 
-   That is not to say this program is inherently unreliable.  Indeed,
-   I very much hope the opposite is true.  bzip2 has been carefully
-   constructed and extensively tested.
+   That is not to say this program is inherently unreliable.  
+   Indeed, I very much hope the opposite is true.  bzip2/libbzip2 
+   has been carefully constructed and extensively tested.
 
 
 PATENTS:
 
-   To the best of my knowledge, bzip2 does not use any patented
-   algorithms.  However, I do not have the resources to carry out
-   a patent search.  Therefore I cannot give any guarantee of the
-   above statement.
+   To the best of my knowledge, bzip2/libbzip2 does not use any 
+   patented algorithms.  However, I do not have the resources 
+   to carry out a patent search.  Therefore I cannot give any 
+   guarantee of the above statement.
 
-End of legalities.
 
 
 WHAT'S NEW IN 0.9.0 (as compared to 0.1pl2) ?
@@ -156,21 +173,23 @@ WHAT'S NEW IN 1.0.3 ?
 
    See the CHANGES file.
 
+WHAT'S NEW IN 1.0.4 ?
+
+   See the CHANGES file.
+
 
 I hope you find bzip2 useful.  Feel free to contact me at
    jseward@bzip.org
 if you have any suggestions or queries.  Many people mailed me with
 comments, suggestions and patches after the releases of bzip-0.15,
-bzip-0.21, and bzip2 versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1 and
-1.0.2, and the changes in bzip2 are largely a result of this feedback.
-I thank you for your comments.
+bzip-0.21, and bzip2 versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1,
+1.0.2 and 1.0.3, and the changes in bzip2 are largely a result of this
+feedback.  I thank you for your comments.
 
-At least for the time being, bzip2's "home" is (or can be reached via)
-http://www.bzip.org
+bzip2's "home" is http://www.bzip.org/
 
 Julian Seward
 jseward@bzip.org
-
 Cambridge, UK.
 
 18     July 1996 (version 0.15)
@@ -183,3 +202,4 @@ Cambridge, UK.
  5      May 2000 (bzip2, version 1.0pre8)
 30 December 2001 (bzip2, version 1.0.2pre1)
 15 February 2005 (bzip2, version 1.0.3)
+20 December 2006 (bzip2, version 1.0.4)
diff --git a/README.COMPILATION.PROBLEMS b/README.COMPILATION.PROBLEMS
index f1bc396..64ab586 100644
--- a/README.COMPILATION.PROBLEMS
+++ b/README.COMPILATION.PROBLEMS
@@ -1,32 +1,47 @@
+------------------------------------------------------------------
+This file is part of bzip2/libbzip2, a program and library for
+lossless, block-sorting data compression.
 
-bzip2-1.0.3 should compile without problems on the vast majority of
+bzip2/libbzip2 version 1.0.4 of 20 December 2006
+Copyright (C) 1996-2006 Julian Seward 
+
+Please read the WARNING, DISCLAIMER and PATENTS sections in the 
+README file.
+
+This program is released under the terms of the license contained
+in the file LICENSE.
+------------------------------------------------------------------
+
+bzip2-1.0.4 should compile without problems on the vast majority of
 platforms.  Using the supplied Makefile, I've built and tested it
-myself for x86-linux and x86_64-linux.  With makefile.msc, Visual C++
+myself for x86-linux and amd64-linux.  With makefile.msc, Visual C++
 6.0 and nmake, you can build a native Win32 version too.  Large file
-support seems to work correctly on at least alpha-tru64unix and
-x86-cygwin32 (on Windows 2000).
+support seems to work correctly on at least on amd64-linux.
 
 When I say "large file" I mean a file of size 2,147,483,648 (2^31)
 bytes or above.  Many older OSs can't handle files above this size,
 but many newer ones can.  Large files are pretty huge -- most files
 you'll encounter are not Large Files.
 
-Earlier versions of bzip2 (0.1, 0.9.0, 0.9.5) compiled on a wide
-variety of platforms without difficulty, and I hope this version will
-continue in that tradition.  However, in order to support large files,
-I've had to include the define -D_FILE_OFFSET_BITS=64 in the Makefile.
-This can cause problems.
+Early versions of bzip2 (0.1, 0.9.0, 0.9.5) compiled on a wide variety
+of platforms without difficulty, and I hope this version will continue
+in that tradition.  However, in order to support large files, I've had
+to include the define -D_FILE_OFFSET_BITS=64 in the Makefile.  This
+can cause problems.
 
 The technique of adding -D_FILE_OFFSET_BITS=64 to get large file
 support is, as far as I know, the Recommended Way to get correct large
 file support.  For more details, see the Large File Support
 Specification, published by the Large File Summit, at
+
    http://ftp.sas.com/standards/large.file
 
 As a general comment, if you get compilation errors which you think
 are related to large file support, try removing the above define from
 the Makefile, ie, delete the line
+
    BIGFILES=-D_FILE_OFFSET_BITS=64 
+
 from the Makefile, and do 'make clean ; make'.  This will give you a
 version of bzip2 without large file support, which, for most
 applications, is probably not a problem.  
@@ -37,3 +52,7 @@ You can use the spewG.c program to generate huge files to test bzip2's
 large file support, if you are feeling paranoid.  Be aware though that
 any compilation problems which affect bzip2 will also affect spewG.c,
 alas.
+
+AIX: I have reports that for large file support, you need to specify
+-D_LARGE_FILES rather than -D_FILE_OFFSET_BITS=64.  I have not tested
+this myself.
diff --git a/README.XML.STUFF b/README.XML.STUFF
index 0ff209f..93bf405 100644
--- a/README.XML.STUFF
+++ b/README.XML.STUFF
@@ -1,3 +1,17 @@
+  ----------------------------------------------------------------
+  This file is part of bzip2/libbzip2, a program and library for
+  lossless, block-sorting data compression.
+
+  bzip2/libbzip2 version 1.0.4 of 20 December 2006
+  Copyright (C) 1996-2006 Julian Seward 
+
+  Please read the WARNING, DISCLAIMER and PATENTS sections in the 
+  README file.
+
+  This program is released under the terms of the license contained
+  in the file LICENSE.
+  ----------------------------------------------------------------
+
 The script xmlproc.sh takes an xml file as input,
 and processes it to create .pdf, .html or .ps output.
 It uses format.pl, a perl script to format 
 blocks nicely,
@@ -9,16 +23,16 @@ version, year, etc.
 
 Usage:
 
-  xmlproc.sh -v manual.xml
+  ./xmlproc.sh -v manual.xml
   Validates an xml file to ensure no dtd-compliance errors
 
-  xmlproc.sh -html manual.xml
+  ./xmlproc.sh -html manual.xml
   Output: manual.html
 
-  xmlproc.sh -pdf manual.xml
+  ./xmlproc.sh -pdf manual.xml
   Output: manual.pdf
 
-  xmlproc.sh -ps manual.xml
+  ./xmlproc.sh -ps manual.xml
   Output: manual.ps
 
 
diff --git a/Y2K_INFO b/Y2K_INFO
deleted file mode 100644
index 55fd56a..0000000
--- a/Y2K_INFO
+++ /dev/null
@@ -1,34 +0,0 @@
-
-Y2K status of bzip2 and libbzip2, versions 0.1, 0.9.0 and 0.9.5
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Informally speaking:
-   bzip2 is a compression program built on top of libbzip2, 
-   a library which does the real work of compression and 
-   decompression.  As far as I am aware, libbzip2 does not have 
-   any date-related code at all.
-
-   bzip2 itself copies dates from source to destination files 
-   when compressing or decompressing, using the 'stat' and 'utime' 
-   UNIX system calls.  It doesn't examine, manipulate or store the 
-   dates in any way.  So as far as I can see, there shouldn't be any 
-   problem with bzip2 providing 'stat' and 'utime' work correctly 
-   on your system.
-
-   On non-unix platforms (those for which BZ_UNIX in bzip2.c is
-   not set to 1), bzip2 doesn't even do the date copying.
-
-   Overall, informally speaking, I don't think bzip2 or libbzip2 
-   have a Y2K problem.
-
-Formally speaking:
-   I am not prepared to offer you any assurance whatsoever 
-   regarding Y2K issues in my software.  You alone assume the 
-   entire risk of using the software.  The disclaimer of liability 
-   in the LICENSE file in the bzip2 source distribution continues 
-   to apply on this issue as with every other issue pertaining 
-   to the software.
-
-Julian Seward
-Cambridge, UK
-25 August 1999
diff --git a/blocksort.c b/blocksort.c
index 33ec9f5..8535c93 100644
--- a/blocksort.c
+++ b/blocksort.c
@@ -4,66 +4,19 @@
 /*---                                           blocksort.c ---*/
 /*-------------------------------------------------------------*/
 
-/*--
-  This file is a part of bzip2 and/or libbzip2, a program and
-  library for lossless, block-sorting data compression.
-
-  Copyright (C) 1996-2005 Julian R Seward.  All rights reserved.
-
-  Redistribution and use in source and binary forms, with or without
-  modification, are permitted provided that the following conditions
-  are met:
-
-  1. Redistributions of source code must retain the above copyright
-     notice, this list of conditions and the following disclaimer.
-
-  2. The origin of this software must not be misrepresented; you must 
-     not claim that you wrote the original software.  If you use this 
-     software in a product, an acknowledgment in the product 
-     documentation would be appreciated but is not required.
-
-  3. Altered source versions must be plainly marked as such, and must
-     not be misrepresented as being the original software.
-
-  4. The name of the author may not be used to endorse or promote 
-     products derived from this software without specific prior written 
-     permission.
-
-  THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
-  OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-  WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-  ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
-  DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
-  DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
-  GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-  INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
-  WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
-  NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
-  SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-  Julian Seward, Cambridge, UK.
-  jseward@bzip.org
-  bzip2/libbzip2 version 1.0 of 21 March 2000
-
-  This program is based on (at least) the work of:
-     Mike Burrows
-     David Wheeler
-     Peter Fenwick
-     Alistair Moffat
-     Radford Neal
-     Ian H. Witten
-     Robert Sedgewick
-     Jon L. Bentley
-
-  For more information on these sources, see the manual.
-
-  To get some idea how the block sorting algorithms in this file 
-  work, read my paper 
-     On the Performance of BWT Sorting Algorithms
-  in Proceedings of the IEEE Data Compression Conference 2000,
-  Snowbird, Utah, USA, 27-30 March 2000.  The main sort in this
-  file implements the algorithm called  cache  in the paper.
---*/
+/* ------------------------------------------------------------------
+   This file is part of bzip2/libbzip2, a program and library for
+   lossless, block-sorting data compression.
+
+   bzip2/libbzip2 version 1.0.4 of 20 December 2006
+   Copyright (C) 1996-2006 Julian Seward 
+
+   Please read the WARNING, DISCLAIMER and PATENTS sections in the 
+   README file.
+
+   This program is released under the terms of the license contained
+   in the file LICENSE.
+   ------------------------------------------------------------------ */
 
 
 #include "bzlib_private.h"
@@ -155,7 +108,7 @@ void fallbackQSort3 ( UInt32* fmap,
 
    while (sp > 0) {
 
-      AssertH ( sp < FALLBACK_QSORT_STACK_SIZE, 1004 );
+      AssertH ( sp < FALLBACK_QSORT_STACK_SIZE - 1, 1004 );
 
       fpop ( lo, hi );
       if (hi - lo < FALLBACK_QSORT_SMALL_THRESH) {
@@ -690,7 +643,7 @@ void mainQSort3 ( UInt32* ptr,
 
    while (sp > 0) {
 
-      AssertH ( sp < MAIN_QSORT_STACK_SIZE, 1001 );
+      AssertH ( sp < MAIN_QSORT_STACK_SIZE - 2, 1001 );
 
       mpop ( lo, hi, d );
       if (hi - lo < MAIN_QSORT_SMALL_THRESH || 
diff --git a/bz-fo.xsl b/bz-fo.xsl
index 7f2a767..ba3e301 100644
--- a/bz-fo.xsl
+++ b/bz-fo.xsl
@@ -254,4 +254,23 @@
 
 
 
+
+
+
+  always
+  
+    
+  
+  
+    
+    pt
+  
+  
+    
+    pt
+  
+  false
+
+
+
 
diff --git a/bzdiff b/bzdiff
index 3c2eb85..6fc38f9 100644
--- a/bzdiff
+++ b/bzdiff
@@ -12,7 +12,7 @@
 # necessary) and fed to cmp or diff.  The exit status from cmp
 # or diff is preserved.
 
-PATH="/usr/bin:$PATH"; export PATH
+PATH="/usr/bin:/bin:$PATH"; export PATH
 prog=`echo $0 | sed 's|.*/||'`
 case "$prog" in
   *cmp) comp=${CMP-cmp}   ;;
@@ -37,7 +37,7 @@ if test -z "$FILES"; then
 	echo "Usage: $prog [${comp}_options] file [file]"
 	exit 1
 fi
-tmp=`tempfile -d /tmp -p bz` || {
+tmp=`mktemp ${TMPDIR:-/tmp}/bzdiff.XXXXXXXXXX` || {
       echo 'cannot create a temporary file' >&2
       exit 1
 }
diff --git a/bzgrep b/bzgrep
index dbfc00e..9a04b83 100644
--- a/bzgrep
+++ b/bzgrep
@@ -63,7 +63,11 @@ for i do
     bzip2 -cdfq "$i" | $grep $opt "$pat"
     r=$?
   else
-    bzip2 -cdfq "$i" | $grep $opt "$pat" | sed "s|^|${i}:|"
+    j=${i//\\/\\\\}
+    j=${j//|/\\|}
+    j=${j//&/\\&}
+    j=`printf "%s" "$j" | tr '\n' ' '`
+    bzip2 -cdfq "$i" | $grep $opt "$pat" | sed "s|^|${j}:|"
     r=$?
   fi
   test "$r" -ne 0 && res="$r"
diff --git a/bzip2.1 b/bzip2.1
index d2c0661..a313f2d 100644
--- a/bzip2.1
+++ b/bzip2.1
@@ -1,7 +1,7 @@
 .PU
 .TH bzip2 1
 .SH NAME
-bzip2, bunzip2 \- a block-sorting file compressor, v1.0.3
+bzip2, bunzip2 \- a block-sorting file compressor, v1.0.4
 .br
 bzcat \- decompresses files to stdout
 .br
@@ -405,11 +405,11 @@ I/O error messages are not as helpful as they could be.
 tries hard to detect I/O errors and exit cleanly, but the details of
 what the problem is sometimes seem rather misleading.
 
-This manual page pertains to version 1.0.3 of
+This manual page pertains to version 1.0.4 of
 .I bzip2.  
 Compressed data created by this version is entirely forwards and
 backwards compatible with the previous public releases, versions
-0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1 and 1.0.2, but with the following
+0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1, 1.0.2 and 1.0.3, but with the following
 exception: 0.9.0 and above can correctly decompress multiple
 concatenated compressed files.  0.1pl2 cannot do this; it will stop
 after decompressing just the first file in the stream.
diff --git a/bzip2.1.preformatted b/bzip2.1.preformatted
index 129ca83..15e16e5 100644
--- a/bzip2.1.preformatted
+++ b/bzip2.1.preformatted
@@ -3,7 +3,7 @@ bzip2(1)                                                 bzip2(1)
 
 
 NNAAMMEE
-       bzip2, bunzip2 − a block‐sorting file compressor, v1.0.3
+       bzip2, bunzip2 − a block‐sorting file compressor, v1.0.4
        bzcat − decompresses files to stdout
        bzip2recover − recovers data from damaged bzip2 files
 
@@ -348,14 +348,14 @@ CCAAVVEEAATTSS
        but  the  details  of  what  the problem is sometimes seem
        rather misleading.
 
-       This manual page pertains to version 1.0.3 of _b_z_i_p_2_.  Com­
+       This manual page pertains to version 1.0.4 of _b_z_i_p_2_.  Com­
        pressed  data created by this version is entirely forwards
        and  backwards  compatible  with   the   previous   public
-       releases,  versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1 and
-       1.0.2, but with the following exception: 0.9.0  and  above
-       can  correctly decompress multiple concatenated compressed
-       files.  0.1pl2 cannot do this; it will stop  after  decom­
-       pressing just the first file in the stream.
+       releases,  versions  0.1pl2,  0.9.0,  0.9.5, 1.0.0, 1.0.1, 
+       1.0.2 and 1.0.3, but with the  following  exception: 0.9.0
+       and above can  correctly decompress  multiple concatenated
+       compressed files.  0.1pl2  cannot do this;  it  will  stop 
+       after  decompressing just the first file in the stream.
 
        _b_z_i_p_2_r_e_c_o_v_e_r  versions prior to 1.0.2 used 32‐bit integers
        to represent bit positions in compressed  files,  so  they
diff --git a/bzip2.c b/bzip2.c
index 79f87a5..5fbae0e 100644
--- a/bzip2.c
+++ b/bzip2.c
@@ -3,118 +3,26 @@
 /*--- A block-sorting, lossless compressor        bzip2.c ---*/
 /*-----------------------------------------------------------*/
 
-/*--
-  This file is a part of bzip2 and/or libbzip2, a program and
-  library for lossless, block-sorting data compression.
-
-  Copyright (C) 1996-2005 Julian R Seward.  All rights reserved.
-
-  Redistribution and use in source and binary forms, with or without
-  modification, are permitted provided that the following conditions
-  are met:
-
-  1. Redistributions of source code must retain the above copyright
-     notice, this list of conditions and the following disclaimer.
-
-  2. The origin of this software must not be misrepresented; you must 
-     not claim that you wrote the original software.  If you use this 
-     software in a product, an acknowledgment in the product 
-     documentation would be appreciated but is not required.
-
-  3. Altered source versions must be plainly marked as such, and must
-     not be misrepresented as being the original software.
-
-  4. The name of the author may not be used to endorse or promote 
-     products derived from this software without specific prior written 
-     permission.
-
-  THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
-  OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-  WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-  ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
-  DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
-  DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
-  GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-  INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
-  WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
-  NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
-  SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-  Julian Seward, Cambridge, UK.
-  jseward@bzip.org
-  bzip2/libbzip2 version 1.0 of 21 March 2000
-
-  This program is based on (at least) the work of:
-     Mike Burrows
-     David Wheeler
-     Peter Fenwick
-     Alistair Moffat
-     Radford Neal
-     Ian H. Witten
-     Robert Sedgewick
-     Jon L. Bentley
-
-  For more information on these sources, see the manual.
---*/
+/* ------------------------------------------------------------------
+   This file is part of bzip2/libbzip2, a program and library for
+   lossless, block-sorting data compression.
 
+   bzip2/libbzip2 version 1.0.4 of 20 December 2006
+   Copyright (C) 1996-2006 Julian Seward 
 
-/*----------------------------------------------------*/
-/*--- IMPORTANT                                    ---*/
-/*----------------------------------------------------*/
+   Please read the WARNING, DISCLAIMER and PATENTS sections in the 
+   README file.
 
-/*--
-   WARNING:
-      This program and library (attempts to) compress data by 
-      performing several non-trivial transformations on it.  
-      Unless you are 100% familiar with *all* the algorithms 
-      contained herein, and with the consequences of modifying them, 
-      you should NOT meddle with the compression or decompression 
-      machinery.  Incorrect changes can and very likely *will* 
-      lead to disasterous loss of data.
-
-   DISCLAIMER:
-      I TAKE NO RESPONSIBILITY FOR ANY LOSS OF DATA ARISING FROM THE
-      USE OF THIS PROGRAM, HOWSOEVER CAUSED.
-
-      Every compression of a file implies an assumption that the
-      compressed file can be decompressed to reproduce the original.
-      Great efforts in design, coding and testing have been made to
-      ensure that this program works correctly.  However, the
-      complexity of the algorithms, and, in particular, the presence
-      of various special cases in the code which occur with very low
-      but non-zero probability make it impossible to rule out the
-      possibility of bugs remaining in the program.  DO NOT COMPRESS
-      ANY DATA WITH THIS PROGRAM AND/OR LIBRARY UNLESS YOU ARE PREPARED 
-      TO ACCEPT THE POSSIBILITY, HOWEVER SMALL, THAT THE DATA WILL 
-      NOT BE RECOVERABLE.
-
-      That is not to say this program is inherently unreliable.
-      Indeed, I very much hope the opposite is true.  bzip2/libbzip2
-      has been carefully constructed and extensively tested.
-
-   PATENTS:
-      To the best of my knowledge, bzip2/libbzip2 does not use any 
-      patented algorithms.  However, I do not have the resources 
-      available to carry out a full patent search.  Therefore I cannot 
-      give any guarantee of the above statement.
---*/
+   This program is released under the terms of the license contained
+   in the file LICENSE.
+   ------------------------------------------------------------------ */
 
 
-
-/*----------------------------------------------------*/
-/*--- and now for something much more pleasant :-) ---*/
-/*----------------------------------------------------*/
-
-/*---------------------------------------------*/
-/*--
-  Place a 1 beside your platform, and 0 elsewhere.
---*/
-
-/*--
-  Generic 32-bit Unix.
-  Also works on 64-bit Unix boxes.
-  This is the default.
---*/
+/* Place a 1 beside your platform, and 0 elsewhere.
+   Generic 32-bit Unix.
+   Also works on 64-bit Unix boxes.
+   This is the default.
+*/
 #define BZ_UNIX      1
 
 /*--
@@ -302,16 +210,17 @@ Char    progNameReally[FILE_NAME_LEN];
 FILE    *outputHandleJustInCase;
 Int32   workFactor;
 
-static void    panic                 ( Char* )   NORETURN;
-static void    ioError               ( void )    NORETURN;
-static void    outOfMemory           ( void )    NORETURN;
-static void    configError           ( void )    NORETURN;
-static void    crcError              ( void )    NORETURN;
-static void    cleanUpAndFail        ( Int32 )   NORETURN;
-static void    compressedStreamEOF   ( void )    NORETURN;
+static void    panic                 ( const Char* ) NORETURN;
+static void    ioError               ( void )        NORETURN;
+static void    outOfMemory           ( void )        NORETURN;
+static void    configError           ( void )        NORETURN;
+static void    crcError              ( void )        NORETURN;
+static void    cleanUpAndFail        ( Int32 )       NORETURN;
+static void    compressedStreamEOF   ( void )        NORETURN;
 
 static void    copyFileName ( Char*, Char* );
 static void*   myMalloc     ( Int32 );
+static void    applySavedFileAttrToOutputFile ( IntNative fd );
 
 
 
@@ -457,6 +366,9 @@ void compressStream ( FILE *stream, FILE *zStream )
    ret = fflush ( zStream );
    if (ret == EOF) goto errhandler_io;
    if (zStream != stdout) {
+      Int32 fd = fileno ( zStream );
+      if (fd < 0) goto errhandler_io;
+      applySavedFileAttrToOutputFile ( fd );
       ret = fclose ( zStream );
       outputHandleJustInCase = NULL;
       if (ret == EOF) goto errhandler_io;
@@ -569,6 +481,11 @@ Bool uncompressStream ( FILE *zStream, FILE *stream )
 
    closeok:
    if (ferror(zStream)) goto errhandler_io;
+   if (stream != stdout) {
+      Int32 fd = fileno ( stream );
+      if (fd < 0) goto errhandler_io;
+      applySavedFileAttrToOutputFile ( fd );
+   }
    ret = fclose ( zStream );
    if (ret == EOF) goto errhandler_io;
 
@@ -826,7 +743,7 @@ void cleanUpAndFail ( Int32 ec )
 
 /*---------------------------------------------*/
 static 
-void panic ( Char* s )
+void panic ( const Char* s )
 {
    fprintf ( stderr,
              "\n%s: PANIC -- internal consistency error:\n"
@@ -1039,6 +956,7 @@ Bool fileExists ( Char* name )
    For non-Unix platforms, if we are not worrying about
    security issues, simple this simply behaves like fopen.
 */
+static
 FILE* fopen_output_safely ( Char* name, const char* mode )
 {
 #  if BZ_UNIX
@@ -1129,7 +1047,7 @@ void saveInputFileMetaInfo ( Char *srcName )
 
 
 static 
-void applySavedMetaInfoToOutputFile ( Char *dstName )
+void applySavedTimeInfoToOutputFile ( Char *dstName )
 {
 #  if BZ_UNIX
    IntNative      retVal;
@@ -1138,13 +1056,21 @@ void applySavedMetaInfoToOutputFile ( Char *dstName )
    uTimBuf.actime = fileMetaInfo.st_atime;
    uTimBuf.modtime = fileMetaInfo.st_mtime;
 
-   retVal = chmod ( dstName, fileMetaInfo.st_mode );
+   retVal = utime ( dstName, &uTimBuf );
    ERROR_IF_NOT_ZERO ( retVal );
+#  endif
+}
 
-   retVal = utime ( dstName, &uTimBuf );
+static 
+void applySavedFileAttrToOutputFile ( IntNative fd )
+{
+#  if BZ_UNIX
+   IntNative retVal;
+
+   retVal = fchmod ( fd, fileMetaInfo.st_mode );
    ERROR_IF_NOT_ZERO ( retVal );
 
-   retVal = chown ( dstName, fileMetaInfo.st_uid, fileMetaInfo.st_gid );
+   (void) fchown ( fd, fileMetaInfo.st_uid, fileMetaInfo.st_gid );
    /* chown() will in many cases return with EPERM, which can
       be safely ignored.
    */
@@ -1175,13 +1101,13 @@ Bool containsDubiousChars ( Char* name )
 /*---------------------------------------------*/
 #define BZ_N_SUFFIX_PAIRS 4
 
-Char* zSuffix[BZ_N_SUFFIX_PAIRS] 
+const Char* zSuffix[BZ_N_SUFFIX_PAIRS] 
    = { ".bz2", ".bz", ".tbz2", ".tbz" };
-Char* unzSuffix[BZ_N_SUFFIX_PAIRS] 
+const Char* unzSuffix[BZ_N_SUFFIX_PAIRS] 
    = { "", "", ".tar", ".tar" };
 
 static 
-Bool hasSuffix ( Char* s, Char* suffix )
+Bool hasSuffix ( Char* s, const Char* suffix )
 {
    Int32 ns = strlen(s);
    Int32 nx = strlen(suffix);
@@ -1192,7 +1118,8 @@ Bool hasSuffix ( Char* s, Char* suffix )
 
 static 
 Bool mapSuffix ( Char* name, 
-                 Char* oldSuffix, Char* newSuffix )
+                 const Char* oldSuffix, 
+                 const Char* newSuffix )
 {
    if (!hasSuffix(name,oldSuffix)) return False;
    name[strlen(name)-strlen(oldSuffix)] = 0;
@@ -1217,8 +1144,8 @@ void compress ( Char *name )
 
    switch (srcMode) {
       case SM_I2O: 
-         copyFileName ( inName, "(stdin)" );
-         copyFileName ( outName, "(stdout)" ); 
+         copyFileName ( inName, (Char*)"(stdin)" );
+         copyFileName ( outName, (Char*)"(stdout)" ); 
          break;
       case SM_F2F: 
          copyFileName ( inName, name );
@@ -1227,7 +1154,7 @@ void compress ( Char *name )
          break;
       case SM_F2O: 
          copyFileName ( inName, name );
-         copyFileName ( outName, "(stdout)" ); 
+         copyFileName ( outName, (Char*)"(stdout)" ); 
          break;
    }
 
@@ -1370,7 +1297,7 @@ void compress ( Char *name )
 
    /*--- If there was an I/O error, we won't get here. ---*/
    if ( srcMode == SM_F2F ) {
-      applySavedMetaInfoToOutputFile ( outName );
+      applySavedTimeInfoToOutputFile ( outName );
       deleteOutputOnInterrupt = False;
       if ( !keepInputFiles ) {
          IntNative retVal = remove ( inName );
@@ -1401,8 +1328,8 @@ void uncompress ( Char *name )
    cantGuess = False;
    switch (srcMode) {
       case SM_I2O: 
-         copyFileName ( inName, "(stdin)" );
-         copyFileName ( outName, "(stdout)" ); 
+         copyFileName ( inName, (Char*)"(stdin)" );
+         copyFileName ( outName, (Char*)"(stdout)" ); 
          break;
       case SM_F2F: 
          copyFileName ( inName, name );
@@ -1415,7 +1342,7 @@ void uncompress ( Char *name )
          break;
       case SM_F2O: 
          copyFileName ( inName, name );
-         copyFileName ( outName, "(stdout)" ); 
+         copyFileName ( outName, (Char*)"(stdout)" ); 
          break;
    }
 
@@ -1548,7 +1475,7 @@ void uncompress ( Char *name )
    /*--- If there was an I/O error, we won't get here. ---*/
    if ( magicNumberOK ) {
       if ( srcMode == SM_F2F ) {
-         applySavedMetaInfoToOutputFile ( outName );
+         applySavedTimeInfoToOutputFile ( outName );
          deleteOutputOnInterrupt = False;
          if ( !keepInputFiles ) {
             IntNative retVal = remove ( inName );
@@ -1593,9 +1520,9 @@ void testf ( Char *name )
    if (name == NULL && srcMode != SM_I2O)
       panic ( "testf: bad modes\n" );
 
-   copyFileName ( outName, "(none)" );
+   copyFileName ( outName, (Char*)"(none)" );
    switch (srcMode) {
-      case SM_I2O: copyFileName ( inName, "(stdin)" ); break;
+      case SM_I2O: copyFileName ( inName, (Char*)"(stdin)" ); break;
       case SM_F2F: copyFileName ( inName, name ); break;
       case SM_F2O: copyFileName ( inName, name ); break;
    }
@@ -1678,11 +1605,11 @@ void license ( void )
     "bzip2, a block-sorting file compressor.  "
     "Version %s.\n"
     "   \n"
-    "   Copyright (C) 1996-2005 by Julian Seward.\n"
+    "   Copyright (C) 1996-2006 by Julian Seward.\n"
     "   \n"
     "   This program is free software; you can redistribute it and/or modify\n"
     "   it under the terms set out in the LICENSE file, which is included\n"
-    "   in the bzip2-1.0 source distribution.\n"
+    "   in the bzip2-1.0.4 source distribution.\n"
     "   \n"
     "   This program is distributed in the hope that it will be useful,\n"
     "   but WITHOUT ANY WARRANTY; without even the implied warranty of\n"
@@ -1885,8 +1812,8 @@ IntNative main ( IntNative argc, Char *argv[] )
 #  endif
 #  endif
 
-   copyFileName ( inName,  "(none)" );
-   copyFileName ( outName, "(none)" );
+   copyFileName ( inName,  (Char*)"(none)" );
+   copyFileName ( outName, (Char*)"(none)" );
 
    copyFileName ( progNameReally, argv[0] );
    progName = &progNameReally[0];
@@ -1898,8 +1825,8 @@ IntNative main ( IntNative argc, Char *argv[] )
         expand filename wildcards in arg list.
    --*/
    argList = NULL;
-   addFlagsFromEnvVar ( &argList,  "BZIP2" );
-   addFlagsFromEnvVar ( &argList,  "BZIP" );
+   addFlagsFromEnvVar ( &argList,  (Char*)"BZIP2" );
+   addFlagsFromEnvVar ( &argList,  (Char*)"BZIP" );
    for (i = 1; i <= argc-1; i++)
       APPEND_FILESPEC(argList, argv[i]);
 
diff --git a/bzip2.txt b/bzip2.txt
index bf895b6..4fb9c74 100644
--- a/bzip2.txt
+++ b/bzip2.txt
@@ -1,6 +1,6 @@
 
 NAME
-       bzip2, bunzip2 - a block-sorting file compressor, v1.0.3
+       bzip2, bunzip2 - a block-sorting file compressor, v1.0.4
        bzcat - decompresses files to stdout
        bzip2recover - recovers data from damaged bzip2 files
 
@@ -345,14 +345,14 @@ CAVEATS
        but  the  details  of  what  the problem is sometimes seem
        rather misleading.
 
-       This manual page pertains to version 1.0.3 of bzip2.  Com-
+       This manual page pertains to version 1.0.4 of bzip2.  Com-
        pressed  data created by this version is entirely forwards
        and  backwards  compatible  with   the   previous   public
-       releases,  versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1 and
-       1.0.2, but with the following exception: 0.9.0  and  above
-       can  correctly decompress multiple concatenated compressed
-       files.  0.1pl2 cannot do this; it will stop  after  decom-
-       pressing just the first file in the stream.
+       releases,  versions  0.1pl2,  0.9.0,  0.9.5, 1.0.0, 1.0.1,
+       1.0.2 and 1.0.3, but with the  following  exception: 0.9.0
+       and above can  correctly decompress  multiple concatenated
+       compressed files.  0.1pl2  cannot do this;  it  will  stop
+       after  decompressing just the first file in the stream.
 
        bzip2recover  versions prior to 1.0.2 used 32-bit integers
        to represent bit positions in compressed  files,  so  they
diff --git a/bzip2recover.c b/bzip2recover.c
index 5cd405d..ffe60bc 100644
--- a/bzip2recover.c
+++ b/bzip2recover.c
@@ -1,56 +1,24 @@
-
 /*-----------------------------------------------------------*/
 /*--- Block recoverer program for bzip2                   ---*/
 /*---                                      bzip2recover.c ---*/
 /*-----------------------------------------------------------*/
 
-/*--
-  This program is bzip2recover, a program to attempt data 
-  salvage from damaged files created by the accompanying
-  bzip2-1.0.3 program.
-
-  Copyright (C) 1996-2005 Julian R Seward.  All rights reserved.
-
-  Redistribution and use in source and binary forms, with or without
-  modification, are permitted provided that the following conditions
-  are met:
-
-  1. Redistributions of source code must retain the above copyright
-     notice, this list of conditions and the following disclaimer.
-
-  2. The origin of this software must not be misrepresented; you must 
-     not claim that you wrote the original software.  If you use this 
-     software in a product, an acknowledgment in the product 
-     documentation would be appreciated but is not required.
-
-  3. Altered source versions must be plainly marked as such, and must
-     not be misrepresented as being the original software.
-
-  4. The name of the author may not be used to endorse or promote 
-     products derived from this software without specific prior written 
-     permission.
-
-  THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
-  OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-  WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-  ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
-  DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
-  DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
-  GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-  INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
-  WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
-  NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
-  SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-  Julian Seward, Cambridge, UK.
-  jseward@bzip.org
-  bzip2/libbzip2 version 1.0.3 of 15 February 2005
---*/
+/* ------------------------------------------------------------------
+   This file is part of bzip2/libbzip2, a program and library for
+   lossless, block-sorting data compression.
 
-/*--
-  This program is a complete hack and should be rewritten
-  properly.  It isn't very complicated.
---*/
+   bzip2/libbzip2 version 1.0.4 of 20 December 2006
+   Copyright (C) 1996-2006 Julian Seward 
+
+   Please read the WARNING, DISCLAIMER and PATENTS sections in the 
+   README file.
+
+   This program is released under the terms of the license contained
+   in the file LICENSE.
+   ------------------------------------------------------------------ */
+
+/* This program is a complete hack and should be rewritten properly.
+	 It isn't very complicated. */
 
 #include 
 #include 
@@ -114,7 +82,7 @@ MaybeUInt64 bytesIn  = 0;
 /*---------------------------------------------------*/
 
 /*---------------------------------------------*/
-void readError ( void )
+static void readError ( void )
 {
    fprintf ( stderr,
              "%s: I/O error reading `%s', possible reason follows.\n",
@@ -127,7 +95,7 @@ void readError ( void )
 
 
 /*---------------------------------------------*/
-void writeError ( void )
+static void writeError ( void )
 {
    fprintf ( stderr,
              "%s: I/O error reading `%s', possible reason follows.\n",
@@ -140,7 +108,7 @@ void writeError ( void )
 
 
 /*---------------------------------------------*/
-void mallocFail ( Int32 n )
+static void mallocFail ( Int32 n )
 {
    fprintf ( stderr,
              "%s: malloc failed on request for %d bytes.\n",
@@ -152,7 +120,7 @@ void mallocFail ( Int32 n )
 
 
 /*---------------------------------------------*/
-void tooManyBlocks ( Int32 max_handled_blocks )
+static void tooManyBlocks ( Int32 max_handled_blocks )
 {
    fprintf ( stderr,
              "%s: `%s' appears to contain more than %d blocks\n",
@@ -183,7 +151,7 @@ typedef
 
 
 /*---------------------------------------------*/
-BitStream* bsOpenReadStream ( FILE* stream )
+static BitStream* bsOpenReadStream ( FILE* stream )
 {
    BitStream *bs = malloc ( sizeof(BitStream) );
    if (bs == NULL) mallocFail ( sizeof(BitStream) );
@@ -196,7 +164,7 @@ BitStream* bsOpenReadStream ( FILE* stream )
 
 
 /*---------------------------------------------*/
-BitStream* bsOpenWriteStream ( FILE* stream )
+static BitStream* bsOpenWriteStream ( FILE* stream )
 {
    BitStream *bs = malloc ( sizeof(BitStream) );
    if (bs == NULL) mallocFail ( sizeof(BitStream) );
@@ -209,7 +177,7 @@ BitStream* bsOpenWriteStream ( FILE* stream )
 
 
 /*---------------------------------------------*/
-void bsPutBit ( BitStream* bs, Int32 bit )
+static void bsPutBit ( BitStream* bs, Int32 bit )
 {
    if (bs->buffLive == 8) {
       Int32 retVal = putc ( (UChar) bs->buffer, bs->handle );
@@ -228,7 +196,7 @@ void bsPutBit ( BitStream* bs, Int32 bit )
 /*--
    Returns 0 or 1, or 2 to indicate EOF.
 --*/
-Int32 bsGetBit ( BitStream* bs )
+static Int32 bsGetBit ( BitStream* bs )
 {
    if (bs->buffLive > 0) {
       bs->buffLive --;
@@ -247,7 +215,7 @@ Int32 bsGetBit ( BitStream* bs )
 
 
 /*---------------------------------------------*/
-void bsClose ( BitStream* bs )
+static void bsClose ( BitStream* bs )
 {
    Int32 retVal;
 
@@ -271,7 +239,7 @@ void bsClose ( BitStream* bs )
 
 
 /*---------------------------------------------*/
-void bsPutUChar ( BitStream* bs, UChar c )
+static void bsPutUChar ( BitStream* bs, UChar c )
 {
    Int32 i;
    for (i = 7; i >= 0; i--)
@@ -280,7 +248,7 @@ void bsPutUChar ( BitStream* bs, UChar c )
 
 
 /*---------------------------------------------*/
-void bsPutUInt32 ( BitStream* bs, UInt32 c )
+static void bsPutUInt32 ( BitStream* bs, UInt32 c )
 {
    Int32 i;
 
@@ -290,7 +258,7 @@ void bsPutUInt32 ( BitStream* bs, UInt32 c )
 
 
 /*---------------------------------------------*/
-Bool endsInBz2 ( Char* name )
+static Bool endsInBz2 ( Char* name )
 {
    Int32 n = strlen ( name );
    if (n <= 4) return False;
@@ -345,7 +313,7 @@ Int32 main ( Int32 argc, Char** argv )
    inFileName[0] = outFileName[0] = 0;
 
    fprintf ( stderr, 
-             "bzip2recover 1.0.3: extracts blocks from damaged .bz2 files.\n" );
+             "bzip2recover 1.0.4: extracts blocks from damaged .bz2 files.\n" );
 
    if (argc != 2) {
       fprintf ( stderr, "%s: usage is `%s damaged_file_name'.\n",
diff --git a/bzlib.c b/bzlib.c
index e9c1e87..79c34a5 100644
--- a/bzlib.c
+++ b/bzlib.c
@@ -4,74 +4,29 @@
 /*---                                               bzlib.c ---*/
 /*-------------------------------------------------------------*/
 
-/*--
-  This file is a part of bzip2 and/or libbzip2, a program and
-  library for lossless, block-sorting data compression.
-
-  Copyright (C) 1996-2005 Julian R Seward.  All rights reserved.
-
-  Redistribution and use in source and binary forms, with or without
-  modification, are permitted provided that the following conditions
-  are met:
-
-  1. Redistributions of source code must retain the above copyright
-     notice, this list of conditions and the following disclaimer.
-
-  2. The origin of this software must not be misrepresented; you must 
-     not claim that you wrote the original software.  If you use this 
-     software in a product, an acknowledgment in the product 
-     documentation would be appreciated but is not required.
-
-  3. Altered source versions must be plainly marked as such, and must
-     not be misrepresented as being the original software.
-
-  4. The name of the author may not be used to endorse or promote 
-     products derived from this software without specific prior written 
-     permission.
-
-  THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
-  OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-  WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-  ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
-  DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
-  DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
-  GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-  INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
-  WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
-  NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
-  SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-  Julian Seward, Cambridge, UK.
-  jseward@bzip.org
-  bzip2/libbzip2 version 1.0 of 21 March 2000
-
-  This program is based on (at least) the work of:
-     Mike Burrows
-     David Wheeler
-     Peter Fenwick
-     Alistair Moffat
-     Radford Neal
-     Ian H. Witten
-     Robert Sedgewick
-     Jon L. Bentley
-
-  For more information on these sources, see the manual.
---*/
+/* ------------------------------------------------------------------
+   This file is part of bzip2/libbzip2, a program and library for
+   lossless, block-sorting data compression.
 
-/*--
-   CHANGES
-   ~~~~~~~
-   0.9.0 -- original version.
+   bzip2/libbzip2 version 1.0.4 of 20 December 2006
+   Copyright (C) 1996-2006 Julian Seward 
 
-   0.9.0a/b -- no changes in this file.
+   Please read the WARNING, DISCLAIMER and PATENTS sections in the 
+   README file.
 
-   0.9.0c
-      * made zero-length BZ_FLUSH work correctly in bzCompress().
-      * fixed bzWrite/bzRead to ignore zero-length requests.
-      * fixed bzread to correctly handle read requests after EOF.
-      * wrong parameter order in call to bzDecompressInit in
-        bzBuffToBuffDecompress.  Fixed.
---*/
+   This program is released under the terms of the license contained
+   in the file LICENSE.
+   ------------------------------------------------------------------ */
+
+/* CHANGES
+   0.9.0    -- original version.
+   0.9.0a/b -- no changes in this file.
+   0.9.0c   -- made zero-length BZ_FLUSH work correctly in bzCompress().
+     fixed bzWrite/bzRead to ignore zero-length requests.
+     fixed bzread to correctly handle read requests after EOF.
+     wrong parameter order in call to bzDecompressInit in
+     bzBuffToBuffDecompress.  Fixed.
+*/
 
 #include "bzlib_private.h"
 
@@ -1394,8 +1349,7 @@ int BZ_API(BZ2_bzBuffToBuffDecompress)
 
 /*---------------------------------------------------*/
 /*--
-   Code contributed by Yoshioka Tsuneo
-   (QWF00133@niftyserve.or.jp/tsuneo-y@is.aist-nara.ac.jp),
+   Code contributed by Yoshioka Tsuneo (tsuneo@rr.iij4u.or.jp)
    to support better zlib compatibility.
    This code is not _officially_ part of libbzip2 (yet);
    I haven't tested it, documented it, or considered the
@@ -1406,7 +1360,7 @@ int BZ_API(BZ2_bzBuffToBuffDecompress)
 
 /*---------------------------------------------------*/
 /*--
-   return version like "0.9.0c".
+   return version like "0.9.5d, 4-Sept-1999".
 --*/
 const char * BZ_API(BZ2_bzlibVersion)(void)
 {
@@ -1559,9 +1513,10 @@ int BZ_API(BZ2_bzflush) (BZFILE *b)
 void BZ_API(BZ2_bzclose) (BZFILE* b)
 {
    int bzerr;
-   FILE *fp = ((bzFile *)b)->handle;
+   FILE *fp;
    
    if (b==NULL) {return;}
+   fp = ((bzFile *)b)->handle;
    if(((bzFile*)b)->writing){
       BZ2_bzWriteClose(&bzerr,b,0,NULL,NULL);
       if(bzerr != BZ_OK){
@@ -1580,7 +1535,7 @@ void BZ_API(BZ2_bzclose) (BZFILE* b)
 /*--
    return last error code 
 --*/
-static char *bzerrorstrings[] = {
+static const char *bzerrorstrings[] = {
        "OK"
       ,"SEQUENCE_ERROR"
       ,"PARAM_ERROR"
diff --git a/bzlib.h b/bzlib.h
index 3237243..fdb0dbe 100644
--- a/bzlib.h
+++ b/bzlib.h
@@ -4,59 +4,19 @@
 /*---                                               bzlib.h ---*/
 /*-------------------------------------------------------------*/
 
-/*--
-  This file is a part of bzip2 and/or libbzip2, a program and
-  library for lossless, block-sorting data compression.
-
-  Copyright (C) 1996-2005 Julian R Seward.  All rights reserved.
-
-  Redistribution and use in source and binary forms, with or without
-  modification, are permitted provided that the following conditions
-  are met:
-
-  1. Redistributions of source code must retain the above copyright
-     notice, this list of conditions and the following disclaimer.
-
-  2. The origin of this software must not be misrepresented; you must 
-     not claim that you wrote the original software.  If you use this 
-     software in a product, an acknowledgment in the product 
-     documentation would be appreciated but is not required.
-
-  3. Altered source versions must be plainly marked as such, and must
-     not be misrepresented as being the original software.
-
-  4. The name of the author may not be used to endorse or promote 
-     products derived from this software without specific prior written 
-     permission.
-
-  THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
-  OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-  WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-  ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
-  DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
-  DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
-  GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-  INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
-  WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
-  NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
-  SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-  Julian Seward, Cambridge, UK.
-  jseward@bzip.org
-  bzip2/libbzip2 version 1.0 of 21 March 2000
-
-  This program is based on (at least) the work of:
-     Mike Burrows
-     David Wheeler
-     Peter Fenwick
-     Alistair Moffat
-     Radford Neal
-     Ian H. Witten
-     Robert Sedgewick
-     Jon L. Bentley
-
-  For more information on these sources, see the manual.
---*/
+/* ------------------------------------------------------------------
+   This file is part of bzip2/libbzip2, a program and library for
+   lossless, block-sorting data compression.
+
+   bzip2/libbzip2 version 1.0.4 of 20 December 2006
+   Copyright (C) 1996-2006 Julian Seward 
+
+   Please read the WARNING, DISCLAIMER and PATENTS sections in the 
+   README file.
+
+   This program is released under the terms of the license contained
+   in the file LICENSE.
+   ------------------------------------------------------------------ */
 
 
 #ifndef _BZLIB_H
@@ -262,8 +222,7 @@ BZ_EXTERN int BZ_API(BZ2_bzBuffToBuffDecompress) (
 
 
 /*--
-   Code contributed by Yoshioka Tsuneo
-   (QWF00133@niftyserve.or.jp/tsuneo-y@is.aist-nara.ac.jp),
+   Code contributed by Yoshioka Tsuneo (tsuneo@rr.iij4u.or.jp)
    to support better zlib compatibility.
    This code is not _officially_ part of libbzip2 (yet);
    I haven't tested it, documented it, or considered the
diff --git a/bzlib_private.h b/bzlib_private.h
index ca76fe6..d0a0554 100644
--- a/bzlib_private.h
+++ b/bzlib_private.h
@@ -4,59 +4,19 @@
 /*---                                       bzlib_private.h ---*/
 /*-------------------------------------------------------------*/
 
-/*--
-  This file is a part of bzip2 and/or libbzip2, a program and
-  library for lossless, block-sorting data compression.
-
-  Copyright (C) 1996-2005 Julian R Seward.  All rights reserved.
-
-  Redistribution and use in source and binary forms, with or without
-  modification, are permitted provided that the following conditions
-  are met:
-
-  1. Redistributions of source code must retain the above copyright
-     notice, this list of conditions and the following disclaimer.
-
-  2. The origin of this software must not be misrepresented; you must 
-     not claim that you wrote the original software.  If you use this 
-     software in a product, an acknowledgment in the product 
-     documentation would be appreciated but is not required.
-
-  3. Altered source versions must be plainly marked as such, and must
-     not be misrepresented as being the original software.
-
-  4. The name of the author may not be used to endorse or promote 
-     products derived from this software without specific prior written 
-     permission.
-
-  THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
-  OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-  WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-  ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
-  DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
-  DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
-  GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-  INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
-  WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
-  NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
-  SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-  Julian Seward, Cambridge, UK.
-  jseward@bzip.org
-  bzip2/libbzip2 version 1.0 of 21 March 2000
-
-  This program is based on (at least) the work of:
-     Mike Burrows
-     David Wheeler
-     Peter Fenwick
-     Alistair Moffat
-     Radford Neal
-     Ian H. Witten
-     Robert Sedgewick
-     Jon L. Bentley
-
-  For more information on these sources, see the manual.
---*/
+/* ------------------------------------------------------------------
+   This file is part of bzip2/libbzip2, a program and library for
+   lossless, block-sorting data compression.
+
+   bzip2/libbzip2 version 1.0.4 of 20 December 2006
+   Copyright (C) 1996-2006 Julian Seward 
+
+   Please read the WARNING, DISCLAIMER and PATENTS sections in the 
+   README file.
+
+   This program is released under the terms of the license contained
+   in the file LICENSE.
+   ------------------------------------------------------------------ */
 
 
 #ifndef _BZLIB_PRIVATE_H
@@ -76,7 +36,7 @@
 
 /*-- General stuff. --*/
 
-#define BZ_VERSION  "1.0.3, 15-Feb-2005"
+#define BZ_VERSION  "1.0.4, 20-Dec-2006"
 
 typedef char            Char;
 typedef unsigned char   Bool;
@@ -94,9 +54,11 @@ typedef unsigned short  UInt16;
 #endif 
 
 #ifndef BZ_NO_STDIO
+
 extern void BZ2_bz__AssertH__fail ( int errcode );
 #define AssertH(cond,errcode) \
    { if (!(cond)) BZ2_bz__AssertH__fail ( errcode ); }
+
 #if BZ_DEBUG
 #define AssertD(cond,msg) \
    { if (!(cond)) {       \
@@ -107,6 +69,7 @@ extern void BZ2_bz__AssertH__fail ( int errcode );
 #else
 #define AssertD(cond,msg) /* */
 #endif
+
 #define VPrintf0(zf) \
    fprintf(stderr,zf)
 #define VPrintf1(zf,za1) \
@@ -119,17 +82,20 @@ extern void BZ2_bz__AssertH__fail ( int errcode );
    fprintf(stderr,zf,za1,za2,za3,za4)
 #define VPrintf5(zf,za1,za2,za3,za4,za5) \
    fprintf(stderr,zf,za1,za2,za3,za4,za5)
+
 #else
+
 extern void bz_internal_error ( int errcode );
 #define AssertH(cond,errcode) \
    { if (!(cond)) bz_internal_error ( errcode ); }
-#define AssertD(cond,msg) /* */
-#define VPrintf0(zf) /* */
-#define VPrintf1(zf,za1) /* */
-#define VPrintf2(zf,za1,za2) /* */
-#define VPrintf3(zf,za1,za2,za3) /* */
-#define VPrintf4(zf,za1,za2,za3,za4) /* */
-#define VPrintf5(zf,za1,za2,za3,za4,za5) /* */
+#define AssertD(cond,msg)                do { } while (0)
+#define VPrintf0(zf)                     do { } while (0)
+#define VPrintf1(zf,za1)                 do { } while (0)
+#define VPrintf2(zf,za1,za2)             do { } while (0)
+#define VPrintf3(zf,za1,za2,za3)         do { } while (0)
+#define VPrintf4(zf,za1,za2,za3,za4)     do { } while (0)
+#define VPrintf5(zf,za1,za2,za3,za4,za5) do { } while (0)
+
 #endif
 
 
diff --git a/compress.c b/compress.c
index 7e0c291..d98d5c0 100644
--- a/compress.c
+++ b/compress.c
@@ -4,71 +4,27 @@
 /*---                                            compress.c ---*/
 /*-------------------------------------------------------------*/
 
-/*--
-  This file is a part of bzip2 and/or libbzip2, a program and
-  library for lossless, block-sorting data compression.
-
-  Copyright (C) 1996-2005 Julian R Seward.  All rights reserved.
-
-  Redistribution and use in source and binary forms, with or without
-  modification, are permitted provided that the following conditions
-  are met:
-
-  1. Redistributions of source code must retain the above copyright
-     notice, this list of conditions and the following disclaimer.
-
-  2. The origin of this software must not be misrepresented; you must 
-     not claim that you wrote the original software.  If you use this 
-     software in a product, an acknowledgment in the product 
-     documentation would be appreciated but is not required.
-
-  3. Altered source versions must be plainly marked as such, and must
-     not be misrepresented as being the original software.
-
-  4. The name of the author may not be used to endorse or promote 
-     products derived from this software without specific prior written 
-     permission.
-
-  THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
-  OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-  WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-  ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
-  DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
-  DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
-  GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-  INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
-  WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
-  NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
-  SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-  Julian Seward, Cambridge, UK.
-  jseward@bzip.org
-  bzip2/libbzip2 version 1.0 of 21 March 2000
-
-  This program is based on (at least) the work of:
-     Mike Burrows
-     David Wheeler
-     Peter Fenwick
-     Alistair Moffat
-     Radford Neal
-     Ian H. Witten
-     Robert Sedgewick
-     Jon L. Bentley
-
-  For more information on these sources, see the manual.
---*/
-
-/*--
-   CHANGES
-   ~~~~~~~
-   0.9.0 -- original version.
-
-   0.9.0a/b -- no changes in this file.
-
-   0.9.0c
-      * changed setting of nGroups in sendMTFValues() so as to 
-        do a bit better on small files
---*/
+/* ------------------------------------------------------------------
+   This file is part of bzip2/libbzip2, a program and library for
+   lossless, block-sorting data compression.
+
+   bzip2/libbzip2 version 1.0.4 of 20 December 2006
+   Copyright (C) 1996-2006 Julian Seward 
+
+   Please read the WARNING, DISCLAIMER and PATENTS sections in the 
+   README file.
+
+   This program is released under the terms of the license contained
+   in the file LICENSE.
+   ------------------------------------------------------------------ */
+
+
+/* CHANGES
+    0.9.0    -- original version.
+    0.9.0a/b -- no changes in this file.
+    0.9.0c   -- changed setting of nGroups in sendMTFValues() 
+                so as to do a bit better on small files
+*/
 
 #include "bzlib_private.h"
 
diff --git a/crctable.c b/crctable.c
index b6dadfc..bc7e2ae 100644
--- a/crctable.c
+++ b/crctable.c
@@ -4,59 +4,19 @@
 /*---                                            crctable.c ---*/
 /*-------------------------------------------------------------*/
 
-/*--
-  This file is a part of bzip2 and/or libbzip2, a program and
-  library for lossless, block-sorting data compression.
-
-  Copyright (C) 1996-2005 Julian R Seward.  All rights reserved.
-
-  Redistribution and use in source and binary forms, with or without
-  modification, are permitted provided that the following conditions
-  are met:
-
-  1. Redistributions of source code must retain the above copyright
-     notice, this list of conditions and the following disclaimer.
-
-  2. The origin of this software must not be misrepresented; you must 
-     not claim that you wrote the original software.  If you use this 
-     software in a product, an acknowledgment in the product 
-     documentation would be appreciated but is not required.
+/* ------------------------------------------------------------------
+   This file is part of bzip2/libbzip2, a program and library for
+   lossless, block-sorting data compression.
 
-  3. Altered source versions must be plainly marked as such, and must
-     not be misrepresented as being the original software.
+   bzip2/libbzip2 version 1.0.4 of 20 December 2006
+   Copyright (C) 1996-2006 Julian Seward 
 
-  4. The name of the author may not be used to endorse or promote 
-     products derived from this software without specific prior written 
-     permission.
+   Please read the WARNING, DISCLAIMER and PATENTS sections in the 
+   README file.
 
-  THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
-  OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-  WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-  ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
-  DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
-  DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
-  GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-  INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
-  WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
-  NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
-  SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-  Julian Seward, Cambridge, UK.
-  jseward@bzip.org
-  bzip2/libbzip2 version 1.0 of 21 March 2000
-
-  This program is based on (at least) the work of:
-     Mike Burrows
-     David Wheeler
-     Peter Fenwick
-     Alistair Moffat
-     Radford Neal
-     Ian H. Witten
-     Robert Sedgewick
-     Jon L. Bentley
-
-  For more information on these sources, see the manual.
---*/
+   This program is released under the terms of the license contained
+   in the file LICENSE.
+   ------------------------------------------------------------------ */
 
 
 #include "bzlib_private.h"
diff --git a/decompress.c b/decompress.c
index 81c3d2c..124cc8d 100644
--- a/decompress.c
+++ b/decompress.c
@@ -4,59 +4,19 @@
 /*---                                          decompress.c ---*/
 /*-------------------------------------------------------------*/
 
-/*--
-  This file is a part of bzip2 and/or libbzip2, a program and
-  library for lossless, block-sorting data compression.
-
-  Copyright (C) 1996-2005 Julian R Seward.  All rights reserved.
-
-  Redistribution and use in source and binary forms, with or without
-  modification, are permitted provided that the following conditions
-  are met:
-
-  1. Redistributions of source code must retain the above copyright
-     notice, this list of conditions and the following disclaimer.
-
-  2. The origin of this software must not be misrepresented; you must 
-     not claim that you wrote the original software.  If you use this 
-     software in a product, an acknowledgment in the product 
-     documentation would be appreciated but is not required.
-
-  3. Altered source versions must be plainly marked as such, and must
-     not be misrepresented as being the original software.
-
-  4. The name of the author may not be used to endorse or promote 
-     products derived from this software without specific prior written 
-     permission.
-
-  THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
-  OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-  WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-  ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
-  DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
-  DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
-  GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-  INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
-  WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
-  NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
-  SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-  Julian Seward, Cambridge, UK.
-  jseward@bzip.org
-  bzip2/libbzip2 version 1.0 of 21 March 2000
-
-  This program is based on (at least) the work of:
-     Mike Burrows
-     David Wheeler
-     Peter Fenwick
-     Alistair Moffat
-     Radford Neal
-     Ian H. Witten
-     Robert Sedgewick
-     Jon L. Bentley
-
-  For more information on these sources, see the manual.
---*/
+/* ------------------------------------------------------------------
+   This file is part of bzip2/libbzip2, a program and library for
+   lossless, block-sorting data compression.
+
+   bzip2/libbzip2 version 1.0.4 of 20 December 2006
+   Copyright (C) 1996-2006 Julian Seward 
+
+   Please read the WARNING, DISCLAIMER and PATENTS sections in the 
+   README file.
+
+   This program is released under the terms of the license contained
+   in the file LICENSE.
+   ------------------------------------------------------------------ */
 
 
 #include "bzlib_private.h"
diff --git a/dlltest.c b/dlltest.c
index 2d7dcca..03fa146 100644
--- a/dlltest.c
+++ b/dlltest.c
@@ -1,9 +1,8 @@
 /*
    minibz2
       libbz2.dll test program.
-      by Yoshioka Tsuneo(QWF00133@nifty.ne.jp/tsuneo-y@is.aist-nara.ac.jp)
-      This file is Public Domain.
-      welcome any email to me.
+      by Yoshioka Tsuneo (tsuneo@rr.iij4u.or.jp)
+      This file is Public Domain.  Welcome any email to me.
 
    usage: minibz2 [-d] [-{1,2,..9}] [[srcfilename] destfilename]
 */
diff --git a/entities.xml b/entities.xml
index 6d0975f..ce2b70d 100644
--- a/entities.xml
+++ b/entities.xml
@@ -1,9 +1,9 @@
 
 
 
-
+
 
-
-
+
+
 
 
diff --git a/format.pl b/format.pl
index 8ab47ac..bd03043 100755
--- a/format.pl
+++ b/format.pl
@@ -1,4 +1,19 @@
 #!/usr/bin/perl -w
+#
+# ------------------------------------------------------------------
+# This file is part of bzip2/libbzip2, a program and library for
+# lossless, block-sorting data compression.
+#
+# bzip2/libbzip2 version 1.0.4 of 20 December 2006
+# Copyright (C) 1996-2006 Julian Seward 
+#
+# Please read the WARNING, DISCLAIMER and PATENTS sections in the 
+# README file.
+#
+# This program is released under the terms of the license contained
+# in the file LICENSE.
+# ------------------------------------------------------------------
+#
 use strict;
 
 # get command line values:
diff --git a/huffman.c b/huffman.c
index 5bf190b..be4dc02 100644
--- a/huffman.c
+++ b/huffman.c
@@ -4,59 +4,19 @@
 /*---                                             huffman.c ---*/
 /*-------------------------------------------------------------*/
 
-/*--
-  This file is a part of bzip2 and/or libbzip2, a program and
-  library for lossless, block-sorting data compression.
-
-  Copyright (C) 1996-2005 Julian R Seward.  All rights reserved.
-
-  Redistribution and use in source and binary forms, with or without
-  modification, are permitted provided that the following conditions
-  are met:
-
-  1. Redistributions of source code must retain the above copyright
-     notice, this list of conditions and the following disclaimer.
-
-  2. The origin of this software must not be misrepresented; you must 
-     not claim that you wrote the original software.  If you use this 
-     software in a product, an acknowledgment in the product 
-     documentation would be appreciated but is not required.
-
-  3. Altered source versions must be plainly marked as such, and must
-     not be misrepresented as being the original software.
-
-  4. The name of the author may not be used to endorse or promote 
-     products derived from this software without specific prior written 
-     permission.
-
-  THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
-  OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-  WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-  ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
-  DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
-  DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
-  GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-  INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
-  WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
-  NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
-  SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-  Julian Seward, Cambridge, UK.
-  jseward@bzip.org
-  bzip2/libbzip2 version 1.0 of 21 March 2000
-
-  This program is based on (at least) the work of:
-     Mike Burrows
-     David Wheeler
-     Peter Fenwick
-     Alistair Moffat
-     Radford Neal
-     Ian H. Witten
-     Robert Sedgewick
-     Jon L. Bentley
-
-  For more information on these sources, see the manual.
---*/
+/* ------------------------------------------------------------------
+   This file is part of bzip2/libbzip2, a program and library for
+   lossless, block-sorting data compression.
+
+   bzip2/libbzip2 version 1.0.4 of 20 December 2006
+   Copyright (C) 1996-2006 Julian Seward 
+
+   Please read the WARNING, DISCLAIMER and PATENTS sections in the 
+   README file.
+
+   This program is released under the terms of the license contained
+   in the file LICENSE.
+   ------------------------------------------------------------------ */
 
 
 #include "bzlib_private.h"
diff --git a/manual.xml b/manual.xml
index 1ab5bd7..b22a6c8 100644
--- a/manual.xml
+++ b/manual.xml
@@ -9,7 +9,7 @@
 
 
  
-  bzip2 and libbzip2, version 1.0.3
+  bzip2 and libbzip2, version 1.0.4
   A program and library for data compression
   
    &bz-lifespan;
@@ -139,7 +139,7 @@ else.
 
  bzip2,
   bunzip2 - a block-sorting file
-  compressor, v1.0.3
+  compressor, v1.0.4
 
  bzcat -
    decompresses files to stdout
@@ -625,13 +625,13 @@ errors and exit cleanly, but the details of what the problem is
 sometimes seem rather misleading.
 
 This manual page pertains to version &bz-version; of
-bzip2.  Compressed data created
-by this version is entirely forwards and backwards compatible
-with the previous public releases, versions 0.1pl2, 0.9.0 and
-0.9.5, 1.0.0, 1.0.1 and 1.0.2, but with the following exception: 0.9.0
-and above can correctly decompress multiple concatenated
-compressed files.  0.1pl2 cannot do this; it will stop after
-decompressing just the first file in the stream.
+bzip2.  Compressed data created by
+this version is entirely forwards and backwards compatible with the
+previous public releases, versions 0.1pl2, 0.9.0 and 0.9.5, 1.0.0,
+1.0.1, 1.0.2 and 1.0.3, but with the following exception: 0.9.0 and
+above can correctly decompress multiple concatenated compressed files.
+0.1pl2 cannot do this; it will stop after decompressing just the first
+file in the stream.
 
 bzip2recover versions
 prior to 1.0.2 used 32-bit integers to represent bit positions in
@@ -819,8 +819,7 @@ understanding the more general but more complex low-level
 interface.
 
 Yoshioka Tsuneo
-(QWF00133@niftyserve.or.jp /
-tsuneo-y@is.aist-nara.ac.jp) has
+(tsuneo@rr.iij4u.or.jp) has
 contributed some functions to give better
 zlib compatibility.  These
 functions are BZ2_bzopen,
@@ -1352,7 +1351,7 @@ FINISHING/BZ_FINISH
   output has been removed
     Next state = IDLE; Return value = BZ_STREAM_END
   else
-    Next state = FINISHING; Return value = BZ_FINISHING
+    Next state = FINISHING; Return value = BZ_FINISH_OK
 
 FINISHING/other
   Illegal.
@@ -2222,7 +2221,7 @@ if ( bzerror != BZ_STREAM_END ) {
    BZ2_bzReadClose ( &bzerror, b );
    /* handle error */
 } else {
-   BZ2_bzReadClose ( &bzerror );
+   BZ2_bzReadClose ( &bzerror, b );
 }
 
 
@@ -2537,8 +2536,7 @@ recovered from.
 
 Everything related to Windows has been contributed by
 Yoshioka Tsuneo
-(QWF00133@niftyserve.or.jp /
-tsuneo-y@is.aist-nara.ac.jp), so
+(tsuneo@rr.iij4u.or.jp), so
 you should send your queries to him (but perhaps Cc: me,
 &bz-email;).
 
diff --git a/mk251.c b/mk251.c
index 205778a..b56d420 100644
--- a/mk251.c
+++ b/mk251.c
@@ -5,6 +5,21 @@
    case, which is fixed in this version (1.0.2) and above.
 */
 
+/* ------------------------------------------------------------------
+   This file is part of bzip2/libbzip2, a program and library for
+   lossless, block-sorting data compression.
+
+   bzip2/libbzip2 version 1.0.4 of 20 December 2006
+   Copyright (C) 1996-2006 Julian Seward 
+
+   Please read the WARNING, DISCLAIMER and PATENTS sections in the 
+   README file.
+
+   This program is released under the terms of the license contained
+   in the file LICENSE.
+   ------------------------------------------------------------------ */
+
+
 #include 
 
 int main ()
diff --git a/randtable.c b/randtable.c
index 940462d..d186335 100644
--- a/randtable.c
+++ b/randtable.c
@@ -4,59 +4,19 @@
 /*---                                           randtable.c ---*/
 /*-------------------------------------------------------------*/
 
-/*--
-  This file is a part of bzip2 and/or libbzip2, a program and
-  library for lossless, block-sorting data compression.
+/* ------------------------------------------------------------------
+   This file is part of bzip2/libbzip2, a program and library for
+   lossless, block-sorting data compression.
 
-  Copyright (C) 1996-2005 Julian R Seward.  All rights reserved.
+   bzip2/libbzip2 version 1.0.4 of 20 December 2006
+   Copyright (C) 1996-2006 Julian Seward 
 
-  Redistribution and use in source and binary forms, with or without
-  modification, are permitted provided that the following conditions
-  are met:
+   Please read the WARNING, DISCLAIMER and PATENTS sections in the 
+   README file.
 
-  1. Redistributions of source code must retain the above copyright
-     notice, this list of conditions and the following disclaimer.
-
-  2. The origin of this software must not be misrepresented; you must 
-     not claim that you wrote the original software.  If you use this 
-     software in a product, an acknowledgment in the product 
-     documentation would be appreciated but is not required.
-
-  3. Altered source versions must be plainly marked as such, and must
-     not be misrepresented as being the original software.
-
-  4. The name of the author may not be used to endorse or promote 
-     products derived from this software without specific prior written 
-     permission.
-
-  THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
-  OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-  WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
-  ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
-  DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
-  DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
-  GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-  INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
-  WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
-  NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
-  SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-  Julian Seward, Cambridge, UK.
-  jseward@bzip.org
-  bzip2/libbzip2 version 1.0 of 21 March 2000
-
-  This program is based on (at least) the work of:
-     Mike Burrows
-     David Wheeler
-     Peter Fenwick
-     Alistair Moffat
-     Radford Neal
-     Ian H. Witten
-     Robert Sedgewick
-     Jon L. Bentley
-
-  For more information on these sources, see the manual.
---*/
+   This program is released under the terms of the license contained
+   in the file LICENSE.
+   ------------------------------------------------------------------ */
 
 
 #include "bzlib_private.h"
diff --git a/spewG.c b/spewG.c
index 7934e76..e1f3032 100644
--- a/spewG.c
+++ b/spewG.c
@@ -9,6 +9,21 @@
    (but is otherwise harmless).
 */
 
+/* ------------------------------------------------------------------
+   This file is part of bzip2/libbzip2, a program and library for
+   lossless, block-sorting data compression.
+
+   bzip2/libbzip2 version 1.0.4 of 20 December 2006
+   Copyright (C) 1996-2006 Julian Seward 
+
+   Please read the WARNING, DISCLAIMER and PATENTS sections in the 
+   README file.
+
+   This program is released under the terms of the license contained
+   in the file LICENSE.
+	 ------------------------------------------------------------------ */
+
+
 #define _FILE_OFFSET_BITS 64
 
 #include 
diff --git a/unzcrash.c b/unzcrash.c
index f0f17fc..1f08a2e 100644
--- a/unzcrash.c
+++ b/unzcrash.c
@@ -8,11 +8,26 @@
    This should not cause any invalid memory accesses.  If it does, 
    I want to know about it!
 
-   p.s.  As you can see from the above description, the process is
+   PS.  As you can see from the above description, the process is
    incredibly slow.  A file of size eg 5KB will cause it to run for
    many hours.
 */
 
+/* ------------------------------------------------------------------
+   This file is part of bzip2/libbzip2, a program and library for
+   lossless, block-sorting data compression.
+
+   bzip2/libbzip2 version 1.0.4 of 20 December 2006
+   Copyright (C) 1996-2006 Julian Seward 
+
+   Please read the WARNING, DISCLAIMER and PATENTS sections in the 
+   README file.
+
+   This program is released under the terms of the license contained
+   in the file LICENSE.
+   ------------------------------------------------------------------ */
+
+
 #include 
 #include 
 #include "bzlib.h"
diff --git a/words0 b/words0
index 164a8ed..fbf442a 100644
--- a/words0
+++ b/words0
@@ -1,5 +1,9 @@
 
-If compilation produces errors, or a large number of warnings, 
+If compilation produces errors, or a large number of warnings,
 please read README.COMPILATION.PROBLEMS -- you might be able to
 adjust the flags in this Makefile to improve matters.
 
+Also in README.COMPILATION.PROBLEMS are some hints that may help
+if your build produces an executable which is unable to correctly
+handle so-called 'large files' -- files of size 2GB or more.
+
diff --git a/words2 b/words2
index 203ee39..caddcf4 100644
--- a/words2
+++ b/words2
@@ -1,5 +1,5 @@
 
 Checking test results.  If any of the four "cmp"s which follow
 report any differences, something is wrong.  If you can't easily
-figure out what, please let me know (jseward@acm.org).
+figure out what, please let me know (jseward@bzip.org).
 
diff --git a/words3 b/words3
index 7a6b462..6972669 100644
--- a/words3
+++ b/words3
@@ -1,22 +1,29 @@
 
-If you got this far and the "cmp"s didn't complain, it looks
+If you got this far and the 'cmp's didn't complain, it looks
 like you're in business.  
 
-To install in /usr/bin, /usr/lib, /usr/man and /usr/include, type
+To install in /usr/local/bin, /usr/local/lib, /usr/local/man and 
+/usr/local/include, type
+
    make install
+
 To install somewhere else, eg, /xxx/yyy/{bin,lib,man,include}, type 
+
    make install PREFIX=/xxx/yyy
+
 If you are (justifiably) paranoid and want to see what 'make install'
 is going to do, you can first do
+
    make -n install                      or
    make -n install PREFIX=/xxx/yyy      respectively.
+
 The -n instructs make to show the commands it would execute, but
 not actually execute them.
 
 Instructions for use are in the preformatted manual page, in the file
 bzip2.txt.  For more detailed documentation, read the full manual.  
 It is available in Postscript form (manual.ps), PDF form (manual.pdf),
-and HTML form (manual_toc.html).
+and HTML form (manual.html).
 
 You can also do "bzip2 --help" to see some helpful information. 
 "bzip2 -L" displays the software license.
diff --git a/xmlproc.sh b/xmlproc.sh
index 6fe4d57..5a1f051 100755
--- a/xmlproc.sh
+++ b/xmlproc.sh
@@ -1,5 +1,20 @@
 #!/bin/bash
-# see the README in this directory for usage etc.
+# see the README file for usage etc.
+#
+# ------------------------------------------------------------------
+#  This file is part of bzip2/libbzip2, a program and library for
+#  lossless, block-sorting data compression.
+#
+#  bzip2/libbzip2 version 1.0.4 of 20 December 2006
+#  Copyright (C) 1996-2006 Julian Seward 
+#
+#  Please read the WARNING, DISCLAIMER and PATENTS sections in the 
+#  README file.
+#
+#  This program is released under the terms of the license contained
+#  in the file LICENSE.
+# ----------------------------------------------------------------
+
 
 usage() {
   echo '';
@@ -45,7 +60,7 @@ export XML_CATALOG_FILES=/etc/xml/catalog
 
 # post-processing tidy up
 cleanup() {
-  echo "Cleaning up: # $@" 
+  echo "Cleaning up: $@" 
   while [ $# != 0 ]
   do
     arg=$1; shift;
-- 
cgit v1.2.3


From a1d78c550180bde11af6c559a01721c962f9f231 Mon Sep 17 00:00:00 2001
From: Julian Seward 
Date: Mon, 10 Dec 2007 22:13:13 +0100
Subject: bzip2-1.0.5

---
 CHANGES                     |  9 +++++++--
 LICENSE                     |  7 +++----
 Makefile                    |  6 +++---
 Makefile-libbz2_so          |  4 ++--
 README                      |  9 +++++++--
 README.COMPILATION.PROBLEMS |  6 +++---
 README.XML.STUFF            |  4 ++--
 blocksort.c                 |  4 ++--
 bzip2.c                     |  8 ++++----
 bzip2recover.c              |  6 +++---
 bzlib.c                     |  7 ++++---
 bzlib.h                     |  4 ++--
 bzlib_private.h             | 16 +++++++++++-----
 compress.c                  |  4 ++--
 crctable.c                  |  4 ++--
 decompress.c                |  4 ++--
 entities.xml                |  6 +++---
 format.pl                   |  4 ++--
 huffman.c                   |  4 ++--
 manual.xml                  |  2 +-
 mk251.c                     |  4 ++--
 randtable.c                 |  4 ++--
 spewG.c                     |  4 ++--
 unzcrash.c                  |  4 ++--
 xmlproc.sh                  |  4 ++--
 25 files changed, 77 insertions(+), 61 deletions(-)

diff --git a/CHANGES b/CHANGES
index 0959ea0..6e4f65e 100644
--- a/CHANGES
+++ b/CHANGES
@@ -2,8 +2,8 @@
  This file is part of bzip2/libbzip2, a program and library for
  lossless, block-sorting data compression.
 
- bzip2/libbzip2 version 1.0.4 of 20 December 2006
- Copyright (C) 1996-2006 Julian Seward 
+ bzip2/libbzip2 version 1.0.5 of 10 December 2007
+ Copyright (C) 1996-2007 Julian Seward 
 
  Please read the WARNING, DISCLAIMER and PATENTS sections in the 
  README file.
@@ -312,3 +312,8 @@ Fixes some minor bugs since the last version, 1.0.3.
 
 * Fix minor doc/comment bugs.
 
+
+1.0.5 (10 Dec 07)
+~~~~~~~~~~~~~~~~~
+Security fix only.  Fixes CERT-FI 20469 as it applies to bzip2.
+
diff --git a/LICENSE b/LICENSE
index 4458e35..f420cff 100644
--- a/LICENSE
+++ b/LICENSE
@@ -2,7 +2,7 @@
 --------------------------------------------------------------------------
 
 This program, "bzip2", the associated library "libbzip2", and all
-documentation, are copyright (C) 1996-2006 Julian R Seward.  All
+documentation, are copyright (C) 1996-2007 Julian R Seward.  All
 rights reserved.
 
 Redistribution and use in source and binary forms, with or without
@@ -36,8 +36,7 @@ WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
 NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
-Julian Seward, Cambridge, UK.
-jseward@bzip.org
-bzip2/libbzip2 version 1.0.4 of 20 December 2006
+Julian Seward, jseward@bzip.org
+bzip2/libbzip2 version 1.0.5 of 10 December 2007
 
 --------------------------------------------------------------------------
diff --git a/Makefile b/Makefile
index c4cbc04..eb09753 100644
--- a/Makefile
+++ b/Makefile
@@ -2,8 +2,8 @@
 # This file is part of bzip2/libbzip2, a program and library for
 # lossless, block-sorting data compression.
 #
-# bzip2/libbzip2 version 1.0.4 of 20 December 2006
-# Copyright (C) 1996-2006 Julian Seward 
+# bzip2/libbzip2 version 1.0.5 of 10 December 2007
+# Copyright (C) 1996-2007 Julian Seward 
 #
 # Please read the WARNING, DISCLAIMER and PATENTS sections in the 
 # README file.
@@ -137,7 +137,7 @@ bzip2recover.o: bzip2recover.c
 distclean: clean
 	rm -f manual.ps manual.html manual.pdf
 
-DISTNAME=bzip2-1.0.4
+DISTNAME=bzip2-1.0.5
 dist: check manual
 	rm -f $(DISTNAME)
 	ln -s -f . $(DISTNAME)
diff --git a/Makefile-libbz2_so b/Makefile-libbz2_so
index 2b65f3b..9a13c77 100644
--- a/Makefile-libbz2_so
+++ b/Makefile-libbz2_so
@@ -10,8 +10,8 @@
 # This file is part of bzip2/libbzip2, a program and library for
 # lossless, block-sorting data compression.
 #
-# bzip2/libbzip2 version 1.0.4 of 20 December 2006
-# Copyright (C) 1996-2006 Julian Seward 
+# bzip2/libbzip2 version 1.0.5 of 10 December 2007
+# Copyright (C) 1996-2007 Julian Seward 
 #
 # Please read the WARNING, DISCLAIMER and PATENTS sections in the 
 # README file.
diff --git a/README b/README
index b18c096..e17a84e 100644
--- a/README
+++ b/README
@@ -6,8 +6,8 @@ This version is fully compatible with the previous public releases.
 This file is part of bzip2/libbzip2, a program and library for
 lossless, block-sorting data compression.
 
-bzip2/libbzip2 version 1.0.4 of 20 December 2006
-Copyright (C) 1996-2006 Julian Seward 
+bzip2/libbzip2 version 1.0.5 of 10 December 2007
+Copyright (C) 1996-2007 Julian Seward 
 
 Please read the WARNING, DISCLAIMER and PATENTS sections in this file.
 
@@ -177,6 +177,10 @@ WHAT'S NEW IN 1.0.4 ?
 
    See the CHANGES file.
 
+WHAT'S NEW IN 1.0.5 ?
+
+   See the CHANGES file.
+
 
 I hope you find bzip2 useful.  Feel free to contact me at
    jseward@bzip.org
@@ -203,3 +207,4 @@ Cambridge, UK.
 30 December 2001 (bzip2, version 1.0.2pre1)
 15 February 2005 (bzip2, version 1.0.3)
 20 December 2006 (bzip2, version 1.0.4)
+10 December 2007 (bzip2, version 1.0.5)
diff --git a/README.COMPILATION.PROBLEMS b/README.COMPILATION.PROBLEMS
index 64ab586..22b95c6 100644
--- a/README.COMPILATION.PROBLEMS
+++ b/README.COMPILATION.PROBLEMS
@@ -2,8 +2,8 @@
 This file is part of bzip2/libbzip2, a program and library for
 lossless, block-sorting data compression.
 
-bzip2/libbzip2 version 1.0.4 of 20 December 2006
-Copyright (C) 1996-2006 Julian Seward 
+bzip2/libbzip2 version 1.0.5 of 10 December 2007
+Copyright (C) 1996-2007 Julian Seward 
 
 Please read the WARNING, DISCLAIMER and PATENTS sections in the 
 README file.
@@ -12,7 +12,7 @@ This program is released under the terms of the license contained
 in the file LICENSE.
 ------------------------------------------------------------------
 
-bzip2-1.0.4 should compile without problems on the vast majority of
+bzip2-1.0.5 should compile without problems on the vast majority of
 platforms.  Using the supplied Makefile, I've built and tested it
 myself for x86-linux and amd64-linux.  With makefile.msc, Visual C++
 6.0 and nmake, you can build a native Win32 version too.  Large file
diff --git a/README.XML.STUFF b/README.XML.STUFF
index 93bf405..1a5b4c5 100644
--- a/README.XML.STUFF
+++ b/README.XML.STUFF
@@ -2,8 +2,8 @@
   This file is part of bzip2/libbzip2, a program and library for
   lossless, block-sorting data compression.
 
-  bzip2/libbzip2 version 1.0.4 of 20 December 2006
-  Copyright (C) 1996-2006 Julian Seward 
+  bzip2/libbzip2 version 1.0.5 of 10 December 2007
+  Copyright (C) 1996-2007 Julian Seward 
 
   Please read the WARNING, DISCLAIMER and PATENTS sections in the 
   README file.
diff --git a/blocksort.c b/blocksort.c
index 8535c93..bd2dec1 100644
--- a/blocksort.c
+++ b/blocksort.c
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.4 of 20 December 2006
-   Copyright (C) 1996-2006 Julian Seward 
+   bzip2/libbzip2 version 1.0.5 of 10 December 2007
+   Copyright (C) 1996-2007 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/bzip2.c b/bzip2.c
index 5fbae0e..3904107 100644
--- a/bzip2.c
+++ b/bzip2.c
@@ -7,8 +7,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.4 of 20 December 2006
-   Copyright (C) 1996-2006 Julian Seward 
+   bzip2/libbzip2 version 1.0.5 of 10 December 2007
+   Copyright (C) 1996-2007 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
@@ -1605,11 +1605,11 @@ void license ( void )
     "bzip2, a block-sorting file compressor.  "
     "Version %s.\n"
     "   \n"
-    "   Copyright (C) 1996-2006 by Julian Seward.\n"
+    "   Copyright (C) 1996-2007 by Julian Seward.\n"
     "   \n"
     "   This program is free software; you can redistribute it and/or modify\n"
     "   it under the terms set out in the LICENSE file, which is included\n"
-    "   in the bzip2-1.0.4 source distribution.\n"
+    "   in the bzip2-1.0.5 source distribution.\n"
     "   \n"
     "   This program is distributed in the hope that it will be useful,\n"
     "   but WITHOUT ANY WARRANTY; without even the implied warranty of\n"
diff --git a/bzip2recover.c b/bzip2recover.c
index ffe60bc..5f6d621 100644
--- a/bzip2recover.c
+++ b/bzip2recover.c
@@ -7,8 +7,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.4 of 20 December 2006
-   Copyright (C) 1996-2006 Julian Seward 
+   bzip2/libbzip2 version 1.0.5 of 10 December 2007
+   Copyright (C) 1996-2007 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
@@ -313,7 +313,7 @@ Int32 main ( Int32 argc, Char** argv )
    inFileName[0] = outFileName[0] = 0;
 
    fprintf ( stderr, 
-             "bzip2recover 1.0.4: extracts blocks from damaged .bz2 files.\n" );
+             "bzip2recover 1.0.5: extracts blocks from damaged .bz2 files.\n" );
 
    if (argc != 2) {
       fprintf ( stderr, "%s: usage is `%s damaged_file_name'.\n",
diff --git a/bzlib.c b/bzlib.c
index 79c34a5..ef86c91 100644
--- a/bzlib.c
+++ b/bzlib.c
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.4 of 20 December 2006
-   Copyright (C) 1996-2006 Julian Seward 
+   bzip2/libbzip2 version 1.0.5 of 10 December 2007
+   Copyright (C) 1996-2007 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
@@ -48,7 +48,7 @@ void BZ2_bz__AssertH__fail ( int errcode )
       "component, you should also report this bug to the author(s)\n"
       "of that program.  Please make an effort to report this bug;\n"
       "timely and accurate bug reports eventually lead to higher\n"
-      "quality software.  Thanks.  Julian Seward, 15 February 2005.\n\n",
+      "quality software.  Thanks.  Julian Seward, 10 December 2007.\n\n",
       errcode,
       BZ2_bzlibVersion()
    );
@@ -598,6 +598,7 @@ Bool unRLE_obuf_to_output_FAST ( DState* s )
       UInt32        c_tPos               = s->tPos;
       char*         cs_next_out          = s->strm->next_out;
       unsigned int  cs_avail_out         = s->strm->avail_out;
+      Int32         ro_blockSize100k     = s->blockSize100k;
       /* end restore */
 
       UInt32       avail_out_INIT = cs_avail_out;
diff --git a/bzlib.h b/bzlib.h
index fdb0dbe..c5b75d6 100644
--- a/bzlib.h
+++ b/bzlib.h
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.4 of 20 December 2006
-   Copyright (C) 1996-2006 Julian Seward 
+   bzip2/libbzip2 version 1.0.5 of 10 December 2007
+   Copyright (C) 1996-2007 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/bzlib_private.h b/bzlib_private.h
index d0a0554..2342787 100644
--- a/bzlib_private.h
+++ b/bzlib_private.h
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.4 of 20 December 2006
-   Copyright (C) 1996-2006 Julian Seward 
+   bzip2/libbzip2 version 1.0.5 of 10 December 2007
+   Copyright (C) 1996-2007 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
@@ -36,7 +36,7 @@
 
 /*-- General stuff. --*/
 
-#define BZ_VERSION  "1.0.4, 20-Dec-2006"
+#define BZ_VERSION  "1.0.5, 10-Dec-2007"
 
 typedef char            Char;
 typedef unsigned char   Bool;
@@ -442,11 +442,15 @@ typedef
 /*-- Macros for decompression. --*/
 
 #define BZ_GET_FAST(cccc)                     \
+    /* c_tPos is unsigned, hence test < 0 is pointless. */ \
+    if (s->tPos >= (UInt32)100000 * (UInt32)s->blockSize100k) return True; \
     s->tPos = s->tt[s->tPos];                 \
     cccc = (UChar)(s->tPos & 0xff);           \
     s->tPos >>= 8;
 
 #define BZ_GET_FAST_C(cccc)                   \
+    /* c_tPos is unsigned, hence test < 0 is pointless. */ \
+    if (c_tPos >= (UInt32)100000 * (UInt32)ro_blockSize100k) return True; \
     c_tPos = c_tt[c_tPos];                    \
     cccc = (UChar)(c_tPos & 0xff);            \
     c_tPos >>= 8;
@@ -469,8 +473,10 @@ typedef
    (((UInt32)s->ll16[i]) | (GET_LL4(i) << 16))
 
 #define BZ_GET_SMALL(cccc)                            \
-      cccc = BZ2_indexIntoF ( s->tPos, s->cftab );    \
-      s->tPos = GET_LL(s->tPos);
+    /* c_tPos is unsigned, hence test < 0 is pointless. */ \
+    if (s->tPos >= (UInt32)100000 * (UInt32)s->blockSize100k) return True; \
+    cccc = BZ2_indexIntoF ( s->tPos, s->cftab );    \
+    s->tPos = GET_LL(s->tPos);
 
 
 /*-- externs for decompression. --*/
diff --git a/compress.c b/compress.c
index d98d5c0..8c80a07 100644
--- a/compress.c
+++ b/compress.c
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.4 of 20 December 2006
-   Copyright (C) 1996-2006 Julian Seward 
+   bzip2/libbzip2 version 1.0.5 of 10 December 2007
+   Copyright (C) 1996-2007 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/crctable.c b/crctable.c
index bc7e2ae..215687b 100644
--- a/crctable.c
+++ b/crctable.c
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.4 of 20 December 2006
-   Copyright (C) 1996-2006 Julian Seward 
+   bzip2/libbzip2 version 1.0.5 of 10 December 2007
+   Copyright (C) 1996-2007 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/decompress.c b/decompress.c
index 124cc8d..bba5e0f 100644
--- a/decompress.c
+++ b/decompress.c
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.4 of 20 December 2006
-   Copyright (C) 1996-2006 Julian Seward 
+   bzip2/libbzip2 version 1.0.5 of 10 December 2007
+   Copyright (C) 1996-2007 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/entities.xml b/entities.xml
index ce2b70d..e9e0553 100644
--- a/entities.xml
+++ b/entities.xml
@@ -1,9 +1,9 @@
 
 
 
-
+
 
-
-
+
+
 
 
diff --git a/format.pl b/format.pl
index bd03043..1928d15 100755
--- a/format.pl
+++ b/format.pl
@@ -4,8 +4,8 @@
 # This file is part of bzip2/libbzip2, a program and library for
 # lossless, block-sorting data compression.
 #
-# bzip2/libbzip2 version 1.0.4 of 20 December 2006
-# Copyright (C) 1996-2006 Julian Seward 
+# bzip2/libbzip2 version 1.0.5 of 10 December 2007
+# Copyright (C) 1996-2007 Julian Seward 
 #
 # Please read the WARNING, DISCLAIMER and PATENTS sections in the 
 # README file.
diff --git a/huffman.c b/huffman.c
index be4dc02..87e79e3 100644
--- a/huffman.c
+++ b/huffman.c
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.4 of 20 December 2006
-   Copyright (C) 1996-2006 Julian Seward 
+   bzip2/libbzip2 version 1.0.5 of 10 December 2007
+   Copyright (C) 1996-2007 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/manual.xml b/manual.xml
index b22a6c8..f224136 100644
--- a/manual.xml
+++ b/manual.xml
@@ -9,7 +9,7 @@
 
 
  
-  bzip2 and libbzip2, version 1.0.4
+  bzip2 and libbzip2, version 1.0.5
   A program and library for data compression
   
    &bz-lifespan;
diff --git a/mk251.c b/mk251.c
index b56d420..39e94c0 100644
--- a/mk251.c
+++ b/mk251.c
@@ -9,8 +9,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.4 of 20 December 2006
-   Copyright (C) 1996-2006 Julian Seward 
+   bzip2/libbzip2 version 1.0.5 of 10 December 2007
+   Copyright (C) 1996-2007 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/randtable.c b/randtable.c
index d186335..068b763 100644
--- a/randtable.c
+++ b/randtable.c
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.4 of 20 December 2006
-   Copyright (C) 1996-2006 Julian Seward 
+   bzip2/libbzip2 version 1.0.5 of 10 December 2007
+   Copyright (C) 1996-2007 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/spewG.c b/spewG.c
index e1f3032..5892b92 100644
--- a/spewG.c
+++ b/spewG.c
@@ -13,8 +13,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.4 of 20 December 2006
-   Copyright (C) 1996-2006 Julian Seward 
+   bzip2/libbzip2 version 1.0.5 of 10 December 2007
+   Copyright (C) 1996-2007 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/unzcrash.c b/unzcrash.c
index 1f08a2e..a1b7546 100644
--- a/unzcrash.c
+++ b/unzcrash.c
@@ -17,8 +17,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.4 of 20 December 2006
-   Copyright (C) 1996-2006 Julian Seward 
+   bzip2/libbzip2 version 1.0.5 of 10 December 2007
+   Copyright (C) 1996-2007 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/xmlproc.sh b/xmlproc.sh
index 5a1f051..5384177 100755
--- a/xmlproc.sh
+++ b/xmlproc.sh
@@ -5,8 +5,8 @@
 #  This file is part of bzip2/libbzip2, a program and library for
 #  lossless, block-sorting data compression.
 #
-#  bzip2/libbzip2 version 1.0.4 of 20 December 2006
-#  Copyright (C) 1996-2006 Julian Seward 
+#  bzip2/libbzip2 version 1.0.5 of 10 December 2007
+#  Copyright (C) 1996-2007 Julian Seward 
 #
 #  Please read the WARNING, DISCLAIMER and PATENTS sections in the 
 #  README file.
-- 
cgit v1.2.3


From 962d60610cb31e0f294a834e55ebb355be55d05a Mon Sep 17 00:00:00 2001
From: Julian Seward 
Date: Mon, 6 Sep 2010 22:13:13 +0200
Subject: bzip2-1.0.6

---
 CHANGES                     | 12 ++++++++++--
 LICENSE                     |  4 ++--
 Makefile                    |  6 +++---
 Makefile-libbz2_so          | 14 +++++++-------
 README                      |  9 +++++++--
 README.COMPILATION.PROBLEMS |  6 +++---
 README.XML.STUFF            |  4 ++--
 blocksort.c                 |  4 ++--
 bzip2.1                     |  6 +++---
 bzip2.1.preformatted        |  6 +++---
 bzip2.c                     |  8 ++++----
 bzip2.txt                   |  6 +++---
 bzip2recover.c              |  6 +++---
 bzlib.c                     |  4 ++--
 bzlib.h                     |  4 ++--
 bzlib_private.h             |  6 +++---
 compress.c                  |  4 ++--
 crctable.c                  |  4 ++--
 decompress.c                | 24 ++++++++++++++++++++++--
 entities.xml                |  6 +++---
 format.pl                   |  4 ++--
 huffman.c                   |  4 ++--
 manual.xml                  | 40 ++++++++++++++++++++--------------------
 mk251.c                     |  4 ++--
 randtable.c                 |  4 ++--
 spewG.c                     |  4 ++--
 unzcrash.c                  |  4 ++--
 xmlproc.sh                  |  4 ++--
 28 files changed, 122 insertions(+), 89 deletions(-)

diff --git a/CHANGES b/CHANGES
index 6e4f65e..81e97ca 100644
--- a/CHANGES
+++ b/CHANGES
@@ -2,8 +2,8 @@
  This file is part of bzip2/libbzip2, a program and library for
  lossless, block-sorting data compression.
 
- bzip2/libbzip2 version 1.0.5 of 10 December 2007
- Copyright (C) 1996-2007 Julian Seward 
+ bzip2/libbzip2 version 1.0.6 of 6 September 2010
+ Copyright (C) 1996-2010 Julian Seward 
 
  Please read the WARNING, DISCLAIMER and PATENTS sections in the 
  README file.
@@ -317,3 +317,11 @@ Fixes some minor bugs since the last version, 1.0.3.
 ~~~~~~~~~~~~~~~~~
 Security fix only.  Fixes CERT-FI 20469 as it applies to bzip2.
 
+
+1.0.6 (6 Sept 10)
+~~~~~~~~~~~~~~~~~
+
+* Security fix for CVE-2010-0405.  This was reported by Mikolaj
+  Izdebski.
+
+* Make the documentation build on Ubuntu 10.04
diff --git a/LICENSE b/LICENSE
index f420cff..cc61417 100644
--- a/LICENSE
+++ b/LICENSE
@@ -2,7 +2,7 @@
 --------------------------------------------------------------------------
 
 This program, "bzip2", the associated library "libbzip2", and all
-documentation, are copyright (C) 1996-2007 Julian R Seward.  All
+documentation, are copyright (C) 1996-2010 Julian R Seward.  All
 rights reserved.
 
 Redistribution and use in source and binary forms, with or without
@@ -37,6 +37,6 @@ NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
 Julian Seward, jseward@bzip.org
-bzip2/libbzip2 version 1.0.5 of 10 December 2007
+bzip2/libbzip2 version 1.0.6 of 6 September 2010
 
 --------------------------------------------------------------------------
diff --git a/Makefile b/Makefile
index eb09753..9754ddf 100644
--- a/Makefile
+++ b/Makefile
@@ -2,8 +2,8 @@
 # This file is part of bzip2/libbzip2, a program and library for
 # lossless, block-sorting data compression.
 #
-# bzip2/libbzip2 version 1.0.5 of 10 December 2007
-# Copyright (C) 1996-2007 Julian Seward 
+# bzip2/libbzip2 version 1.0.6 of 6 September 2010
+# Copyright (C) 1996-2010 Julian Seward 
 #
 # Please read the WARNING, DISCLAIMER and PATENTS sections in the 
 # README file.
@@ -137,7 +137,7 @@ bzip2recover.o: bzip2recover.c
 distclean: clean
 	rm -f manual.ps manual.html manual.pdf
 
-DISTNAME=bzip2-1.0.5
+DISTNAME=bzip2-1.0.6
 dist: check manual
 	rm -f $(DISTNAME)
 	ln -s -f . $(DISTNAME)
diff --git a/Makefile-libbz2_so b/Makefile-libbz2_so
index 9a13c77..e58791b 100644
--- a/Makefile-libbz2_so
+++ b/Makefile-libbz2_so
@@ -1,6 +1,6 @@
 
 # This Makefile builds a shared version of the library, 
-# libbz2.so.1.0.4, with soname libbz2.so.1.0,
+# libbz2.so.1.0.6, with soname libbz2.so.1.0,
 # at least on x86-Linux (RedHat 7.2), 
 # with gcc-2.96 20000731 (Red Hat Linux 7.1 2.96-98).  
 # Please see the README file for some important info 
@@ -10,8 +10,8 @@
 # This file is part of bzip2/libbzip2, a program and library for
 # lossless, block-sorting data compression.
 #
-# bzip2/libbzip2 version 1.0.5 of 10 December 2007
-# Copyright (C) 1996-2007 Julian Seward 
+# bzip2/libbzip2 version 1.0.6 of 6 September 2010
+# Copyright (C) 1996-2010 Julian Seward 
 #
 # Please read the WARNING, DISCLAIMER and PATENTS sections in the 
 # README file.
@@ -35,13 +35,13 @@ OBJS= blocksort.o  \
       bzlib.o
 
 all: $(OBJS)
-	$(CC) -shared -Wl,-soname -Wl,libbz2.so.1.0 -o libbz2.so.1.0.4 $(OBJS)
-	$(CC) $(CFLAGS) -o bzip2-shared bzip2.c libbz2.so.1.0.4
+	$(CC) -shared -Wl,-soname -Wl,libbz2.so.1.0 -o libbz2.so.1.0.6 $(OBJS)
+	$(CC) $(CFLAGS) -o bzip2-shared bzip2.c libbz2.so.1.0.6
 	rm -f libbz2.so.1.0
-	ln -s libbz2.so.1.0.4 libbz2.so.1.0
+	ln -s libbz2.so.1.0.6 libbz2.so.1.0
 
 clean: 
-	rm -f $(OBJS) bzip2.o libbz2.so.1.0.4 libbz2.so.1.0 bzip2-shared
+	rm -f $(OBJS) bzip2.o libbz2.so.1.0.6 libbz2.so.1.0 bzip2-shared
 
 blocksort.o: blocksort.c
 	$(CC) $(CFLAGS) -c blocksort.c
diff --git a/README b/README
index e17a84e..9fb0f63 100644
--- a/README
+++ b/README
@@ -6,8 +6,8 @@ This version is fully compatible with the previous public releases.
 This file is part of bzip2/libbzip2, a program and library for
 lossless, block-sorting data compression.
 
-bzip2/libbzip2 version 1.0.5 of 10 December 2007
-Copyright (C) 1996-2007 Julian Seward 
+bzip2/libbzip2 version 1.0.6 of 6 September 2010
+Copyright (C) 1996-2010 Julian Seward 
 
 Please read the WARNING, DISCLAIMER and PATENTS sections in this file.
 
@@ -181,6 +181,10 @@ WHAT'S NEW IN 1.0.5 ?
 
    See the CHANGES file.
 
+WHAT'S NEW IN 1.0.6 ?
+
+   See the CHANGES file.
+
 
 I hope you find bzip2 useful.  Feel free to contact me at
    jseward@bzip.org
@@ -208,3 +212,4 @@ Cambridge, UK.
 15 February 2005 (bzip2, version 1.0.3)
 20 December 2006 (bzip2, version 1.0.4)
 10 December 2007 (bzip2, version 1.0.5)
+ 6     Sept 2010 (bzip2, version 1.0.6)
diff --git a/README.COMPILATION.PROBLEMS b/README.COMPILATION.PROBLEMS
index 22b95c6..667d0d6 100644
--- a/README.COMPILATION.PROBLEMS
+++ b/README.COMPILATION.PROBLEMS
@@ -2,8 +2,8 @@
 This file is part of bzip2/libbzip2, a program and library for
 lossless, block-sorting data compression.
 
-bzip2/libbzip2 version 1.0.5 of 10 December 2007
-Copyright (C) 1996-2007 Julian Seward 
+bzip2/libbzip2 version 1.0.6 of 6 September 2010
+Copyright (C) 1996-2010 Julian Seward 
 
 Please read the WARNING, DISCLAIMER and PATENTS sections in the 
 README file.
@@ -12,7 +12,7 @@ This program is released under the terms of the license contained
 in the file LICENSE.
 ------------------------------------------------------------------
 
-bzip2-1.0.5 should compile without problems on the vast majority of
+bzip2-1.0.6 should compile without problems on the vast majority of
 platforms.  Using the supplied Makefile, I've built and tested it
 myself for x86-linux and amd64-linux.  With makefile.msc, Visual C++
 6.0 and nmake, you can build a native Win32 version too.  Large file
diff --git a/README.XML.STUFF b/README.XML.STUFF
index 1a5b4c5..3a57f3f 100644
--- a/README.XML.STUFF
+++ b/README.XML.STUFF
@@ -2,8 +2,8 @@
   This file is part of bzip2/libbzip2, a program and library for
   lossless, block-sorting data compression.
 
-  bzip2/libbzip2 version 1.0.5 of 10 December 2007
-  Copyright (C) 1996-2007 Julian Seward 
+  bzip2/libbzip2 version 1.0.6 of 6 September 2010
+  Copyright (C) 1996-2010 Julian Seward 
 
   Please read the WARNING, DISCLAIMER and PATENTS sections in the 
   README file.
diff --git a/blocksort.c b/blocksort.c
index bd2dec1..d0d662c 100644
--- a/blocksort.c
+++ b/blocksort.c
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.5 of 10 December 2007
-   Copyright (C) 1996-2007 Julian Seward 
+   bzip2/libbzip2 version 1.0.6 of 6 September 2010
+   Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/bzip2.1 b/bzip2.1
index a313f2d..ce3a78e 100644
--- a/bzip2.1
+++ b/bzip2.1
@@ -1,7 +1,7 @@
 .PU
 .TH bzip2 1
 .SH NAME
-bzip2, bunzip2 \- a block-sorting file compressor, v1.0.4
+bzip2, bunzip2 \- a block-sorting file compressor, v1.0.6
 .br
 bzcat \- decompresses files to stdout
 .br
@@ -405,11 +405,11 @@ I/O error messages are not as helpful as they could be.
 tries hard to detect I/O errors and exit cleanly, but the details of
 what the problem is sometimes seem rather misleading.
 
-This manual page pertains to version 1.0.4 of
+This manual page pertains to version 1.0.6 of
 .I bzip2.  
 Compressed data created by this version is entirely forwards and
 backwards compatible with the previous public releases, versions
-0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1, 1.0.2 and 1.0.3, but with the following
+0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1, 1.0.2 and above, but with the following
 exception: 0.9.0 and above can correctly decompress multiple
 concatenated compressed files.  0.1pl2 cannot do this; it will stop
 after decompressing just the first file in the stream.
diff --git a/bzip2.1.preformatted b/bzip2.1.preformatted
index 15e16e5..63c33be 100644
--- a/bzip2.1.preformatted
+++ b/bzip2.1.preformatted
@@ -3,7 +3,7 @@ bzip2(1)                                                 bzip2(1)
 
 
 NNAAMMEE
-       bzip2, bunzip2 − a block‐sorting file compressor, v1.0.4
+       bzip2, bunzip2 − a block‐sorting file compressor, v1.0.6
        bzcat − decompresses files to stdout
        bzip2recover − recovers data from damaged bzip2 files
 
@@ -348,11 +348,11 @@ CCAAVVEEAATTSS
        but  the  details  of  what  the problem is sometimes seem
        rather misleading.
 
-       This manual page pertains to version 1.0.4 of _b_z_i_p_2_.  Com­
+       This manual page pertains to version 1.0.6 of _b_z_i_p_2_.  Com­
        pressed  data created by this version is entirely forwards
        and  backwards  compatible  with   the   previous   public
        releases,  versions  0.1pl2,  0.9.0,  0.9.5, 1.0.0, 1.0.1, 
-       1.0.2 and 1.0.3, but with the  following  exception: 0.9.0
+       1.0.2 and above, but with the  following  exception: 0.9.0
        and above can  correctly decompress  multiple concatenated
        compressed files.  0.1pl2  cannot do this;  it  will  stop 
        after  decompressing just the first file in the stream.
diff --git a/bzip2.c b/bzip2.c
index 3904107..6de9d1d 100644
--- a/bzip2.c
+++ b/bzip2.c
@@ -7,8 +7,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.5 of 10 December 2007
-   Copyright (C) 1996-2007 Julian Seward 
+   bzip2/libbzip2 version 1.0.6 of 6 September 2010
+   Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
@@ -1605,11 +1605,11 @@ void license ( void )
     "bzip2, a block-sorting file compressor.  "
     "Version %s.\n"
     "   \n"
-    "   Copyright (C) 1996-2007 by Julian Seward.\n"
+    "   Copyright (C) 1996-2010 by Julian Seward.\n"
     "   \n"
     "   This program is free software; you can redistribute it and/or modify\n"
     "   it under the terms set out in the LICENSE file, which is included\n"
-    "   in the bzip2-1.0.5 source distribution.\n"
+    "   in the bzip2-1.0.6 source distribution.\n"
     "   \n"
     "   This program is distributed in the hope that it will be useful,\n"
     "   but WITHOUT ANY WARRANTY; without even the implied warranty of\n"
diff --git a/bzip2.txt b/bzip2.txt
index 4fb9c74..d2deb39 100644
--- a/bzip2.txt
+++ b/bzip2.txt
@@ -1,6 +1,6 @@
 
 NAME
-       bzip2, bunzip2 - a block-sorting file compressor, v1.0.4
+       bzip2, bunzip2 - a block-sorting file compressor, v1.0.6
        bzcat - decompresses files to stdout
        bzip2recover - recovers data from damaged bzip2 files
 
@@ -345,11 +345,11 @@ CAVEATS
        but  the  details  of  what  the problem is sometimes seem
        rather misleading.
 
-       This manual page pertains to version 1.0.4 of bzip2.  Com-
+       This manual page pertains to version 1.0.6 of bzip2.  Com-
        pressed  data created by this version is entirely forwards
        and  backwards  compatible  with   the   previous   public
        releases,  versions  0.1pl2,  0.9.0,  0.9.5, 1.0.0, 1.0.1,
-       1.0.2 and 1.0.3, but with the  following  exception: 0.9.0
+       1.0.2 and above, but with the  following  exception: 0.9.0
        and above can  correctly decompress  multiple concatenated
        compressed files.  0.1pl2  cannot do this;  it  will  stop
        after  decompressing just the first file in the stream.
diff --git a/bzip2recover.c b/bzip2recover.c
index 5f6d621..f9de049 100644
--- a/bzip2recover.c
+++ b/bzip2recover.c
@@ -7,8 +7,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.5 of 10 December 2007
-   Copyright (C) 1996-2007 Julian Seward 
+   bzip2/libbzip2 version 1.0.6 of 6 September 2010
+   Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
@@ -313,7 +313,7 @@ Int32 main ( Int32 argc, Char** argv )
    inFileName[0] = outFileName[0] = 0;
 
    fprintf ( stderr, 
-             "bzip2recover 1.0.5: extracts blocks from damaged .bz2 files.\n" );
+             "bzip2recover 1.0.6: extracts blocks from damaged .bz2 files.\n" );
 
    if (argc != 2) {
       fprintf ( stderr, "%s: usage is `%s damaged_file_name'.\n",
diff --git a/bzlib.c b/bzlib.c
index ef86c91..bd358a7 100644
--- a/bzlib.c
+++ b/bzlib.c
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.5 of 10 December 2007
-   Copyright (C) 1996-2007 Julian Seward 
+   bzip2/libbzip2 version 1.0.6 of 6 September 2010
+   Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/bzlib.h b/bzlib.h
index c5b75d6..8277123 100644
--- a/bzlib.h
+++ b/bzlib.h
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.5 of 10 December 2007
-   Copyright (C) 1996-2007 Julian Seward 
+   bzip2/libbzip2 version 1.0.6 of 6 September 2010
+   Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/bzlib_private.h b/bzlib_private.h
index 2342787..5d0217f 100644
--- a/bzlib_private.h
+++ b/bzlib_private.h
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.5 of 10 December 2007
-   Copyright (C) 1996-2007 Julian Seward 
+   bzip2/libbzip2 version 1.0.6 of 6 September 2010
+   Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
@@ -36,7 +36,7 @@
 
 /*-- General stuff. --*/
 
-#define BZ_VERSION  "1.0.5, 10-Dec-2007"
+#define BZ_VERSION  "1.0.6, 6-Sept-2010"
 
 typedef char            Char;
 typedef unsigned char   Bool;
diff --git a/compress.c b/compress.c
index 8c80a07..caf7696 100644
--- a/compress.c
+++ b/compress.c
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.5 of 10 December 2007
-   Copyright (C) 1996-2007 Julian Seward 
+   bzip2/libbzip2 version 1.0.6 of 6 September 2010
+   Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/crctable.c b/crctable.c
index 215687b..1fea7e9 100644
--- a/crctable.c
+++ b/crctable.c
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.5 of 10 December 2007
-   Copyright (C) 1996-2007 Julian Seward 
+   bzip2/libbzip2 version 1.0.6 of 6 September 2010
+   Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/decompress.c b/decompress.c
index bba5e0f..311f566 100644
--- a/decompress.c
+++ b/decompress.c
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.5 of 10 December 2007
-   Copyright (C) 1996-2007 Julian Seward 
+   bzip2/libbzip2 version 1.0.6 of 6 September 2010
+   Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
@@ -381,6 +381,13 @@ Int32 BZ2_decompress ( DState* s )
             es = -1;
             N = 1;
             do {
+               /* Check that N doesn't get too big, so that es doesn't
+                  go negative.  The maximum value that can be
+                  RUNA/RUNB encoded is equal to the block size (post
+                  the initial RLE), viz, 900k, so bounding N at 2
+                  million should guard against overflow without
+                  rejecting any legitimate inputs. */
+               if (N >= 2*1024*1024) RETURN(BZ_DATA_ERROR);
                if (nextSym == BZ_RUNA) es = es + (0+1) * N; else
                if (nextSym == BZ_RUNB) es = es + (1+1) * N;
                N = N * 2;
@@ -485,15 +492,28 @@ Int32 BZ2_decompress ( DState* s )
          RETURN(BZ_DATA_ERROR);
 
       /*-- Set up cftab to facilitate generation of T^(-1) --*/
+      /* Check: unzftab entries in range. */
+      for (i = 0; i <= 255; i++) {
+         if (s->unzftab[i] < 0 || s->unzftab[i] > nblock)
+            RETURN(BZ_DATA_ERROR);
+      }
+      /* Actually generate cftab. */
       s->cftab[0] = 0;
       for (i = 1; i <= 256; i++) s->cftab[i] = s->unzftab[i-1];
       for (i = 1; i <= 256; i++) s->cftab[i] += s->cftab[i-1];
+      /* Check: cftab entries in range. */
       for (i = 0; i <= 256; i++) {
          if (s->cftab[i] < 0 || s->cftab[i] > nblock) {
             /* s->cftab[i] can legitimately be == nblock */
             RETURN(BZ_DATA_ERROR);
          }
       }
+      /* Check: cftab entries non-descending. */
+      for (i = 1; i <= 256; i++) {
+         if (s->cftab[i-1] > s->cftab[i]) {
+            RETURN(BZ_DATA_ERROR);
+         }
+      }
 
       s->state_out_len = 0;
       s->state_out_ch  = 0;
diff --git a/entities.xml b/entities.xml
index e9e0553..4b28f34 100644
--- a/entities.xml
+++ b/entities.xml
@@ -1,9 +1,9 @@
 
 
 
-
+
 
-
-
+
+
 
 
diff --git a/format.pl b/format.pl
index 1928d15..f169fd9 100755
--- a/format.pl
+++ b/format.pl
@@ -4,8 +4,8 @@
 # This file is part of bzip2/libbzip2, a program and library for
 # lossless, block-sorting data compression.
 #
-# bzip2/libbzip2 version 1.0.5 of 10 December 2007
-# Copyright (C) 1996-2007 Julian Seward 
+# bzip2/libbzip2 version 1.0.6 of 6 September 2010
+# Copyright (C) 1996-2010 Julian Seward 
 #
 # Please read the WARNING, DISCLAIMER and PATENTS sections in the 
 # README file.
diff --git a/huffman.c b/huffman.c
index 87e79e3..2283fdb 100644
--- a/huffman.c
+++ b/huffman.c
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.5 of 10 December 2007
-   Copyright (C) 1996-2007 Julian Seward 
+   bzip2/libbzip2 version 1.0.6 of 6 September 2010
+   Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/manual.xml b/manual.xml
index f224136..a7fbcb3 100644
--- a/manual.xml
+++ b/manual.xml
@@ -9,7 +9,7 @@
 
 
  
-  bzip2 and libbzip2, version 1.0.5
+  bzip2 and libbzip2, version 1.0.6
   A program and library for data compression
   
    &bz-lifespan;
@@ -139,7 +139,7 @@ else.
 
  bzip2,
   bunzip2 - a block-sorting file
-  compressor, v1.0.4
+  compressor, v1.0.6
 
  bzcat -
    decompresses files to stdout
@@ -1033,7 +1033,7 @@ kind.
 
 
 
-<computeroutput>BZ2_bzCompressInit</computeroutput>
+BZ2_bzCompressInit
 
 
 typedef struct {
@@ -1183,7 +1183,7 @@ BZ2_bzCompress
 
 
 
-<computeroutput>BZ2_bzCompress</computeroutput>
+BZ2_bzCompress
 
 
 int BZ2_bzCompress ( bz_stream *strm, int action );
@@ -1408,7 +1408,7 @@ BZ_PARAM_ERROR
 
 
 
-<computeroutput>BZ2_bzCompressEnd</computeroutput>
+BZ2_bzCompressEnd
 
 
 int BZ2_bzCompressEnd ( bz_stream *strm );
@@ -1428,7 +1428,7 @@ BZ_OK           otherwise
 
 
 
-<computeroutput>BZ2_bzDecompressInit</computeroutput>
+BZ2_bzDecompressInit
 
 
 int BZ2_bzDecompressInit ( bz_stream *strm, int verbosity, int small );
@@ -1492,7 +1492,7 @@ BZ2_bzDecompress
 
 
 
-<computeroutput>BZ2_bzDecompress</computeroutput>
+BZ2_bzDecompress
 
 
 int BZ2_bzDecompress ( bz_stream *strm );
@@ -1587,7 +1587,7 @@ BZ2_bzDecompressEnd
 
 
 
-<computeroutput>BZ2_bzDecompressEnd</computeroutput>
+BZ2_bzDecompressEnd
 
 
 int BZ2_bzDecompressEnd ( bz_stream *strm );
@@ -1682,7 +1682,7 @@ general points.
 
 
 
-<computeroutput>BZ2_bzReadOpen</computeroutput>
+BZ2_bzReadOpen
 
 
 typedef void BZFILE;
@@ -1765,7 +1765,7 @@ BZ2_bzClose
 
 
 
-<computeroutput>BZ2_bzRead</computeroutput>
+BZ2_bzRead
 
 
 int BZ2_bzRead ( int *bzerror, BZFILE *b, void *buf, int len );
@@ -1859,7 +1859,7 @@ BZ2_bzReadClose
 
 
 
-<computeroutput>BZ2_bzReadGetUnused</computeroutput>
+BZ2_bzReadGetUnused
 
 
 void BZ2_bzReadGetUnused( int* bzerror, BZFILE *b, 
@@ -1903,7 +1903,7 @@ BZ2_bzReadClose
 
 
 
-<computeroutput>BZ2_bzReadClose</computeroutput>
+BZ2_bzReadClose
 
 
 void BZ2_bzReadClose ( int *bzerror, BZFILE *b );
@@ -1937,7 +1937,7 @@ none
 
 
 
-<computeroutput>BZ2_bzWriteOpen</computeroutput>
+BZ2_bzWriteOpen
 
 
 BZFILE *BZ2_bzWriteOpen( int *bzerror, FILE *f, 
@@ -2003,7 +2003,7 @@ BZ2_bzWriteClose
 
 
 
-<computeroutput>BZ2_bzWrite</computeroutput>
+BZ2_bzWrite
 
 
 void BZ2_bzWrite ( int *bzerror, BZFILE *b, void *buf, int len );
@@ -2031,7 +2031,7 @@ BZ_OK
 
 
 
-<computeroutput>BZ2_bzWriteClose</computeroutput>
+BZ2_bzWriteClose
 
 
 void BZ2_bzWriteClose( int *bzerror, BZFILE* f,
@@ -2235,7 +2235,7 @@ if ( bzerror != BZ_STREAM_END ) {
 
 
 
-<computeroutput>BZ2_bzBuffToBuffCompress</computeroutput>
+BZ2_bzBuffToBuffCompress
 
 
 int BZ2_bzBuffToBuffCompress( char*         dest,
@@ -2302,7 +2302,7 @@ BZ_OK
 
 
 
-<computeroutput>BZ2_bzBuffToBuffDecompress</computeroutput>
+BZ2_bzBuffToBuffDecompress
 
 
 int BZ2_bzBuffToBuffDecompress( char*         dest,
@@ -2377,7 +2377,7 @@ BZ_OK
 
 
 
-<computeroutput>zlib</computeroutput> compatibility functions
+zlib compatibility functions
 
 Yoshioka Tsuneo has contributed some functions to give
 better zlib compatibility.
@@ -2444,11 +2444,11 @@ value.
 
 
-Using the library in a <computeroutput>stdio</computeroutput>-free environment
+Using the library in a stdio-free environment
 
 
 
-Getting rid of <computeroutput>stdio</computeroutput>
+Getting rid of stdio
 
 In a deeply embedded application, you might want to use
 just the memory-to-memory functions.  You can do this
diff --git a/mk251.c b/mk251.c
index 39e94c0..c9c36f6 100644
--- a/mk251.c
+++ b/mk251.c
@@ -9,8 +9,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.5 of 10 December 2007
-   Copyright (C) 1996-2007 Julian Seward 
+   bzip2/libbzip2 version 1.0.6 of 6 September 2010
+   Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/randtable.c b/randtable.c
index 068b763..6d62459 100644
--- a/randtable.c
+++ b/randtable.c
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.5 of 10 December 2007
-   Copyright (C) 1996-2007 Julian Seward 
+   bzip2/libbzip2 version 1.0.6 of 6 September 2010
+   Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/spewG.c b/spewG.c
index 5892b92..14a3649 100644
--- a/spewG.c
+++ b/spewG.c
@@ -13,8 +13,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.5 of 10 December 2007
-   Copyright (C) 1996-2007 Julian Seward 
+   bzip2/libbzip2 version 1.0.6 of 6 September 2010
+   Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/unzcrash.c b/unzcrash.c
index a1b7546..7041da5 100644
--- a/unzcrash.c
+++ b/unzcrash.c
@@ -17,8 +17,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.5 of 10 December 2007
-   Copyright (C) 1996-2007 Julian Seward 
+   bzip2/libbzip2 version 1.0.6 of 6 September 2010
+   Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/xmlproc.sh b/xmlproc.sh
index 5384177..ca284ea 100755
--- a/xmlproc.sh
+++ b/xmlproc.sh
@@ -5,8 +5,8 @@
 #  This file is part of bzip2/libbzip2, a program and library for
 #  lossless, block-sorting data compression.
 #
-#  bzip2/libbzip2 version 1.0.5 of 10 December 2007
-#  Copyright (C) 1996-2007 Julian Seward 
+#  bzip2/libbzip2 version 1.0.6 of 6 September 2010
+#  Copyright (C) 1996-2010 Julian Seward 
 #
 #  Please read the WARNING, DISCLAIMER and PATENTS sections in the 
 #  README file.
-- 
cgit v1.2.3


From 5b923861db7bdd0560f3ecbc8be6e91e830066eb Mon Sep 17 00:00:00 2001
From: Mark Wielaard 
Date: Sat, 30 Mar 2019 22:56:16 +0100
Subject: Change Julian's email address to jseward@acm.org

---
 CHANGES                     | 2 +-
 LICENSE                     | 2 +-
 Makefile                    | 2 +-
 Makefile-libbz2_so          | 2 +-
 README                      | 6 +++---
 README.COMPILATION.PROBLEMS | 2 +-
 README.XML.STUFF            | 2 +-
 blocksort.c                 | 2 +-
 bzip2.1                     | 2 +-
 bzip2.1.preformatted        | 2 +-
 bzip2.c                     | 8 ++++----
 bzip2.txt                   | 2 +-
 bzip2recover.c              | 2 +-
 bzlib.c                     | 4 ++--
 bzlib.h                     | 2 +-
 bzlib_private.h             | 2 +-
 compress.c                  | 2 +-
 crctable.c                  | 2 +-
 decompress.c                | 2 +-
 entities.xml                | 2 +-
 format.pl                   | 2 +-
 huffman.c                   | 2 +-
 mk251.c                     | 2 +-
 randtable.c                 | 2 +-
 spewG.c                     | 2 +-
 unzcrash.c                  | 2 +-
 words2                      | 2 +-
 xmlproc.sh                  | 2 +-
 28 files changed, 34 insertions(+), 34 deletions(-)

diff --git a/CHANGES b/CHANGES
index 81e97ca..94a9b66 100644
--- a/CHANGES
+++ b/CHANGES
@@ -3,7 +3,7 @@
  lossless, block-sorting data compression.
 
  bzip2/libbzip2 version 1.0.6 of 6 September 2010
- Copyright (C) 1996-2010 Julian Seward 
+ Copyright (C) 1996-2010 Julian Seward 
 
  Please read the WARNING, DISCLAIMER and PATENTS sections in the 
  README file.
diff --git a/LICENSE b/LICENSE
index cc61417..0a10ee6 100644
--- a/LICENSE
+++ b/LICENSE
@@ -36,7 +36,7 @@ WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
 NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
-Julian Seward, jseward@bzip.org
+Julian Seward, jseward@acm.org
 bzip2/libbzip2 version 1.0.6 of 6 September 2010
 
 --------------------------------------------------------------------------
diff --git a/Makefile b/Makefile
index 9754ddf..c61f867 100644
--- a/Makefile
+++ b/Makefile
@@ -3,7 +3,7 @@
 # lossless, block-sorting data compression.
 #
 # bzip2/libbzip2 version 1.0.6 of 6 September 2010
-# Copyright (C) 1996-2010 Julian Seward 
+# Copyright (C) 1996-2010 Julian Seward 
 #
 # Please read the WARNING, DISCLAIMER and PATENTS sections in the 
 # README file.
diff --git a/Makefile-libbz2_so b/Makefile-libbz2_so
index e58791b..cff37c2 100644
--- a/Makefile-libbz2_so
+++ b/Makefile-libbz2_so
@@ -11,7 +11,7 @@
 # lossless, block-sorting data compression.
 #
 # bzip2/libbzip2 version 1.0.6 of 6 September 2010
-# Copyright (C) 1996-2010 Julian Seward 
+# Copyright (C) 1996-2010 Julian Seward 
 #
 # Please read the WARNING, DISCLAIMER and PATENTS sections in the 
 # README file.
diff --git a/README b/README
index 9fb0f63..771205a 100644
--- a/README
+++ b/README
@@ -7,7 +7,7 @@ This file is part of bzip2/libbzip2, a program and library for
 lossless, block-sorting data compression.
 
 bzip2/libbzip2 version 1.0.6 of 6 September 2010
-Copyright (C) 1996-2010 Julian Seward 
+Copyright (C) 1996-2010 Julian Seward 
 
 Please read the WARNING, DISCLAIMER and PATENTS sections in this file.
 
@@ -187,7 +187,7 @@ WHAT'S NEW IN 1.0.6 ?
 
 
 I hope you find bzip2 useful.  Feel free to contact me at
-   jseward@bzip.org
+   jseward@acm.org
 if you have any suggestions or queries.  Many people mailed me with
 comments, suggestions and patches after the releases of bzip-0.15,
 bzip-0.21, and bzip2 versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1,
@@ -197,7 +197,7 @@ feedback.  I thank you for your comments.
 bzip2's "home" is http://www.bzip.org/
 
 Julian Seward
-jseward@bzip.org
+jseward@acm.org
 Cambridge, UK.
 
 18     July 1996 (version 0.15)
diff --git a/README.COMPILATION.PROBLEMS b/README.COMPILATION.PROBLEMS
index 667d0d6..14ddd1b 100644
--- a/README.COMPILATION.PROBLEMS
+++ b/README.COMPILATION.PROBLEMS
@@ -3,7 +3,7 @@ This file is part of bzip2/libbzip2, a program and library for
 lossless, block-sorting data compression.
 
 bzip2/libbzip2 version 1.0.6 of 6 September 2010
-Copyright (C) 1996-2010 Julian Seward 
+Copyright (C) 1996-2010 Julian Seward 
 
 Please read the WARNING, DISCLAIMER and PATENTS sections in the 
 README file.
diff --git a/README.XML.STUFF b/README.XML.STUFF
index 3a57f3f..f5dc3a4 100644
--- a/README.XML.STUFF
+++ b/README.XML.STUFF
@@ -3,7 +3,7 @@
   lossless, block-sorting data compression.
 
   bzip2/libbzip2 version 1.0.6 of 6 September 2010
-  Copyright (C) 1996-2010 Julian Seward 
+  Copyright (C) 1996-2010 Julian Seward 
 
   Please read the WARNING, DISCLAIMER and PATENTS sections in the 
   README file.
diff --git a/blocksort.c b/blocksort.c
index d0d662c..830d1b4 100644
--- a/blocksort.c
+++ b/blocksort.c
@@ -9,7 +9,7 @@
    lossless, block-sorting data compression.
 
    bzip2/libbzip2 version 1.0.6 of 6 September 2010
-   Copyright (C) 1996-2010 Julian Seward 
+   Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/bzip2.1 b/bzip2.1
index ce3a78e..d71c9b2 100644
--- a/bzip2.1
+++ b/bzip2.1
@@ -427,7 +427,7 @@ with MaybeUInt64 set to be an unsigned 64-bit integer.
 
 
 .SH AUTHOR
-Julian Seward, jsewardbzip.org.
+Julian Seward, jseward@acm.org.
 
 http://www.bzip.org
 
diff --git a/bzip2.1.preformatted b/bzip2.1.preformatted
index 63c33be..da67b66 100644
--- a/bzip2.1.preformatted
+++ b/bzip2.1.preformatted
@@ -372,7 +372,7 @@ CCAAVVEEAATTSS
 
 
 AAUUTTHHOORR
-       Julian Seward, jsewardbzip.org.
+       Julian Seward, jseward@acm.org.
 
        http://www.bzip.org
 
diff --git a/bzip2.c b/bzip2.c
index 6de9d1d..59f59ff 100644
--- a/bzip2.c
+++ b/bzip2.c
@@ -8,7 +8,7 @@
    lossless, block-sorting data compression.
 
    bzip2/libbzip2 version 1.0.6 of 6 September 2010
-   Copyright (C) 1996-2010 Julian Seward 
+   Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
@@ -749,7 +749,7 @@ void panic ( const Char* s )
              "\n%s: PANIC -- internal consistency error:\n"
              "\t%s\n"
              "\tThis is a BUG.  Please report it to me at:\n"
-             "\tjseward@bzip.org\n",
+             "\tjseward@acm.org\n",
              progName, s );
    showFileNames();
    cleanUpAndFail( 3 );
@@ -829,7 +829,7 @@ void mySIGSEGVorSIGBUScatcher ( IntNative n )
       "   The user's manual, Section 4.3, has more info on (1) and (2).\n"
       "   \n"
       "   If you suspect this is a bug in bzip2, or are unsure about (1)\n"
-      "   or (2), feel free to report it to me at: jseward@bzip.org.\n"
+      "   or (2), feel free to report it to me at: jseward@acm.org.\n"
       "   Section 4.3 of the user's manual describes the info a useful\n"
       "   bug report should have.  If the manual is available on your\n"
       "   system, please try and read it before mailing me.  If you don't\n"
@@ -852,7 +852,7 @@ void mySIGSEGVorSIGBUScatcher ( IntNative n )
       "   The user's manual, Section 4.3, has more info on (2) and (3).\n"
       "   \n"
       "   If you suspect this is a bug in bzip2, or are unsure about (2)\n"
-      "   or (3), feel free to report it to me at: jseward@bzip.org.\n"
+      "   or (3), feel free to report it to me at: jseward@acm.org.\n"
       "   Section 4.3 of the user's manual describes the info a useful\n"
       "   bug report should have.  If the manual is available on your\n"
       "   system, please try and read it before mailing me.  If you don't\n"
diff --git a/bzip2.txt b/bzip2.txt
index d2deb39..3122c31 100644
--- a/bzip2.txt
+++ b/bzip2.txt
@@ -367,7 +367,7 @@ CAVEATS
 
 
 AUTHOR
-       Julian Seward, jsewardbzip.org.
+       Julian Seward, jseward@acm.org
 
        http://www.bzip.org
 
diff --git a/bzip2recover.c b/bzip2recover.c
index f9de049..06ac1f5 100644
--- a/bzip2recover.c
+++ b/bzip2recover.c
@@ -8,7 +8,7 @@
    lossless, block-sorting data compression.
 
    bzip2/libbzip2 version 1.0.6 of 6 September 2010
-   Copyright (C) 1996-2010 Julian Seward 
+   Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/bzlib.c b/bzlib.c
index bd358a7..8aa6ae7 100644
--- a/bzlib.c
+++ b/bzlib.c
@@ -9,7 +9,7 @@
    lossless, block-sorting data compression.
 
    bzip2/libbzip2 version 1.0.6 of 6 September 2010
-   Copyright (C) 1996-2010 Julian Seward 
+   Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
@@ -43,7 +43,7 @@ void BZ2_bz__AssertH__fail ( int errcode )
    fprintf(stderr, 
       "\n\nbzip2/libbzip2: internal error number %d.\n"
       "This is a bug in bzip2/libbzip2, %s.\n"
-      "Please report it to me at: jseward@bzip.org.  If this happened\n"
+      "Please report it to me at: jseward@acm.org.  If this happened\n"
       "when you were using some program which uses libbzip2 as a\n"
       "component, you should also report this bug to the author(s)\n"
       "of that program.  Please make an effort to report this bug;\n"
diff --git a/bzlib.h b/bzlib.h
index 8277123..c2c4b72 100644
--- a/bzlib.h
+++ b/bzlib.h
@@ -9,7 +9,7 @@
    lossless, block-sorting data compression.
 
    bzip2/libbzip2 version 1.0.6 of 6 September 2010
-   Copyright (C) 1996-2010 Julian Seward 
+   Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/bzlib_private.h b/bzlib_private.h
index 5d0217f..06ee0b4 100644
--- a/bzlib_private.h
+++ b/bzlib_private.h
@@ -9,7 +9,7 @@
    lossless, block-sorting data compression.
 
    bzip2/libbzip2 version 1.0.6 of 6 September 2010
-   Copyright (C) 1996-2010 Julian Seward 
+   Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/compress.c b/compress.c
index caf7696..5d4af91 100644
--- a/compress.c
+++ b/compress.c
@@ -9,7 +9,7 @@
    lossless, block-sorting data compression.
 
    bzip2/libbzip2 version 1.0.6 of 6 September 2010
-   Copyright (C) 1996-2010 Julian Seward 
+   Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/crctable.c b/crctable.c
index 1fea7e9..8daae06 100644
--- a/crctable.c
+++ b/crctable.c
@@ -9,7 +9,7 @@
    lossless, block-sorting data compression.
 
    bzip2/libbzip2 version 1.0.6 of 6 September 2010
-   Copyright (C) 1996-2010 Julian Seward 
+   Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/decompress.c b/decompress.c
index 311f566..ab6a624 100644
--- a/decompress.c
+++ b/decompress.c
@@ -9,7 +9,7 @@
    lossless, block-sorting data compression.
 
    bzip2/libbzip2 version 1.0.6 of 6 September 2010
-   Copyright (C) 1996-2010 Julian Seward 
+   Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/entities.xml b/entities.xml
index 4b28f34..808a96a 100644
--- a/entities.xml
+++ b/entities.xml
@@ -1,6 +1,6 @@
 
 
-
+
 
 
 
diff --git a/format.pl b/format.pl
index f169fd9..0a56cba 100755
--- a/format.pl
+++ b/format.pl
@@ -5,7 +5,7 @@
 # lossless, block-sorting data compression.
 #
 # bzip2/libbzip2 version 1.0.6 of 6 September 2010
-# Copyright (C) 1996-2010 Julian Seward 
+# Copyright (C) 1996-2010 Julian Seward 
 #
 # Please read the WARNING, DISCLAIMER and PATENTS sections in the 
 # README file.
diff --git a/huffman.c b/huffman.c
index 2283fdb..fc396be 100644
--- a/huffman.c
+++ b/huffman.c
@@ -9,7 +9,7 @@
    lossless, block-sorting data compression.
 
    bzip2/libbzip2 version 1.0.6 of 6 September 2010
-   Copyright (C) 1996-2010 Julian Seward 
+   Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/mk251.c b/mk251.c
index c9c36f6..aefd855 100644
--- a/mk251.c
+++ b/mk251.c
@@ -10,7 +10,7 @@
    lossless, block-sorting data compression.
 
    bzip2/libbzip2 version 1.0.6 of 6 September 2010
-   Copyright (C) 1996-2010 Julian Seward 
+   Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/randtable.c b/randtable.c
index 6d62459..35b29e1 100644
--- a/randtable.c
+++ b/randtable.c
@@ -9,7 +9,7 @@
    lossless, block-sorting data compression.
 
    bzip2/libbzip2 version 1.0.6 of 6 September 2010
-   Copyright (C) 1996-2010 Julian Seward 
+   Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/spewG.c b/spewG.c
index 14a3649..08fb3fb 100644
--- a/spewG.c
+++ b/spewG.c
@@ -14,7 +14,7 @@
    lossless, block-sorting data compression.
 
    bzip2/libbzip2 version 1.0.6 of 6 September 2010
-   Copyright (C) 1996-2010 Julian Seward 
+   Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/unzcrash.c b/unzcrash.c
index 7041da5..acdb94d 100644
--- a/unzcrash.c
+++ b/unzcrash.c
@@ -18,7 +18,7 @@
    lossless, block-sorting data compression.
 
    bzip2/libbzip2 version 1.0.6 of 6 September 2010
-   Copyright (C) 1996-2010 Julian Seward 
+   Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/words2 b/words2
index caddcf4..203ee39 100644
--- a/words2
+++ b/words2
@@ -1,5 +1,5 @@
 
 Checking test results.  If any of the four "cmp"s which follow
 report any differences, something is wrong.  If you can't easily
-figure out what, please let me know (jseward@bzip.org).
+figure out what, please let me know (jseward@acm.org).
 
diff --git a/xmlproc.sh b/xmlproc.sh
index ca284ea..c9dc790 100755
--- a/xmlproc.sh
+++ b/xmlproc.sh
@@ -6,7 +6,7 @@
 #  lossless, block-sorting data compression.
 #
 #  bzip2/libbzip2 version 1.0.6 of 6 September 2010
-#  Copyright (C) 1996-2010 Julian Seward 
+#  Copyright (C) 1996-2010 Julian Seward 
 #
 #  Please read the WARNING, DISCLAIMER and PATENTS sections in the 
 #  README file.
-- 
cgit v1.2.3


From 032a25eee83fd7556cd9e7b1829d56dcc627ddcf Mon Sep 17 00:00:00 2001
From: Mark Wielaard 
Date: Sat, 30 Mar 2019 23:09:35 +0100
Subject: Change all bzip.org URLs to sourceware.org/bzip2

---
 README               | 4 ++--
 bzip2.1              | 2 +-
 bzip2.1.preformatted | 2 +-
 bzip2.txt            | 2 +-
 entities.xml         | 2 +-
 5 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/README b/README
index 771205a..c3bcbeb 100644
--- a/README
+++ b/README
@@ -73,7 +73,7 @@ HOW TO BUILD -- Windows 95, NT, DOS, Mac, etc.
 
 It's difficult for me to support compilation on all these platforms.
 My approach is to collect binaries for these platforms, and put them
-on the master web site (http://www.bzip.org).  Look there.  However
+on the master web site (https://sourceware.org/bzip2/).  Look there.  However
 (FWIW), bzip2-1.0.X is very standard ANSI C and should compile
 unmodified with MS Visual C.  If you have difficulties building, you
 might want to read README.COMPILATION.PROBLEMS.
@@ -194,7 +194,7 @@ bzip-0.21, and bzip2 versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1,
 1.0.2 and 1.0.3, and the changes in bzip2 are largely a result of this
 feedback.  I thank you for your comments.
 
-bzip2's "home" is http://www.bzip.org/
+bzip2's "home" is https://sourceware.org/bzip2/
 
 Julian Seward
 jseward@acm.org
diff --git a/bzip2.1 b/bzip2.1
index d71c9b2..e386fa3 100644
--- a/bzip2.1
+++ b/bzip2.1
@@ -429,7 +429,7 @@ with MaybeUInt64 set to be an unsigned 64-bit integer.
 .SH AUTHOR
 Julian Seward, jseward@acm.org.
 
-http://www.bzip.org
+https://sourceware.org/bzip2/
 
 The ideas embodied in
 .I bzip2
diff --git a/bzip2.1.preformatted b/bzip2.1.preformatted
index da67b66..5f8e38e 100644
--- a/bzip2.1.preformatted
+++ b/bzip2.1.preformatted
@@ -374,7 +374,7 @@ CCAAVVEEAATTSS
 AAUUTTHHOORR
        Julian Seward, jseward@acm.org.
 
-       http://www.bzip.org
+       https://sourceware.org/bzip2/
 
        The ideas embodied in _b_z_i_p_2 are due to (at least) the fol­
        lowing  people: Michael Burrows and David Wheeler (for the
diff --git a/bzip2.txt b/bzip2.txt
index 3122c31..0b9f74a 100644
--- a/bzip2.txt
+++ b/bzip2.txt
@@ -369,7 +369,7 @@ CAVEATS
 AUTHOR
        Julian Seward, jseward@acm.org
 
-       http://www.bzip.org
+       https://sourceware.org/bzip2/
 
        The ideas embodied in bzip2 are due to (at least) the fol-
        lowing  people: Michael Burrows and David Wheeler (for the
diff --git a/entities.xml b/entities.xml
index 808a96a..0ff303e 100644
--- a/entities.xml
+++ b/entities.xml
@@ -1,5 +1,5 @@
 
-
+
 
 
 
-- 
cgit v1.2.3


From ddd8bae9d51c1b5d04c26be666625d559505fee4 Mon Sep 17 00:00:00 2001
From: Mark Wielaard 
Date: Sun, 23 Jun 2019 00:47:20 +0200
Subject: Adjust bzip.css images to new https://sourceware.org/bzip2/ location.

---
 bzip.css | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/bzip.css b/bzip.css
index 43193d8..2113118 100644
--- a/bzip.css
+++ b/bzip.css
@@ -25,7 +25,7 @@ dd {
 /* -- ruler -- */
 div.hr_blue { 
   height:  3px; 
-  background:#ffffff url("/images/hr_blue.png") repeat-x; }
+  background:#ffffff url("../images/hr_blue.png") repeat-x; }
 div.hr_blue hr { display:none; }
 
 /* release styles */
@@ -38,7 +38,7 @@ div.hr_blue hr { display:none; }
 ul  { 
  margin:     0px 4px 16px 16px;
  padding:    0px;
- list-style: url("/images/li-blue.png"); 
+ list-style: url("../images/li-blue.png"); 
 }
 ul li { 
  margin-bottom: 10px;
-- 
cgit v1.2.3


From ac9b3847405574dfd6022c1a9347a4449e8c1510 Mon Sep 17 00:00:00 2001
From: Mark Wielaard 
Date: Sun, 23 Jun 2019 02:14:23 +0200
Subject: Use UTF-8 encoding and include bzip.css as link for HTML manual.

---
 bz-html.xsl | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/bz-html.xsl b/bz-html.xsl
index 1785fff..b6000d5 100644
--- a/bz-html.xsl
+++ b/bz-html.xsl
@@ -7,11 +7,14 @@
 
 
 
-
-
+
+
 
-
-  
+
+
+  
+link rel="stylesheet" type="text/css" href="bzip.css" />
+  
   
-- 
cgit v1.2.3


From 32db5b677a55fc32de6588deae2ac27c28728a05 Mon Sep 17 00:00:00 2001
From: Mark Wielaard 
Date: Sun, 23 Jun 2019 22:18:58 +0200
Subject: Add release-update.sh script.

Script to run after a release has been tagged, signed and pushed
to git.  Will do a fresh checkout, verify the git tag, do fresh
build/dist, sign the dist with gpg, create a backup copy in HOME,
upload the tar.gz and sig to sourceware, checkout bzip2-htdocs,
copy over the new changes, manual, etc. and git push that to update
https://sourceware.org/bzip2/
---
 release-update.sh | 85 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 85 insertions(+)
 create mode 100755 release-update.sh

diff --git a/release-update.sh b/release-update.sh
new file mode 100755
index 0000000..2ad32ce
--- /dev/null
+++ b/release-update.sh
@@ -0,0 +1,85 @@
+#!/bin/bash
+
+# Script to run after a release has been tagged, signed and pushed
+# to git.  Will do a fresh checkout, verify the git tag, do fresh
+# build/dist, sign the dist with gpg, create a backup copy in HOME,
+# upload the tar.gz and sig to sourceware, checkout bzip2-htdocs,
+# copy over the new changes, manual, etc. and git push that to update
+# https://sourceware.org/bzip2/
+
+# Any error is fatal
+set -e
+
+# We take one argument, the version (e.g. 1.0.7)
+if [ $# -ne 1 ]; then
+  echo "$0  (e.g. 1.0.7)"
+  exit 1
+fi
+
+VERSION="$1"
+echo
+echo "  === NOTE ===  "
+echo
+echo "Requires a sourceware account in the bzip2 group."
+echo
+echo "Make sure the git repo was tagged, signed and pushed"
+echo "If not, please double check the source tree is release ready first"
+echo "Then do:"
+echo
+echo "    git tag -s -m \"bzip2 $VERSION release\" bzip2-$VERSION"
+echo "    git push --tags"
+echo
+read -p "Do you want to continue creating/uploading the release (yes/no)? "
+
+if [ "x$REPLY" != "xyes" ]; then
+  echo "OK, till next time."
+  exit
+fi
+
+echo "OK, creating and updating the release."
+
+# Create a temporary directoy and make sure it is cleaned up.
+tempdir=$(mktemp -d) || exit
+trap "rm -rf -- ${tempdir}" EXIT
+
+pushd "${tempdir}"
+
+# Checkout
+git clone git://sourceware.org/git/bzip2.git
+cd bzip2
+git tag --verify "bzip2-${VERSION}"
+git checkout -b "$VERSION" "bzip2-${VERSION}"
+
+# Create dist (creates bzip2-${VERSION}.tar.gz)
+make dist
+
+# Sign (creates bzip2-${VERSION}.tar.gz.sig)
+gpg -b bzip2-${VERSION}.tar.gz
+
+# Create backup copy
+echo "Putting a backup copy in $HOME/bzip2-$VERSION"
+mkdir $HOME/bzip2-$VERSION
+cp bzip2-${VERSION}.tar.gz bzip2-${VERSION}.tar.gz.sig $HOME/bzip2-$VERSION/
+
+# Upload
+scp bzip2-${VERSION}.tar.gz bzip2-${VERSION}.tar.gz.sig \
+    sourceware.org:/sourceware/ftp/pub/bzip2/
+ssh sourceware.org "(cd /sourceware/ftp/pub/bzip2 \
+  && ln -sf bzip2-$VERSION.tar.gz bzip2-latest.tar.gz \
+  && ln -sf bzip2-$VERSION.tar.gz.sig bzip2-latest.tar.gz.sig \
+  && ls -lah bzip2-latest*)"
+
+# Update homepage, manual, etc.
+cd "${tempdir}"
+git clone ssh://sourceware.org/git/bzip2-htdocs.git
+cp bzip2/CHANGES bzip2/bzip.css bzip2-htdocs/
+cp bzip2/bzip.css bzip2/bzip2.txt bzip2/manual.{html,pdf} bzip2-htdocs/manual/
+cd bzip2-htdocs
+git commit -a -m "Update for bzip2 $VERSION release"
+git show
+git push
+
+# Cleanup
+popd
+trap - EXIT
+exit
-- 
cgit v1.2.3


From 02fe3ca2349e45eee6dff6ca46bf9a9187f382c5 Mon Sep 17 00:00:00 2001
From: Mark Wielaard 
Date: Sun, 23 Jun 2019 23:52:03 +0200
Subject: bzip2.c (testStream): Remove set, but not used nread variable.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Modern GCC warns:

bzip2.c: In function ‘testStream’:
bzip2.c:557:37: warning: variable ‘nread’ set but not used
[-Wunused-but-set-variable]
    Int32   bzerr, bzerr_dummy, ret, nread, streamNo, i;
                                     ^~~~~

GCC is correct. In testStream we don't care about the number of bytes
read by BZ2_bzRead. So just remove the variable and the assignment.
---
 bzip2.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/bzip2.c b/bzip2.c
index 59f59ff..854a2bb 100644
--- a/bzip2.c
+++ b/bzip2.c
@@ -554,7 +554,7 @@ static
 Bool testStream ( FILE *zStream )
 {
    BZFILE* bzf = NULL;
-   Int32   bzerr, bzerr_dummy, ret, nread, streamNo, i;
+   Int32   bzerr, bzerr_dummy, ret, streamNo, i;
    UChar   obuf[5000];
    UChar   unused[BZ_MAX_UNUSED];
    Int32   nUnused;
@@ -577,7 +577,7 @@ Bool testStream ( FILE *zStream )
       streamNo++;
 
       while (bzerr == BZ_OK) {
-         nread = BZ2_bzRead ( &bzerr, bzf, obuf, 5000 );
+         BZ2_bzRead ( &bzerr, bzf, obuf, 5000 );
          if (bzerr == BZ_DATA_ERROR_MAGIC) goto errhandler;
       }
       if (bzerr != BZ_STREAM_END) goto errhandler;
-- 
cgit v1.2.3


From 833548edc0eb4af85ce8da193835f0f31a6c300f Mon Sep 17 00:00:00 2001
From: Mark Wielaard 
Date: Mon, 24 Jun 2019 00:14:02 +0200
Subject: bzip2recover: Fix buffer overflow for large argv[0].

bzip2recover.c (main) copies argv[0] to a statically sized buffer
without checking whether argv[0] might be too big (> 2000 chars).

This patch comes from Fedora and was originally reported at
https://bugzilla.redhat.com/show_bug.cgi?id=226979
---
 bzip2recover.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/bzip2recover.c b/bzip2recover.c
index 06ac1f5..1a70e04 100644
--- a/bzip2recover.c
+++ b/bzip2recover.c
@@ -309,7 +309,8 @@ Int32 main ( Int32 argc, Char** argv )
    UInt32      buffHi, buffLo, blockCRC;
    Char*       p;
 
-   strcpy ( progName, argv[0] );
+   strncpy ( progName, argv[0], BZ_MAX_FILENAME-1);
+   progName[BZ_MAX_FILENAME-1]='\0';
    inFileName[0] = outFileName[0] = 0;
 
    fprintf ( stderr, 
-- 
cgit v1.2.3


From c1cdd98db3238cb711c7d9cdc5671452ce2822cb Mon Sep 17 00:00:00 2001
From: Mark Wielaard 
Date: Mon, 24 Jun 2019 00:45:32 +0200
Subject: bzip2recover: Fix use after free issue with outFile.

bzip2recover.c (main): Make sure to set outFile to NULL when done.

This was reported as CVE-2016-3189 and found in multiple distributions.
https://seclists.org/oss-sec/2016/q2/568

Some more analysis can be found in:
https://bugzilla.redhat.com/show_bug.cgi?id=1319648
---
 bzip2recover.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/bzip2recover.c b/bzip2recover.c
index 1a70e04..a955d60 100644
--- a/bzip2recover.c
+++ b/bzip2recover.c
@@ -458,6 +458,7 @@ Int32 main ( Int32 argc, Char** argv )
             bsPutUChar ( bsWr, 0x50 ); bsPutUChar ( bsWr, 0x90 );
             bsPutUInt32 ( bsWr, blockCRC );
             bsClose ( bsWr );
+            outFile = NULL;
          }
          if (wrBlock >= rbCtr) break;
          wrBlock++;
-- 
cgit v1.2.3


From f51f164df0e8a2c0e055174b328b4038a040e547 Mon Sep 17 00:00:00 2001
From: Mark Wielaard 
Date: Mon, 24 Jun 2019 09:31:16 +0200
Subject: bzip2: Fix return value when combining --test,-t and -q.

When passing -q to get quiet output --test would not display an error
message, but would also suppress the exit 2 code to indicate the file
was corrupt. Only suppress the error message with -q, not the exit value.

This patch comes from Debian.
"bunzip2 -qt returns 0 for corrupt archives"
https://bugs.debian.org/279025
---
 bzip2.c | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/bzip2.c b/bzip2.c
index 854a2bb..63649f6 100644
--- a/bzip2.c
+++ b/bzip2.c
@@ -2003,12 +2003,14 @@ IntNative main ( IntNative argc, Char *argv[] )
             testf ( aa->name );
 	 }
       }
-      if (testFailsExist && noisy) {
-         fprintf ( stderr,
-           "\n"
-           "You can use the `bzip2recover' program to attempt to recover\n"
-           "data from undamaged sections of corrupted files.\n\n"
-         );
+      if (testFailsExist) {
+	 if (noisy) {
+            fprintf ( stderr,
+               "\n"
+               "You can use the `bzip2recover' program to attempt to recover\n"
+               "data from undamaged sections of corrupted files.\n\n"
+            );
+	 }
          setExit(2);
          exit(exitValue);
       }
-- 
cgit v1.2.3


From 16f2c753f9959e8d7c7e1fa771b8ccc5821427aa Mon Sep 17 00:00:00 2001
From: Paul Kehrer 
Date: Sat, 8 Jun 2019 10:06:40 -0400
Subject: Fix undefined behavior in the macros SET_BH, CLEAR_BH, & ISSET_BH

These macros contain this pattern:
1 << ((Int32_value) & 31

This causes the undefined behavior sanitizers in clang and gcc to
complain because the shift, while ultimately stored to an unsigned
variable, is done as a signed value. Adding a cast to unsigned for
the int32 value resolves this issue.
---
 blocksort.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/blocksort.c b/blocksort.c
index 830d1b4..aa36766 100644
--- a/blocksort.c
+++ b/blocksort.c
@@ -202,9 +202,9 @@ void fallbackQSort3 ( UInt32* fmap,
       bhtab [ 0 .. 2+(nblock/32) ] destroyed
 */
 
-#define       SET_BH(zz)  bhtab[(zz) >> 5] |= (1 << ((zz) & 31))
-#define     CLEAR_BH(zz)  bhtab[(zz) >> 5] &= ~(1 << ((zz) & 31))
-#define     ISSET_BH(zz)  (bhtab[(zz) >> 5] & (1 << ((zz) & 31)))
+#define       SET_BH(zz)  bhtab[(zz) >> 5] |= ((UInt32)1 << ((zz) & 31))
+#define     CLEAR_BH(zz)  bhtab[(zz) >> 5] &= ~((UInt32)1 << ((zz) & 31))
+#define     ISSET_BH(zz)  (bhtab[(zz) >> 5] & ((UInt32)1 << ((zz) & 31)))
 #define      WORD_BH(zz)  bhtab[(zz) >> 5]
 #define UNALIGNED_BH(zz)  ((zz) & 0x01f)
 
-- 
cgit v1.2.3


From 7ed62bfb46e87a9e878712603469440e6882b184 Mon Sep 17 00:00:00 2001
From: Albert Astals Cid 
Date: Tue, 28 May 2019 19:35:18 +0200
Subject: Make sure nSelectors is not out of range

nSelectors is used in a loop from 0 to nSelectors to access selectorMtf
which is
	UChar    selectorMtf[BZ_MAX_SELECTORS];
so if nSelectors is bigger than BZ_MAX_SELECTORS it'll do an invalid memory
access

Fixes out of bounds access discovered while fuzzying karchive

This was reported as CVE-2019-12900
BZ2_decompress in decompress.c in bzip2 through 1.0.6 has an
out-of-bounds write when there are many selectors.
---
 decompress.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/decompress.c b/decompress.c
index ab6a624..f3db91d 100644
--- a/decompress.c
+++ b/decompress.c
@@ -287,7 +287,7 @@ Int32 BZ2_decompress ( DState* s )
       GET_BITS(BZ_X_SELECTOR_1, nGroups, 3);
       if (nGroups < 2 || nGroups > 6) RETURN(BZ_DATA_ERROR);
       GET_BITS(BZ_X_SELECTOR_2, nSelectors, 15);
-      if (nSelectors < 1) RETURN(BZ_DATA_ERROR);
+      if (nSelectors < 1 || nSelectors > BZ_MAX_SELECTORS) RETURN(BZ_DATA_ERROR);
       for (i = 0; i < nSelectors; i++) {
          j = 0;
          while (True) {
-- 
cgit v1.2.3


From ff986850159a1ea0c75617ffa792d1bb2069856e Mon Sep 17 00:00:00 2001
From: Federico Mena Quintero 
Date: Wed, 29 May 2019 17:14:27 -0500
Subject: Change a magic number (6) for a constant (BZ_N_GROUPS).

decompress.c (BZ2_decompress): Check nGroups against BZ_N_GROUPS.
---
 decompress.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/decompress.c b/decompress.c
index f3db91d..7351134 100644
--- a/decompress.c
+++ b/decompress.c
@@ -285,7 +285,7 @@ Int32 BZ2_decompress ( DState* s )
 
       /*--- Now the selectors ---*/
       GET_BITS(BZ_X_SELECTOR_1, nGroups, 3);
-      if (nGroups < 2 || nGroups > 6) RETURN(BZ_DATA_ERROR);
+      if (nGroups < 2 || nGroups > BZ_N_GROUPS) RETURN(BZ_DATA_ERROR);
       GET_BITS(BZ_X_SELECTOR_2, nSelectors, 15);
       if (nSelectors < 1 || nSelectors > BZ_MAX_SELECTORS) RETURN(BZ_DATA_ERROR);
       for (i = 0; i < nSelectors; i++) {
-- 
cgit v1.2.3


From f1e937776c5f331e24cc63a9d8e7ae9445a76761 Mon Sep 17 00:00:00 2001
From: Mark Wielaard 
Date: Tue, 25 Jun 2019 19:22:37 +0200
Subject: Add prepare-release.sh script.

Script to run to prepare a new release.
It will update the release number and tell you to update the
CHANGES file and to double check everything looks before doing
the release commit and tagging.

Afterwards you probably want to run release-update.sh to upload
the release and update the website at https://sourceware.org/bzip2/

There are embedded version strings and dates in a couple of places.
To keep the script simple remove some that aren't absolutely necessary.

README now just points to CHANGES.
README.COMPILATION.PROBLEMS only mentions the version once at the top.
bzip2.c only mentions the version once when doing --version.
manual.xml now doesn't have any embedded versions, just uses &bz-version;
everywhere.
---
 README                      | 23 +------------
 README.COMPILATION.PROBLEMS |  2 +-
 bzip2.c                     |  2 +-
 manual.xml                  |  4 +--
 prepare-release.sh          | 80 +++++++++++++++++++++++++++++++++++++++++++++
 release-update.sh           |  1 +
 6 files changed, 86 insertions(+), 26 deletions(-)
 create mode 100755 prepare-release.sh

diff --git a/README b/README
index c3bcbeb..bf95ad7 100644
--- a/README
+++ b/README
@@ -161,31 +161,10 @@ WHAT'S NEW IN 0.9.5 ?
    * Many small improvements in file and flag handling.
    * A Y2K statement.
 
-WHAT'S NEW IN 1.0.0 ?
+WHAT'S NEW IN 1.0.x ?
 
    See the CHANGES file.
 
-WHAT'S NEW IN 1.0.2 ?
-
-   See the CHANGES file.
-
-WHAT'S NEW IN 1.0.3 ?
-
-   See the CHANGES file.
-
-WHAT'S NEW IN 1.0.4 ?
-
-   See the CHANGES file.
-
-WHAT'S NEW IN 1.0.5 ?
-
-   See the CHANGES file.
-
-WHAT'S NEW IN 1.0.6 ?
-
-   See the CHANGES file.
-
-
 I hope you find bzip2 useful.  Feel free to contact me at
    jseward@acm.org
 if you have any suggestions or queries.  Many people mailed me with
diff --git a/README.COMPILATION.PROBLEMS b/README.COMPILATION.PROBLEMS
index 14ddd1b..d420854 100644
--- a/README.COMPILATION.PROBLEMS
+++ b/README.COMPILATION.PROBLEMS
@@ -12,7 +12,7 @@ This program is released under the terms of the license contained
 in the file LICENSE.
 ------------------------------------------------------------------
 
-bzip2-1.0.6 should compile without problems on the vast majority of
+bzip2 should compile without problems on the vast majority of
 platforms.  Using the supplied Makefile, I've built and tested it
 myself for x86-linux and amd64-linux.  With makefile.msc, Visual C++
 6.0 and nmake, you can build a native Win32 version too.  Large file
diff --git a/bzip2.c b/bzip2.c
index 63649f6..4804e80 100644
--- a/bzip2.c
+++ b/bzip2.c
@@ -1609,7 +1609,7 @@ void license ( void )
     "   \n"
     "   This program is free software; you can redistribute it and/or modify\n"
     "   it under the terms set out in the LICENSE file, which is included\n"
-    "   in the bzip2-1.0.6 source distribution.\n"
+    "   in the bzip2 source distribution.\n"
     "   \n"
     "   This program is distributed in the hope that it will be useful,\n"
     "   but WITHOUT ANY WARRANTY; without even the implied warranty of\n"
diff --git a/manual.xml b/manual.xml
index a7fbcb3..b177bbf 100644
--- a/manual.xml
+++ b/manual.xml
@@ -9,7 +9,7 @@
 
 
  
-  bzip2 and libbzip2, version 1.0.6
+  bzip2 and libbzip2, version &bz-version;
   A program and library for data compression
   
    &bz-lifespan;
@@ -139,7 +139,7 @@ else.
 
  bzip2,
   bunzip2 - a block-sorting file
-  compressor, v1.0.6
+  compressor, v&bz-version;
 
  bzcat -
    decompresses files to stdout
diff --git a/prepare-release.sh b/prepare-release.sh
new file mode 100755
index 0000000..db736b7
--- /dev/null
+++ b/prepare-release.sh
@@ -0,0 +1,80 @@
+#!/bin/bash
+
+# Script to run to prepare a new release.
+# It will update the release number and tell you to update the
+# CHANGES file and to double check everything looks before doing
+# the release commit and tagging.
+
+# Afterwards you probably want to run release-update.sh to upload
+# the release and update the website at https://sourceware.org/bzip2/
+
+# Any error is fatal
+set -e
+
+# We take one argument, the version (e.g. 1.0.7)
+if [ $# -ne 1 ]; then
+  echo "$0  (e.g. 1.0.7)"
+  exit 1
+fi
+
+LANG=C
+VERSION="$1"
+DATE=$(date +"%d %B %Y")
+
+# Replace the version strings in the comments
+VER_PREFIX="bzip2/libbzip2 version "
+sed -i -e "s@${VER_PREFIX}[0-9].*@${VER_PREFIX}${VERSION} of ${DATE}@" \
+  CHANGES LICENSE README* *.c *.h *.pl *.sh
+
+# Add an entry to the README
+DAY=$(date +"%d")
+MONTH=$(date +"%B")
+SHORTMONTH=$(date +"%b")
+YEAR=$(date +"%Y")
+printf "%2s %8s %s\n" "$DAY" "$MONTH" "$YEAR (bzip2, version $VERSION)" \
+  >> README
+
+# Update manual
+sed -i -e "s@ENTITY bz-version \".*\"@ENTITY bz-version \"$VERSION\"@" \
+       -e "s@ENTITY bz-date \".*\"@ENTITY bz-date \"$DAY $MONTH $YEAR\"@" \
+  entities.xml
+
+# bzip2.1 should really be generated from the manual.xml, but currently
+# isn't, so explicitly change it here too.
+sed -i -e "s@This manual page pertains to version .* of@This manual page pertains to version $VERSION of@" \
+       -e "s@sorting file compressor, v.*@sorting file compressor, v$VERSION@" \
+  bzip2.1* bzip2.txt
+
+# Update sources. All sources, use bzlib_private.
+# Except bzip2recover, which embeds a version string...
+sed -i -e "s@^#define BZ_VERSION  \".*\"@#define BZ_VERSION  \"${VERSION}, ${DAY}-${SHORTMONTH}-${YEAR}\"@" \
+  bzlib_private.h
+sed -i -e "s@\"bzip2recover .*: extracts blocks from damaged@\"bzip2recover ${VERSION}: extracts blocks from damaged@" \
+  bzip2recover.c
+
+# And finally update the version/dist/so_name in the Makefiles.
+sed -i -e "s@^DISTNAME=bzip2-.*@DISTNAME=bzip2-${VERSION}@" \
+  Makefile
+sed -i -e "s@libbz2\.so\.[0-9]\.[0-9]\.[0-9]*@libbz2\.so\.${VERSION}@" \
+  Makefile-libbz2_so
+
+echo "Now make sure the diff looks correct:"
+echo "  git diff"
+echo
+echo "And make sure there is a $VERSION section in the CHANGES file."
+echo
+echo "Double check:"
+echo "  make clean && make dist && make clean && make -f Makefile-libbz2_so"
+echo
+echo "Does everything look fine?"
+echo
+echo "git commit -a -m \"Prepare for $VERSION release.\""
+echo "git push"
+echo
+echo "Wait for the buildbot to give the all green!"
+echo "Then..."
+echo
+echo "git tag -s -m \"bzip2 $VERSION release\" bzip2-$VERSION"
+echo "git push --tags"
+echo
+echo "./release-update.sh"
diff --git a/release-update.sh b/release-update.sh
index 2ad32ce..d641ca6 100755
--- a/release-update.sh
+++ b/release-update.sh
@@ -24,6 +24,7 @@ echo "Requires a sourceware account in the bzip2 group."
 echo
 echo "Make sure the git repo was tagged, signed and pushed"
 echo "If not, please double check the source tree is release ready first"
+echo "You probably want to run ./prepare-release.sh $VERSION first."
 echo "Then do:"
 echo
 echo "    git tag -s -m \"bzip2 $VERSION release\" bzip2-$VERSION"
-- 
cgit v1.2.3


From f319b98aade2a337c74b9a3b48c6daffb7809cda Mon Sep 17 00:00:00 2001
From: Mark Wielaard 
Date: Thu, 27 Jun 2019 20:10:07 +0200
Subject: Prepare for 1.0.7 release.

---
 CHANGES                     | 15 ++++++++++++++-
 LICENSE                     |  2 +-
 Makefile                    |  2 +-
 Makefile-libbz2_so          | 10 +++++-----
 README                      |  3 ++-
 README.COMPILATION.PROBLEMS |  2 +-
 README.XML.STUFF            |  2 +-
 blocksort.c                 |  2 +-
 bzip2.1                     |  4 ++--
 bzip2.1.preformatted        |  4 ++--
 bzip2.c                     |  2 +-
 bzip2.txt                   |  4 ++--
 bzip2recover.c              |  4 ++--
 bzlib.c                     |  2 +-
 bzlib.h                     |  2 +-
 bzlib_private.h             |  4 ++--
 compress.c                  |  2 +-
 crctable.c                  |  2 +-
 decompress.c                |  2 +-
 entities.xml                |  4 ++--
 format.pl                   |  2 +-
 huffman.c                   |  2 +-
 mk251.c                     |  2 +-
 randtable.c                 |  2 +-
 spewG.c                     |  2 +-
 unzcrash.c                  |  2 +-
 xmlproc.sh                  |  2 +-
 27 files changed, 51 insertions(+), 37 deletions(-)

diff --git a/CHANGES b/CHANGES
index 94a9b66..d9b4c05 100644
--- a/CHANGES
+++ b/CHANGES
@@ -2,7 +2,7 @@
  This file is part of bzip2/libbzip2, a program and library for
  lossless, block-sorting data compression.
 
- bzip2/libbzip2 version 1.0.6 of 6 September 2010
+ bzip2/libbzip2 version 1.0.7 of 27 June 2019
  Copyright (C) 1996-2010 Julian Seward 
 
  Please read the WARNING, DISCLAIMER and PATENTS sections in the 
@@ -325,3 +325,16 @@ Security fix only.  Fixes CERT-FI 20469 as it applies to bzip2.
   Izdebski.
 
 * Make the documentation build on Ubuntu 10.04
+
+1.0.7 (27 Jun 19)
+~~~~~~~~~~~~~~~~~
+
+* Fix undefined behavior in the macros SET_BH, CLEAR_BH, & ISSET_BH
+
+* bzip2: Fix return value when combining --test,-t and -q.
+
+* bzip2recover: Fix buffer overflow for large argv[0]
+
+* bzip2recover: Fix use after free issue with outFile (CVE-2016-3189)
+
+* Make sure nSelectors is not out of range (CVE-2019-12900)
diff --git a/LICENSE b/LICENSE
index 0a10ee6..95f9598 100644
--- a/LICENSE
+++ b/LICENSE
@@ -37,6 +37,6 @@ NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
 Julian Seward, jseward@acm.org
-bzip2/libbzip2 version 1.0.6 of 6 September 2010
+bzip2/libbzip2 version 1.0.7 of 27 June 2019
 
 --------------------------------------------------------------------------
diff --git a/Makefile b/Makefile
index c61f867..6706685 100644
--- a/Makefile
+++ b/Makefile
@@ -137,7 +137,7 @@ bzip2recover.o: bzip2recover.c
 distclean: clean
 	rm -f manual.ps manual.html manual.pdf
 
-DISTNAME=bzip2-1.0.6
+DISTNAME=bzip2-1.0.7
 dist: check manual
 	rm -f $(DISTNAME)
 	ln -s -f . $(DISTNAME)
diff --git a/Makefile-libbz2_so b/Makefile-libbz2_so
index cff37c2..862eb7d 100644
--- a/Makefile-libbz2_so
+++ b/Makefile-libbz2_so
@@ -1,6 +1,6 @@
 
 # This Makefile builds a shared version of the library, 
-# libbz2.so.1.0.6, with soname libbz2.so.1.0,
+# libbz2.so.1.0.7, with soname libbz2.so.1.0,
 # at least on x86-Linux (RedHat 7.2), 
 # with gcc-2.96 20000731 (Red Hat Linux 7.1 2.96-98).  
 # Please see the README file for some important info 
@@ -35,13 +35,13 @@ OBJS= blocksort.o  \
       bzlib.o
 
 all: $(OBJS)
-	$(CC) -shared -Wl,-soname -Wl,libbz2.so.1.0 -o libbz2.so.1.0.6 $(OBJS)
-	$(CC) $(CFLAGS) -o bzip2-shared bzip2.c libbz2.so.1.0.6
+	$(CC) -shared -Wl,-soname -Wl,libbz2.so.1.0 -o libbz2.so.1.0.7 $(OBJS)
+	$(CC) $(CFLAGS) -o bzip2-shared bzip2.c libbz2.so.1.0.7
 	rm -f libbz2.so.1.0
-	ln -s libbz2.so.1.0.6 libbz2.so.1.0
+	ln -s libbz2.so.1.0.7 libbz2.so.1.0
 
 clean: 
-	rm -f $(OBJS) bzip2.o libbz2.so.1.0.6 libbz2.so.1.0 bzip2-shared
+	rm -f $(OBJS) bzip2.o libbz2.so.1.0.7 libbz2.so.1.0 bzip2-shared
 
 blocksort.o: blocksort.c
 	$(CC) $(CFLAGS) -c blocksort.c
diff --git a/README b/README
index bf95ad7..64873f9 100644
--- a/README
+++ b/README
@@ -6,7 +6,7 @@ This version is fully compatible with the previous public releases.
 This file is part of bzip2/libbzip2, a program and library for
 lossless, block-sorting data compression.
 
-bzip2/libbzip2 version 1.0.6 of 6 September 2010
+bzip2/libbzip2 version 1.0.7 of 27 June 2019
 Copyright (C) 1996-2010 Julian Seward 
 
 Please read the WARNING, DISCLAIMER and PATENTS sections in this file.
@@ -192,3 +192,4 @@ Cambridge, UK.
 20 December 2006 (bzip2, version 1.0.4)
 10 December 2007 (bzip2, version 1.0.5)
  6     Sept 2010 (bzip2, version 1.0.6)
+27     June 2019 (bzip2, version 1.0.7)
diff --git a/README.COMPILATION.PROBLEMS b/README.COMPILATION.PROBLEMS
index d420854..9e6ac5a 100644
--- a/README.COMPILATION.PROBLEMS
+++ b/README.COMPILATION.PROBLEMS
@@ -2,7 +2,7 @@
 This file is part of bzip2/libbzip2, a program and library for
 lossless, block-sorting data compression.
 
-bzip2/libbzip2 version 1.0.6 of 6 September 2010
+bzip2/libbzip2 version 1.0.7 of 27 June 2019
 Copyright (C) 1996-2010 Julian Seward 
 
 Please read the WARNING, DISCLAIMER and PATENTS sections in the 
diff --git a/README.XML.STUFF b/README.XML.STUFF
index f5dc3a4..b0575c3 100644
--- a/README.XML.STUFF
+++ b/README.XML.STUFF
@@ -2,7 +2,7 @@
   This file is part of bzip2/libbzip2, a program and library for
   lossless, block-sorting data compression.
 
-  bzip2/libbzip2 version 1.0.6 of 6 September 2010
+  bzip2/libbzip2 version 1.0.7 of 27 June 2019
   Copyright (C) 1996-2010 Julian Seward 
 
   Please read the WARNING, DISCLAIMER and PATENTS sections in the 
diff --git a/blocksort.c b/blocksort.c
index aa36766..2657901 100644
--- a/blocksort.c
+++ b/blocksort.c
@@ -8,7 +8,7 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.6 of 6 September 2010
+   bzip2/libbzip2 version 1.0.7 of 27 June 2019
    Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
diff --git a/bzip2.1 b/bzip2.1
index e386fa3..f49a78c 100644
--- a/bzip2.1
+++ b/bzip2.1
@@ -1,7 +1,7 @@
 .PU
 .TH bzip2 1
 .SH NAME
-bzip2, bunzip2 \- a block-sorting file compressor, v1.0.6
+bzip2, bunzip2 \- a block-sorting file compressor, v1.0.7
 .br
 bzcat \- decompresses files to stdout
 .br
@@ -405,7 +405,7 @@ I/O error messages are not as helpful as they could be.
 tries hard to detect I/O errors and exit cleanly, but the details of
 what the problem is sometimes seem rather misleading.
 
-This manual page pertains to version 1.0.6 of
+This manual page pertains to version 1.0.7 of
 .I bzip2.  
 Compressed data created by this version is entirely forwards and
 backwards compatible with the previous public releases, versions
diff --git a/bzip2.1.preformatted b/bzip2.1.preformatted
index 5f8e38e..a2279ad 100644
--- a/bzip2.1.preformatted
+++ b/bzip2.1.preformatted
@@ -3,7 +3,7 @@ bzip2(1)                                                 bzip2(1)
 
 
 NNAAMMEE
-       bzip2, bunzip2 − a block‐sorting file compressor, v1.0.6
+       bzip2, bunzip2 − a block‐sorting file compressor, v1.0.7
        bzcat − decompresses files to stdout
        bzip2recover − recovers data from damaged bzip2 files
 
@@ -348,7 +348,7 @@ CCAAVVEEAATTSS
        but  the  details  of  what  the problem is sometimes seem
        rather misleading.
 
-       This manual page pertains to version 1.0.6 of _b_z_i_p_2_.  Com­
+       This manual page pertains to version 1.0.7 of _b_z_i_p_2_.  Com­
        pressed  data created by this version is entirely forwards
        and  backwards  compatible  with   the   previous   public
        releases,  versions  0.1pl2,  0.9.0,  0.9.5, 1.0.0, 1.0.1, 
diff --git a/bzip2.c b/bzip2.c
index 4804e80..e362c65 100644
--- a/bzip2.c
+++ b/bzip2.c
@@ -7,7 +7,7 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.6 of 6 September 2010
+   bzip2/libbzip2 version 1.0.7 of 27 June 2019
    Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
diff --git a/bzip2.txt b/bzip2.txt
index 0b9f74a..c449a1a 100644
--- a/bzip2.txt
+++ b/bzip2.txt
@@ -1,6 +1,6 @@
 
 NAME
-       bzip2, bunzip2 - a block-sorting file compressor, v1.0.6
+       bzip2, bunzip2 - a block-sorting file compressor, v1.0.7
        bzcat - decompresses files to stdout
        bzip2recover - recovers data from damaged bzip2 files
 
@@ -345,7 +345,7 @@ CAVEATS
        but  the  details  of  what  the problem is sometimes seem
        rather misleading.
 
-       This manual page pertains to version 1.0.6 of bzip2.  Com-
+       This manual page pertains to version 1.0.7 of bzip2.  Com-
        pressed  data created by this version is entirely forwards
        and  backwards  compatible  with   the   previous   public
        releases,  versions  0.1pl2,  0.9.0,  0.9.5, 1.0.0, 1.0.1,
diff --git a/bzip2recover.c b/bzip2recover.c
index a955d60..c0b9eac 100644
--- a/bzip2recover.c
+++ b/bzip2recover.c
@@ -7,7 +7,7 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.6 of 6 September 2010
+   bzip2/libbzip2 version 1.0.7 of 27 June 2019
    Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
@@ -314,7 +314,7 @@ Int32 main ( Int32 argc, Char** argv )
    inFileName[0] = outFileName[0] = 0;
 
    fprintf ( stderr, 
-             "bzip2recover 1.0.6: extracts blocks from damaged .bz2 files.\n" );
+             "bzip2recover 1.0.7: extracts blocks from damaged .bz2 files.\n" );
 
    if (argc != 2) {
       fprintf ( stderr, "%s: usage is `%s damaged_file_name'.\n",
diff --git a/bzlib.c b/bzlib.c
index 8aa6ae7..f9da295 100644
--- a/bzlib.c
+++ b/bzlib.c
@@ -8,7 +8,7 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.6 of 6 September 2010
+   bzip2/libbzip2 version 1.0.7 of 27 June 2019
    Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
diff --git a/bzlib.h b/bzlib.h
index c2c4b72..8cf0791 100644
--- a/bzlib.h
+++ b/bzlib.h
@@ -8,7 +8,7 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.6 of 6 September 2010
+   bzip2/libbzip2 version 1.0.7 of 27 June 2019
    Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
diff --git a/bzlib_private.h b/bzlib_private.h
index 06ee0b4..7975552 100644
--- a/bzlib_private.h
+++ b/bzlib_private.h
@@ -8,7 +8,7 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.6 of 6 September 2010
+   bzip2/libbzip2 version 1.0.7 of 27 June 2019
    Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
@@ -36,7 +36,7 @@
 
 /*-- General stuff. --*/
 
-#define BZ_VERSION  "1.0.6, 6-Sept-2010"
+#define BZ_VERSION  "1.0.7, 27-Jun-2019"
 
 typedef char            Char;
 typedef unsigned char   Bool;
diff --git a/compress.c b/compress.c
index 5d4af91..237620d 100644
--- a/compress.c
+++ b/compress.c
@@ -8,7 +8,7 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.6 of 6 September 2010
+   bzip2/libbzip2 version 1.0.7 of 27 June 2019
    Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
diff --git a/crctable.c b/crctable.c
index 8daae06..746efac 100644
--- a/crctable.c
+++ b/crctable.c
@@ -8,7 +8,7 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.6 of 6 September 2010
+   bzip2/libbzip2 version 1.0.7 of 27 June 2019
    Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
diff --git a/decompress.c b/decompress.c
index 7351134..20ce493 100644
--- a/decompress.c
+++ b/decompress.c
@@ -8,7 +8,7 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.6 of 6 September 2010
+   bzip2/libbzip2 version 1.0.7 of 27 June 2019
    Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
diff --git a/entities.xml b/entities.xml
index 0ff303e..3016674 100644
--- a/entities.xml
+++ b/entities.xml
@@ -3,7 +3,7 @@
 
 
 
-
-
+
+
 
 
diff --git a/format.pl b/format.pl
index 0a56cba..74d7eda 100755
--- a/format.pl
+++ b/format.pl
@@ -4,7 +4,7 @@
 # This file is part of bzip2/libbzip2, a program and library for
 # lossless, block-sorting data compression.
 #
-# bzip2/libbzip2 version 1.0.6 of 6 September 2010
+# bzip2/libbzip2 version 1.0.7 of 27 June 2019
 # Copyright (C) 1996-2010 Julian Seward 
 #
 # Please read the WARNING, DISCLAIMER and PATENTS sections in the 
diff --git a/huffman.c b/huffman.c
index fc396be..0fd6fd7 100644
--- a/huffman.c
+++ b/huffman.c
@@ -8,7 +8,7 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.6 of 6 September 2010
+   bzip2/libbzip2 version 1.0.7 of 27 June 2019
    Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
diff --git a/mk251.c b/mk251.c
index aefd855..9528b92 100644
--- a/mk251.c
+++ b/mk251.c
@@ -9,7 +9,7 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.6 of 6 September 2010
+   bzip2/libbzip2 version 1.0.7 of 27 June 2019
    Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
diff --git a/randtable.c b/randtable.c
index 35b29e1..726d62f 100644
--- a/randtable.c
+++ b/randtable.c
@@ -8,7 +8,7 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.6 of 6 September 2010
+   bzip2/libbzip2 version 1.0.7 of 27 June 2019
    Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
diff --git a/spewG.c b/spewG.c
index 08fb3fb..320dc85 100644
--- a/spewG.c
+++ b/spewG.c
@@ -13,7 +13,7 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.6 of 6 September 2010
+   bzip2/libbzip2 version 1.0.7 of 27 June 2019
    Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
diff --git a/unzcrash.c b/unzcrash.c
index acdb94d..6e7c631 100644
--- a/unzcrash.c
+++ b/unzcrash.c
@@ -17,7 +17,7 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.6 of 6 September 2010
+   bzip2/libbzip2 version 1.0.7 of 27 June 2019
    Copyright (C) 1996-2010 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
diff --git a/xmlproc.sh b/xmlproc.sh
index c9dc790..ea6e7e6 100755
--- a/xmlproc.sh
+++ b/xmlproc.sh
@@ -5,7 +5,7 @@
 #  This file is part of bzip2/libbzip2, a program and library for
 #  lossless, block-sorting data compression.
 #
-#  bzip2/libbzip2 version 1.0.6 of 6 September 2010
+#  bzip2/libbzip2 version 1.0.7 of 27 June 2019
 #  Copyright (C) 1996-2010 Julian Seward 
 #
 #  Please read the WARNING, DISCLAIMER and PATENTS sections in the 
-- 
cgit v1.2.3


From 35dd3cf94f3008a3bc6e8c6c964f091e8025570f Mon Sep 17 00:00:00 2001
From: Joshua Watt 
Date: Tue, 2 Jul 2019 15:05:53 -0500
Subject: Update .gitignore

Updates the .gitignore file to ignore many build artifacts
---
 .gitignore | 9 +++++++++
 1 file changed, 9 insertions(+)
 create mode 100644 .gitignore

diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..da23814
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,9 @@
+*.o
+*.obj
+*.rb2
+*.tst
+*.a
+*.lib
+*.exe
+bzip2
+bzip2recover
-- 
cgit v1.2.3


From 61b434b4ef28b5e3cbf8665d6829b08b405e0899 Mon Sep 17 00:00:00 2001
From: Joshua Watt 
Date: Tue, 2 Jul 2019 15:06:01 -0500
Subject: Always treat .ref files as binary

.ref files should always be treated as binary files so that git does not
attempt to convert the line endings if core.autocrlf is set.
---
 .gitattributes | 1 +
 1 file changed, 1 insertion(+)
 create mode 100644 .gitattributes

diff --git a/.gitattributes b/.gitattributes
new file mode 100644
index 0000000..f3af893
--- /dev/null
+++ b/.gitattributes
@@ -0,0 +1 @@
+*.ref binary
-- 
cgit v1.2.3


From 53f3d0d71c3c445bc4ac413d42c4386381e47b49 Mon Sep 17 00:00:00 2001
From: Joshua Watt 
Date: Tue, 2 Jul 2019 15:05:44 -0500
Subject: Fix include path separator

Changes the include path separator for Windows builds to use "/" instead
of "\". Windows has no problems with using a forward slash as a path
separator, but using a backslash causes problems when attempting to
cross compile for other platforms (for example, when trying to cross
compile for MinGW from Linux).
---
 bzip2.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/bzip2.c b/bzip2.c
index e362c65..be3b3be 100644
--- a/bzip2.c
+++ b/bzip2.c
@@ -128,7 +128,7 @@
 #if BZ_LCCWIN32
 #   include 
 #   include 
-#   include 
+#   include 
 
 #   define NORETURN       /**/
 #   define PATH_SEP       '\\'
-- 
cgit v1.2.3


From 5a4a6f44f0748840a7abe341098445c42d8d979a Mon Sep 17 00:00:00 2001
From: Mark Wielaard 
Date: Wed, 3 Jul 2019 22:22:16 +0200
Subject: Update prepare-release.sh for Makefile* and date ranges.

Also update the version number in the Makefile comments.
And update any date ranges to include the current year.
---
 prepare-release.sh | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/prepare-release.sh b/prepare-release.sh
index db736b7..a1ae375 100755
--- a/prepare-release.sh
+++ b/prepare-release.sh
@@ -20,23 +20,25 @@ fi
 LANG=C
 VERSION="$1"
 DATE=$(date +"%d %B %Y")
+DAY=$(date +"%d")
+MONTH=$(date +"%B")
+SHORTMONTH=$(date +"%b")
+YEAR=$(date +"%Y")
 
-# Replace the version strings in the comments
+# Replace the version strings and date ranges in the comments
 VER_PREFIX="bzip2/libbzip2 version "
 sed -i -e "s@${VER_PREFIX}[0-9].*@${VER_PREFIX}${VERSION} of ${DATE}@" \
-  CHANGES LICENSE README* *.c *.h *.pl *.sh
+       -e "s@ (C) \([0-9]\+\)-[0-9]\+ @ (C) \1-$YEAR @" \
+  CHANGES LICENSE Makefile* README* *.c *.h *.pl *.sh
 
 # Add an entry to the README
-DAY=$(date +"%d")
-MONTH=$(date +"%B")
-SHORTMONTH=$(date +"%b")
-YEAR=$(date +"%Y")
 printf "%2s %8s %s\n" "$DAY" "$MONTH" "$YEAR (bzip2, version $VERSION)" \
   >> README
 
 # Update manual
 sed -i -e "s@ENTITY bz-version \".*\"@ENTITY bz-version \"$VERSION\"@" \
        -e "s@ENTITY bz-date \".*\"@ENTITY bz-date \"$DAY $MONTH $YEAR\"@" \
+       -e "s@ENTITY bz-lifespan \"\([0-9]\+\)-[0-9]\+\"@ENTITY bz-filespan \"\1-$YEAR\"@"\
   entities.xml
 
 # bzip2.1 should really be generated from the manual.xml, but currently
-- 
cgit v1.2.3


From 13d8bce0393ca21cbca1e8ba7692466a64fd46dd Mon Sep 17 00:00:00 2001
From: Phil Ross 
Date: Tue, 21 May 2019 20:46:14 +0100
Subject: Fix a 'not a normal file' error when compressing large files.

The bzip2 command line would report 'not a normal file' for files of
size larger than 2^32 - 1 bytes.

Patch bzip2.c to use _stati64 instead of _stat so that a successful
result is returned for large files.

Resolves https://github.com/philr/bzip2-windows/issues/3.
---
 bzip2.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/bzip2.c b/bzip2.c
index be3b3be..76c6c94 100644
--- a/bzip2.c
+++ b/bzip2.c
@@ -132,8 +132,8 @@
 
 #   define NORETURN       /**/
 #   define PATH_SEP       '\\'
-#   define MY_LSTAT       _stat
-#   define MY_STAT        _stat
+#   define MY_LSTAT       _stati64
+#   define MY_STAT        _stati64
 #   define MY_S_ISREG(x)  ((x) & _S_IFREG)
 #   define MY_S_ISDIR(x)  ((x) & _S_IFDIR)
 
-- 
cgit v1.2.3


From b07b105d1b66e32760095e3602261738443b9e13 Mon Sep 17 00:00:00 2001
From: Mark Wielaard 
Date: Wed, 3 Jul 2019 01:28:11 +0200
Subject: Accept as many selectors as the file format allows.

But ignore any larger than the theoretical maximum, BZ_MAX_SELECTORS.

The theoretical maximum number of selectors depends on the maximum
blocksize (900000 bytes) and the number of symbols (50) that can be
encoded with a different Huffman tree. BZ_MAX_SELECTORS is 18002.

But the bzip2 file format allows the number of selectors to be encoded
with 15 bits (because 18002 isn't a factor of 2 and doesn't fit in
14 bits). So the file format maximum is 32767 selectors.

Some bzip2 encoders might actually have written out more selectors
than the theoretical maximum because they rounded up the number of
selectors to some convenient factor of 8.

The extra 14766 selectors can never be validly used by the decompression
algorithm. So we can read them, but then discard them.

This is effectively what was done (by accident) before we added a
check for nSelectors to be at most BZ_MAX_SELECTORS to mitigate
CVE-2019-12900.

The extra selectors were written out after the array inside the
EState struct. But the struct has extra space allocated after the
selector arrays of 18060 bytes (which is larger than 14766).
All of which will be initialized later (so the overwrite of that
space with extra selector values would have been harmless).
---
 compress.c   |  2 +-
 decompress.c | 10 ++++++++--
 2 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/compress.c b/compress.c
index 237620d..76adee6 100644
--- a/compress.c
+++ b/compress.c
@@ -454,7 +454,7 @@ void sendMTFValues ( EState* s )
 
    AssertH( nGroups < 8, 3002 );
    AssertH( nSelectors < 32768 &&
-            nSelectors <= (2 + (900000 / BZ_G_SIZE)),
+            nSelectors <= BZ_MAX_SELECTORS,
             3003 );
 
 
diff --git a/decompress.c b/decompress.c
index 20ce493..3303499 100644
--- a/decompress.c
+++ b/decompress.c
@@ -287,7 +287,7 @@ Int32 BZ2_decompress ( DState* s )
       GET_BITS(BZ_X_SELECTOR_1, nGroups, 3);
       if (nGroups < 2 || nGroups > BZ_N_GROUPS) RETURN(BZ_DATA_ERROR);
       GET_BITS(BZ_X_SELECTOR_2, nSelectors, 15);
-      if (nSelectors < 1 || nSelectors > BZ_MAX_SELECTORS) RETURN(BZ_DATA_ERROR);
+      if (nSelectors < 1) RETURN(BZ_DATA_ERROR);
       for (i = 0; i < nSelectors; i++) {
          j = 0;
          while (True) {
@@ -296,8 +296,14 @@ Int32 BZ2_decompress ( DState* s )
             j++;
             if (j >= nGroups) RETURN(BZ_DATA_ERROR);
          }
-         s->selectorMtf[i] = j;
+         /* Having more than BZ_MAX_SELECTORS doesn't make much sense
+            since they will never be used, but some implementations might
+            "round up" the number of selectors, so just ignore those. */
+         if (i < BZ_MAX_SELECTORS)
+           s->selectorMtf[i] = j;
       }
+      if (nSelectors > BZ_MAX_SELECTORS)
+        nSelectors = BZ_MAX_SELECTORS;
 
       /*--- Undo the MTF values for the selectors. ---*/
       {
-- 
cgit v1.2.3


From 04e979201daffd511826ed1c196c9f5fe7d14a89 Mon Sep 17 00:00:00 2001
From: Mark Wielaard 
Date: Thu, 11 Jul 2019 02:40:18 +0200
Subject: release-update.sh should update version number in website pages too.

---
 release-update.sh | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/release-update.sh b/release-update.sh
index d641ca6..d860f67 100755
--- a/release-update.sh
+++ b/release-update.sh
@@ -76,6 +76,10 @@ git clone ssh://sourceware.org/git/bzip2-htdocs.git
 cp bzip2/CHANGES bzip2/bzip.css bzip2-htdocs/
 cp bzip2/bzip.css bzip2/bzip2.txt bzip2/manual.{html,pdf} bzip2-htdocs/manual/
 cd bzip2-htdocs
+
+# Update version in html pages.
+sed -i -e "s/The current stable version is bzip2 [0-9]\.[0-9]\.[0-9]\+/The current stable version is bzip2 ${VERSION}/" *.html */*.html
+
 git commit -a -m "Update for bzip2 $VERSION release"
 git show
 git push
-- 
cgit v1.2.3


From d50cc4b0e700f8f8285b02d28db595e805e3b627 Mon Sep 17 00:00:00 2001
From: Mark Wielaard 
Date: Thu, 11 Jul 2019 19:54:37 +0200
Subject: Replace project contact email with bzip2-devel@sourceware.org.

Keep Julian's email as author information, but redirect general
project feedback in the code and manual to the community mailinglist.
---
 README       | 4 ++--
 bzip2.c      | 8 ++++----
 bzlib.c      | 4 ++--
 entities.xml | 3 ++-
 manual.xml   | 8 ++++----
 5 files changed, 14 insertions(+), 13 deletions(-)

diff --git a/README b/README
index 64873f9..42543d1 100644
--- a/README
+++ b/README
@@ -165,8 +165,8 @@ WHAT'S NEW IN 1.0.x ?
 
    See the CHANGES file.
 
-I hope you find bzip2 useful.  Feel free to contact me at
-   jseward@acm.org
+I hope you find bzip2 useful.  Feel free to contact the developers at
+   bzip2-devel@sourceware.org
 if you have any suggestions or queries.  Many people mailed me with
 comments, suggestions and patches after the releases of bzip-0.15,
 bzip-0.21, and bzip2 versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1,
diff --git a/bzip2.c b/bzip2.c
index 76c6c94..01a41d2 100644
--- a/bzip2.c
+++ b/bzip2.c
@@ -748,8 +748,8 @@ void panic ( const Char* s )
    fprintf ( stderr,
              "\n%s: PANIC -- internal consistency error:\n"
              "\t%s\n"
-             "\tThis is a BUG.  Please report it to me at:\n"
-             "\tjseward@acm.org\n",
+             "\tThis is a BUG.  Please report it to:\n"
+             "\tbzip2-devel@sourceware.org\n",
              progName, s );
    showFileNames();
    cleanUpAndFail( 3 );
@@ -829,7 +829,7 @@ void mySIGSEGVorSIGBUScatcher ( IntNative n )
       "   The user's manual, Section 4.3, has more info on (1) and (2).\n"
       "   \n"
       "   If you suspect this is a bug in bzip2, or are unsure about (1)\n"
-      "   or (2), feel free to report it to me at: jseward@acm.org.\n"
+      "   or (2), feel free to report it to: bzip2-devel@sourceware.org.\n"
       "   Section 4.3 of the user's manual describes the info a useful\n"
       "   bug report should have.  If the manual is available on your\n"
       "   system, please try and read it before mailing me.  If you don't\n"
@@ -852,7 +852,7 @@ void mySIGSEGVorSIGBUScatcher ( IntNative n )
       "   The user's manual, Section 4.3, has more info on (2) and (3).\n"
       "   \n"
       "   If you suspect this is a bug in bzip2, or are unsure about (2)\n"
-      "   or (3), feel free to report it to me at: jseward@acm.org.\n"
+      "   or (3), feel free to report it to: bzip2-devel@sourceware.org.\n"
       "   Section 4.3 of the user's manual describes the info a useful\n"
       "   bug report should have.  If the manual is available on your\n"
       "   system, please try and read it before mailing me.  If you don't\n"
diff --git a/bzlib.c b/bzlib.c
index f9da295..2dbe666 100644
--- a/bzlib.c
+++ b/bzlib.c
@@ -43,12 +43,12 @@ void BZ2_bz__AssertH__fail ( int errcode )
    fprintf(stderr, 
       "\n\nbzip2/libbzip2: internal error number %d.\n"
       "This is a bug in bzip2/libbzip2, %s.\n"
-      "Please report it to me at: jseward@acm.org.  If this happened\n"
+      "Please report it to: bzip2-devel@sourceware.org.  If this happened\n"
       "when you were using some program which uses libbzip2 as a\n"
       "component, you should also report this bug to the author(s)\n"
       "of that program.  Please make an effort to report this bug;\n"
       "timely and accurate bug reports eventually lead to higher\n"
-      "quality software.  Thanks.  Julian Seward, 10 December 2007.\n\n",
+      "quality software.  Thanks.\n\n",
       errcode,
       BZ2_bzlibVersion()
    );
diff --git a/entities.xml b/entities.xml
index 3016674..a415062 100644
--- a/entities.xml
+++ b/entities.xml
@@ -1,6 +1,7 @@
 
 
-
+
+
 
 
 
diff --git a/manual.xml b/manual.xml
index b177bbf..b99e480 100644
--- a/manual.xml
+++ b/manual.xml
@@ -653,7 +653,7 @@ unsigned 64-bit integer.
 AUTHOR
 
 Julian Seward,
-&bz-email;
+&bz-author;
 
 The ideas embodied in
 bzip2 are due to (at least) the
@@ -2487,12 +2487,12 @@ message:
 

 bzip2/libbzip2: internal error number N.
 This is a bug in bzip2/libbzip2, &bz-version; of &bz-date;.
-Please report it to me at: &bz-email;.  If this happened
+Please report it to: &bz-email;.  If this happened
 when you were using some program which uses libbzip2 as a
 component, you should also report this bug to the author(s)
 of that program.  Please make an effort to report this bug;
 timely and accurate bug reports eventually lead to higher
-quality software.  Thanks.  Julian Seward, &bz-date;.
+quality software.  Thanks.
 

 
 where N is some error code
@@ -2537,7 +2537,7 @@ recovered from.
 Everything related to Windows has been contributed by
 Yoshioka Tsuneo
 (tsuneo@rr.iij4u.or.jp), so
-you should send your queries to him (but perhaps Cc: me,
+you should send your queries to him (but please Cc:
 &bz-email;).
 
 My vague understanding of what to do is: using Visual C++
-- 
cgit v1.2.3


From f7d209bfde37cf337f769531f9c7a29b8bb6ae7f Mon Sep 17 00:00:00 2001
From: Mark Wielaard 
Date: Thu, 11 Jul 2019 23:24:29 +0200
Subject: fix bzdiff when TMPDIR contains spaces

The bzdiff script doesn't contain enough quotes, so that it doesn't
work if the TMPDIR environment variable is defined and contains
spaces.

https://bugs.debian.org/493710
Author: Vincent Lefevre 
---
 bzdiff | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
 mode change 100644 => 100755 bzdiff

diff --git a/bzdiff b/bzdiff
old mode 100644
new mode 100755
index 6fc38f9..bd96c27
--- a/bzdiff
+++ b/bzdiff
@@ -37,10 +37,6 @@ if test -z "$FILES"; then
 	echo "Usage: $prog [${comp}_options] file [file]"
 	exit 1
 fi
-tmp=`mktemp ${TMPDIR:-/tmp}/bzdiff.XXXXXXXXXX` || {
-      echo 'cannot create a temporary file' >&2
-      exit 1
-}
 set $FILES
 if test $# -eq 1; then
 	FILE=`echo "$1" | sed 's/.bz2$//'`
@@ -53,10 +49,14 @@ elif test $# -eq 2; then
                 case "$2" in
 	        *.bz2)
 			F=`echo "$2" | sed 's|.*/||;s|.bz2$||'`
-                        bzip2 -cdfq "$2" > $tmp
-                        bzip2 -cdfq "$1" | $comp $OPTIONS - $tmp
+			tmp=`mktemp "${TMPDIR:-/tmp}"/bzdiff.XXXXXXXXXX` || {
+			      echo 'cannot create a temporary file' >&2
+			      exit 1
+			}
+                        bzip2 -cdfq "$2" > "$tmp"
+                        bzip2 -cdfq "$1" | $comp $OPTIONS - "$tmp"
                         STAT="$?"
-			/bin/rm -f $tmp;;
+			/bin/rm -f "$tmp";;
 
                 *)      bzip2 -cdfq "$1" | $comp $OPTIONS - "$2"
                         STAT="$?";;
@@ -69,8 +69,8 @@ elif test $# -eq 2; then
                         STAT="$?";;
                 esac;;
 	esac
-        exit "$STAT"
 else
 	echo "Usage: $prog [${comp}_options] file [file]"
 	exit 1
 fi
+exit "$STAT"
-- 
cgit v1.2.3


From 33414da1d2bedf2cbe693f0e21fdaef11d221b1d Mon Sep 17 00:00:00 2001
From: Mark Wielaard 
Date: Fri, 12 Jul 2019 00:50:54 +0200
Subject: Fix bashism in bzgrep

bzgrep uses ${var//} which is a bashism.
Replace by calling sed so other POSIX shells work.

Patch from openSUSE by Led 
---
 bzgrep | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/bzgrep b/bzgrep
index 9a04b83..5080afd 100644
--- a/bzgrep
+++ b/bzgrep
@@ -63,9 +63,7 @@ for i do
     bzip2 -cdfq "$i" | $grep $opt "$pat"
     r=$?
   else
-    j=${i//\\/\\\\}
-    j=${j//|/\\|}
-    j=${j//&/\\&}
+    j=$(echo "$i" | sed 's/\\/&&/g;s/|/\\&/g;s/&/\\&/g')
     j=`printf "%s" "$j" | tr '\n' ' '`
     bzip2 -cdfq "$i" | $grep $opt "$pat" | sed "s|^|${j}:|"
     r=$?
-- 
cgit v1.2.3


From d0b47bde0e2f5bae7502ca55b80ab006523db820 Mon Sep 17 00:00:00 2001
From: Mark Wielaard 
Date: Fri, 12 Jul 2019 01:06:33 +0200
Subject: Fix bzgrep so it doesn't always return a 0 exit code with multiple
 archives
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The bzgrep wrapper always returns 0 as exit code when working on
multiple archives, even when the pattern is not found.

Fix from openSUSE by Kristýna Streitová 
https://bugzilla.suse.com/970260
---
 bzgrep | 16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)
 mode change 100644 => 100755 bzgrep

diff --git a/bzgrep b/bzgrep
old mode 100644
new mode 100755
index 5080afd..0314ca6
--- a/bzgrep
+++ b/bzgrep
@@ -65,8 +65,20 @@ for i do
   else
     j=$(echo "$i" | sed 's/\\/&&/g;s/|/\\&/g;s/&/\\&/g')
     j=`printf "%s" "$j" | tr '\n' ' '`
-    bzip2 -cdfq "$i" | $grep $opt "$pat" | sed "s|^|${j}:|"
-    r=$?
+    # A trick adapted from
+    # https://groups.google.com/forum/#!original/comp.unix.shell/x1345iu10eg/Nn1n-1r1uU0J
+    # that has the same effect as the following bash code:
+    # bzip2 -cdfq "$i" | $grep $opt "$pat" | sed "s|^|${j}:|"
+    # r=${PIPESTATUS[1]}
+    exec 3>&1
+    eval `
+      exec 4>&1 >&3 3>&-
+      {
+        bzip2 -cdfq "$i" 4>&-
+      } | {
+        $grep $opt "$pat" 4>&-; echo "r=$?;" >&4
+      } | sed "s|^|${j}:|"
+    `
   fi
   test "$r" -ne 0 && res="$r"
 done
-- 
cgit v1.2.3


From 48e4d87489c1e8ba6c77b7ed6217eb3c9ea519bc Mon Sep 17 00:00:00 2001
From: Mark Wielaard 
Date: Thu, 11 Jul 2019 20:05:42 +0200
Subject: manual: Add id to legalnotice.

Otherwise the generated HTML will have a different randomly generated
name id which generates spurious diffs.
---
 manual.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/manual.xml b/manual.xml
index b99e480..ea9fca2 100644
--- a/manual.xml
+++ b/manual.xml
@@ -27,7 +27,7 @@
    
   
 
-  
+  
 
   This program, bzip2, the
   associated library libbzip2, and
-- 
cgit v1.2.3


From 1c8dd8698c1c84c39c25d950edf6aec4eca70901 Mon Sep 17 00:00:00 2001
From: Mark Wielaard 
Date: Sat, 13 Jul 2019 17:06:25 +0200
Subject: prepare-release.sh: Fix bz-lifespan typo.

---
 prepare-release.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/prepare-release.sh b/prepare-release.sh
index a1ae375..12c29f7 100755
--- a/prepare-release.sh
+++ b/prepare-release.sh
@@ -38,7 +38,7 @@ printf "%2s %8s %s\n" "$DAY" "$MONTH" "$YEAR (bzip2, version $VERSION)" \
 # Update manual
 sed -i -e "s@ENTITY bz-version \".*\"@ENTITY bz-version \"$VERSION\"@" \
        -e "s@ENTITY bz-date \".*\"@ENTITY bz-date \"$DAY $MONTH $YEAR\"@" \
-       -e "s@ENTITY bz-lifespan \"\([0-9]\+\)-[0-9]\+\"@ENTITY bz-filespan \"\1-$YEAR\"@"\
+       -e "s@ENTITY bz-lifespan \"\([0-9]\+\)-[0-9]\+\"@ENTITY bz-lifespan \"\1-$YEAR\"@"\
   entities.xml
 
 # bzip2.1 should really be generated from the manual.xml, but currently
-- 
cgit v1.2.3


From 6a8690fc8d26c815e798c588f796eabe9d684cf0 Mon Sep 17 00:00:00 2001
From: Mark Wielaard 
Date: Sat, 13 Jul 2019 17:17:58 +0200
Subject: Prepare for 1.0.8 release.

---
 CHANGES                     | 20 ++++++++++++++++++--
 LICENSE                     |  4 ++--
 Makefile                    |  6 +++---
 Makefile-libbz2_so          | 14 +++++++-------
 README                      |  5 +++--
 README.COMPILATION.PROBLEMS |  4 ++--
 README.XML.STUFF            |  4 ++--
 blocksort.c                 |  4 ++--
 bzip2.1                     |  4 ++--
 bzip2.1.preformatted        |  4 ++--
 bzip2.c                     |  6 +++---
 bzip2.txt                   |  4 ++--
 bzip2recover.c              |  6 +++---
 bzlib.c                     |  4 ++--
 bzlib.h                     |  4 ++--
 bzlib_private.h             |  6 +++---
 compress.c                  |  4 ++--
 crctable.c                  |  4 ++--
 decompress.c                |  4 ++--
 entities.xml                |  6 +++---
 format.pl                   |  4 ++--
 huffman.c                   |  4 ++--
 mk251.c                     |  4 ++--
 randtable.c                 |  4 ++--
 spewG.c                     |  4 ++--
 unzcrash.c                  |  4 ++--
 xmlproc.sh                  |  4 ++--
 27 files changed, 81 insertions(+), 64 deletions(-)

diff --git a/CHANGES b/CHANGES
index d9b4c05..30afead 100644
--- a/CHANGES
+++ b/CHANGES
@@ -2,8 +2,8 @@
  This file is part of bzip2/libbzip2, a program and library for
  lossless, block-sorting data compression.
 
- bzip2/libbzip2 version 1.0.7 of 27 June 2019
- Copyright (C) 1996-2010 Julian Seward 
+ bzip2/libbzip2 version 1.0.8 of 13 July 2019
+ Copyright (C) 1996-2019 Julian Seward 
 
  Please read the WARNING, DISCLAIMER and PATENTS sections in the 
  README file.
@@ -338,3 +338,19 @@ Security fix only.  Fixes CERT-FI 20469 as it applies to bzip2.
 * bzip2recover: Fix use after free issue with outFile (CVE-2016-3189)
 
 * Make sure nSelectors is not out of range (CVE-2019-12900)
+
+1.0.8 (13 Jul 19)
+~~~~~~~~~~~~~~~~~
+
+* Accept as many selectors as the file format allows.
+  This relaxes the fix for CVE-2019-12900 from 1.0.7
+  so that bzip2 allows decompression of bz2 files that
+  use (too) many selectors again.
+
+* Fix handling of large (> 4GB) files on Windows.
+
+* Cleanup of bzdiff and bzgrep scripts so they don't use
+  any bash extensions and handle multiple archives correctly.
+
+* There is now a bz2-files testsuite at
+  https://sourceware.org/git/bzip2-tests.git
diff --git a/LICENSE b/LICENSE
index 95f9598..81a37ea 100644
--- a/LICENSE
+++ b/LICENSE
@@ -2,7 +2,7 @@
 --------------------------------------------------------------------------
 
 This program, "bzip2", the associated library "libbzip2", and all
-documentation, are copyright (C) 1996-2010 Julian R Seward.  All
+documentation, are copyright (C) 1996-2019 Julian R Seward.  All
 rights reserved.
 
 Redistribution and use in source and binary forms, with or without
@@ -37,6 +37,6 @@ NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
 Julian Seward, jseward@acm.org
-bzip2/libbzip2 version 1.0.7 of 27 June 2019
+bzip2/libbzip2 version 1.0.8 of 13 July 2019
 
 --------------------------------------------------------------------------
diff --git a/Makefile b/Makefile
index 6706685..f8a1772 100644
--- a/Makefile
+++ b/Makefile
@@ -2,8 +2,8 @@
 # This file is part of bzip2/libbzip2, a program and library for
 # lossless, block-sorting data compression.
 #
-# bzip2/libbzip2 version 1.0.6 of 6 September 2010
-# Copyright (C) 1996-2010 Julian Seward 
+# bzip2/libbzip2 version 1.0.8 of 13 July 2019
+# Copyright (C) 1996-2019 Julian Seward 
 #
 # Please read the WARNING, DISCLAIMER and PATENTS sections in the 
 # README file.
@@ -137,7 +137,7 @@ bzip2recover.o: bzip2recover.c
 distclean: clean
 	rm -f manual.ps manual.html manual.pdf
 
-DISTNAME=bzip2-1.0.7
+DISTNAME=bzip2-1.0.8
 dist: check manual
 	rm -f $(DISTNAME)
 	ln -s -f . $(DISTNAME)
diff --git a/Makefile-libbz2_so b/Makefile-libbz2_so
index 862eb7d..fb0f230 100644
--- a/Makefile-libbz2_so
+++ b/Makefile-libbz2_so
@@ -1,6 +1,6 @@
 
 # This Makefile builds a shared version of the library, 
-# libbz2.so.1.0.7, with soname libbz2.so.1.0,
+# libbz2.so.1.0.8, with soname libbz2.so.1.0,
 # at least on x86-Linux (RedHat 7.2), 
 # with gcc-2.96 20000731 (Red Hat Linux 7.1 2.96-98).  
 # Please see the README file for some important info 
@@ -10,8 +10,8 @@
 # This file is part of bzip2/libbzip2, a program and library for
 # lossless, block-sorting data compression.
 #
-# bzip2/libbzip2 version 1.0.6 of 6 September 2010
-# Copyright (C) 1996-2010 Julian Seward 
+# bzip2/libbzip2 version 1.0.8 of 13 July 2019
+# Copyright (C) 1996-2019 Julian Seward 
 #
 # Please read the WARNING, DISCLAIMER and PATENTS sections in the 
 # README file.
@@ -35,13 +35,13 @@ OBJS= blocksort.o  \
       bzlib.o
 
 all: $(OBJS)
-	$(CC) -shared -Wl,-soname -Wl,libbz2.so.1.0 -o libbz2.so.1.0.7 $(OBJS)
-	$(CC) $(CFLAGS) -o bzip2-shared bzip2.c libbz2.so.1.0.7
+	$(CC) -shared -Wl,-soname -Wl,libbz2.so.1.0 -o libbz2.so.1.0.8 $(OBJS)
+	$(CC) $(CFLAGS) -o bzip2-shared bzip2.c libbz2.so.1.0.8
 	rm -f libbz2.so.1.0
-	ln -s libbz2.so.1.0.7 libbz2.so.1.0
+	ln -s libbz2.so.1.0.8 libbz2.so.1.0
 
 clean: 
-	rm -f $(OBJS) bzip2.o libbz2.so.1.0.7 libbz2.so.1.0 bzip2-shared
+	rm -f $(OBJS) bzip2.o libbz2.so.1.0.8 libbz2.so.1.0 bzip2-shared
 
 blocksort.o: blocksort.c
 	$(CC) $(CFLAGS) -c blocksort.c
diff --git a/README b/README
index 42543d1..b9c6099 100644
--- a/README
+++ b/README
@@ -6,8 +6,8 @@ This version is fully compatible with the previous public releases.
 This file is part of bzip2/libbzip2, a program and library for
 lossless, block-sorting data compression.
 
-bzip2/libbzip2 version 1.0.7 of 27 June 2019
-Copyright (C) 1996-2010 Julian Seward 
+bzip2/libbzip2 version 1.0.8 of 13 July 2019
+Copyright (C) 1996-2019 Julian Seward 
 
 Please read the WARNING, DISCLAIMER and PATENTS sections in this file.
 
@@ -193,3 +193,4 @@ Cambridge, UK.
 10 December 2007 (bzip2, version 1.0.5)
  6     Sept 2010 (bzip2, version 1.0.6)
 27     June 2019 (bzip2, version 1.0.7)
+13     July 2019 (bzip2, version 1.0.8)
diff --git a/README.COMPILATION.PROBLEMS b/README.COMPILATION.PROBLEMS
index 9e6ac5a..fa317a5 100644
--- a/README.COMPILATION.PROBLEMS
+++ b/README.COMPILATION.PROBLEMS
@@ -2,8 +2,8 @@
 This file is part of bzip2/libbzip2, a program and library for
 lossless, block-sorting data compression.
 
-bzip2/libbzip2 version 1.0.7 of 27 June 2019
-Copyright (C) 1996-2010 Julian Seward 
+bzip2/libbzip2 version 1.0.8 of 13 July 2019
+Copyright (C) 1996-2019 Julian Seward 
 
 Please read the WARNING, DISCLAIMER and PATENTS sections in the 
 README file.
diff --git a/README.XML.STUFF b/README.XML.STUFF
index b0575c3..1503476 100644
--- a/README.XML.STUFF
+++ b/README.XML.STUFF
@@ -2,8 +2,8 @@
   This file is part of bzip2/libbzip2, a program and library for
   lossless, block-sorting data compression.
 
-  bzip2/libbzip2 version 1.0.7 of 27 June 2019
-  Copyright (C) 1996-2010 Julian Seward 
+  bzip2/libbzip2 version 1.0.8 of 13 July 2019
+  Copyright (C) 1996-2019 Julian Seward 
 
   Please read the WARNING, DISCLAIMER and PATENTS sections in the 
   README file.
diff --git a/blocksort.c b/blocksort.c
index 2657901..92d81fe 100644
--- a/blocksort.c
+++ b/blocksort.c
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.7 of 27 June 2019
-   Copyright (C) 1996-2010 Julian Seward 
+   bzip2/libbzip2 version 1.0.8 of 13 July 2019
+   Copyright (C) 1996-2019 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/bzip2.1 b/bzip2.1
index f49a78c..0cbcdab 100644
--- a/bzip2.1
+++ b/bzip2.1
@@ -1,7 +1,7 @@
 .PU
 .TH bzip2 1
 .SH NAME
-bzip2, bunzip2 \- a block-sorting file compressor, v1.0.7
+bzip2, bunzip2 \- a block-sorting file compressor, v1.0.8
 .br
 bzcat \- decompresses files to stdout
 .br
@@ -405,7 +405,7 @@ I/O error messages are not as helpful as they could be.
 tries hard to detect I/O errors and exit cleanly, but the details of
 what the problem is sometimes seem rather misleading.
 
-This manual page pertains to version 1.0.7 of
+This manual page pertains to version 1.0.8 of
 .I bzip2.  
 Compressed data created by this version is entirely forwards and
 backwards compatible with the previous public releases, versions
diff --git a/bzip2.1.preformatted b/bzip2.1.preformatted
index a2279ad..787f1c6 100644
--- a/bzip2.1.preformatted
+++ b/bzip2.1.preformatted
@@ -3,7 +3,7 @@ bzip2(1)                                                 bzip2(1)
 
 
 NNAAMMEE
-       bzip2, bunzip2 − a block‐sorting file compressor, v1.0.7
+       bzip2, bunzip2 − a block‐sorting file compressor, v1.0.8
        bzcat − decompresses files to stdout
        bzip2recover − recovers data from damaged bzip2 files
 
@@ -348,7 +348,7 @@ CCAAVVEEAATTSS
        but  the  details  of  what  the problem is sometimes seem
        rather misleading.
 
-       This manual page pertains to version 1.0.7 of _b_z_i_p_2_.  Com­
+       This manual page pertains to version 1.0.8 of _b_z_i_p_2_.  Com­
        pressed  data created by this version is entirely forwards
        and  backwards  compatible  with   the   previous   public
        releases,  versions  0.1pl2,  0.9.0,  0.9.5, 1.0.0, 1.0.1, 
diff --git a/bzip2.c b/bzip2.c
index 01a41d2..d95d280 100644
--- a/bzip2.c
+++ b/bzip2.c
@@ -7,8 +7,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.7 of 27 June 2019
-   Copyright (C) 1996-2010 Julian Seward 
+   bzip2/libbzip2 version 1.0.8 of 13 July 2019
+   Copyright (C) 1996-2019 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
@@ -1605,7 +1605,7 @@ void license ( void )
     "bzip2, a block-sorting file compressor.  "
     "Version %s.\n"
     "   \n"
-    "   Copyright (C) 1996-2010 by Julian Seward.\n"
+    "   Copyright (C) 1996-2019 by Julian Seward.\n"
     "   \n"
     "   This program is free software; you can redistribute it and/or modify\n"
     "   it under the terms set out in the LICENSE file, which is included\n"
diff --git a/bzip2.txt b/bzip2.txt
index c449a1a..a50570b 100644
--- a/bzip2.txt
+++ b/bzip2.txt
@@ -1,6 +1,6 @@
 
 NAME
-       bzip2, bunzip2 - a block-sorting file compressor, v1.0.7
+       bzip2, bunzip2 - a block-sorting file compressor, v1.0.8
        bzcat - decompresses files to stdout
        bzip2recover - recovers data from damaged bzip2 files
 
@@ -345,7 +345,7 @@ CAVEATS
        but  the  details  of  what  the problem is sometimes seem
        rather misleading.
 
-       This manual page pertains to version 1.0.7 of bzip2.  Com-
+       This manual page pertains to version 1.0.8 of bzip2.  Com-
        pressed  data created by this version is entirely forwards
        and  backwards  compatible  with   the   previous   public
        releases,  versions  0.1pl2,  0.9.0,  0.9.5, 1.0.0, 1.0.1,
diff --git a/bzip2recover.c b/bzip2recover.c
index c0b9eac..a8131e0 100644
--- a/bzip2recover.c
+++ b/bzip2recover.c
@@ -7,8 +7,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.7 of 27 June 2019
-   Copyright (C) 1996-2010 Julian Seward 
+   bzip2/libbzip2 version 1.0.8 of 13 July 2019
+   Copyright (C) 1996-2019 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
@@ -314,7 +314,7 @@ Int32 main ( Int32 argc, Char** argv )
    inFileName[0] = outFileName[0] = 0;
 
    fprintf ( stderr, 
-             "bzip2recover 1.0.7: extracts blocks from damaged .bz2 files.\n" );
+             "bzip2recover 1.0.8: extracts blocks from damaged .bz2 files.\n" );
 
    if (argc != 2) {
       fprintf ( stderr, "%s: usage is `%s damaged_file_name'.\n",
diff --git a/bzlib.c b/bzlib.c
index 2dbe666..2178655 100644
--- a/bzlib.c
+++ b/bzlib.c
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.7 of 27 June 2019
-   Copyright (C) 1996-2010 Julian Seward 
+   bzip2/libbzip2 version 1.0.8 of 13 July 2019
+   Copyright (C) 1996-2019 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/bzlib.h b/bzlib.h
index 8cf0791..8966a6c 100644
--- a/bzlib.h
+++ b/bzlib.h
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.7 of 27 June 2019
-   Copyright (C) 1996-2010 Julian Seward 
+   bzip2/libbzip2 version 1.0.8 of 13 July 2019
+   Copyright (C) 1996-2019 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/bzlib_private.h b/bzlib_private.h
index 7975552..3755a6f 100644
--- a/bzlib_private.h
+++ b/bzlib_private.h
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.7 of 27 June 2019
-   Copyright (C) 1996-2010 Julian Seward 
+   bzip2/libbzip2 version 1.0.8 of 13 July 2019
+   Copyright (C) 1996-2019 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
@@ -36,7 +36,7 @@
 
 /*-- General stuff. --*/
 
-#define BZ_VERSION  "1.0.7, 27-Jun-2019"
+#define BZ_VERSION  "1.0.8, 13-Jul-2019"
 
 typedef char            Char;
 typedef unsigned char   Bool;
diff --git a/compress.c b/compress.c
index 76adee6..5dfa002 100644
--- a/compress.c
+++ b/compress.c
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.7 of 27 June 2019
-   Copyright (C) 1996-2010 Julian Seward 
+   bzip2/libbzip2 version 1.0.8 of 13 July 2019
+   Copyright (C) 1996-2019 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/crctable.c b/crctable.c
index 746efac..2b33c25 100644
--- a/crctable.c
+++ b/crctable.c
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.7 of 27 June 2019
-   Copyright (C) 1996-2010 Julian Seward 
+   bzip2/libbzip2 version 1.0.8 of 13 July 2019
+   Copyright (C) 1996-2019 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/decompress.c b/decompress.c
index 3303499..a1a0bac 100644
--- a/decompress.c
+++ b/decompress.c
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.7 of 27 June 2019
-   Copyright (C) 1996-2010 Julian Seward 
+   bzip2/libbzip2 version 1.0.8 of 13 July 2019
+   Copyright (C) 1996-2019 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/entities.xml b/entities.xml
index a415062..dd69924 100644
--- a/entities.xml
+++ b/entities.xml
@@ -2,9 +2,9 @@
 
 
 
-
+
 
-
-
+
+
 
 
diff --git a/format.pl b/format.pl
index 74d7eda..2734ded 100755
--- a/format.pl
+++ b/format.pl
@@ -4,8 +4,8 @@
 # This file is part of bzip2/libbzip2, a program and library for
 # lossless, block-sorting data compression.
 #
-# bzip2/libbzip2 version 1.0.7 of 27 June 2019
-# Copyright (C) 1996-2010 Julian Seward 
+# bzip2/libbzip2 version 1.0.8 of 13 July 2019
+# Copyright (C) 1996-2019 Julian Seward 
 #
 # Please read the WARNING, DISCLAIMER and PATENTS sections in the 
 # README file.
diff --git a/huffman.c b/huffman.c
index 0fd6fd7..43a1899 100644
--- a/huffman.c
+++ b/huffman.c
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.7 of 27 June 2019
-   Copyright (C) 1996-2010 Julian Seward 
+   bzip2/libbzip2 version 1.0.8 of 13 July 2019
+   Copyright (C) 1996-2019 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/mk251.c b/mk251.c
index 9528b92..6c5bbf9 100644
--- a/mk251.c
+++ b/mk251.c
@@ -9,8 +9,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.7 of 27 June 2019
-   Copyright (C) 1996-2010 Julian Seward 
+   bzip2/libbzip2 version 1.0.8 of 13 July 2019
+   Copyright (C) 1996-2019 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/randtable.c b/randtable.c
index 726d62f..bdc6d4a 100644
--- a/randtable.c
+++ b/randtable.c
@@ -8,8 +8,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.7 of 27 June 2019
-   Copyright (C) 1996-2010 Julian Seward 
+   bzip2/libbzip2 version 1.0.8 of 13 July 2019
+   Copyright (C) 1996-2019 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/spewG.c b/spewG.c
index 320dc85..65d24c8 100644
--- a/spewG.c
+++ b/spewG.c
@@ -13,8 +13,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.7 of 27 June 2019
-   Copyright (C) 1996-2010 Julian Seward 
+   bzip2/libbzip2 version 1.0.8 of 13 July 2019
+   Copyright (C) 1996-2019 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/unzcrash.c b/unzcrash.c
index 6e7c631..c68f93c 100644
--- a/unzcrash.c
+++ b/unzcrash.c
@@ -17,8 +17,8 @@
    This file is part of bzip2/libbzip2, a program and library for
    lossless, block-sorting data compression.
 
-   bzip2/libbzip2 version 1.0.7 of 27 June 2019
-   Copyright (C) 1996-2010 Julian Seward 
+   bzip2/libbzip2 version 1.0.8 of 13 July 2019
+   Copyright (C) 1996-2019 Julian Seward 
 
    Please read the WARNING, DISCLAIMER and PATENTS sections in the 
    README file.
diff --git a/xmlproc.sh b/xmlproc.sh
index ea6e7e6..16fe72b 100755
--- a/xmlproc.sh
+++ b/xmlproc.sh
@@ -5,8 +5,8 @@
 #  This file is part of bzip2/libbzip2, a program and library for
 #  lossless, block-sorting data compression.
 #
-#  bzip2/libbzip2 version 1.0.7 of 27 June 2019
-#  Copyright (C) 1996-2010 Julian Seward 
+#  bzip2/libbzip2 version 1.0.8 of 13 July 2019
+#  Copyright (C) 1996-2019 Julian Seward 
 #
 #  Please read the WARNING, DISCLAIMER and PATENTS sections in the 
 #  README file.
-- 
cgit v1.2.3