bzip3: A BZip2 Successor Trading Memory for Much Tighter Text and Code Compression
bzip3 is a new general-purpose compressor positioned as the spiritual heir to bzip2, built by Kamila Szewczyk and released under LGPLv3. It pairs a Burrows-Wheeler transform (using Ilya Grebnov’s libsais suffix-array implementation) with an order-0 context-mixing entropy coder and an LZP-style pass that combines LZ77 string matching with PPM-style context modeling. The design targets the same sweet spot as its predecessor — highly redundant text and source code — but pushes for both higher ratios and faster throughput.
The author’s benchmark, compressing a tarball of every Perl 5 release ever shipped, illustrates the tradeoffs. bzip3 finishes in roughly seven minutes, comparable to or faster than zstd -16 and far quicker than bzip2 -9 or xz -9, while its larger block sizes deliver strong ratios. The catch is memory: bzip3 runs consumed 12–18 GB of RAM versus bzip2’s 8 MB, so its gains come at a steep resource cost that will matter on constrained systems. A follow-up test layering lrzip long-range deduplication ahead of bzip3 shows the two can be chained effectively.
Performance is heavily compiler- and platform-dependent — clang-built x64 Linux hits around 17 MiB/s compression and 23 MiB/s decompression per thread, while Windows and 32-bit builds lag. The project has been tested across a wide range of architectures (x86, ARM, aarch64, ppc64le, MIPS, SPARC, s390x) and installs via autotools or Homebrew. As with bzip2, the documentation carries a blunt warning that no compressor is provably bug-free and users should keep originals of irreplaceable data.
Read the full article
Continue reading at Hacker News →This is an AI-generated summary. Read the original for the full story.