ziptastic

One self-contained binary that compresses a game asset tree 20–36% smaller than xz -9, and tells you when it cannot. It recognises the container and the payload — a ZIP of PNGs, a raw CD image, a spreadsheet, a packed asset tree — undoes the compression already there, and re-compresses what is underneath. No external processes, no network, no model files.

−35.7%against xz -9e on a mixed game-asset tree, 6.3 MB, byte-exact round trip

−29.6%on a raw CD image, because the 276 bytes of Reed-Solomon parity per sector are regenerated instead of stored

0.3–0.6MB/s at -9, against xz's 1–2. The ratio is bought with time, deliberately

Source Every measured number Releases

A diagram of the compression pipeline: input, parser, backends, one stream

What it is actually good at

Measured on real content, each row byte-exact on the round trip. Format-aware transforms under a strong backend is the whole idea; where no transform fires it is level with xz, and that is in the table too.

case	ziptastic -9	xz -9e	vs xz
Mixed game-asset tree, 6.3 MB, as `.tar`	1,275,048	1,983,636	−35.7%
The same tree as a `.zip`	1,284,836	1,610,228	−20.2%
6.8 MB spreadsheet sheet XML	70,090	163,656	−57.2%
4.7 MB raw CD image (Mode 2, with ECC)	1,417,452	2,014,224	−29.6%
2 MB English prose	408,421	543,572	−24.9%
4.8 MB cooked ISO	3,333,904	3,339,112	−0.2%
1.4 MB floppy image	1,257,830	1,257,792	+0.0%

The last two rows are the honest half: on data with neither a format to parse nor redundancy a general compressor misses, this is an LZMA-class compressor with more machinery and no advantage. Speed is not in this table because it does not flatter us — see the claim above.

What it does, in one transcript

A native binary cannot run in a browser, so this is a real transcript rather than a live demo — copied from a run on this machine, not written by hand.

$ ziptastic c -9 assets.zip
assets.zip  2,797,544 -> 1,284,836   -54.1%   layers=0x40  (1.2 s)

$ ziptastic d assets.zip.dr
assets.zip  1,284,836 -> 2,797,544   identical to the original

$ ziptastic c -9 --split-size 100M big.iso
big.iso  ->  big.iso.dr.001  big.iso.dr.002  big.iso.dr.003

$ ziptastic d big.iso.dr.002          # any part joins its siblings
big.iso  restored

$ ziptastic info blocks.dr
blocks         3
block table    verified against the manifest digest, 204 bytes read of 3145932
block access   each block decodes on its own (--parallel)
  #    offset       csize      usize      mode
  0    0x00000016 1048576    1048576    RAW
  1    0x0010003d 1048576    1048576    RAW
  2    0x00200064 1048576    1048576    RAW

$ ziptastic d --block 1 blocks.dr      # one block, without reading the other two
blocks 1..1 of 3 -> 1048576 bytes (verified)

Addressable blocks

The trailer carries a table of every block's offset, so a block can be reached without walking the stream: --block N decodes one on its own, --from-block N resumes a decode that stopped. Reading the index touches under 1% of an archive — 0.76% on a 225-block, 1.4 MB one — and the table costs 0.01–0.13% of an archive to store. It needs --parallel blocks with no format transform: a block of an archive-mode stream may reference an earlier one, and asking for it alone is refused rather than guessed at.

The extension

.dr. Detection is by the CHCE magic, never by the name: a .dr renamed to .bin still decodes, and a file that is not a CHCE stream is refused. Split parts keep the HJSplit convention — archive.dr.001, archive.dr.002 — so cat and every existing split tool work unchanged.

Build it

cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build build -j
ctest --test-dir build        # 34 suites, ~4 minutes

CMake ≥ 3.25, Ninja and a C++20 compiler. Nothing else — brotli, brunsli, libbsc, zpaq, ppmd, preflate and liblzma are vendored and built by the same CMake run. Linux and Windows; macOS is not supported. The long form, including the traps that cost a debugging session, is in docs/BUILD.md.

What it is made of

PreprocessorsDelta, x86 and base64 transforms chosen per block, before the match finder sees the data.

Dual-resolution match finderLong matches found coarsely, short ones exactly, without a second pass over the input.

Joint optimal parserLiteral and match decisions priced together, so a cheap match never hides an expensive literal.

Per-block entropy backendsRange coding or Huffman per block, whichever the block actually favours.

What it refuses to do

No network calls. No model files to download. No external processes to shell out to. One binary in, one stream out — which is why it can be dropped into an offline pipeline without a supply chain to audit.