https://jason-grey.com/tags/compression/Recent content in Compression on jason greyHugoenFri, 15 Nov 2024 00:00:00 +0000https://jason-grey.com/posts/2024/gzinspector/Fri, 15 Nov 2024 00:00:00 +0000https://jason-grey.com/posts/2024/gzinspector/<p>I published an open source tool &ldquo;<a href="https://github.com/jt55401/gzinspector" class="external-link" target="_blank" rel="noopener">gzinspector</a>&rdquo; to inspect gzip streams - specifically those encoded with many chunks.</p> <blockquote> <p>A robust command-line tool for inspecting and analyzing GZIP/ZLIB compressed files. GZInspector provides detailed information about compression chunks, headers, and content previews with support for both human-readable and JSON output formats.</p> </blockquote> <p>I did this due to the work I&rsquo;ve been doing for <a href="https://commoncrawl.org/" class="external-link" target="_blank" rel="noopener">CommonCrawl</a> - specifically around processing &ldquo;<a href="https://github.com/webrecorder/pywb/wiki/CDX-Index-Format#zipnum-sharded-cdx" class="external-link" target="_blank" rel="noopener">ZipNum</a>&rdquo; format <a href="https://github.com/webrecorder/cdxj-indexer" class="external-link" target="_blank" rel="noopener">CDXJ</a> indexes.</p> <p>If you find it useful, let me know (reach out or star it on github.)</p>