How we made our MMDB files a 33% smaller

IPinfo reduced its 89 MMDB database files from 91.2 GB to 62.8 GB, a 31% overall saving, without changing their contents or requiring a new file format or reader. The key insight was that conventional MMDB writers duplicate identical parts of the search tree: different IP ranges can lead through the same sequence of branches to the same records. IPinfo’s open-source tool, mmdbshrink, processes the tree from the leaves upward and uses hash-consing to identify and store each identical subtree just once. The data section, which already supports shared records, remains unchanged; the savings come entirely from the tree. Reductions vary by database, reaching 80% for the privacy database and 54% for the location database. In the Lite example, the file shrank by 33.6%, with 41.8% fewer tree nodes.

The optimized files remain standard MMDBs, and lookups return the same results through existing reader libraries. IPinfo checked equivalence by comparing prefixes in both directions, testing millions of random IPv4 and IPv6 addresses, and running established verification tools and 12 reader SDKs; the tests found no mismatches. Lookup speed was unchanged or slightly better, while Lite’s reported memory footprint fell from 36.02 MiB to 23.92 MiB. The smaller files are now used across IPinfo’s API traffic and database downloads. Because the optimization applies to MMDB files generally, IPinfo released the tool under the Apache 2.0 license for others to use; database customers need take no action.