I Replaced JSON With a Custom Binary Format in PHP
We need to store rich text for web and mobile apps.
For years, we used pure HTML in our database. Later, we switched to JSON to separate editing from printing.
JSON became a problem as our data grew. To change a single link, we had to parse, rebuild, and stringify the entire array. This process was slow and inefficient.
I decided to build a custom binary format using PHP.
Here is why I made the switch:
- Speed: Parsing and searching became much faster.
- Control: I can manipulate specific bytes without loading everything into memory.
- Structure: I moved from a nested tree to a flat list of elements.
Most rich text tasks do not need a complex tree. You often just need to find text or specific tags. A flat structure of elements with a simple tree of offsets works better.
PHP is not a low-level language. It is not optimized for byte manipulation like Zig or C. However, PHP has the right tools:
- pack() turns numbers or strings into raw bytes.
- unpack() turns those bytes back into usable data.
I stopped using multibyte string functions. Instead, I read specific chunks of bytes. For example, an unsigned 64-bit integer requires exactly 8 bytes.
The results after 10,000 loops:
Old JSON encoding: 2.18s Old JSON decoding: 0.86s New Binary encoding: 1.19s New Binary decoding: 0.67s
The binary format is faster. It is also larger. It is 2x the size of JSON and 3x the size of HTML. Since we use server-side rendering, this does not matter to us.
If you build a custom format, follow one rule: write a specification. Document your format clearly. You will regret it if you do not.
It took one week of work. It was worth it.
Source: https://dev.to/tomj/i-replaced-json-with-a-custom-binary-format-in-php-mok