𝗠𝘆 𝗥𝗘𝗔𝗗𝗠𝗘 𝗦𝗮𝗶𝗱 𝟴𝟬 𝗧𝗼𝗼𝗹𝘀. 𝗠𝘆 𝗖𝗼𝗱𝗲 𝗛𝗮𝗱 𝟵𝟲. 𝗡𝗼𝗯𝗼𝗱𝘆 𝗡𝗼𝘁𝗶𝗰𝗲𝗱 𝗙𝗼𝗿 𝗪𝗲𝗲𝗸𝘀.

I run an open-source project. It uses browser automation for AI agents.

My README said the project had 80 tools. I put that number in the tagline, the navigation, and the social media text.

Last week I ran an audit. I compared my README against my actual code.

The code registered 96 tools.

I had three different truths in one file:

  • The marketing number: 80
  • The documented list: 83
  • The actual code: 96

Thirteen tools were completely undocumented. Users did not know they existed.

The most dangerous part was the gap. Four of those undocumented tools were high-level system tools. They used OS-level events to control the keyboard and mouse. This is the most powerful and sensitive part of my project.

This taught me a hard lesson about documentation drift.

Drift is not random. It is biased. You document the easy, boring stuff. You forget to document the new, powerful, or sensitive tools you ship in a hurry.

If you want to find the riskiest part of a project, do not read the docs. Look at the gap between the docs and the code.

I tried to fix the number. That was a mistake. Fixing the number only treats the symptom.

The problem is manual maintenance. If a human types a fact, that fact will eventually become wrong.

The only real fix is to make it impossible to get the number wrong.

You must derive facts from the source of truth.

My project already had one solution: the smoke test. The test asks the server how many tools it has and checks the count. It does not use a hardcoded number. The test can never drift because it never stores a static number.

Stop trying to be more disciplined with your writing. Discipline fails. Instead, automate your facts:

  • Generate counts using a script at build time.
  • Generate tool lists directly from your code registry.
  • Use humans only for the prose and explanations.

If a fact is derivable, do not type it by hand. It is just drift waiting to happen.

Run this audit on your own project today:

  1. Count the real items in your code.
  2. Count what your documentation claims.
  3. If they disagree, find out what is in that gap.

The items in the gap are usually the most important parts of your project.

Source: https://dev.to/achiya-automation/my-readme-said-80-tools-my-code-had-96-nobody-noticed-for-weeks-1f3e

Optional learning community: https://t.me/GyaanSetuAi