𝗠𝘆 𝗕𝗶𝗮𝘀 𝗗𝗲𝘁𝗲𝗰𝘁𝗼𝗿 𝗙𝗼𝘂𝗻𝗱 𝗖𝗵𝗲𝗿𝗿𝘆-𝗣𝗶𝗰𝗸𝗶𝗻𝗴 𝗶𝗻 "𝗡𝗼 𝗜𝗻𝗳𝗼"

I built a project called Biassemble.

The goal is simple. You feed it a story. It asks questions. It flags cognitive biases in how you reason.

I tested it with a story about a woman named Anna. Her story contained only facts. She drank coffee. She caught a bus. She walked to her office. There were no opinions or emotions in the text.

My first prompt had one rule: Detect at least one bias from a list.

This rule forced the system to find something. Even when nothing existed, the system tried to invent a bias.

I updated the prompt to version 1.0.0. I added a requirement for evidence. The system had to cite specific parts of the story to prove a bias existed.

I thought this fixed the problem. It did not.

When I answered "no info" to follow-up questions, the system flagged me for Cherry-Picking. It argued that my refusal to speculate was a deliberate choice to omit data.

The system was technically correct. I did say "no info" four times. It cited my exact words. But the conclusion was wrong. It turned a lack of information into a reasoning error.

Grounding made the citations honest. It did not make the conclusions right.

I released version 1.1.0. I added three new checks:

The results changed. With the same "no info" answers, version 1.1.0 reported no bias. It recognized that my answers were descriptive rather than interpretive.

The difference between the two versions is not the evidence. The difference is how the system treats "I don't know."

One version sees "I don't know" as a sign of bias. The other sees it as a valid answer. You must tell the model how to handle this. It will not decide on its own.

I am now building metrics to test this. I need to know if the system actually improved or if it just learned to ignore this specific story.

I thought I was building a bias detector. Instead, I learned why reasoning systems struggle with "I don't know."

Source: https://dev.to/lemind/my-bias-detector-found-cherry-picking-in-the-answer-no-info-9hf

Optional learning community: https://t.me/GyaanSetuAi