I wanted to see if running the same question 51 times would make the answer more reliable. I took a local LLM, fed it a

I wanted to see if running the same question 51 times would make the answer more reliable. I took a local LLM, fed it a chunk of production code, and asked for a review. Then I did it again. And again. Fifty-one times total, using majority voting to pick the "best" response. The idea was simple: if the model stumbles on one run, maybe the wisdom of the crowd across 51 generations would cancel out the noise and surface the correct analysis. That is not what happened. The experiment showed that majority voting does not select for correctness. It selects for whatever the model is most stubborn about.

This distinction matters because majority voting has become a popular hack in LLM pipelines. The pattern is straightforward. You run the model several times on the same prompt, collect the outputs, and keep the answer that appears most frequently. In fields like medical imaging or spam detection, ensemble methods work because different models, or different views of the data, produce independent errors that actually do cancel out. Large language models are not independent voters. They are one system with one training history, one set of weights, and one universe of biases. When you ask the same model the same question fifty-one times, you are not convening a committee. You are taking a poll of the same respondent under slightly different moods.

The Persistence Problem

The core issue is that LLM errors are rarely random. They are patterns baked into the training data and the architecture. A model that misreads a specific Python decorator in one run is likely to misread it in the next. A model that hallucinates a security vulnerability because the variable name looks like a password will probably hallucinate it again on run forty-seven. The noise you are averaging out is usually surface-level variation in wording or formatting. The underlying reasoning often stays locked in place.

Consider a concrete example. Imagine a function that parses log files using a regular expression. The regex is strict and safe. But the string r'...' contains characters that, in a different context, might invite injection. Ask an LLM to review this. If the model has seen a thousand Stack Overflow posts warning against regex injection in log parsers, it may flag this safe snippet as a risk. Run it once, you get a false positive. Run it 51 times, and there is a good chance you get 51 false positives, or at least a strong majority. The majority vote now entrenches the hallucination. The model is persistent, so the "consensus" is persistent too.

This happens because temperature and sampling tricks do not change what the model knows. They only rearrange how it speaks. A high temperature might make the explanation chatty or terse. It might swap synonyms. It does not suddenly teach the model that the regex is actually harmless. The variations you are voting on are cosmetic. The error is structural.

What 51 Runs Reveal

When I spread those 51 outputs across my desk, the pattern was obvious. The model did not explore 51 different interpretations of the code. It rehearsed the same interpretation in slightly different voices. A handful of runs went off-script, suggesting edge-case fixes or noticing unrelated style issues. But the dominant cluster, the clear majority, kept returning to the same incorrect central claim. That claim was not correct. It was just familiar.

The mathematics of majority voting assumes independent Bernoulli trials. You need unrelated errors for the majority to outperform the individual. In my experiment, the errors were deeply related. They shared the same root cause: the model's training distribution overweights certain coding tropes. The majority vote, therefore, did not reduce error. It amplified the majority bias. It gave a false sense of certainty to a flawed analysis.

This is especially dangerous in code review because developers treat unanimous or near-unanimous AI output as authoritative. A single hesitant suggestion is easy to dismiss. A recommendation that holds steady across fifty-one runs feels like ground truth. It is not. It is ground loop.

Where Voting Actually Works

Nic z tego nie oznacza, że nigdy nie powinieneś uruchamiać modelu więcej niż raz. Głosowanie większościowe może pomóc w ograniczonych sytuacjach, w których zadanie jest powierzchowne, a błędy są całkowicie losowe. Poproszenie modelu o wybór między dwoma formatami składniowymi, wybór konwencji nazywania zmiennych lub wyodrębnienie ciągu daty z linii logu — te zadania o niskiej stawce czasem zyskują na wielokrotnym próbkowaniu. Zmienność to rzeczywisty szum, a szybkie głosowanie pozwala go wyeliminować.

Problemy zaczynają się, gdy zadanie wymaga rozumowania na temat intencji. Czy ta kontrola uwierzytelniania powinna się tutaj znaleźć? Czy to wywołanie asynchroniczne jest bezpieczne? Czy ta kolizja kluczy cache jest faktycznie podatna na wykorzystanie? Te pytania wymagają zrozumienia kontekstu, a nie tylko dopasowywania wzorców. Mechanizm dopasowywania wzorców modelu jest deterministyczny w swoim uprzedzeniu. Model sięgnie po najczęstszą odpowiedź ze swoich danych treningowych, a nie po najdokładniejszą odpowiedź dla Twojej bazy kodu.

Sprytniejsze sposoby na wykorzystanie mocy obliczeniowej

Pięćdziesiąt jeden uruchomień lokalnego modelu kosztuje realny czas i energię elektryczną. Istnieją lepsze sposoby na zainwestowanie tej mocy obliczeniowej. Jeśli chcesz poprawić niezawodność, różnorodność wygrywa z ilością. Uruchom dwa różne modele z