If you have ever uploaded a resume to an applicant tracking system and wondered why a human never saw it, you already un

If you have ever uploaded a resume to an applicant tracking system and wondered why a human never saw it, you already understand the black-box problem of AI hiring. Most resume-scoring tools hide their logic behind SaaS dashboards and polite rejection emails. HackerRank took a different route. Its Hiring Agent is open source, which means anyone can crack it open, trace the code, and see exactly how an LLM turns a PDF and a GitHub link into a number. One developer did exactly that. What they found is not a polished recruiting framework. It is a mirror showing us how easily automation codifies personal opinions.

Under the Hood

The pipeline is deceptively simple. A candidate’s PDF resume is converted to Markdown, then parsed into a rigid JSON structure with fields for work history, skills, education, and side projects. Python scripts shuttle the data from one stage to the next, but the actual thinking happens inside a chain of prompts. Each section gets its own prompt. The LLM reads the structured data, applies scoring rules written in plain English, and returns a grade.

This architecture matters. The heavy lifting is not happening in clever algorithms or training loops. It is happening in the wording of the prompts. Change a few adjectives in the instruction set, and the same engineer goes from a strong hire to a weak candidate. That makes the tool fragile. It also makes it honest. Most AI hiring vendors would never let you see the prompts. HackerRank’s prototype exposes the truth that resume scoring has always been about the rubric, not the code.

The Tyranny of the 35 Percent

The most striking bias hides in the scoring rubric. Open source contributions account for 35 percent of the total score. That is an enormous weight. To put it in perspective, a candidate’s entire work history, education, and skill set must compete with one slice of their extracurricular coding life for the other 65 percent.

The rules are even stricter than the weighting suggests. Personal GitHub repositories do not count. Maintaining your own library, no matter how useful, scores zero. The tool only rewards contributions to other people’s projects. The candidate must be a committer on someone else’s codebase to earn those points.

That preference carries real demographic weight. Engineers who maintain their own tools often do so because they solved a problem no one else was solving. They might also hold jobs that forbid external contribution, work in regions with fewer large open-source communities, or simply have family obligations that make unpaid coding after hours impossible. By writing the prompt this way, the tool does not measure raw engineering ability. It measures participation in a specific coding culture, then calls it objectivity.

When the Instructions Don't Land

The rubric also tries to reward startup experience. The prompt explicitly suggests giving extra points to founders and early-stage engineers. That sounds reasonable in theory. Startup veterans often wear many hats and ship under pressure. So the tester tried an experiment. They took a single resume and changed nothing except the most recent job title, running it through the agent three times with three different labels: Senior Java Engineer, Founding Engineer, and Co-founder / CTO.

The scores barely moved. The LLM essentially ignored the instruction.

This is one of the most important findings from the entire audit. It proves that a prompt rule is only a suggestion. Large language models are trained on vast corpora of text that contain their own stubborn biases about what signals quality. If the model’s training data associates prestige with certain titles, company names, or keywords rather than the phrase “founding engineer,” your carefully written instruction may simply bounce off. The prompt tells the model to care about startup titles, but the model has its own ideas, and it wins. That gap between human intent and machine behavior is dangerous when the output is a hiring score.

Points Without Purpose

Beyond the major weightings, the rubric is full of oddly specific micro-rules that feel less like data-driven decisions and more like someone’s late-night brainstorming session.

A LinkedIn profile is worth exactly one point. Not the quality of the profile. Not the number of recommendations or the depth of the work history. Simply having a URL on the resume adds a single point to the total. Meanwhile, being a Google Summer of Code participant is worth five points. And if a candidate has forked repositories on GitHub, the agent ignores any fork with fewer than five forks of its own.

Each of these rules makes a loud value judgment disguised as a quiet coefficient. Why is LinkedIn presence worth a point at all? It signals that a candidate knows how to fill out a social network, not that they can architect a distributed system. Why is GSoC worth five times as much as a LinkedIn link? Perhaps because the prompt author respects the program. That respect is now a hiring policy. And why draw the line at five forks? A tool with ten users might solve a critical niche problem. Under this system, it might as well not exist.

These numbers do not emerge from regression analysis. They were chosen by individuals. One person decided open source participation is more than a third of an engineer’s worth. Another person decided a LinkedIn profile is worth 1 point. When you automate those guesses, you give them the authority of software.

Every Prompt Is a Prejudice

The hardest part of building a resume-scoring agent is not parsing PDFs or calling an API. It is deciding what matters. Every word in a scoring prompt is a value judgment about what makes a good engineer. Should side projects outweigh day jobs? Should public code matter more than private enterprise work? Should a social media profile matter at all? There are no mathematically correct answers to these questions. There are only cultural preferences.

When a recruiting team does this manually, at least the biases are distributed across many reviewers who can disagree, calibrate, and learn. When an LLM does it, the biases of one prompt engineer harden into a repeatable function that runs at scale. The tool does not eliminate subjectivity. It archives it.

Use It as a Mirror, Not a Filter

HackerRank’s Hiring Agent is best understood as a prototype. It feels like a first draft, which is exactly what it is. It offers a fascinating early look at how AI hiring tools are constructed, but it lacks the calibration, testing, and diverse input of a real recruiting organization.

If you are building hiring tech, study it carefully. It shows how quickly arbitrary rules turn into automated gatekeeping. If you are a candidate,remember that these systems are not oracles. They are spreadsheets dressed up in natural language, and they carry the assumptions of whoever wrote the prompts.

Until these tools are tested for bias as rigorously as the engineers they judge, they should inform human conversation, not replace it.