HR Tech & Recruiting LLM Archived

Why can't a hiring manager tell if a dev can still code when they only direct agents?

Most developer candidates now say they direct coding agents instead of writing code, and interviewers have no format that tells them whether the person can reason about a system or would be lost when the agent is wrong.

Who has it: Engineering managers and tech leads running developer interviews at small and mid-size companies.

7.8Score / 10

Severity8/10
Feasible today8/10
Market signal8/10
Competition gap7/10

Heard in public

“about 80% of the dev candidates that I interview tell me that they aren't writing much code themselves anymore”

Quoted word for word from the public post linked below. Nobody submitted it to Real Problem AI.

Ask HN thread on interviewing developers in a post-AI world, 19 September 2026, 47 points and 43 comments.

What nothing does yet

An interview harness where the candidate works with a real coding agent on a seeded task, the agent is deliberately wrong at planted points, and the recording is scored on whether the candidate caught the errors, explained trade-offs and understood the result.

Tools that solve this

4 named so far, and where each one stops.

HackerRank and CodeSignal
Timed coding tests built for the pre-agent era, easy to pass with an assistant and weak on judgment.
Karat
Outsourced live interviews, still centred on classic algorithm questions.
CoderPad
A shared editor for live coding, with no structure for assessing how a candidate steers and corrects an agent.
Paid work trials
The most trusted signal in the thread, and too slow and costly to run for every candidate.

Why it is archived

replaced by a fresher problem

How you might build it

01Sandboxed repo with seeded bugs
02Instrumented coding agent session recording
03LLM rubric scoring of transcript and diffs
04Interviewer review dashboard

#N29B · Canonical URL: https://www.realproblem.ai/archive/why-can-t-a-hiring-manager-tell-if-a-dev-can-still-code-when-they-only-direct-agents