Why can't a hiring manager tell if a dev can still code when they only direct agents?
Most developer candidates now say they direct coding agents instead of writing code, and interviewers have no format that tells them whether the person can reason about a system or would be lost when the agent is wrong.
Category: HR Tech & Recruiting · Trend: LLM · Opportunity score: 7.8 / 10
What is the “Why can't a hiring manager tell if a dev can still code when they only direct agents?” problem in 2026?
Most developer candidates now say they direct coding agents instead of writing code, and interviewers have no format that tells them whether the person can reason about a system or would be lost when the agent is wrong.
Who has this problem?
Engineering managers and tech leads running developer interviews at small and mid-size companies.
Recorded source context
Dataset source note: about 80% of the dev candidates that I interview tell me that they aren't writing much code themselves anymore
This note may summarize the referenced material rather than quote it verbatim. Source label: Ask HN thread on interviewing developers in a post-AI world, 19 September 2026, 47 points and 43 comments. (primary source).
Existing players in this space
- HackerRank and CodeSignal: Timed coding tests built for the pre-agent era, easy to pass with an assistant and weak on judgment.
- Karat: Outsourced live interviews, still centred on classic algorithm questions.
- CoderPad: A shared editor for live coding, with no structure for assessing how a candidate steers and corrects an agent.
- Paid work trials: The most trusted signal in the thread, and too slow and costly to run for every candidate.
What existing players are missing
An interview harness where the candidate works with a real coding agent on a seeded task, the agent is deliberately wrong at planted points, and the recording is scored on whether the candidate caught the errors, explained trade-offs and understood the result.
How Real Problem AI scores this opportunity
Aggregate score: 7.8 / 10. Four-axis rubric:
- Problem severity: 8 / 10
- AI feasibility today: 8 / 10
- Market signal: 8 / 10
- Competition gap: 7 / 10
How to build a solution: stack hints
- Sandboxed repo with seeded bugs
- Instrumented coding agent session recording
- LLM rubric scoring of transcript and diffs
- Interviewer review dashboard
Related HR Tech & Recruiting problems on Real Problem AI
- Why does every job application want me to retype my resume into 14 boxes? (8.5/10)
- Why can't a 200-person company tell if their final-round candidate is a deepfake proxy? (8.5/10)
- Why is one missed work-visa date a $2,500 ICE fine for HR sitting in a spreadsheet? (8.5/10)
- Why does an AI screener reject me in 12 seconds and tell me nothing about why? (8.4/10)
- Why does an ADA accommodation request still depend on someone forwarding an email to four different stovepipe inboxes? (8.3/10)