How to Write Technical Interview Questions That AI Can't Answer

By Vaibhav Devere, Founder, Zero Assist · 2025-02-05 · 6 min read

Why Most Technical Questions Are AI-Solvable

ChatGPT was trained on essentially every public coding challenge, LeetCode problem, and Stack Overflow thread. If your interview question has a canonical answer that exists anywhere on the public internet, an AI tool can produce it.

This does not mean you need to invent entirely novel algorithms. It means you need to combine problem elements in ways that require contextual judgment rather than pattern matching.

Techniques for AI-Resistant Questions

Ground the Problem in Your Actual Stack

Instead of: "Implement a rate limiter." Try: "We have a Redis-backed rate limiter in our API gateway that is failing silently under high load. Here is the relevant code. What is wrong and how would you fix it?"

An AI tool can describe rate limiter concepts. A candidate who understands the code can diagnose the specific failure.

Require Constraint Trade-Off Decisions

Instead of: "Design a URL shortener." Try: "Design a URL shortener that needs to handle 1M writes per day but where reads must have P99 latency under 50ms and storage costs must stay under $200/month. What do you give up?"

This question requires reasoning about specific constraints that do not have a single correct answer. AI tools produce generic designs; this requires a real decision.

Ask About Failure

Instead of: "How does your solution handle errors?" Try: "Walk me through the failure mode where your approach breaks under realistic production conditions. What would you monitor to detect it before customers notice?"

Failure reasoning is a domain where genuine experience is irreplaceable. AI tools describe happy paths fluently but are weak on battle-tested failure intuition.

Use Your Codebase Directly

Share a real (anonymized) piece of code from your production system. Ask the candidate to:

  • Identify a performance issue
  • Extend it with a new feature
  • Explain a design decision that looks wrong but was intentional

This cannot be prepared for in advance. It requires genuine understanding of the code in front of them.

The Follow-Up Is Everything

Even a well-designed question can be defeated with enough AI preparation. The follow-up cannot be. After any answer:

  1. Ask why they chose that approach over the obvious alternative
  2. Ask what would need to change if [constraint X] were different
  3. Ask them to predict the behavior of a specific edge case you introduce on the spot

Someone who received AI assistance will be working from a mental model they do not own. The follow-ups expose it quickly.