How will technical interviews (coding, system design) change in the world of Ai?
(3 Min Read) The New Interview Format thats compatible with Ai
As hiring managers, we may have questions about what to do if the interview candidate uses AI. Is it ethical? Should it be allowed? What does it say about the quality of the candidate?
In my current view (as of writing), AI is a tool. If you allow using other tools in your interview loop and the usage of those tools does not raise concerns about candidates' ability to perform the job, then AI should be allowed, and in fact encouraged, and made part of the interview loop.
What are some other tools we use in Interview today? Code collaborators, IDEs, syntax/regex checker, formatting, etc. So why not Ai?
Some may say that AI can actually give the answer, but other tools can't.
That's a fair point. But this means that we need to change the interview format and how interviews are conducted in the world of AI rather than blocking AI.
How do we conduct interviews in the world of AI?
One way to think about this is to merge coding and system design into a single interview. At the end of the day, in technical interviews, we are trying to figure out if the candidate can build a system that solves our business problem and code it up. Today, we do that in two separate interviews; with AI, we can do it in one. Here's how.
Instead of asking someone to share how they would code up a binary tree or design an automated cron system on a whiteboard, we ask them to build the live working app.
Imagine this new interview format.
"Hey, candidate. Welcome to the Interview. We have 1 hour. I want you to build a working and live website to collect leads for an event that signups drivers for Lyft. You can use whatever AI tools you want, such as Replit, Cursor, ChatGPT, Claude, etc., and use whatever language you want. We want to understand how you can quickly go from problem to prototype solution. In the end, we will review the design together, understand the tradeoffs you made, your understanding of code that your collaborator Ai wrote, and the system you built."
As a hiring manager, this will give you far better signals in far less time on the candidates' ability to:
If they know and can use AI in their workflow because that's what your company is using.
How they think about system design as thats what their prompts will be.
How do they debug the issue when Ai doesn't do what they expect it to do?
Can they explain what AI did at a high level?
Do they understand the end-to-end lifecycle of what it takes to ship a product?
Based on questions they asked about the system, were they able to reduce the problem's scope to complete the project in 1 hour?
How close did they get to shipping the app?
Where do they think the current system bottlenecks are?
How would they change the design if this were across US and international markets?
Did they think about observability (as in, did they ask the AI in the prompt?)
And so much more
In my view, companies and hiring managers are needlessly worrying if AI is breaking the interview process. We all know that the interview process is NOT perfect. We need to update it.
I personally would like to spend more time understanding whether candidates are a culture fit, how they think about their growth, why they believe this company is the right company to support them in their career, etc., and less time identifying whether they can code and build systems.
Yes, being able to code and build systems is a critical part of software engineering. However, I have never managed an IC because they could not code or build systems. However, I have managed ICs/EMs that were not a cultural/team/org/company fit.
If this was helpful, I'm happy to chat about what other people think on this topic.