What a take-home is actually for

Every hiring conversation I've had this year eventually arrives at the same worry. Candidates have agents now, so how do you stop them using one on the take-home?

I think that's the wrong thing to be anxious about, and the anxiety is diagnostic. If a model can complete your assignment and you can't tell, your assignment was measuring whether the candidate could produce a specific artifact. That was never the interesting question, and it's been a weak signal for a lot longer than agents have existed.

I like take-homes. I want to defend them, which means being honest about the version of them that deserves the bad reputation.

They were a real improvement

Start with what the take-home replaced.

The whiteboard interview asked you to solve an algorithms problem, standing up, in front of strangers, with no references, while narrating your thought process. Nothing about that resembles the job. It measured composure under artificial pressure and how recently you'd studied, and it filtered hard for people who interview well rather than people who work well.

Against that, "go build something small and we'll talk about it" was a genuine improvement. You work in your own environment, with your own tools, at your own pace, on something that at least rhymes with real work. The people who pushed for take-homes in the first place were right.

Then the format got abused, and it got abused in a specific way that has nothing to do with AI.

Scope is the actual problem

The failure mode is that "a couple of hours" quietly becomes a weekend.

You get an assignment estimated at four hours. It has six requirements, three of which are underspecified, and a rubric that mentions test coverage and documentation. Doing it properly takes twelve. Doing it in four means submitting something you know is thin, and you have no idea whether the reviewer will read it as pragmatic scoping or as sloppy work. So you spend the twelve, because you want the job, and because the person on the other side has given you no way to calibrate.

That's the real complaint, and it predates every model. It's the same asymmetry I keep coming back to with hiring. The candidate absorbs a cost the company would never pay in the other direction. A company that asks for twelve unpaid hours is spending twenty minutes reviewing the result. Nobody at that company is going to work a weekend to win the right to interview you.

Two rules fix most of this, and neither is complicated.

Cap it honestly. Say two hours, mean two hours, and design something that fits. If you catch yourself writing "this shouldn't take more than four hours" about something you know takes eight, you already know.

Pay for it past a threshold. Anything beyond a couple of hours should come with money attached. The amount matters less than the signal, which is that you understand you're asking for something. I've never once resented a paid exercise, however long.

Neither of those has anything to do with AI. Get them wrong and your process is bad regardless of what tools the candidate uses.

Two shapes, and only one of them survives

Take-homes come in two varieties that get discussed as though they're the same thing.

The first hands you a repository. Clone it, install the dependencies, implement the missing methods, make the existing test suite pass. Everything is scaffolded. The stack is chosen, the architecture is chosen, the interfaces are chosen. Your job is to fill in the blanks correctly.

The second hands you a paragraph. Build a service that does this. Your stack, your structure, your call on what's in scope. Here's what we care about, here's roughly how long we expect you to spend, show us what you'd do.

The first one is easier to grade, which is why it's popular. It's also the one that falls apart completely the moment the candidate has an agent, because filling in blanks against a known test suite is precisely the task these tools are best at. You are grading an artifact that can be generated, and the generation is indistinguishable from the real thing because it is the real thing. The test passes either way.

The second one holds up. When the assignment is "make a set of choices," using a model to help make them is what you'd do on the job on Monday. The work being evaluated is the judgment, and the judgment is still yours. A candidate who used an agent well and can tell me why they structured it that way has demonstrated the thing I actually wanted to know.

One more mark against the clone-this-repo format. An unsolicited repository you are expected to clone and run without reading is also, structurally, how a certain category of fake-recruiter attack gets delivered. I've written about that elsewhere and I don't want to make it the whole argument here, because the format has plenty of problems that don't require anyone to be malicious. But if you're choosing between the two shapes anyway, it's another mark against the one that was already losing.

The conversation is the interview

Everything above is setup for this part.

Treat the submission as material rather than as the answer. The evaluation is the conversation you have about it.

Sit down for forty-five minutes and ask why. Why this structure. What did you leave out, and how did you decide. Where does this fall over if traffic goes up tenfold. What would you do with another week. What's the worst part of what you submitted, and why is it still in there.

Nobody bluffs their way through that, and you never have to detect anything to make it work. The questions are about a set of decisions, and you can only defend decisions you understand. Somebody who prompted their way to a working submission without following along will stall on the second question. Somebody who used every tool available and stayed engaged will have opinions about all of them, including opinions about what the tool got wrong.

That conversation is also a much better predictor of what working with the person is like. I have never learned anything useful about a colleague from whether their test suite went green. I learn a lot from watching them explain a tradeoff.

And it's cheap. Forty-five minutes of your time against two hours of theirs is a defensible ratio, which is more than most processes can claim.

Give me a paragraph, not a repository

Tell me honestly how long you expect it to take, and mean it. Pay me if it runs long. Then get on a call and grill me about what I built, because that's the part where you'll learn something.

If a candidate can produce a good submission with an agent and then defend every decision in it, they have shown me exactly what I want to see, which is somebody who uses the tools well and understands what came out. That's the job now. Screening against it is screening for the wrong century.

The take-home is fine. The unpaid weekend building someone else's scaffolding was always the problem, and we noticed for the wrong reason.