I started writing candidate notes the way the AI does

Open notebook with handwritten candidate notes beside a printed AI scorecard, warm morning light on cream paper

Three weeks ago, I was typing up candidate summaries for the hiring manager on a senior ops role we'd been trying to fill since February. It was nearly 10pm. She'd asked for my read on the final four before our call the next morning.

I wrote up the first candidate and stopped.

The note read: "Strong domain knowledge. Clear under pressure, particularly when the question shifted. Weaker on stakeholder complexity. Handles familiar teams well but hesitates when the org structure is new."

I looked at it. Something felt off and I couldn't place what. Then I recognised it. That was the shape of the AI's scoring breakdowns. Not the same words. The same architecture. Signal, communication observation, gap. I'd been reading AI-generated candidate summaries for about seven months at that point, and somewhere in those seven months the structure had migrated into how I think.

I didn't know what to do with that.

when I went back and looked

The next day I pulled two months of the notes I write before reading any AI assessments. I do this deliberately, because reading how patterns in transcripts start to change what you notice in a screening made me want to keep at least one pre-AI layer in my process. Write my own read first. Then look at the model's scores. Keep them separate.

The separation, it turned out, had been leaking.

About 40% of those pre-AI notes had the same structure as the AI output. Signal-first, gap-last. The compressed final sentence. I'd even started using "domain strength" in my own notes, which is not a phrase I would have reached for eighteen months ago.

I went back further. Notes from fourteen months back, before we were running AI screening across the full pipeline. Different. Looser. Some started with impressions rather than categories: "feels like the last strong hire we made for this role" or "something off about the pace, answers were fine but never quite landed." Subjective. Sometimes wrong. But a different kind of thinking.

I'm not sure the older notes were more useful. But they were mine.

the question I can't answer yet

What I keep circling back to is whether I've learned something or been shaped by something.

When I now look at a candidate and mentally sort them into signal, communication, gap, is that because I've absorbed a framework that's genuinely cleaner? Or because I've absorbed the tool's lens and now see everything through it?

I agreed with 83% of the AI recommendations in the last cohort we ran. Twenty-three out of twenty-eight candidates. At the time I read that as the model being well-calibrated. Maybe I had it backwards. Maybe the 83% is me being calibrated to the model.

There's a specific type of candidate this worries me about. Someone with a genuinely non-standard path. Not unqualified, just structured differently than the roles the AI has been trained on. My older notes sometimes flagged those candidates. "Unusual background, worth a longer conversation." Recent notes don't do that as often. They score lower on domain fit and move on, because the model scores them lower, and somewhere in the past year I picked up the model's framing without noticing.

If you've ever tried to figure out which of your opinions are actually yours, you know what I mean.

The thing I've been meaning to do, for about three weeks now, is force myself to write raw notes on the first five candidates of every new role before I let the AI touch anything. Run both tracks. Compare them after sixty days. See where they diverge and whether the divergences turn out to matter.

I keep not doing it.

That's probably telling me something.

Talk soon.

— the recruiter

---

The Diary of an AI Recruiter is a daily column from the team at Ployo. If you're thinking about what AI screening does to how your team evaluates candidates, book a call.