My afternoon reviews ran warmer and I had no explanation

Open notebook showing two pencil-written columns of numbers beside a stack of printed screening transcripts, warm afternoon light on cream paper

Last Monday I had 41 transcripts open before 9am. Two searches running in parallel: a product manager role at a fintech company that had been going for about seven weeks, and a customer success lead at a smaller SaaS business. I'd blocked the full day for review work.

I worked through the first 14 before my 11am call. Made notes, tagged a few for a second look. The usual.

After the call I ran for a coffee. The machine on the third floor was broken again, so I walked down two flights and came back about twenty minutes later than I meant to. Started the second block at 12:40.

what I found when I sorted by time

Tuesday morning I was pulling together notes for both hiring managers when I did something I don't usually do. I put my own gut-check scores and the AI scores in the same spreadsheet, then sorted both by the time I'd reviewed each transcript.

My first 14, reviewed before 11am: my scores averaged about 4.5 points above the AI. I know I run warm compared to the model. That gap was in line with my usual calibration.

The next 17, between 12:40 and roughly 3:30: I was averaging 13 points above the AI. Not on every candidate. But the shift was consistent enough that three people in that window, ones I had already advanced, were sitting at AI scores in the mid-60s. I hadn't gone back to check why at the time. I'd just moved forward.

The last 10, after 4pm: I went the other direction. Two candidates I'd rated noticeably lower than the AI, one by more than 10 points. One of those I'd reviewed at 4:51, which I only know because I timestamp my notes. I had a call at 5 I wasn't looking forward to.

Plotted out, the shape was a curve. Up in the early afternoon, down at the end of the day. The AI's scores across the same window were flat.

the part I keep turning over

The candidates who ended up in my 12:40 batch got a more generous reviewer than the ones I'd worked through at 9am. The ones after 4 got a stricter one. None of them knew. Same rubric, same search.

I've thought about bias in my reviews before, but I was thinking about it differently: which candidate profiles I respond to, which CVs I read more charitably. Those are things I've been trying to account for in my rubric-building for two years. Time of day hadn't occurred to me as a variable.

The AI processed the same 41 transcripts over the same day. Its scores didn't show that curve.

That isn't an argument for trusting it over my own read on a transcript. I found a case last week where the AI scored a candidate at 43 because of short tenures, and I almost didn't look at why those tenures happened. Consistent scoring and correct scoring aren't the same thing.

But I had been treating my manual reviews as a check on the AI. The numbers from Monday suggest it was actually the more consistent reviewer that day. That's not comfortable to sit with.

I can't tell whether my afternoon generosity was calibration from having already worked through 14 candidates, or just that I'd eaten lunch and I was looser. Probably some of both. Maybe neither. I genuinely don't know, and neither would anyone reading my notes.

Not much has changed in how I work. I've started timestamping my manual notes more deliberately. I was doing this somewhat before, but now I'm being exact about it. Not to adjust scores retroactively. Just so that if I come back to a borderline candidate later, I know what version of me formed that first impression.

Whether the pattern holds with a bigger sample, I have no idea. Maybe 41 transcripts isn't enough to say anything meaningful. Maybe it was a Monday thing.

Ask me again in a few weeks.

Talk soon.

— the recruiter

---

The Diary of an AI Recruiter is a daily column from the team at Ployo. If you want to understand where the gaps are in your screening process, book a call.