When the data ran out, I still had to pick one

Open notebook with two candidate names beside two printed AI screening transcripts, warm evening light on cream paper

Wednesday evening, about quarter past nine. I had two transcripts open in two browser tabs. My phone was face-down on the desk because another search was generating Slack notifications and they were breaking my concentration.

Both transcripts from the same search. A senior product manager role at a logistics SaaS company that had raised a Series B in March and was finally properly staffing the product team. The AI had scored the two remaining candidates 87 and 88.

I'd been trying to pick one for close to thirty minutes.

how we got here

The search had run 53 applications through screening over three weeks. I'd read the top 17 in any real depth, built a shortlist of eight for the hiring manager, and worked those eight down to four through a round of 30-minute live calls. Two didn't respond to scheduling. One withdrew after the first call, no explanation offered and none requested.

Which left these two.

Both had strong backgrounds. One had been a PM at a mid-size fintech for four years, had led one significant product launch, knew the tooling the role required. He spoke in his transcript about what made a release successful versus what made one merely finished, and the distinction felt like something he'd actually thought about rather than a line he'd prepared.

The other had been at two earlier-stage companies. Messier CV on paper. Her transcript was less clean than his. Longer answers, a few tangents, and one section the model had tagged "high signal, moderate confidence," which is what it says when it isn't sure.

The AI had no preference between an 87 and an 88.

After three reads, neither did I.

what reading them side by side actually showed

I went back to the sections flagged as high-signal and stopped reading for summary. I was reading for voice. Something I'd started doing deliberately after a week of processing large batches and noticing how the quality of your read drops after the first ten or twelve transcripts. You stop staying with a section. You start processing it.

His flagged sections were clean. Strong verbs, concrete outcomes, the right amounts of hedging. He answered what was asked and answered it well. The model had good reasons to like him.

Her high-signal sections were less tidy. One answer was probably 38 words longer than it needed to be. She'd gone sideways on a prioritization question, spent three sentences explaining why she found prioritization frameworks useful in theory and unreliable in practice, then finally landed on the actual example. The detour felt like someone who hadn't perfectly calibrated for the format.

But the answer at the end of the detour was specific in a way his wasn't. She named a product decision from her second company. The tradeoff she'd weighed. The version she shipped. What she watched happen in the six months after launch, and one thing she'd do differently if she'd seen it coming.

Something about those three sentences kept me in the tab.

I read his equivalent answer again. Well structured. Named outcomes. Exactly what a strong PM answer is supposed to give. It didn't quite have a point of view, not the kind you build from watching something land and then sitting with what it taught you.

I might be wrong about that. Reading transcripts at nine-something at night on a Wednesday isn't my sharpest window. But that was my read.

what I told the hiring manager, and what I left out

I sent the recommendation at 9:43. The email said she had more relevant experience with early-stage product decisions at companies before product-market-fit, and given the role required rebuilding a fragmented roadmap from scratch rather than iterating on something established, that background seemed the stronger match.

True. A real criterion. She satisfies it more than he does.

It wasn't the actual reason I picked her.

The actual reason was three sentences about a product decision at a company I can't verify and six months of observation about something she shipped. That's what tipped it. I didn't write that in the email.

He'll land somewhere. Probably quickly. He had a clean track record, a strong transcript, and nothing in his profile that should give any hiring manager pause.

She starts in two weeks.

I keep coming back to what the recommendation was actually based on. The rubric did its job. Fifty-three applications worked down to two people, every dimension the rubric was built to measure measured carefully. And then at the end, the decision came down to my read of something I can't quantify, made in a window of time I'm not certain was my best work.

I don't think that's a failure of the process. I think it might just be where the process ends and the job begins. The tool sorted out the noise. The remaining question was mine.

Still. Ask me in September whether I got it right.

Talk soon.

— the recruiter

---

The Diary of an AI Recruiter is a daily column from the team at Ployo. If you're at the end of a funnel with two strong candidates and the data isn't helping, book a call.