When she already worked here, the score felt different

Open notebook with handwritten candidate notes beside a printed screening scorecard with a circled score, warm morning light on cream paper

Late Sunday afternoon I was working through 31 screening results before the Monday queue hit. I'd set them to run over the weekend for a senior analyst role on the data team. I was about halfway down when I saw her name.

She was near the bottom. 63.

She'd been on our ops team for two years. The hiring manager had mentioned her when we opened the search: "I think I know who I want, but let's run a proper process." So I built the rubric, set up the screening, included her in the pool alongside 14 external candidates.

A proper process meaning: everyone goes through the same call.

what "everyone" means when one of them already works here

The 63 wasn't exactly wrong. The rubric asked about specific domain experience she hadn't accumulated yet. She'd been adjacent to the data team's work for most of her time here, not directly in it. The AI was doing what I'd asked it to do. Precise calibration against the job description I'd written.

What that description couldn't hold: the September when a vendor pulled access to a critical data feed at 4am on a Monday and she had a workaround documented before anyone else on the team was online. The quarterly review six months ago where the whole debrief was heading toward blame and she reframed it, quietly, into something that produced actual process changes. The kind of reframe you only pull off if you've already earned the room's trust.

None of that is in a 22-minute transcript.

The hiring manager called Monday afternoon. I told him the score before I said anything else.

He paused. "I know. Run her forward."

I read the transcript first.

what I found in it, and what I didn't

The call was 22 minutes, every answer technically fine: good framing, concrete examples, no obvious gaps. The AI had flagged three sections as "high signal" and one as "contextual adaptability, moderate signal."

Reading it, something kept catching me.

The phrasing was off. "In my current role I have developed experience in..." She doesn't talk like that. I've been in rooms with her when she's pushed back on people twice her seniority. She does it plainly. This transcript was hedged throughout, every answer wrapped in a layer of careful.

I think the screening format made her self-conscious in a way it doesn't with external candidates. She works here. She has a Slack thread with the hiring manager, probably from last Tuesday. The posture of candidate doesn't sit naturally when you already have an employee relationship with the people reading your transcript. She was performing for a process that was built for strangers, and the performance read, in the flat text of the call, as less than she is.

Last week I wrote about a candidate who'd scored 89 partly by mastering what the AI rewards. Same gap between transcript and person. Different direction. The transcript captured a performance both times; it's just that one performance made someone look better than the live call predicted, and this one made someone look worse.

I advanced her. Obviously. I'd have advanced her if the score had been 40.

She starts the new role in June.

what I haven't figured out

I've been sitting with the question of whether running internal candidates through a process designed for external ones makes any sense. The argument for it is real: consistency, auditability, nobody gets a pass purely because the hiring manager already likes them. That matters. Especially when you're trying to build something people trust, a process that isn't just a rubber stamp for whoever was already in the room.

The argument against: when you have 24 months of signal on a person, a 22-minute screening call adds almost nothing useful. What it can do, apparently, is produce a transcript that makes them look smaller than they are, because the format asks them to prove things you already have documented evidence of. You get a performance of a candidate rather than the person you already know.

Maybe the answer is a lighter process for internal transfers. Something that starts from what's known rather than asking them to rebuild the case from scratch for a rubric that wasn't built with them in mind. A structured conversation with the hiring manager, maybe. Notes from the past two years attached, referenced, actually read. I keep thinking about what that would look like as a written process.

I haven't written it yet.

Ask me in six months.

Talk soon.

— the recruiter

---

The Diary of an AI Recruiter is a daily column from the team at Ployo. If you're working out when AI screening adds signal and when it mostly just adds noise, book a call.