Careers in AI Aren’t Just for Engineers: Building the Workforce Behind AI Systems

Part 1 established that AI performance depends on delivery: the structured execution that keeps annotation quality stable as guidelines shift. 

Part 2 explored how operations teams redesign workflows to enable projects to scale without fragmenting under pressure.

Both stages depend on qualified contributors being ready before launch. How well a pipeline holds under growth comes down to decisions made in sourcing and assessment, long before the first contributor goes live.

Sophie Byfield, our former Talent and Partnership Manager, built the contributor pipelines that keep delivery on track as projects grow.

"Projects cannot scale without the right people. Project managers can have a great plan, a timeline, and a budget prepared, but without our team, it falls flat," says Sophie. 

Sourcing starts long before a project receives official approval.

Once delivery timelines are estimated, the talent team begins a multi-stage process:

• Sourcing and screening: Identifying candidates across specialized communities, platforms, and referral networks.
• Project-specific assessments: Running evaluations tailored to the technical, linguistic, or domain-specific requirements of the project.
• Trial and training: Testing candidates through simulated project work and feedback cycles.
• Qualification: Confirming candidates only after they meet quality benchmarks under real project conditions.

“We usually start building pipelines long before projects officially begin. By launch time, it’s already too late to start sourcing from scratch.”

The team tracks pipeline health by stage and region before a project is approved.

Figure 1. Contributor Pipeline Overview dashboard

A sample dashboard view illustrating how pipeline stages, qualification rates, and regional splits are tracked. Figures shown are for illustration only.

Lead time is what gives delivery teams a functioning pipeline on day one.

Contributor requirements shift as projects expand into new languages or specialized domains. This creates constant pressure to hire quickly without dropping standards.

The Speed vs. Quality Trade-off

On projects with smaller talent pools, contributors who score around 70% on qualification benchmarks may enter training if they show potential for improvement. Leaving roles unfilled slows delivery further when qualified candidates are scarce.

Structured onboarding and feedback cycles help contributors reach project-ready standards before entering live work.

“There’s always a trade-off between speed and quality. It’s something we don’t want to compromise on too much.”

Training does more than raise skills. It's also where Chemin tests whether contributors can adapt fast enough to hit that bar.

Testing for Adaptability

Guidelines and review standards shift after onboarding begins. Contributors who struggle to adjust quickly create quality gaps on live projects.

To identify adaptability before placement, contributors complete three evaluation stages:

• Trial tasks: Measure how contributors respond to feedback and whether performance improves after guidance.
• Simulated project work: Test learning speed and the ability to adjust to tools and changing requirements.
• Project-specific assessments: Evaluate task quality, communication standards, and alignment with project expectations.

During the final training stage, contributors also work on live project tasks under QC review. This gives delivery teams direct visibility into how contributors perform before formal onboarding begins.

Contributors are confirmed on a project only after passing all three stages. As contributor volume grew, the challenge shifted from sourcing people to tracking them.

During the rapid expansion of a large-scale audio AI project, the team onboarded 16 contributor cohorts of 30-50 people across multiple regions.

Delivery teams needed immediate answers:

• How many contributors qualified this week?
• How many contributors will be needed next week?
• Were assessment scores accurately predicting live project performance?

The earlier hiring systems could no longer support that level of coordination.

“We needed better ways to track hiring progress, onboarding status, assessment outcomes, and live project performance across extremely large groups of annotators.”

Figure 2. Hiring Intelligence Center dashboard

Used to track qualification outcomes, cohort performance, and hiring trends as contributor volumes grow across projects. Figures shown are for illustration only.

Across the Southeast Asia cohorts, we qualified more than 250 annotators from roughly 378 final assessment attendees. The average conversion rate was 66%, with the strongest cohort reaching 87%.

Every application originally moved through separate spreadsheets, Google Forms, and manual follow-ups.

Once onboarding cohorts began running simultaneously across multiple regions, teams spent more time coordinating contributor movement than evaluating candidates.

An Applicant Tracking System (ATS) runs the hiring funnel. It tracks each candidate from sourcing through screening, assessment, and qualification, with communications and reporting in one place. A separate in-house database manages the active pool after hiring. It holds each contributor's tier, accuracy, availability, and project history, so the broader annotator pool stays visible beyond hiring alone.

“For projects like this, we built a pipeline of around 13,000 people through the ATS, which would have been impossible to manage manually.”

Expanding into Latin America showed how quickly hiring assumptions built around Southeast Asia break in a new market.

The data annotation market in LATAM is less established than in SEA, so sourcing leaned on relationships. Chemin brought in a local project manager based in the region. LinkedIn outreach and referrals drove most sourcing, and referred candidates often introduced colleagues into the pipeline.

Early conversations surfaced concerns that the SEA team would not have flagged. Payment fees in Brazil ran high, and many professional annotators there freelance across several platforms, so they weighed each opportunity on stability, clear expectations, and fair pay. That shaped how the team communicated from the first message.

A thin pool of C1-level candidates pushed the team to lower the English requirement to B2, then train candidates to the task standard.

Onboarding sessions ran in LATAM time zones, and every candidate received status updates throughout, including those who did not pass the assessments. The pipeline grew from fewer than 10 annotators in the first sourcing round to cohorts of more than 40. Across the project, the team onboarded 82 contributors, with 43 active through delivery.

Transparency from the first touchpoint reduces attrition at high volumes. Timelines, KPIs, and commitment levels are communicated before onboarding begins.

Contributors encounter deliberate exit points after project briefings and assessments. Those who opt out do so before live delivery begins.

“By the time contributors are onboarded, they already understand what they’re signing up for.”

Diagnosing Workflow Bottlenecks

When projects miss targets, teams evaluate problems across three areas:

• Process: Too many approval steps, duplicated workflows, or slow manual handoffs between teams.
• People: Qualification gaps, mismatched contributor profiles, or communication breakdowns after requirements shift.
• Tooling: Manual coordination failing under larger project volumes, requiring automation for outreach and reporting.

Teams review these breakdowns after projects close and apply the findings to the next launch.

Technical hiring moves faster at Chemin because the company already has an established network and reputation among technical candidates.

Project managers and delivery leads are harder to hire. The role calls for someone who is people-focused and can navigate fast-moving operations without a fixed set of guidelines.

Critical thinking is increasingly valued as AI-generated content becomes harder to distinguish from original work. The strongest contributors use AI tools to sharpen their thinking rather than replace it. Keeping an AI project stable at scale is a people problem before it’s a technical one. 

Engineers build the models. Teams in Delivery, Operations, and Talent at Chemin keep projects stable once thousands of contributors enter the workflow.

If you enjoy building structure inside fast-moving environments and solving ambiguity, explore open roles at Chemin.