The short answer
Record the change you are investigating, which visitors are relevant and what counts as a suitable response or appointment. Prefer changing one element and note differences in reach and conditions. With few observations, the outcome remains uncertain. Conversations and candidate feedback can suggest explanations, but do not establish that a change causes more hires.
Bring one vacancy. In 30 minutes, we discuss where your recruitment is getting stuck, which approach could fit and your recruitment fee.
Start with a hypothesis, not a feeling
A recruitment test without a hypothesis is not a test – it is guesswork. Write down what you expect to see and why: "If we change the job title from 'Site Preparation Engineer' to 'Project Coordinator Construction', we expect more clicks from candidates with a bachelor's background." That sentence forces clarity before you change anything. It also tells you what you actually learned afterwards.
Change one thing at a time
The classic mistake: rewriting the job ad, switching channels, and broadening the target group all at once. Then you never know what worked. Choose one variable – copy, channel, target group, or timing – and keep everything else the same. A practical example: a contractor changes only the opening sentence of the vacancy. Two weeks later, he compares the number of relevant responses with the two weeks before.
Measure relevant visitors, not just numbers
With few applications, raw numbers are misleading. Five relevant candidates are worth more than fifty who do not fit. Look at the quality of who responds: does their background match the role? Are candidates asking good questions? Those signals tell you more than a conversion rate based on twenty visitors. Qualitative observations are your most important measurement tool on small volumes.
Reading qualitative signals as supplementary data
Ask candidates who drop out why they did so. Ask those who do apply what appealed to them. Structured conversations – where you ask every candidate the same questions – make that feedback comparable. This builds insight without waiting for statistically significant numbers. The structured interviews guide (OPM) provides a practical foundation for consistent interviewing.
Discuss the test and its uncertainty
Red Rocket connects campaigns, the AI recruiter and organisational context with personal follow-up by the employer. Agree which information is needed to assess the test and who reviews candidate quality. Discuss a focused recruitment test for your vacancy. OPM’s method guidance on structured interviews offers background on collecting comparable qualitative information.
Frequently asked questions
How many applications do you need before drawing a conclusion?
There is no universal minimum. The data required depend in part on the starting position and the difference you want to distinguish. A small number of responses does not justify a precise growth claim.
Can you test several channels at once?
Yes, but distinguish a practical channel comparison from a controlled experiment. Keep the vacancy, period and definitions comparable and record differences in audience and conditions.
What if the test does not produce a clear result?
Record what you changed and what remains uncertain. No visible difference does not establish that two approaches work equally well or that the problem lies elsewhere.
