Picture a hypothetical school group with three campuses. Two of them run the Grade 4 benchmark on paper, because the internet is patchy and the children are still young. The third campus sits the same benchmark on tablets. When the results arrive, the tablet campus is a few points behind in reading. The academic director now has to decide whether that campus needs a literacy push.
Before anyone writes an action plan, there is a quieter question to answer.
Is that gap about reading, or about reading on a screen?
So, can schools compare paper and online results fairly? Yes, but only under certain conditions. Paper vs online assessment comparability depends less on the format itself than on everything around it: the same blueprint, the same questions, the same scoring rules and similar testing conditions. When those hold, the comparison is reasonable. When they don’t, the format becomes one more unknown hiding inside the score.
Most Unfair Comparisons Have Little to Do With Paper
When leaders worry about mixed-mode assessment, they usually picture the screen as the problem. In practice, the bigger problems are often more ordinary:
- The paper version was an older form with different questions.
- The online version had a timer and the paper version didn’t.
- Teachers marked handwritten answers against a slightly different rubric.
- One campus allowed calculators and another didn’t.
None of that is a mode effect. It is two different assessments carrying the same name. If the goal is a fair comparison, the first job is to make sure the only thing that changed is how the student answered. This is the same discipline that sits behind any work on cross-school comparability: a common blueprint, common items and common rules.
Where the Format Itself Can Shift a Result
Once the test really is the same, the format can still leave a mark. Assessment specialists call this a mode effect. It tends to show up in particular places rather than evenly across a paper.
Long reading passages. On paper, a child can flip back, underline and see the whole text at once. On a screen they may be scrolling, and some students find it harder to keep their place.
Maths working. Paper gives space to scribble. An on-screen question may offer a small box, or no working space at all.
Extended writing. This matters most for younger students, whose typing is often slower than their handwriting.
Diagrams. A diagram that fits neatly on a printed page can be split awkwardly on a small tablet.
How large these effects are depends on age, subject, question type and how familiar students are with the device. They are not a reason to distrust every online result. They are a reason to look at where a gap appears before deciding what it means.
The Same Report Does Not Mean the Same Measurement
Many schools face the opposite problem first. Paper results sit in spreadsheets or folders and never reach the analytics at all. The youngest learners, formally invigilated exams and low-connectivity campuses quietly drop out of the picture.
Bringing those results into the same system, with the same competency tags and the same growth scale, is a real step forward. It is the practical case for integrating paper and digital assessment.
But putting the data in one place solves a data problem. It does not automatically solve a measurement problem.
If paper and online results arrive in the same report, have we shown that they measure the same thing, or only that they were processed the same way?
That distinction is where careful leaders earn their keep.
Five Checks Before Putting Two Formats Side by Side

- Is it the same assessment? Check for the same blueprint, the same questions or properly equivalent forms, and the same mix of difficulty and DOK levels. If the paper version was built separately, compare with real caution.
- Was it scored the same way? Multiple-choice questions are rarely the issue. Open responses are. Handwritten and typed answers should be judged against the same rubric, and a teacher should be able to see how a score was reached and question it.
- Were the conditions comparable? Look at time allowed, tools permitted, invigilation, and whether students could go back and change answers.
- Where does the gap sit? If an online group trails mainly on long reading or extended writing questions, suspect the format first. If the gap is spread evenly across competencies and question types, it is more likely to reflect learning.
- Who sat which format? If paper is used only for the youngest grades or the campuses with the weakest infrastructure, the format is tangled up with context. A difference between those groups may say more about who the students are than about how they answered. Every format decision is also a comparability decision. The same is true when a school weighs adaptive vs standard assessment.
When the Format Changes Halfway Through a Growth Story
The hardest case is not comparing two campuses. It is comparing a student with their own earlier result.
Suppose a learner sits the beginning-of-year baseline on paper and the end-of-year assessment online. The growth figure looks modest.
When a student moves between a paper baseline and an online follow-up, how much of the change belongs to the student, and how much belongs to the switch in format?
There is no neat formula for separating the two. Where possible, schools can keep a student’s baseline and follow-up in the same format. Where that isn’t possible, they can at least flag records where the format changed, so that nobody reads them without context.
Growth also needs a scale that is built to show movement. Percentages from two different papers rarely do that well, which is why scaled scores for growth comparisons matter here.
A Fair Answer Is a Conditional Yes
Paper and online results can sit side by side honestly. The fairness does not come from the scanner or the screen. It comes from holding the test, the scoring and the conditions steady, and then checking whether the format has left a fingerprint on the result. A school that does both can keep using paper where paper makes sense, and still see every learner in the same picture.
FAQs
Can schools compare paper and online test results fairly?
Yes, if the blueprint, questions, scoring rules and testing conditions are the same in both formats. Even then, leaders should check whether gaps cluster in specific question types, such as long reading or extended writing, before treating a difference as a learning issue.
What is a mode effect in assessment?
A mode effect is a difference in results caused by the format of the test rather than by what students know. For example, a student may score differently on a long passage read on a screen than on the same passage read on paper.
Which questions are most affected by the test format?
Long reading passages, questions that need written working, extended writing and items with large diagrams tend to be the most sensitive. Short multiple-choice questions are usually less affected, though device familiarity can still play a part.
Does scanning paper answer sheets make results comparable?
Scanning puts paper results into the same system and scoring process, and that is necessary for comparison. It does not, on its own, prove that both formats measured the same thing. That still depends on using the same test, the same scoring and similar conditions.
Should a school group move every campus online to make results comparable?
Not necessarily. Paper often suits younger students, formal exams and low-connectivity campuses. A shared blueprint and shared scoring can make mixed formats workable, provided leaders keep track of which students sat which format.
How should schools read growth when a student switches from paper to online?
With caution. Keep the baseline and the follow-up in the same format where possible. Where that isn’t possible, flag the change so the growth figure is read in context. Use a scaled measure rather than raw percentages.