Work
Nearly 8,000 Students, Graded Question By Question, Anonymously
The platform that runs Montenegro's national primary school final exam: papers print without a name on them, every written answer is marked twice by examiners who never see whose it is and each student gets the scan of their own handwriting back.
Project snapshot
- Location
- Montenegro
- Services provided
- Development period
- Oct 2025 – Aug 2026
Project
The Examination Centre of Montenegro runs the national final examination that every child takes at the end of primary school. The result decides which secondary school they go to, so it is a legally consequential number attached to a fourteen-year-old, and the Centre has one shot at producing it correctly each session.
We built the software that runs it. Children still sit a paper exam, and the platform owns everything around that paper: it pulls each candidate from the national education register, lets subject commissions author the papers in two languages, lays out and prints the booklets, packs and labels the sealed bags, ingests the scans and cuts every handwritten answer out of the page automatically.
The challenge
The Centre had a fixed national date, a legally binding output and a process that ran largely on paper and judgement. The part that mattered most was also the hardest to defend: an examiner receiving a whole test paper knows whose it is and marks it against their own reading of the answer key. Nobody finds out that their reading drifted from a colleague's until results are already published and a parent lodges an appeal. The Centre needed the whole process digitised, and it needed the result to survive a challenge, which meant an examiner could never know whose paper they were holding.
Roles and permissions
Fourteen roles, and each one sees only the stage it is accountable for.
Fourteen staff roles are modelled in the system, and thirteen are wired to real permissions. They cover the subject adviser who authors a paper, the operator who feeds the scanner and a print shop deliberately split in two, so the people who print exam papers are not the people who can print candidate lists.
The marking screen is where that discipline pays off. An examiner sees one question, the scanned answer, the answer key and the scoring criteria, and can switch to the whole scanned page the answer sits on. Nothing on screen says whose paper it is, because that was never sent. Marking is keyboard-first, and a score outside the permitted range raises an explicit error rather than quietly clamping to the maximum, a behaviour we reversed after watching real examiners work.
Anonymity in the data model
Nobody can leak a name that was never sent.
Papers are printed with no identity on them at all. A candidate is bound to a paper only afterwards, by a sticker carrying a seven-digit candidate number, never the national ID. From there, identity stays on one side of an anonymity line throughout live marking: the paper crosses to the examiners, the name does not. The marking payload carries no student name, surname, national ID or register number, and the query behind it holds no relation to the student record at all, so an examiner cannot deanonymise a paper even by opening developer tools. The identity was never serialised into the response. Marks and names meet again once, when results are generated.
When a question is taller than a page
- Split by document structure, never by the browser
- Every part gets its own barcode
- Marking shows the parts in order
The physical production chain
A question never lands on a page without its own barcode.
The layout engine renders every question, measures its true height in a headless browser at the real content width, then packs questions against a page budget, splitting anything taller than a page by walking the document structure rather than letting the browser break it wherever it likes. Browser page breaking cannot promise where a question splits, and a part that lands on a page without its own barcode breaks the barcode contract that marking depends on. Every part the engine splits off carries its own barcode, and marking shows the parts in order.
The paper is the message bus
No scanner calibration, because there is nothing to calibrate.
Most systems of this kind hold a template of where each answer sits on the page, then fight the scanner with deskew, affine correction and a per-device profile to make reality match the template. This one has none of that. The print side stamps a barcode on every page and on every question. The scan side finds each answer between the bottom edge of its own barcode and the top edge of the next one and cuts it along the question's printed border, all measured on the scanned image itself at the moment it is read. A stack of sheets can come back in any order, from any commodity duplex scanner, and still reassemble correctly.
Test creation and management
Papers are written, scored and locked in the browser.
Subject advisers build each paper in the platform. A test is tied to its subject in the national register and carries its own cover pages and points-to-grade scale. It moves from draft to final check to finalised, a step only the director or an administrator can take.
Questions come in three types: closed, open and essay. Text blocks sit between them for instructions and reading passages, and the editor handles tables, images and LaTeX for maths. Once a test is finalised its questions lock in the editor. The answer key can still be corrected, but no question or text block can be added, removed, reordered or reworded. Every paper downloads as an A4 PDF, or as A3 for the print shop.
Three marking screens in order: a paper's questions by round, the packages for one question and an examiner marking a single scanned answer.
Two blind examiners, an arbitrator, a commission
Both examiners get the same package, and the system can prove it.
Answers are batched per question rather than per paper: the answers to question seven are split into packages of up to 300, or 100 for an essay, so an examiner marks the same question hundreds of times instead of one paper's worth of different questions. Round two creates no new packages. It reassigns the existing ones to different people, so both examiners work on provably identical material, the same items in the same shuffled order.
What happens next is a short set of rules anyone can audit. Closed questions resolve from round one. Open questions require the two examiners to agree exactly or the answer escalates. Essay questions carry a per-question tolerance the Centre configures itself, and where two marks fall inside it the system takes the higher one, so the benefit of the doubt goes to the child. Anything still disputed goes to a master arbitrator in round three, the only examiner shown both earlier marks side by side. An accepted appeal in round four overrides everything above it.
The name was never sent. An examiner could not leak it if they tried.
A paper every child can sit
The enlarged-font paper is a different paper, carried as data rather than a rendering flag.
Adaptation arrives from the national register as a number against the child, and the system takes it seriously enough to change the artefact. The enlarged-font paper paginates differently, which changes which physical page every question lands on, which changes what the scanner must cut and what the student later sees. The platform carries both layouts for the same paper in one manifest, merged on write so reprinting one variant never erases the other, and marking, examiner calibration and the student portal each pick the right variant per individual paper. A per-student font override rides inside the ordinary bags, while a paper set up as its own enlarged-font test goes out in separate bags.
Marking measured while it is still running
The drift shows up while there is still time to act on it.
A configurable share of answers, never fewer than one per package, is duplicated inside an examiner's own package and shuffled in among their real work. Because marks are keyed on the package item rather than on the answer, the same person can mark the same answer twice in one round and both marks are separately addressable. When the package is submitted the system pairs each duplicate with its twin inside the same transaction, counts a match only when the score and the status both agree and writes two things: an agreement percentage banded into three states and a comparison row for every pair. The second output is the one that matters. An adviser can open any disagreement, put the two markings of the same handwriting side by side and defend the number to whoever asks.
Scanning
Scanning details
Scanned bags
Exam scanning and readiness
Every bag accounted for before marking starts.
Operators track scanning by test, school and bag, each with the papers expected against the papers scanned. A bag lists every paper it should hold, spares included, and the operator marks any candidate who did not attend or was removed from the exam, so every missing script has a recorded reason. A bad scan is uploaded again. Before anything moves on, a readiness check counts the answers each paper should carry and shows which papers are ready, which can still be completed and which need scanning again. Only then is the set confirmed for marking. An administrator or the director can withdraw that confirmation until marking starts.
Signed in from the state register
A legacy government register that only speaks its own protocol, handled defensively.
Registrations arrive from the national education register over its own SOAP interface, with envelopes hand-built and parsed rather than fought through a generated client, and everything the ministry returns normalised defensively because a single element and a list of one are not the same shape. If the classroom feed fails, the import does not abort. It records the warning and synthesises rooms so the schedule still gets produced. Every candidate gets a seven-digit number that is unique nationally and carries no personal data.
Reporting and exports
Every report is generated for one exam session and downloads as an Excel sheet, a CSV file or a ZIP of PDFs per school. The Centre can compare registered candidates with those who sat each subject, see how many answers each examiner marked and how often they differed from the master examiner, count appeals by subject and outcome and pull every student's results for the reports that go to the ministry.
Raw answer data, down to the option circled on every closed question, goes out as CSV for analysis in the Centre's own tools.
The student sees their own handwriting
The answer they wrote, next to the mark it got.
The results portal opens only once every student's results are ready. A background job builds the whole cohort's results first and has to pass an error budget, so a country's worth of fourteen-year-olds never refreshes a half-ready portal.
Then a student logs in and sees, question by question, the scanned crop of their own handwriting with the points awarded against the maximum. The portal picks the right layout variant for that paper, checks that each image exists and falls back to the full page rather than a broken image. It offers the same crop and full-page toggle the examiners used. A fourteen-year-old can look at what they wrote and the number it received and decide whether they agree.
Identity side
National register
Student record: name, national ID
Candidate binding
Sticker carries a seven-digit candidate number, never the national ID
Printed in three identical copies
Printed exam paper
Goes out with no name on it
Anonymity line: identity stops here
Examiner side, zero identity
Scan and barcode cut
Each answer cut into its own card
Batched per question
Each package holds answers to one question only
Two blind examiners
For each written answer, never told whose it is
Marking payload: 0 identity fields
Arbitrator and rounds
Resolves disputes across three rounds
Student portal
The student sees their own handwriting and the mark
Results generation
Joins the marks back to the student
Where the name stops
Identity lives on one side of the line. Marking lives on the other.
The national register, the student record and the sticker that binds a candidate to a paper all sit on one side. Everything an examiner touches in live marking sits on the other: the scan, the cut answers, the packages, the rounds and a marking payload that carries no identity fields at all. The portal is where a student finally sees their own answer beside the mark it received.
We took the name off the paper. Then we gave the paper back.
Every written answer, checked twice
- Students assessed across the 2026 sessions
- ~8,000
- Rounds a disputed answer can pass through
- 4
- Examiners marking at the same time in the first session
- 140+
- Students on the results portal at once, right after publication
- 7,500
Links
- Websiteiccg.co.me
- FacebookIspitni centar CG
- Instagramispitni.centar
Work with us
Ready to digitise a process that has to survive an appeal?
Whether you're running national exams, public registrations or any process with a fixed date and a legally binding result, we'll help you choose the smallest sensible starting point and a delivery model that fits your team.
Build faster with AI
Our playbook for integrating AI into product design and development workflows.
Download the playbook