Setting Question Papers

Setting question papers is a complex and demanding process which normally starts at least two years before the papers are due to be taken. One principal examiner has responsibility for setting each question paper and its associated mark scheme. Some principal examiners work with a team of question writers but most produce the complete question paper (and associated mark scheme) themselves.

Requirements

When setting a question paper, the principal examiner is required to ensure that the paper covers the appropriate assessment objectives and criteria and subject content as set down in the subject specification. In most cases the principal examiner demonstrates that this has been achieved by means of a setting grid. The principal examiner must also ensure that the level of demand is appropriate, that, where there is a choice of questions, alternative questions make equivalent demands. The principal examiner is also required to check that all instructions and advice to candidates are clear.

First draft

Once the principal examiner has drafted a question paper and mark scheme (that indicates the types of answers that the paper is designed to elicit and how they will be rewarded with marks), the drafts are submitted to the awarding body. They are then reviewed by a reviser, a subject expert who is familiar with the requirements of the specification and its components. The reviser writes a set of comments on the appropriateness of the question paper as a whole, of the individual questions and of the suggested responses in the mark scheme. These comments are sent to the principal examiner who revises the question paper and mark scheme in the light of the reviser's views.

Question paper evaluation committee

It is now time for the question paper evaluation committee (QPEC) to meet to review the full set of draft papers and mark schemes for a particular examination. The members of the QPEC included the chair of examiners for the subject, the chief examiner for the specification, the principal examiners and revisers for all the constituent question papers and possibly other subject experts too. It is serviced by an officer of the awarding body. QPEC members review in turn each draft question paper and mark scheme, together with the relevant reviser's comments. This can be a lengthy process which can take several days, with QPEC members commenting on subject content, level of demand, wording of questions, comparability of questions and papers, and so on.

After the question paper evaluation committee

After the QPEC meeting, the principal examiner again revises the question paper and mark scheme in the light of comments made at the meeting. A revised paper and mark scheme are submitted to the awarding body. Only at this stage are the drafts sent for type-setting by the awarding body officer. Where the paper includes diagrams, maps or other graphics, this can be a highly complex - and highly skilled - process which can take several days. It may also be necessary to get copyright clearance for certain materials included in the question paper.

Scrutineer

Once the question paper has been type-set and proof-read, a scrutineer who has had no previous involvement with the process reviews it. Scrutineers are required to put themselves in the place of a candidate and to work through the question paper and associated mark scheme to ensure that the whole thing works as an assessment instrument. The scrutineer's comments may lead to further revisions to the question paper and/or mark scheme.

Printing

After a series of final checks by principal examiner and chair of examiners, the paper will eventually be sent for print - probably about 18 months after the first drafts were submitted to the awarding body! After a final check on the printed version of the question paper, the papers are stored securely until such time as they are needed for despatch to school and colleges.