Assessment Design Considerations

When designing assessments the following principles should be considered:

Validity

Does the assessment task assess what you want it to assess?  If students are asked to ‘evaluate’ or ‘analyse’ are these skills going to be assessed or are they able to provide a perfect answer by regurgitating lecture material? 

Validity ensures that assessment tasks and the assessment criteria effectively measure the student’s attainment of the intended learning outcomes at an appropriate level.

Reliability

Total reliability of a particular assessment would mean that different assessors using the same assessment criteria and mark scheme would arrive at the same results.  This may be the case in some quantitative assessments.  Complete objectivity is otherwise hard to achieve.  With summative assessment it is, however, necessary that we aim for the goal of complete objectivity.  This means that there need to be explicit intended learning outcomes and assessment criteria.  Students should have access to them when the assessment task is set.  Where there are multiple markers they should be discussed.  In an ideal world they should be ‘tested’ on a sample of cases to ensure that all markers are applying the criteria consistently. 

Moderation and/or double marking are means of ensuring consistency between markers and internal consistency for an individual marker. 

Transparency

It is important that all those involved in an assessment – students, tutors, external examiners – receive clear, accurate, consistent and timely information on the assessment tasks and procedures.  Are they aware of the purpose of the assessment; the associated assessment criteria; and the assessment regulations?  Do students receive detailed briefs on the task(s)?

Inclusive and equitable

Assessment tasks should be designed to ensure that individuals or groups are not disadvantaged.  Questions to ask include:

Do tasks limit or unfairly benefit a particular group?  For example, does one group have an advantage over another because of work previously done?

Is a task accessible by all regardless of their physical abilities? 

Are different learning styles accommodated across a programme? 

Relevant

Academic assessment should be about assessing both knowledge and skills.  When devising assessment tasks it is important that it addresses the skills you want the student to develop.  In addition, they should be set in a context that is seen as having ‘real purpose’ behind the task and that there is a sense of a ‘real audience’ – one beyond the tutor – for whom the task would be done.

Manageable

The scheduling of assignments and the amount of assessed work required should provide a reliable and valid profile of achievement without overloading staff or students. 

Practicable

Can the task(s) be done in the time available?  Can the task(s) be achieved within existing constraints such as student numbers, accommodation facilities etc?  Are the tasks achievable by the students at their level of study?

It is important that the overall workload is examined from the point of view of both staff and students.  Does all the work come at the end of the modules?  Are students over assessed?  Is it necessary for each intended learning outcome to be assessed separately? 

Range of assessment methods

We all have learning style preferences.  Equally, we have preferred ways to communicate our learning.  Are students exposed to a range of assessment methods across their programme?  Do they have opportunities to practise a new assessment method before a summative assessment?

Assessment criteria

It is important that students are aware of the criteria against which their work will be judged.  This is part of transparency.  Are students able to use the criteria to judge their own work?  Are they involved in the formulation of assessment criteria?

Design out plagiarism

As well as ensuring that students are aware of the consequences of academic misconduct it is important that they understand fully what academic misconduct is.  Engaging students in discussion about what is considered to be plagiarism.  This should happen early in a student's career.  As well as defining the term and exploring the ethical issues involved, it is important that students know how to paraphrase and summarise what they read and reference correctly.  Many Schools set out guidelines for students to follow for referencing their work.  The Learning Development Service also provides assistance with essay/report writing and referencing.

Other ways of engaging students with good practice are:

  • to provide students with examples of good and poor practice
  • to ask students to write a short (500 word) formative essay on a topic early in the course and submit it to Turnitin software so that they can see the extent of work they may have plagiarised.

It is also good practice to set assignments which minimise the opportunities for plagiarism.  Ways to do this are:

  • changing assignments and /or data yearly to prevent work being passed from one group to another.  This does not, however, remove the problem of 'cutting and pasting' or collusion.  
  • requiring students to submit their work electronically through Turnitin software