Demystifying Assessment Criteria

Assessment Criteria serve a number of functions: to render the marking process transparent; to provide clarity about what is being assessed how; to ensure fairness across all submissions; and to provide quality assurance in terms of the subject benchmark statements. While all these reasons are valid and honourable, there are a number of issues at play:

  1. Staff have greater or lesser control of the assessment criteria they are asked to use in marking student work and interpretations of criteria may vary between different staff marking the same assessment.
  2. Assessment criteria are different from standards and the difference between the two must be clearly communicated to students (ie. what is being assessed versus how well a criterion has been met).
  3. Students are often assessment motivated (cf. Worth, 2014) and overemphasis of criteria or overly detailed assessment criteria can lead to a box ticking-type approach.
  4. Conversely, criteria that are too vague or too reliant on tacit subject knowledge can be mystifying and inaccessible to students, especially at the beginning of their degree.

This blog post will not pretend to solve all the issues surrounding assessment criteria but will offer a number of potential strategies staff and departments more widely may employ to demystify assessment criteria, and marking processes, for students. Thus, students become involved in a community of practice, rather than being treated as consumers (cf. Worth, 2014; Molesworth, Scullion & Nixon, 2011). Such activities can roughly be grouped chronologically in terms of happening before, during, or after an assessment.

Before

  • Use assessment criteria to identify goals and outcomes at the beginning of a module, with check-in points in the run up to a deadline.
  • Identify the difficulty in understanding marking criteria. Students are often used to very narrow definitions of success with clear statements that “earn” them points. Combined with a prevalent fear of failure, this can undermine their understanding of the criteria. Additionally, they may feel that they cannot judge their own abilities well in this new context (university). Group discussions not of what criteria mean, but what students understand them to mean, can help identify jargon that requires clarification, allow staff to explain their personal understanding (if they are the marker) and allow students to seek clarification before embarking on an assessment.
  • Highlight the difference between criteria and standards to students (the what and the how well – and how this is distinguished in your discipline).
  • Allocating time to a peer marking exercise using the provided criteria with subsequent group discussion will help students better understand the process.
  • Encouraging students to self mark their work pre-submission using the provided criteria will also help them better understand the process.
  • Using exemplars to illustrate both criteria and standards with concrete examples can be very helpful. This might involve students marking an exemplar in session, with subsequent discussion; annotated exemplars where students gain insights into the marking process; or live feedback sessions where students submit extracts of their work-in-progress that are used (anonymised) to show the whole group the marking process. This then allows for questions and clarification on the judgements a marker makes when working through a submission. Staff may worry that students consider exemplars as “the only right way” to respond to an assessment brief – providing a range of exemplars, especially good ones, can counteract this tendency. Different types of exemplars can be used:
    • ‘Real’ assignments may be best for their inherent complexity (so long as students whose work is used consent to this use and their work is properly anonymised).
    • Constructed exemplars may make assessment qualities more visible.
    • Constructed excerpts (rather than full-length pieces) may be more appropriate when students first learn to look for criteria and how they translate into work as well as allay staff concerns about plagiarism.

During

  • Use the same language: making the links between assessment criteria, subject standards, and university standards clear through using the same terminology in feedback as appears in assessment criteria and subject benchmark statements.
  • Where multiple markers engage with different groups of students on the same assessment, having exemplars to refer to can help ensure clear standards across larger cohorts.

After

  • Refer students back to the assessment criteria and preceding discussions thereof when they engage with feedback and marks.
  • Reiterate the difference between criteria and standards.

Simply providing students with access to assessment criteria is not enough. It is essential that staff identify and clarify the distinction between criteria and standards and demystify the language of assessment criteria by examining tacit subject knowledge staff possess by virtue of experience. Using exemplars and group discussion of these in concretising how criteria and standards translate into a submission will provide students with insights into the marking process that enables them to better understand what they are being asked to do. Lastly, staff should repeatedly encourage students to make use of the availability of assessment criteria while they work on their assessments, which should enable students to feel better prepared and more focussed in their responses.

References:

Molesworth, M., Scullion, R., and Nixon, E. (eds.) (2011) The Marketisation of Higher Education and the Student as Consumer, London: Routledge

Worth, N. (2014) ‘Student-focused Assessment Criteria: Thinking Through Best Practice’, Journal of Geography in Higher Education, 38:3, pp. 361-372; DOI: 10.1080/03098265.2014.919441

Vevox for Students

In the past month, the university has rolled out Vevox and begun training staff in its use. Vevox combines polls, surveys and Q&As in one interactive software and has integrations with Microsoft Teams and PowerPoint. We are glad to see that staff are already adopting this new software in their teaching and want to encourage students to do the same.
Aberystwyth University’s subscription to Vevox comes with Single Sign-On, meaning students can securely log in with their Aber Uni ID and password. Just as there are multiple ways that staff may use Vevox, so students might find it a useful tool.
For example, students could use Vevox polls and the Q&A function when presenting in seminars or workshops, especially in assessments that feature audience engagement as a criterion. Equally, Vevox offers great opportunities for student research, both in terms of analysing and interrogating survey design (e.g. by using Vevox’s existing sample surveys), and for creating and running their own surveys. Further, Vevox might be used in group work, offering students the opportunity to gather ideas and encourage diverse input from more quiet group members. This is especially useful at the moment, where student groups may include members from different households.
These are just a few examples of how students might use Vevox in their learning and we encourage staff to alert students to the fact that they, too, can access this software for free. Our Vevox guides are here (English and Cymraeg) and How-To videos are here (English and Cymraeg). The Learning and Teaching Unit is at hand alongside the Vevox Team to support any technical queries that may arise, providing student with support that is not available when using free alternatives.

How to make asynchronous recordings engaging and interactive

Pre-recorded asynchronous content has become a key factor in delivering courses and enabling the best learning experience for students at Aberystwyth University. There are several strategies that lecturers can use to make these recordings both engaging and interactive.

The benefits of asynchronous pre-recorded lectures are manifold, and most students – as the so-called YouTube generation – know this mode of learning extremely well (Scagnoli, Choo & Tian, 2019). Benefits include that students control their engagement with the content and value the convenience and flexibility that asynchronous recordings provide them with, in particular regarding the pace of their learning, and the repeatability of their engagement (Dale & Pymm, 2009; Ramlogan et al., 2014; Scagnoli, Choo & Tian, 2019). It is therefore essential that staff outline what is expected of students in terms of engaging with learning materials, both in pre-recorded videos and in-person sessions.

Read More

Advice on managing face-to-face and HyFlex teaching successfully

Advice on managing face-to-face teaching successfully:
All staff should strive to maximise the amount of time that students are working back to back or side to side, wherever possible. However, where this is not possible, students may turn to one another, for example for seminar discussion, provided other mitigating practices remain in place (ventilation, masks, social distancing).

A short (10 minute) discussion among students can then be opened up by using interactive technologies such as polling software to allow students to pool their knowledge and begin a plenary discussion, for which all students will face forward again. The majority of in-person sessions should take place with students positioned back to back or side to side.

Please note:
• Any activities in which students face each other should be in very small groups (pairs or groups of three) to minimise the overall volume and ensure everyone can contribute.
• Reminding students of good conversational etiquette, in which people take turns to speak, is essential to minimising the volume of conversations, and thus the projection of aerosol droplets.
• In rooms with fixed and/or tiered seating, such discussion may prove difficult, as students are not permitted to change seats.
• In rooms with mobile seating, the layout of the room must not be changed, and staff must ensure that students maintain social distancing at all times when turning to others.

Advice on managing HyFlex teaching successfully:
• Set expectations clearly: what can student joining remotely expect? Will they be in an observer role? Will they be active participants? What are the limits of remote participation?
• Enable interactive tasks that bring remote and in situ students together, e.g. interactive polls that all can access synchronously
• If numbers are very uneven and the majority of students is present in one mode (e.g. only one student is joining remotely from quarantine), invite in situ students to the online session using their own devices, to enable peer discussion

Digital Insights 2018/19: Digital tools and apps useful for learning

In Digital Insights 2018/19 survey, we asked students to give an example of a digital tool or app they find really useful for learning. We thought we will share some of the examples on our blog.

Access AU core e-learning services

 

Research

  • Endnote – reference management software (free to download for AU students and staff)
  • Mendeley – reference management software & researcher network

 

Organize & monitor your progress

  • ApAber– check your timetable, find available computers on campus, see your Aber Card balance, look at local bus timetables and much more
  • GradeHub – a tool to track your progress and predict what marks you need to achieve your degree
  • Asana – is a web and mobile application designed to help teams organize, track, and manage their work
  • MyStudyLife – unfortunately this service is shutting down but try myHomework (app) instead, it will help you to organize your workload

 

Taking notes

 

Study better

  • Forest App – is an app helping you stay away from your smartphone and stay focused on your work
  • GetRevising – revision tools
  • Anki – software for making flashcards
  • Study Blue – online flashcards, homework help & textbook solutions
  • Quora – a platform to ask questions and connect with people who contribute unique insights and quality answers
  • Memrise – a language platform which uses flashcards as memory aids, but also offers user-generated content on a wide range of other subjects
  • GeoGebra – an interactive geometry, algebra, statistics and calculus application
  • KhanAcademy – free online courses, lessons & practice
  • Tomato Timers – ‘Pomodoro Technique’ is a time management method, the technique uses a timer to break down work into intervals, traditionally 25 minutes in length, separated by short breaks

 

 

 

Findings of the Digital Insights survey running at AU for the second time!

Last year Aberystwyth University took part in the pilot of JISC Student Digital Experience Tracker – an online survey designed by JISC to collect information about students’ expectations and experiences of technology. The 2017/18 pilot has led to a new Jisc service now called Digital Experience Insights.

Digital Insights survey for students run at AU in January 2019. We were very excited about running this survey for the second time, as it enabled us to compare the findings with last year’s result and track our progress on digital provisions.

Below you will see a short summary of some of the key findings. If you wish to discuss them further or get more information on the project, please contact us at elearning@aber.ac.uk.

As you may be aware the Digital Experience Insights survey comes with a benchmarking data from other Higher Education institutions in our sector. The benchmarking data has been now made available and we will share it with you in the next Digital Insights post.

If you wish to read about AU experience of running Digital Insights in academic year 2017/18, take a look at the article published on Jisc website or browse through our previous posts:


Digital Experience Insights 2018/19

 

WiFi

Students’ satisfaction with WiFi increased by 7.3% in comparison to last year’s survey. Although WiFi is still the most common theme in students comment, the number of comments regarding WiFi decreased from 66 last year to 38 this year.

 

 

 

 

 

 

 

E-books & E-journals

7.7% less students responded that they have access to e-books and journals whenever they need them, this issue has been also mentioned in 19 of students’ comments.

 

 

 

 

 

 

 

 

Blackboard

The issue regarding a navigation in Blackboard seemed to improve. There were only 3 comments about this issue in comparison to 20 last year and 8.2% increase in the question on Blackboard navigation.

 

 

 

 

 

 

 

 

*The question wording has changed since the 2017/18 survey which could have impacted the ratings.

Security
Students are more satisfied with the provisions regarding security issues.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Mobile devices
The use of smartphone to support learning increased slightly. In the comments, students talked about the need of core services such as Panopto and Blackboard being mobile friendly and about usefulness of apps helping them with their studies. Interestingly, when asked whether they would prefer to be allowed to use their own mobile devices in class only 49% answered ‘At any time’, 45.4% answered ‘Only to carry out class activities’ and 5.6% ‘None of the time’.

 

 

 

 

 

 

 

 

Use of technology
There is a shift towards using more technology, there were quite a few comments about staff needing more training on the use of technology and there was an increase of nearly 10% of students wanting more technology to be used on their course.

 

 

 

Choosing an online polling tool

 

Image of students using polling handsets
https://flic.kr/p/9wNtHp

 

 

 

 

In-class polling or voting is great way to increase student engagement and interactivity in the classroom (for example see: Shaw et al, 2015; Boyle and Nicol 2003; Habel and Stubbs, 2014; Stratling, 2015). It is used widely in both higher and further education, and number of staff at AU make use of in-class polling on a regular basis. In addition to physical Qwizdom handsets available in loan stock, staff are more and more using online polling services such as Poll Everywhere, Socrative and Mentimeter (amongst others). These services allow students to use their own devices (such as mobile phones, tablets and laptops) to take part in polls, give feedback, and ask questions.

The E-learning Group can provide a wide range of information and support for anyone interested in using polling in their teaching. This ranges from advice on how to embed polling into your teaching practice successfully, to practical help on creating and using polls in the classroom.

At present AU doesn’t offer a centrally supported online polling tool for mobile devices. However, there are a wide range of services available, many with free or trial versions. This blog post is designed to help you assess which tool suits you and your students’ best.

  1. What do you want to do? As with all learning technology implementation, the first question you need to ask is ‘what do I want my students to do?’ The service you select will depend on the answer you have. For example, if you want your students to submit questions, or provide written feedback, look for a service that offers more than multiple choice questions.
  2. How many students will be in the class? Many of the free or limited versions of paid-for software have a limit on the number of students they can be used it. Look carefully at the details of what the free version does or doesn’t include.
  3. We also strongly recommend that you look at the Privacy Policy of the service to ensure you know what personal data is collected about you and your students (have a look at our blog post on this issue).

The E-learning Group has produced some information on some services which you may want to look at.

Once you have decided on which service you are using, here’s some of our top tips on successfully using voting in the classroom

  1. Think about your question/s. There’s lots of resources on designing good questions, particularly multiple choice questions. Don’t feel that you have to ask a question that has a correct or incorrect answer. Sometimes a question that sparks debate or shows the breadth of opinions on a subject can be useful.
  2. Using polling as a discussion starter. There are a variety of ways that you can use polling and group discussions together – two popular ways are Peer Instruction (especially the work of Eric Mazur) or Class-Wide Instruction (Dufresne, 1996)
  3. Practice. Have a practice before the session so that you are comfortable and familiar with using the questions and displaying the results. You can do this from your office using a mobile device such as a tablet or mobile phone.
  4. Make time in the lecture. If you are using polling activities in the classroom, make sure you leave enough time to give students to access on their devices, think about the answers and respond. You may also need time to correct misunderstandings or explain the answers.
  5. Let your students know in advance. Make sure that your students know to bring their devices and have them available in class. You can do this using the announcement function in Blackboard. You can also provide links to relevant FAQs such as how to connect to the AU wifi (Android: https://faqs.aber.ac.uk/index.php?id=692, Windows: https://faqs.aber.ac.uk/index.php?id=870, iOS: https://faqs.aber.ac.uk/index.php?id=700 )

There are a whole range of opportunities for using polling – from collecting information on how much the students know at the start of a module, to finding out what topics you need to cover in a revision session. You can also collect opinions, gain feedback on how the lecture is going, or collect anonymous questions. If you’re using polling in your teaching get in touch and tell us more – we may even feature your work on the blog!

 

 

Webinar: Instilling Self-Regulation in Learners & Using Sway for Online Learning

Academy Showcase is a space for sharing good practise among staff from Aberystwyth, Bangor and other Higher Education institutions. Every year we run two sessions with two presentations each, one from Aber and one from Bangor. Anybody can join Academy Showcase from their own machines using the link available here

We look forward this year’s presentations and we hope some of you will be able to join us.

 


20 March 2019 at 1pm -2pm

Instilling Self-Regulation in Learners by Dr Simon Payne (Aberystwyth)

We asked AU students and staff questions such as, “Why do students underachieve or even drop out?,” “What distractions do students face that interfere with their best intentions to study and improve?,” and “What happens to ‘turn students off’ from learning and striving to achieve?” The answers were remarkably similar from both groups, suggesting agreement on the problem and potential alignment on solutions. Self-regulation is the voluntary control of impulses which can facilitate or hinder us from achieving our goals. Hence, self-regulation includes the ability to regulate cognitive processes and activities, e.g. to plan, monitor and reflect on problem solving activities. Self-regulation also includes the control of one’s competing/conflicting motivational and emotional impulses and processes, e.g., overcoming social anxiety to contribute in class. Clearly, the development of self-regulation skills will help students achieve their objectives for entering HE. This presentation will provide techniques for tutors to help their students and tutees to be better self-regulators, and introduce and rationalise an ambitious AU-wide programme of studies that target student self-regulation ability.


Using Sway for Online Learning by Helen Munro (Bangor)

Sessions will be provided in English.

 

Polling software: Mentimeter and Poll Everywhere

The E-leaning Group is looking into how polling software can be used in lectures and seminars. Polling software is a great way to increase classroom engagement as it provides interactive presentations ranging from multiple choice questions to live word clouds. With their personal devices (such as mobiles, tablets etc.), students will be able to answer questions, vote and ask queries,which will appear on the presentation slides.  The recent Digital Insights survey, overseen by Information Services, showed that fifty-seven percent of lectures already use some sort of polling software in the classroom.

Some examples of positive comments from students include:

 “Provided quick feedback on what lecture we needed help with”

“Online poll, on parts of the subject asking the class how much they understood. This made it so people put how they actually felt as they didn’t have to speak in class”

“Polls in lecturers keep the students interested”

“It was fun last year when we did online quizzes in the lecture, interactive with each other and then went through the answers question by question on the big screen”

“Method of reviewing prescribed reading material”

The E-Learning Group has found Mentimeter and Poll Everywhere to be especially accessible and reliable:

  • Mentimeter is best used for lectures with larger audiences as it has no limit on participants. With Mentimeter you can create: quick slides, questions and quizzes. There is no limit on the number of quick slides, however with the free version you only be able to create two questions and five quizzes.
  • Poll Everywhere caps its audience at twenty-five so can best work in seminars and workshops. Poll Everywhere provides much of what Mentimeter does with the benefit of having no limit on the number of questions/activities.

There is a guide to creating presentations with both Mentimeter and Poll Everywhere available on our webpages.

 

Digital Experience Insights 2018-19: How do you rate technology at Aberystwyth University?

For the second time Aberystwyth is taking part in the Digital Experience Insights project aiming to explore our students’ experiences of technology. The project is based on online surveys designed by Jisc and used by different institutions across UK.

It allows us to get better insights on how students use technology and benchmark our results against other HE intuitions in our sector.

We would greatly appreciate your help in promoting this survey to all students: