University of Groningen Educational value of digital examination

Maat: px

Weergave met pagina beginnen:

Download "University of Groningen Educational value of digital examination"

Quinten Jonker
6 jaren geleden
Aantal bezoeken:

1 University of Groningen Educational value of digital examination

Benefits Digital Examination HANDWRITING CORRECTING 1 2 3 Do you remember the Correcting the essay exams in handwriting from your students Nestor is easier for a teacher. at essay exams?

2 Benefits Digital Examination HANDWRITING CORRECTING Do you remember the Correcting the essay exams in handwriting from your students Nestor is easier for a teacher. at essay exams? Digital No more paperwork! examination is our solution! FLEXIBLE EXAMES Flexible exams with advanced options like adaptive release for additional questions based on multiple choice exam scores. MEDIA You can use both video and audio in exams. The computers are equipped with a headphone plug. STUDENTS Students are able to rearrange text fragments easily. No more strikethrough and arrows! REDUCE PAPER Digital exams reduce amount of paper stored in archives etcetera, so it is a sustainable solution!

3 Visible before exam Omschrijving veel voorkomende fouten bij construeren van meerkeuze items Komt voor bij 36 65% van de toetsitems De vraag bevat geen stam, de vraagstelling is verplaatst naar de antwoordalternatieven De vraag bevat een ontkenning (in de stam of in de antwoordalternatieven) Als laatste antwoordalternatief wordt geen van de antwoorden gebruikt Als laatste antwoordalternatief wordt alle antwoorden zijn goed gebruikt Complex ontworpen item format wordt gebruikt Deze items zijn moeilijker te beantwoorden: studenten met hoge totaalscore doen het niet beter dan studenten met een lage totaalscore Vrij grote groep (14%) zakt op deze items en slaagt op goede items (Downing, 2006)

4 Improve Quality of MC test before exam Steps in the proces of testing Plan and prepare for assessment-- Why assessment? - Align outcomes and assessment - Choose assessment method - Construct and review test questions Compose tests Written mc tests - Peer review with written material - Managing item database is hard - Structure of test implicit - Randomization limited -1 out of n correct - max 4 answer alternatives Online mc - Item database in same system as test administration (QMP) - Several possibilities for labeling (statistical variables in item database) - Peer review digital - Randomization - Various question templates - > 4 answer alternatives

5 Improve quality mc during and after exam Steps in the proces of testing Written mc tests Online mc Test administration OMR-forms Test software in AJH (Bb, QMP) Rate test with quality check - Processing mc exams 3 days - Re-analysis 2 days - Test analysis: p, pc, rit, labeling, frequency of scores and answers -Score immediate available - Analysis by lecturer - Test analysis: p, rit, time (QMP) - No re-analysis (Bb, QMP) Determine pass / fail threshold Transform scores to grades Justify decision / report test results ESI software several methods Taking in account guessing score ESI software Two ranges: 1 5.5; Taking in account guessing score Feedback to students Bb: limited methods based on US education system QMP: no Bb: limited methods based on US education system QMP: no (Elaborated) feedback to students

6 Improve quality essay exams before and during test Steps in process of testing Written essay exams Online essay exams Plan and prepare for assessment - Why assessment - Align outcomes and assessment - Choose assessment method - Construct and review test questions Peer review - Item database in same system as test administration (QMP) - Several possibilities for labeling (statistical variables in item database) - Peer review digital Compose test Short answers essays Short answers essays Test administration Written text Online text with rich text editor (Bb)

7 Improve quality essay exams before and during test Steps in process of testing Rate test with preventing pitfalls in rating Determine pass / fail threshold Transform score - grade Justify decision Report test result Written essay exams Short answers: test analysis possible after importing scores by questions manually Online essay exams Short answers -Scoring immediate -Test analysis Essays - Prevent pitfalls of rating question by question, anonymous, - no influence of bad handwriting; - structure text better by correcting in text editor -Tools for annotating and automatic scoring double check scores around pass / fail threshold Elaborated feedback to students with digital annotating

8 Pitfalls during correction of exams Focusverschuiving: verschil in focus beoordelaars Normverschuiving Sequentie-effect Contaminatie-effect: andere doelen spelen mee bij beoordeling Halo-effect: irrelevant kenmerk student beinvloedt beoordeling Persoonlijke tendens: Haviken en duiven

9 Item analysis

10 Thank you for your attention. UNIVERSITY OF GRONINGEN DIGITAL EXAMINATION

Vergelijkbare documenten

Developing an adaptive, diagnostic test of. English writing skills

Developing an adaptive, diagnostic test of English writing skills Development of the DET Objectives Consultation IT Student model Consultation External committee Research Student models Psychometric Automatic