Is an exam a valid and reliable assessment method in an online course ?
Posted by: Jane Dillon on 12 March 2009 - 4:29pm
Hi LDNet Community !
Thanks John for the opportunity to kick off a discussion on a topic that is of increasing interest here at the Nelson Marlborough Institute of Technology (NMIT).
This issue came into focus (in my work as a Flexible Learning Advisor) in the development of an online programme to run concurrently with an established campus programme. Should online students be assessed in the same way with an exam when the nature of their learning environment and activities might suggest this was less appropriate ?
Look forward to your views !
Cheers
Jane Dillon
- If you're member of this group, log-in to post a reply
useful resource
Hi Jane
Discussion seems to have dried up somewhat - maybe everyone is busy marking! But thanks Jane for stimulating some interesting ideas here.
I came across this site today from AUTC, well worth exploring in terms of tertiary assessment in general, and some useful material on online assessment in particular: useful in that the resources are well set out, downloadable for printing, and the powerpoints can be downloaded and adapted. See what you think.
http://www.cshe.unimelb.edu.au/assessinglearning/index.html
Cheers
John
Some ironies
Hi David
Many thanks for your reply. There can be ironies in all of this. I have been reflecting on Gilly Salmon's comment that "we need to move towards 'aligning' our assessment with our online teaching aproaches" (Salmon 2003). Elsewhere she mentions a fundamental irony in students studying online only to have to appear in the exam room with pen and paper ...... As you point out when this is required there are ways of making it happen - complex as the arrangements might be !
Cheers
Jane
Out of my depth
Good questions Jane. Sorry I am showing my age :) and I have not participated in any online assesment. Although I am being encouraged to put my multichoice questions onto Moodle here at UoW. I am going to leave that adventure till next year as there has been many other changes in the course that have kept me busy.
I have had a very small example of none traditional exam situation. Last year in one of my courses there was a writer for a student, and for both the tests and the exams she sat them at same time, but in another part of the university. I am unsure if they had a writer for the test/exam or just extra time.
In the test situation I emailed or sent via internal mail (can't remember) to a person in student services who looked after these special case tests etc. It also struck me as been ironic that there is all this procedure and controls etc around exams and handling of questions and scripts, and there is this exam going in internal mail in a envelope with holes in it, and the exam script comes back the same way.
I would suspect that most tertiarys have the ability to handle test/exams in the way mentioned above. So if your students could easily travel to a tertiary then they could sit the exam there. However I doubt this would happen due to the politics of this kind of thing across tertiaries......
David
Related questions ......
Do we need exams at all any mo...?
This is a good topic for debate. As Nick says, there may well be a place for exams in general, if the SLOs warrant it - ie there is a certain body of knowledge that needs to be learned - vocabulary, process whatever - and the actual student needs to demonstrate that under pressure. So we have to have a process that is authentic as well - and identify the student taking the exam - another reason they maybe are not the best option at a distance, and even face to face sometimes.
But, increasingly exams in general are less use - the skills students need more now are where to find the information, how to evaluate and synthesise it into something else that they can apply - and information readily available doubles every day, and half of it is wrong! Only we don't know which half, whether its in the teacher's notes or on the web either!
Another problem I have with exams - and this is personal experience - is that they don't necessarily test a student's knowledge of the subject......sometimes all they test is your knowledge of Greek and Latin derived vocabulary, if the exam questions are written in exam speak. 7th form history exam - I knew all there was to know about Henry 7th - complete bastard, tyrant, etc. Exam question "How despotic was Henry 7th?" Hand up, what does despotic mean ? Can't tell you, it's an exam. I didn't have enough confidence to write "If despotic means tyrant then this is my answer..." so I chose another question I was less prepared for and didn't get a chance to show what I knew. (David Corson did a thesis on this, if you can't understand the words in the question, how can you answer it?). Hopefully exam questions these days are a little more fair to students???? But do we need them at all??
Oriel
My 2 cents!
<!--[if gte mso 9]>
LatentStyleCount="267">
<!--[if gte mso 10]>
<![endif]-->
Hi Jane
I believe that assessment has to be tightly aligned with the specific learning outcome(s) (SLO) and the students learning dimension i.e. the initial acquisition > independence> mastery continuum.
Once the SLOs and the students learning dimension have been identified assessments maybe designed and rolled out utilising appropriate supportive technologies.
So in response to your question I say yes. An exam CAN be a valid and reliable assessment method in an online course. However as with anything this is context specific with the contexts being slo's, learning dimensions and subject domain.
Do you think online learning changes the notion of assessment needing to be aligned with slo’s and student learning dimensions or are your concerns related to validity?
Cheers, Nick
The Laidlaw experience
Hi Jane,
At Laidlaw College we are preparing a course on the ancient Greek language. We are using online tests in a low stakes (that is, small percentage of final grade) way whereby students are tested on their memorisation of key vocabulary and noun/verb paradigms in timed tests that they can take again and again, based ona rich pool of randomsied questions.This works for us in that:
A final exam will also be a part of the assessment, under proctored conditions. The system is this: weeks before the exam, the student is required to find an exam supervisor (we do set conditions on who is suitable). We assess the relationship between the student and their proposed supervisor and authorise the supervisor to, well, supervise the student under exam conditions that we set. This requires a high level of trust, and clear instructions for supervisors. The supervisor returns the stduent script and provides a signature confirming that exam conditions were in place.
The key terms in your own question are 'should' and 'appropriate'. There are times when exams and tests make good sense, others when they do not. High-stakes exams can still be done, even at a distance. Administratively It is a lot of hassle, but the assessment type suits many of our early-level courses where memorised knowledge in the student is a defensible objective.
In your situation I would be inclined to:
I wouldn't be too quick to dismiss the value of exams in e-learning. There are ways and means, and it does provide a verifiable outcome where you know that the student has engaged with course concepts.
Hope this helps!
A very thorny issue
Jane
There would appear to be two distinct topics, (1) should the on-line assesment be same as the on campus course and (2) is an exam a valid assesment tool. I will limit my comments here to the 2nd point. So people know were I come from - after my Masters I spend approx 10 years in industry before becoming a contract lecturer -teaching on the business of science topics.
As a Sever to extreme dyslexic that was diagnosed while at tertiary (UoW) I found the concept of rote learning to regurgitate in an exam an abhorrent concept ( and no surprise I was not that good at it). And after a decade in industry I experienced nothing that would validate exams as a measure or knowing - In fact I came to realizes that some of the best academic students made some of the worst researchers.. Because research is about the unknown and they struggled with the fact there was no "right" answer - only steps on the journey.
However in saying that, I think assesment is important as it motivates people to learn ! I found the Physics department has the best idea (and this became my major). they set hard/thinking exam's that were open book. This meant that the you had to know the materiel and could apply it, but didn't need to memorize fact to regurgitate them.
Also in some feilds eg medical where large volumes of data floating around in the mind is important. When one goes to a doctor / medical person and ask for a diagnoses, you wouldn't want them taking 1/2 and hour to figure out you had a sore throat caused by an infection....... so exams might be valid.