Assessing what’s important in online discussion

One of the problems with assessment is common to all forms of measurement: ”what is easy to measure may not be important, what is important is often hard to measure.” This is true of online assessment as well as more traditional forms.

The assessment dilemma

This dilemma can cause assessment developers to assess what is less important: they focus on easily quantifiable criteria or standards but neglect to include more important criteria because they are not so easily quantified or measured.

In assessment of online discussion, we see the effects of this dilemma when assessment criteria call for a quantifiable contribution from learners: ‘At least 3 contributions to the forum’ or ‘One forum thread started and responses to at least three others’. This can lead to lightweight discussion as learners post superficial responses to achieve their tally. And this of course is contagious – every lightweight posting says to everyone else: ‘Look, this is all you need to do’. Even worse, genuinely thoughtful postings can get lost in the tide of mediocre ones.

To avoid this, we need to focus on what really is important: usually, what we’d like to see each learner achieve is something like ‘Make a significant contribution to the development of ideas through the discussion forum‘. But since that is clearly more subjective and harder to measure, we need to spell out how this might occur and what it might look like:

• Contributes original ideas which are relevant and well-developed
• Provides significant insights into the existing body of literature
• Helps synthesise ideas and concepts
• Contributes detailed and relevant examples from their own practice
• Helps others to explore ideas in depth

Of course, such a list is still relatively subjective – but it does start to establish what is important to me as an educator. And since I’d rather see one ‘significant’ posting than ten which don’t really add anything substantial, I prefer to avoid the use of quantitative measures when developing rubrics and other assessment tools.