Challenges for an online community for teaching and learning

Unitec’s Diana Ayling (pictured) spoke at an Elearning Community workshop about an online community she’s involved with which focuses on teaching and learning. These are my brief reflections on her presentation.

Diana and audience members identified some challenges for members of a teaching and learning community:

• Teachers take time to develop a voice online because creating and managing content is a complex skill set. There is a growing need for teachers to develop ‘real-world’ technology skills such as working with social network technology. We need to ‘go to where the learners are’ so need to move beyond the institutional  Learning Management System.
• Teachers have varying levels of resilience – when something goes wrong such as a technical problem, some are inclined to give up straight away while others see it as only a temporary setback.
• There is a tendency to form splinter groups, as some are more comfortable with interacting a small group. This may have the effect of decreasing overall activity and interaction.
• Data protection and copyright are ongoing issues as teachers move to more open technologies such as social network tools.
• Online safety and privacy is an issue for both teachers and learners. Separating the personal and professional online presence is complicated but necessary.
• When working with teachers as community members, we should not make assumptions about their level of technical skills – patronising them is a real turn-off.
• Finding time to contribute actively to multiple communities and online spaces is difficult for busy teachers. RSS is a great tool for managing all the sources you read, but it doesn’t really help with contributing through writing.

 

8 tips for online learning community activities

In a purely social community, collaboration and communication can be very open and unstructured. But when incorporating online community approaches into courses, we normally need to adopt a more structured approach to meet needs of the curriculum and the learners.

Here are some tips for this situation:

1. The start of the activity is important – so welcome people and set a positive tone. Don’t overload people with information – start gently!
2. Focus on participant needs – so exploring and sharing existing knowledge is often a great place to start.
3. Give clear instructions and information about the activity – break the activity into digestible chunks so participants can focus on one thing at a time. Move supplementary information (ie info which is not key to the activity) to subsidiary pages.
4. Use triggers to build engagement – eg provide a video or a link to an online survey, then follow with a directly-related question.
5. Use open questions – eg ‘why do you think…?‘ These are more likely to generate higher-level thinking and in-depth discussion.
6. Be a bit provocative – a trigger or question which is controversial is likely to generate engagement. But avoid topics which are too risky as these can lead to dangerous conflict – if you’re inexperienced as a facilitator this can be hard to deal with.
7. Consider the flow of the activity – it is often preferable to deal with one question at a time, allowing participants to focus on one question at a time. You can start with prior experience or knowledge, then move in a developmental sequence to more in-depth questions. If you pose multiple questions from the beginning, especially with smaller groups, the community focus can be easily dissipated and lose momentum.
8. The end of the activity is important – you should at least farewell participants and thank them for taking part. Consider also how the key points that arose can be summarised and published – true collaboration is not just discussing but generating new and powerful knowledge that should not be lost. You might ask for a volunteer to do this.

Image: Augustin Théodule Ribot: The Conversation

Evaluating online community activities

The purpose of evaluation

A key question for anyone managing or facilitating an online community is how to make it sustainable. Sustainable communities need to maintain (and grow) an active and engaged membership. Structured online events or activities can play a very important role in engaging community members and ensuring their regular and active involvement. However, these events must be effective – badly planned and/or facilitated events can turn members off and lead to the failure of the community.

The facilitator’s overall impression of whether an event is effective or not is useful. But without a more rigorous evaluation, we can miss underlying issues which have the potential to damage members’ ongoing engagement in the community. So some kind of evaluation process is vital to its ongoing success and sustainability.

This article focuses on evaluating community events: not to check that they meet a minimum standard, but as a way of engaging in a process of ‘continuous improvement’.

The evaluation process

Many educational and management approaches employ cyclical models involving reflection or evaluation – eg Kolb’s experiential learning cycle and the action research cycle. The PDSA Cycle shown here represents the stages of Deming’s approach to quality improvement in business, but can be adapted to provide a model useful in relation to online community events and activities: PLAN: plan the community event DO: facilitate the community event STUDY: evaluate the event ACT: feed the evaluation results back into further community development

Carrying out the evaluation

Most of the evaluation will take place as part of the STUDY phase. But during the event (the DO phase) the facilitator should keep notes on what’s going well and what isn’t. If it’s not too big a group, it can also be really helpful to keep notes on the level of engagement of each member – not as a form of assessment, but as data that may be useful later.

During the STUDY phase:

Decide how you will gather feedback

Without its members, the community does not exist. So feedback from participants in an event is an essential component of evaluation:

• Online tools such as surveymonkey or polldaddy are quick to complete, so community members are usually happy to complete a survey. But make the survey brief, and tell them how long it’ll take.
• Individual interviews: you may get much more meaningful information about the event’s effectiveness if you personally approach participants. This may be a phone call or skype, or using an asynchronous method such as email. Ideally, a neutral 3rd party will gather the data, since participants may be unwilling to open up to the facilitator. You may be able to set up a reciprocal arrangement with another facilitator to gather data from each other’s participants.
• Collaborative feedback: you could set up a wiki that participants can use to record their feedback. Or you could set up a forum or a synchronous discussion space that participants use to discuss the event. Ideally this would be anonymous, so you may appoint one member to gather the raw data and provide you with a summary. Again, you may be able to arrange with another facilitator to manage this and gather feedback.

Develop evaluation questions and tools

• Communicate clearly what you are evaluating – point the participants back to the activity if possible so they can reflect on it. Be specific – make sure the questions clearly identify aspects of the activity that you want feedback on.
• Focus on how well the activity met their needs, not just on how much they enjoyed the process. If possible, ask for feedback at the higher levels of Kirkpatrick’s model – eg has the activity had a positive result, has it made a difference, have they been able to apply what they learned during the activity?
• Focus separately on the design of the activity and its facilitation: they are distinct, and the effectiveness of each is essential.
• Include questions that are open and qualitative so you can find out why things happened the way they did.

Gather feedback from community members.

• Tell participants how you will make use of the feedback: if you focus on improvement and they believe you are sincerely interested in making things better next time, they are more likely to engage in the evaluation process.
• Gather feedback not just from those that took part in the event, but also from those who chose not to. Asking those who didn’t take part why they didn’t can tell you a lot about the design of the event and the way it was communicated to community members!

Reflect and evaluate
Feedback from participants is just one form of information on which to base the evaluation. Your own reflection is another essential component.

• You may want to reflect on the activity before you gather data from participants – that way your own thoughts won’t be overly influenced by feedback. But once you have gathered the data, that’s a chance to reflect on and learn from the feedback from participants.
• In your reflection, avoid placing blame on participants. Assume you can do better and learn from mistakes. If people go wrong, ask yourself how could I have communicated more clearly?
• Use your own recollections and notes from during the activity to triangulate – compare them with what participants have told you about the activity, what happened and why.
• Focus on improvement – even where you think the activity was effective, try to identify specific things you could do to make it better next time.
• Share your evaluation with the participants – it’s a community, right? Sharing the evaluation means it becomes part of the larger community collaboration and conversation, and can enhance member commitment. But… if there are comments or conclusions that you feel are private or could offend members, leave them out of the published version.

Image: A Midnight Modern Conversation by William Hogarth.

Bibliography

Davies, C (nd). Kolb Learning Cycle Tutorial. Downloaded 30 May 2010 from http://www.ldu.leeds.ac.uk/ldu/sddu_multimedia/kolb/static_version.php

Unknown author, Business Performance Ltd (nd). Why Measure Training Effectiveness? Downloaded 30 May 2010 from http://www.businessperform.com/workplace-training/evaluating_training_effectiven.html

Unknown author, Carpenter group (nd). The Deming Cycle. Downloaded 30 May 2010 from http://www.quality-improvement-matters.com/deming-cycle.html

Unknown author, Warwick University (2008). Action Research. Downloaded 30 May 2010 from http://www2.warwick.ac.uk/services/ldc/resource/evaluation/tools/action/

Planning online learning activities: problems with technology

In my work with educators who are new to planning and facilitating online learning community events, I often see activities which disappoint participants because of simple problems with the technology. Some community members are positive and accepting, others are not so forgiving and lose interest if the facilitator doesn’t seem well prepared.

Make sure the technology will work

In my experience, most of the technical problems that new facilitators encounter are avoidable. Many can be avoided if you:

• Check and check again that the technology works before a synchronous event. First, before you publish the activity details so you know that what you have planned is feasible. And again just before the event – with enough time to repair or work around any fault that occurs.
• Before the event, tell participants clearly what they need to have and do to be ready. eg will they need a headset with microphone? And advise them to check beforehand it’s all working. Establish a fallback position – eg if their webcam doesn’t work, can they just use audio?
• Do a test drive to make sure that participants can access the event – but be aware that what you can see and do with a ‘teacher’ account is not always the same as what ‘student’ or guest account can. Log in using a dummy student or guest account and check that what you planned is possible.

Have a contingency plan

No matter how well you are prepared, technology can still cause problems, especially with synchronous activities. So you need to be prepared:

• Have a plan B on what you will do if the technology goes wrong. How will you facilitate the event if the chosen technology fails?
• Have a plan B for participants who can’t take part. If it’s an asynchronous forum they can access it later, but if it’s a synchronous activity you should create an archive for those who missed the synchronous event. eg in Wimba, click on the archive button as soon as the discussion gets underway. In a Skype chat, you can save the transcript as an HTML file and upload it to a web page.

Effective online facilitators:

• Avoid many technology problems by making sure it works beforehand
• Have a plan on how to continue when unavoidable problems do arise

Image source: Wikimedia Foundation