The development of methods for eliciting learner narratives within a framework for embedded evaluation for fully online distance learners
Caroline Daly, The Centre for Excellence: work-based learning for Educational Professional, Institute of Education
Norbert Pachler, Jon Pickering, Jeff Bezemer, Institute of Education
Jill Russell, Open Learning Unit, University College London
Jon Wardle, CEMP, Bournemouth University
The findings of this project have been developed into an article published in Professional Development in Education, Vol 33 Issue 4 December 2007: Teachers as e-learners: exploring the experiences of teachers in an online professional master’s programme (external link).
The project developed and trialled evaluation methods to investigate the experiences of fully distance e-learners studying within asynchronous environments. It has been conducted with students following the postgraduate programmes in International Primary Health Care at UCL and Creative Media Practice, Bournemouth University. Fifteen learners’ experiences have been examined by narrative investigative methods. These methods have been developed and evaluated to provide a format for developing embedded evaluation in e-learning course design and provide a significant advancement in approaches to evaluating the learner experience in distance contexts. The project builds on previous research funded by the CDE (Daly et. al 2005-06) into learner experiences in the mixed-mode context of the IoE Master of Teaching degree, and addressed the challenge of conducting meaningful evaluation in fully distance contexts.
By focusing on the experiences of e-learners in these contexts, the project has investigated an area identified by the Higher Education Academy (2005) and JISC (2005) as under-researched and has responded to the need for far greater understanding of the pedagogical potentials of new technologies to enhance and deepen the student learning experience. The project sought to develop evaluation methods which provide an alternative to ‘exit’ and ‘survey’ models for evaluating learner experiences, and which challenge conventional ‘purposes’ of evaluation by focusing on achieving dialogue within ‘evaluation for learning’ as a prime factor in improving the student learning experience, and providing evaluation material which can serve QA purposes.
The project presents findings from data collection conducted with the students and their tutors during the first year of their distance programmes.
Major achievements from the project are:
- The identification in detail of narrative evaluation methods which can be applied in fully distance contexts, and the analysis of their effectiveness;
- The identification of key features of a wide range of narrative evaluation methods, their benefits and disadvantages, for wider trial and adoption;
- A proposal, based on the research, for a development in evaluation practices to enhance the learning experience of distance students. Such an approach to evaluation argues for an alternative to the traditional purposes for evaluation and proposes ‘evaluation for learning’ as a rationale underpinning the design of distance courses to include embedded opportunities for participation in narrative evaluation practices.