Final Report


Managing Connections: Using e-Learning tracking Information to improve retention rates in higher education

The Final Report provides an outline of the research design, the findings and the recommendations which arose from the ‘Managing Connections: using e-learning tracking information to improve retention rates in higher education’ project, which was funded by the Higher Education Academy and Middlesex University between 2007 and 2008. A full copy of the report is available to download here:







Executive Summary

This research was undertaken by Middlesex University as a part of the project funded by the Higher Education Academy, Managing Connections: using e-learning tracking data to improve retention rates in higher education. Bringing together the areas of e-learning and student retention, it provides the basis for practical recommendations to aid student progression, especially during the first year of study.
The project was informed by literature in three research domains:
· Retention and progression
· Learning and teaching through the use of technology
· Identity and behaviour

Aims

The aims of the project were to identify the ways in which students engage with their institutional virtual learning environment (VLE) and to explore what can be learnt from the behaviours of students who had withdrawn from their programme. This was then compared and contrasted with the behaviours of students who remained. By investigating the ways in which these two groups of students engaged with learning, the project aimed to provide recommendations on how to provide better support for all students to engage with learning through the use of technology. Specifically, the aims were:

· To assist educational institutions in improving the learning experience of first-year students
· To learn about factors which contribute to withdrawal and progression
· To identify how students at risk of withdrawing from their programmes manifest themselves online

Sample

The sample comprised two groups:
· All first-year undergraduates (totalling 92) in one HE institution who withdrew from or interrupted their studies during the academic year 2007-8
· A sample of 130 first-year undergraduate students at the same institution who persisted with their programme

Methods

Data collection methods included:
· Tracking data extracted from the server logs of the institutional VLE
· Telephone interviews with students who had withdrawn from their programmes
· Surveys among students who were persisting with their programme


Key Findings

Quantitative findings

Descriptive analysis of the withdrawn participants and their actions are as follows:
· The majority of the withdrawn participants (57.2%) withdrew during the first four weeks of their course (mean = 4.69, median = 4, Standard deviation = 2.74)
· Exactly half the participants (50%) never logged onto the VLE
· For the participants who did log on at least once, a total of 473 logins was recorded, with a mean score of 10.28 and a median score of 5.5 (multiple modes exist) and a standard deviation score of 13.05. As the number of logons per participant varied from minimum = 1 to maximum = 65 times, the median score was used for comparative purposes as it is insensitive to extreme scores.
· Withdrawn participants who did not log on to the VLE at least once did not fall into any specific age bracket or sex (male = 50%, female = 50%)

When looking at access trends of individual students who withdrew from their programmes it was noted that students more often than not displayed the following two behaviours:
· VLE Behaviour A: to begin with, withdrawn students accessed the institutional VLE many more times than the average student on that programme and then their access dramatically dropped to zero as they approached the withdrawal date held by the student management system
· VLE Behaviour B: withdrawn students followed the same access trends as the average student on the programme they were studying, but logged on to the institutional VLE much less

Qualitative findings

Qualitative findings have been categorised and are presented here under seven themes.

Theme 1: Perceptions of learning

There were marked differences in the ways in which withdrawn and persisting students described their experiences and perceptions of learning.

Withdrawn students
When asked about their personal experiences of learning, withdrawn students tended to describe learning situations in which they were active participants and experiences which were of a more social constructivist nature; the experiences included their peers and required them to be active. However, when they were asked about how they perceived they learnt best they often defaulted to describing an information transmission model of learning, thus highlighting a gap between their model of learning derived from their previous experience and their perceived model of effective learning.

Withdrawn students also made a clear distinction between passively learning from (being taught, watching demonstrations, etc), learning through (by sitting next to or in the vicinity of intelligent peers) and actively learning with (discussions, group work, etc) others. Occasionally they perceived their peers as obstacles in their learning experiences.

Persisting students
In contrast to the students who withdrew, students who remained showed an awareness of how they learn as individuals. Current students had a greater awareness of 'self' as a learner and expressed a richer description of how they learn, which included a range of methods. When asked about the skills needed in order to learn, they were able to identify many of them without any prompts. However, when responding to questions with regard to learning through the use of technology, persistent students held similar views to those held by withdrawn students. The research showed that they were equally as naïve in the way they used technology in their learning as those who withdrew, as demonstrated below.

Theme 2: Technology as a means for learning

The experience of engaging with technology for learning was expressed in two main ways across both groups of students:

· e-learning was seen as a remedial. In this deficit model, e-learning was experienced as aimed at solving a problem, especially when this involved correcting or improving the student’s performance
· technology was seen as a medium to which many attributes were assigned. Students saw technology as a medium holding certain features that served as aids, altered students and influenced their actions

Theme 3: The institutional VLE and learning

Persisting students’ perceptions of the VLE were expressed with relatively strong opinions. Students frequently described this facility as a tool that helped in their learning or as a service requiring further improvement. In contrast, students who withdrew seemed to have a less specific engagement with the different resources within the VLE, presenting only their perception of it as a remedial tool, which was reinforced by their views on the use of technology.

Theme 4: Deficit approach to e-learning

The evidence of a deficit approach to e-learning appeared to manifest itself in tutor actions as well as in the views of both groups of students of how technology can be used in learning. Specifically, the way in which the use of technology is introduced within particular learning situations and integrated into face-to-face practice influenced the way it was perceived and used by the students.

Theme 5: Technology and new skills

Participants acknowledged the need to engage with technology in order to develop new skills. The issue of whether they themselves actually pursued this engagement and, if so, the way in which they did this were not reported.

Theme 6: Technology in (re)search

Both withdrawn and persisting students referred to research as the search for information only. They made no reference to nor gave account of any further selection, analysis or critical engagement with the material collected.

Theme 7: Range of technologies used

Persisting students appeared to engage with a wider range of technologies than students who withdrew (in both their social and academic lives). Although it is possible to make a link between the engagement with a broader range of technologies and the progression of students, the extent of this relationship, the reasons for it (ieg socio-economic factors) and the way in which the technologies were used to access information and/or support, etc pose issues for further investigation.



Recommendations

Based on the literature review, the findings from this project and the difficulties encountered during data collection and analysis, the following suggestions are offered to institutions, academic staff, researchers and learning technologists:

For institutions

· Make explicit in learning, teaching and assessment strategies and in e-learning policies the benefits of using technology with the curriculum.
· Student support for e-learning should focus on how to learn with technology and on transferring existing skills into the learning situation, not just on how to use the technology.
· Embrace technologies which students bring with them, rather than excluding them. For example, mobile phones could be used for in-class voting rather than requiring students to switch them off.
· The accuracy of personal information is key when integrating student management systems and learning and teaching systems. Build into your induction or enrolment processes a check of personal data. Also, promote ownership and communicate early on to students their responsibility to maintain the accuracy of their personal records.
· Institutions should look into enforcing penalties such as restricting access to IT systems and other resources as late in the academic year as is feasible.
· Discourage the sharing of account details between students.
· Ensure that links are made between the academic and social aspects of your students' lives in order that they see the student experience as a whole.

For academic staff

· Facilitate the development of meta-cognitive learning skills at the beginning of or prior to the start of the academic year, through workshops.
· Clearly articulate to your students the reasons why you have chosen to use learning technologies in your module or programme.
· Not all students like working collaboratively. Carefully consider the balance of group work when designing your learning materials.
· Consider how your teaching acknowledges lurking as a valid way of learning.
· Engage students with their VLE from the start so that logging into the institutional VLE becomes part of their daily routine. Design learning experiences which closely integrate the physical and virtual components of your teaching.

For researchers

· When attempting to map the levels of interaction of students, there is a definite need for including not only the institutional VLE and online instruction as part of e-learning, but also other ICT such as software, electronic deliverables (eg podcasts, electronic articles and electronic handouts) and electronic devices (eg mobile phones, mp3 and mp4 players, USB pen drives).
· Although a single definition of retention is not available, it would be useful to have clearly stated criteria for considering a student as withdrawer.
· A possible approach to tackling the issue of the different conceptions of the term ‘e-learning’ could be to use more dynamic perspectives in which varying degrees of interaction are included, instead of a fixed definition. This trend has already been initiated by the University of Glamorgan (Jones, Skinner & Blackey, 2007).
· When reporting research outcomes, clearly state how retention is defined and measured.
· Further work needs to be done to relate longitudinal statistical analysis of students who dropped out to local, context-specific quantitative and qualitative analyses.
· The issue of students’ self-identity in relation to the social identity generated by the peer group and the HE institution deserves to be further explored by including technology usage, expertise and ownership of students as variables.
· The scarcity of information regarding the interaction of e-learning, student retention and identity points to the need for further exploration of the topic and to determine to what extent and under which circumstances each issue affects the other two, which might lead to a dynamic characterisation of their connection.
· Carefully consider the timing when you interrogate live databases as this only provides a snapshot of reality. Data are continuously updated.

For learning technologists

· Staff development should focus not only on how to use technology but also on how to use it appropriately to enhance the student learning experience.
· Learn how technologies which promote social interaction are used and incorporate the lessons learnt into the design of e-learning experiences.