Building a Learning Analytics Prototype

I’ve been pretty quiet on the blog over the past few months because I’ve been working on a project, so thought it was about time I set the ball rolling again! I tend to find with blogging that its one of those activities that gets easier the more that you do it, and because I haven’t written a post in a while it feels like I’m doing this for the first time again! Timing couldn’t be better though because I can write about the project I am working on.

If you’ve read my previous posts, you’ll know that a big interest of mine is Learning Analytics, and its probably no surprise to learn that this is going to be the subject of this post and the next few posts as I fully intend on keeping this blog ball rolling now I’ve started!

There’s an increasing amount of interest in Learning Analytics at the moment. Its a wide area and there is still lots of debate over exactly what the term means. I am especially interested in exploring how we can learn from how organisations like Google use Big Data and Machine Learning and seeing how these techniques can be used more widely in education.

At the moment, I’m working with the University of Wolverhampton who use our Higher Education student administration system, SITS:Vision. Together, we’re working on a project to explore how Learning Analytics can be used to help understand and improve student retention and encourage students to change their approach to learning.

Project Goals

The aim of the project is to develop a Learning Analytics prototype which uses data collected from university administrative and learning systems to help staff understand students better and students to understand how the way they learn impacts on their success. In particular, we want to see whether this data can be used to:

Predict which students are at risk of withdrawing from their course so that the university can give these students additional support and guidance.
Enable students to see how the way that they use learning resources affects their outcomes. Challenge students to change their learning behaviour by allowing them to compare themselves with their peers.

Student and Activity Data

What makes this project really interesting is that we are combining administrative data held in SITS:Vision with data which tells us how students interact with university services. Data from SITS tells us who the students are, what they are like, what courses and modules they are enrolled in, and module results (amongst other things!). Data from university services tells us how the students have interacted with those services. This allows us to ask some interesting questions. How have they approached learning? How often have they used the library? What content have they accessed on the VLE? Are there periods where they used the VLE more often? This type of data is often called Activity Data. If you want to find out more about activity data, check out this websitewhich has some good background about it and links to learning analytics research projects which have used activity data.

Combining activity and administrative data should allow us to make some powerful insights. For example, it allows us to see the affect that the student’s demographics have on the way that the student uses learning resources. In turn this should enable us to see how the use of learning resources impacts the grades that the student achieves in modules that they have been studying.

Generally, administrative data allows us to make an initial assessment of risk of withdrawal when the student enrols at the university. This is helpful for identifying students who may require additional support at the outset of their course, before any activity data has been collected. Activity data gives an indication of how the student is interacting with university services and we can use it to regularly assess how a student’s risk is changing over time.

Learning Analytics - Types of Data

In this project, we will be collecting activity data which gives us information on student Library, VLE and campus PC usage. This data comes in the form of transaction logs so needs to be transformed to turn it into something which is more useful. I’ll write up some posts looking at what I’ve done with some of this data soon.

This image gives an overview of the data sources we are using and are under consideration for the future:

Machine Learning

The project will explore how Machine Learning can be used to build predictive models which use the student and activity data. In this project we are interested in seeing how we can give both staff and students information which can be used to improve student retention.

From a staff point of view, our requirement will be to predict the probability that a student will withdraw from their course, based on their background, enrolments, results and service interaction. In this case, it seems relevant to allow staff to see a probability rating for each student. This allows the staff to make informed decisions about whether an intervention is necessary and what intervention is required, by combining the risk information with their knowledge of the student and their experience. Machine Learning will be used to build a predictive model which can predict probability of withdrawal.

Whilst planning the project we have questioned whether giving students a probability is an effective way to flag to the student whether their learning behaviour is suggestive of “risky” learning behaviour. We want to give the student some information which is more effective at helping to change the way that they use learning resources. Research has shown that the effective use of learning resources, such as the library and VLE, is a powerful indicator of student success.

Instead we plan to explore how we can challenge a student to improve their approach to learning by showing them what they are likely to achieve if they carry on as they are, and allowing them to compare themselves to students who are similar to them but who’s learning behaviour suggests that they will do better. It is well known that self-efficacy (or your belief in your own competence) is related to success in learning. Setting a challenge and allowing the student to see how their peers are doing promotes the student’s self-efficacy and could lead to improved learning behaviours and therefore student success.