Learning curves

While learning analytics has swept through many sectors of education, only recently have libraries truly begun to show interest. 

From the UK to the US, the last few years have seen more and more university libraries joining a growing number of societies, services and projects that aim to educate the library on how to use data to measure and promote student success.

As early as 2011, the Canada-based Society for Learning Analytics Research, SoLAR, had organised its first conference on Learning Analytics and Knowledge, predominantly for researchers. More recently and following several years of development, UK-based Jisc launched what it describes as the 'world's first national learning analytics service' in 2018 for institutions and academic libraries.

And in the same year, the US-based Library Integration in Institutional Learning Analytics (LIILA) project issued a weighty report on how libraries can become more involved in learning analytics programs while the Michigan University-led Library Learning Analytics Project was formed to understand how libraries impact learning.

'This is a really interesting time as the topic of student success and learning analytics are just starting to really gather speed,' says Steve McCann, product manager at global library cooperative, OCLC. 'Recent studies in the US predict that in the next several years, universities could lose around 15% of their student body, just because of demographics, so learning analytics use is important right now.'

But what exactly are learning analytics? With roots in the deluge of educational data on students, these practices aim to gather this information with a view to improving learning experiences and outcomes. 

Typical data sources include virtual learning environment activity, attendance data, and information from student registry systems such as assessment submission and grades. Library visits, borrowing, use of e-resources and information skills sessions can also be used to to shed light on how students interact with the learning environment.

And importantly, this data can then be neatly packaged in the form of graphs or tables that provide a bird's-eye view of, how, say, a student's assignment is going or an entire class is progressing. Any patterns in student output can then be used to improve course design to bolster support.

To this end, OCLC has collaborated with the UK-based University of Gloucestershire as part of the Jisc learning analytics project. To this end, the organisation shared datasets from its cloud-based WorldShare Management Services and EZProxy access and authentication software, with its partners to establish the necessary processes to analyse data, to better understand the library's impact on student learning and success.

As part of this, WMS provided a means to build learning analytics capabilities in the university's library management system while EZproxy logs provided access to detailed usage logs. Importantly, circulation statistics and user transactions from WMS and EZproxy datasets as well additional contextual information were captured to build up a picture of student engagement with the library. And data was also integrated with the student record system to discover, for example, how many students from a certain course accessed ScienceDirect, shedding new light on library resource use across a range of demographics.

The collaboration was a success, with James Hodgkin, associate director of library technology and information, and university librarian at the University of Gloucestershire, saying at the time: 'When OCLC came on board and worked with us on the data and standard plug-in, they really set the bar.' And OCLC is now due to take part in a pilot alongside additional organisations also working with Jisc on its learning analytics service.

In a further learning analytics development, OCLC is poised to launch a new EZProxy analytics service in 2020, which will allow a library to specifically use its proxying service to investigate user behaviour information and decide how best to measure student engagement.

'Libraries really are sitting on great data and it is serendipitous that we are launching this new EZProxy analytics service next year,' says McCann. 'EZProxy especially can provide a wealth of information; for example certain titles can be targeted with the librarian then deciding if they want to track that activity down to a department or a student level, and then communicate how that activity is happening within the student body.' 

Developed alongside the France-based Couperin.org consortium, the analytics features are based on open source ezPAARSE software, from Couperin.org, which ingests, filters and enriches proxy log files to show users access to subscribed electronic resources. Given this, the new analytics option can augment EZproxy log data to illustrate, say, how a library user is accessing subscribed electronic resources via, for example, dashboard-style analytics and graphs.

Excitingly, earlier this year, a recent pilot with six universities, including the University of Manchester, Vrije Universiteit Amsterdam and the University of New England, delivered promising results. OCLC's analytics were used to glean information on logins from around the world as well as access events and insights into the top ten journals, publishers and platforms across a ninety day time-frame. Librarians could also query service availability at different times, security incidents, user preferences for, say, HTML, PDF or Readcube, look at vendor downtime and gauge student success versus library use.

As McCann highlights: 'Libraries can choose to keep a user anonymous, but they can track the urls that a user is coming from and going to, and these will contain information such as the user's platform, the subject's title whether a PDF or and image is being looked at.'

Clearly a resource such as EZProxy Analytics is set to help librarians tackle the challenge of using and communicating such data. However, past research has, time and time again, shown that libraries in higher education must communicate such data as a means to demonstrating value and showing how the library contributes to student success.

According to McCann: 'One of the challenges that libraries have is that they need to be able to essentially speak the language of the administration, so that they can match the library's activities to the institution's mission... but I think there has been some breakdown here regarding this communication.'

Indeed, while libraries often work with, say, other student learning groups, learning analytics still take place without the library's input or data. So, going forward, McCann believes collaboration between different groups on campus is crucial. 'We are actively talking to librarians right now and asking them questions such as how is it that you want to work with us, which stakeholders do you want to collaborate, and what is the data that you want to communicate and how do you want to communicate it?” he says. “We have this data that we can [analyse] using EZProxy and package up in a way that it can be easily communicated to an administration.'

Analytics ethics

But as academic libraries warm to learning analytics, the thorny issue of privacy remains. Due to sensitive data practices, these learning measures clearly challenge student privacy and raise intellectual freedom issues, so how should the library professional respond?

Earlier this year, Professor Kyle Jones, from the School of Informatics and Computing at Indiana University-Indianapolis published an article, Just because you can doesn't mean you should: Practitioner Perceptions of Learning Analytics Ethics, in Portal, Libraries and the Academy. Jones points out how analytic possibilities created by granular data and information flows raise ethical questions, with student privacy rights being a key concern.

'Given that students have neither the opportunity to consent to learning analytics nor much (if any) control over how their institutions use identifiable data about them, concerns have grown that learning analytics might deleteriously affect student autonomy due to paternalistic or institution-centric technological designs or both,' he writes.

Echoing past research, his analyses also highlighted a lack of ethical guidance for librarians, and a need to document and address the potential harms and benefits of learning analytics, so practitioners could work through the ethical unknowns.

Professor Lisa Janicke Hinchliffe from the University of Illinois Library agrees. As she puts it: 'User privacy is a very long-held value within libraries so this question of user data immediately raises questions of how do we negotiate the tension between privacy, which seeks to protect people from scrutiny, and wanting to understand people's experience, which depends on tracking.'

And, as she adds: 'We have this continuous call for training... Time and time again, I hear librarians say: "I feel like I need to understand more about how these two things interact". And how do I as an ethical practitioner align my work to both these values?'

Given their concerns, Jones and Hinchliffe joined forces to apply for a Laura Bush 21st Century Librarian grant, from the Institute of Museum and Library Services, to develop an education program for librarians to address these learning analytics issues. They won, and 'Prioritizing Privacy: Training to Improve Practice in Library Analytics Projects' is now underway.

Prioritising Privacy is described as a three-year continuing education program that will teach academic library practitioners about privacy and other related ethical issues associated with learning analytics. The program aims to provide librarians with structured experiences to reflect on ethical issues intentionally and purposefully, and support the development of privacy protections for their own learning analytics projects.

As part of this, Hinchliffe intends to look at intentionality, transparency and consent. On intentionality, she believes librarians need to be well-informed of the data they are collecting and associated risks so they can make informed choices on how to protect and secure that data, and mitigate risk.

Meanwhile, she also believes transparency and consent are crucial; library users should be aware that data on themselves is being collected, and librarians should also seek consent from the user. 'Your ability to ask a question at the reference desk should not be dependent on your willingness to have that question and your response recorded in a dataset,' she says.

'We protect reader privacy so much as we want people to have the intellectual freedom to pursue their interests without scrutiny,' she adds. 'So from my perspective, the only option is to step up to engaging with these issues intentionally so that we are at least creating a transparent environment with the greatest degree of consent possible.'

As part of the latest grant, face-to-face as well as online training on privacy protections for up to 400 participants will be provided. And Hinchliffe and Jones are also creating an open educational resource package including the training curriculum and guidelines for facilitating training.

'This allows our materials 'to live' if you will,' says Hinchliffe. 'Librarians tend to work with campus partners on learning analytics projects and such projects may involve data from thousands of students.'

'If, for example, each participant in the training conducts a leaning analytics project with 2500 students, the Prioritizing Privacy training will impact the privacy protections offered to one million students,” she adds. “Once you release this into an open educational resource, then the reach becomes very large.'

But as librarians grapple with privacy and ethics, learning analytics are developing fast. Predictive learning analytics take historical and current data on learners and the learning process, and use this to create models to predict how to improve the learning environment.

Indeed, as part of its collaborations with Jisc, the University of Gloucestershire intends to develop a fully predictive learning analytics model. Meanwhile, in its Library Learning Analytics Project, the University of Michigan is to formulate predictive models of the links between learning outcomes and library user-types – with results being shared using only aggregated and anonymised data.

As OCLC's McCann points out, predictive analytics are probably still a way off and many libraries will want to stick with simple descriptive analytics in the meantime, but he is watching this space with interest. 'I would be very surprised if we didn't get into predictive analytics,' he says.

'You can interpolate into existing data to fill any gaps and then extrapolate some of that data to make predictions on what, say, a student has a problem with or what does a certain topic need,' he adds.

But, as McCann points out, the onus lies on the library, as to how to intervene here. 'We will need the librarians to tell us, how do they want to interpret those data, either as descriptive of student needs or as predictive,' he says. 'A question that will be very important for librarians is - will they want to intervene in a student's study? And of course this is a question that we just can't answer.'