Wondering if learning analytics is worth all the hype? Unfortunately, there is no easy answer. The potential benefits and drawbacks of learning analytics should be considered. Examine our break down of some of the potential benefits, challenges, and limitations of learning analytics. We’ve included an infographic with key points as a quick and shareable reference.
Most sources agree learning analytics’ main benefit is that the information allows for actionable insight—or—the educator will be armed with information about their learners and their habits and be able to predict ways to improve their learning experience. Improvements may occur in the form of learner intervention for struggling students, assessing usefulness of learning materials, discovering patterns in learning activities, and more (Yupangco, 2017). Improved learning experience can also lead to cost savings of time and money as learners have more successful experiences.
However, the potential benefits are not assured as learning analytics comes with its challenges to overcome before the benefits can be realized. The foremost challenge relates to privacy concerns. From collecting data all the way to wanting to share data to further research on learning analytics, every step of the process requires careful consideration learners’ privacy. Siemens (2013) goes as far to suggest that we treat the data collected like money—well protected and with clear guidelines on how to handle it(p.1394). However, with the variances in privacy laws in different jurisdictions, guidelines for handling data and ensuring privacy laws are followed can become very complicated ( Ferguson, et al., 2015).
Beyond privacy concerns, issues surrounding the scope and analysis of data are another challenge to overcome. Truly accurate models require high quality data with a broad scope (Siemens, 2013). Institutions may not have the capacity to collect, store, clean, analyze, and visualize data needed to produce accurate models. Additionally, there is not a one-size-fits-all approach to the data needed for analysis. Different disciplines require and deploy different learning activities and analytics of these activities must be tailored for the specific context (Wilson, et al., 2017, p.998).
The use of learning analytics also comes with limitations. A common critique of learning analytics is that the data is analyzed through a business intelligence lens. While this approach provides personalized recommendations, it does not necessarily lead to better results as it is more of a guide than a sure thing (Wilson, et al. 2017, p.995). Even the idea of personalized learning comes with limitations. Learning analytics cannot account for external factors affecting performance (Campbell, et al., 2017). Ultimately, learning analytics is predictive, not psychic. It can predict learner outcomes, but it cannot foresee things like future training needs, or future courses of action based on learner performance (Siemens, 2013).
There are thousands of papers about learning analytics, but few are from the students' point of view. How do students feel about learning analytics?
Roberts (2016) tries to answer this question. He explored students' knowledge, attitudes, and concerns about big data and learning analytics in higher education by conducting four focus groups with 41 undergraduate and postgraduate students from one Australian university. Six key themes emerged from the analysis.
The first theme, “Uninformed and Uncertain,” represents students’ limited knowledge on learning analytics at the beginning of the focus group. The remaining themes emerged following the provision of information, viewing of videos, and discussion of learning analytics scenarios. Three of these themes; “Help or Hindrance to Learning,” “More than a Number,” and “Impending Independence”; relate to students’ perceptions of the likely impact of learning analytics on their learning. Students were in general quite positive regarding the benefit of learning analytics by giving feedback and by supporting personalized learning. But students also feared that getting feedback on bad performance may increase the pressure and lead to negative emotions. Also, students expressed the fear that learning analytics will diminish self-responsibility of their own learning.
The two remaining themes; “Driving Inequality” and “Where Will it Stop?” represent ethical concerns raised by the students. For example, students feared that instructors may have preconceived judgements of a student when they know this student will probably fail. They also were wary on the impact on student’s privacy and stressed the need for informed consent and opt-out options.
Roberts (2016) summarizes that the study showed several areas of tension: Tension between the wish for personalized feedback and the demand that this feedback is unseen by the instructor; tension between the wish for an opt-out-option and the demand to provide equal information to all students; and tension between the wish for personalized feedback and the fear of losing student’s autonomy and self-regulation.
A recent standardized U.K survey confirms the positive notion of students towards learning analytics (https://www.linkedin.com/pulse/what-do-students-think-subjects-learning-analytics-value-glenn-thomas) In particular, 91% of students would be happy for their lecturer to track their progress week by week if it helped them to achieve better grades, and 76% believe better use of learning analytics could be the key to improving retention rates.
Students feel positive about learning analytics if it helps them in their learning.
Source: http://www.kortext.com/wp-content/uploads/2016/12/Kortext-University-of-the-Future.pdf
What does all this mean for higher education institutions? Roberts (2016) argue that the student’s voice need to be included in the decision making process regarding learning analytics in an institution. Educational institutions should develop a code of practice/ethics as well as procedures regarding informed consent before starting with learning analytics. And Prinsloo (2016) makes aware of the fact that learning analytics brings responsibility for higher education institutions: What do we do with the information that (we think) we can derive from learning analytics?
Source: https://www.slideshare.net/prinsp/learning-analytics-and-big-data-a-tentative- exploration
Bennett (2017) presents an example on how to take into account student’s wishes. She conducted interviews with students to identify their preferences regarding a planned student dashboard for learning analytics. Students liked dashboards in general and found it motivating, particularly the on-track scores. But Bennet (2017) found mixed response regarding to the positional (ie norm referenced) information. She concludes that student’s response to dashboard, like all feedback, is personal and depends on a student’s attitudes, beliefs and dispositions towards learning.