Thinking Out Loud, aka The Broad Gaze

Competency based curriculums are on the rise in higher education. By decoupling the learning outcomes that a student must achieve from the class schedule, it is thought that universities might more efficiently use their resources while also providing students the convenience of a flexible schedule. Yet, I’m my experience the technology that might support such a radical change is lacking — particularly technologies that support the assessment of student learning.

Assessment is crucial in competency based programs because, absent the time structure of a face to face class, two things must be present. First, the program structure must be clearly planned in advance, and students must have a way to progress when they have achieved competence in a particular learning goal. Second, there must be a way to assess a student’s achievement of a particular competency and also to report that success to some sort of accrediting system (like the student system for grades).

While there are systems that do all of these tasks, there is not a standard language that each can use to communicate. There currently is no way for learning  management systems to understand the competency structures of one another, or for automated progression through competencies, our for the grading of the  competencies to be meaningfully tracked (meaning, the systems largely do not provide data beyond whether a student passed an assignment — data that could be used to improve student learning outcomes).

I was very interested to see that a standard may be emerging to fill this very gap of a standard for assessment data. IMS Global, a consortium focusing on learning technology standards recently announced their “Caliper” framework for “learning analytics interoperability.”

The issue of growing concern is that the collecting and sharing of data across disparate systems is a real challenge for faculty, administrators and IT leaders. The need for data sharing between applications, platforms and content is vital in order for institutions to evolve learning analytics to improve learner success.

Unfortunately, the details of the standard will not emerge until a demonstration in May, 2014.

One of the thought leaders on learning management systems, Michael Feldstein, sees Caliper as the “learning management operating system” (LMOS). He describes:

an LMOS service broker would have different adapters to accept data from different kinds of learning applications and pass that data to whatever other apps needed it. These adaptors would ideally be standards-based so that it would be easy to plug in new applications from different sources.

He then goes into describe in fairly technical detail how Caliper might solve some of the complexities of sharing data among systems that define their data in different ways.

…what Caliper adds to a triple structure. It adds a collection of ‘entities,’ or things, that all interoperating computers agree have certain properties. And that, my friends, is what makes time travel work. With both a grammar and a lexicon, learning applications can start talking to each other.

Both Feldstein and IMS Global give specific focus to how Caliper might solve the needs for learning analytics, but I agree that it might rise to the level of the organizing “operating system” of learning. While analytics offer a powerful way to use data to help instructors and students improve their learning,  I hope that Caliper might help to bring what Feldstein calls “a more unified learning experience.”

I believe students, instructors, and administrators all have needs for connecting the data from the various systems that they use to teach and learn — especially for assessment or certification. Instructors might appreciate the ability to use the tools that fit their teaching needs, students appreciate the convenience of a unified experience, and administrators might have a single source of data for institutional improvement and accreditation.

In digging out of unread/unwatched material for the holidays, I happened across a gem that perfectly describes the idea of the “learning map” which has been a theme in recent posts. I’d suggest watching all of Danny Hillis’ “The Learning Map” (from OSCON 2012, embedded).

In case you can’t devote 15 minutes to watching, Hillis describes a special librarian who steered him towards new interests at a young age (I wish I had a Mrs. Wilner!). He asks the audience what it might be like if everyone could learn whatever they want, and to have a personalized guide to help one chart their learning. Hillis then describes an idea for organizing online materials to help individuals discover relevant and increasingly complex content, and ultimately to assessing their knowledge to guide future learning.

Hillis envisions the material laid out in a way that would make sense to a fourth grader — as an actual map, where one could see their way forward as well as past progress (he calls it “a map for my ignorance”).

Hillis ends on two topics that are near to my interests — assessment and standards. For example, he points out that assessment might help one who is new to a particular field figure out where to start based on their prior knowledge.

In looking at the open source standards to connect the pieces, he notes that LRMI (the Learning Metadata Resource Initiative …not sure how I haven’t blogged about that yet) might provide a framework for organizing existing online material.

In connecting an organizational strategy for the wide web of educational materials with a means to help one know the depth of their knowledge of a particular area, Hillis provides a path to more holistic learning that one could not gain from taking the occasional MOOC.

Perhaps traditional education institutions still have a role to play in the world Hillis describes. There would be much work to do, such as charting the map, creating assessments, and accrediting learning. Maybe a traditional residential education would still be valued for providing an environment where people can connect to walk through the map together. Or, perhaps there are problem solving skills that could not be learned through the knowledge of the map, and the (likely) automated assessments.

In any case, this is a vision for online learning that might be worth the decades of investment that it would take to implement. I hope to pick up the thread of standards — especially for assessment — in future posts.