Have you ever been tasked with writing up a summary for an event and found it tough, because literally everything shared was noteworthy? +1
CLO Coffee Club VII: Connecting content with skills featured David Taylor (Degreed) setting the stage with content evolution over the last 20 years, Protima Talapatra (Ericsson) tackled managing content curation with the help of Filtered’s Content Intelligence, and Andrew Barry (Curious Lion) was on-hand to get us thinking about the role the learner has in all of this during the workshop and panel discussion.
Suffering from FOMO? No sweat. See what you missed by checking out the speaker slides and full recorded session here.
Here is my best effort (as well as the fourth edit) targeting my favourite bits. Grab your iced latte and settle in!
Content strategy as a strategic imperative
David Taylor, Vice President of Strategic Partnerships at Degreed, started us off with a historical review of learning content, reviewing how how content has developed from an orientation towards transactional consumption to being a kind of systemic enabler. Learning content, according to David, has passed through the phases of Education (digitising classroom content), Engagement (exploring new modalities), Experience (systems and user experience) and Enablement (driving individual and organisational performance).
Around 2014 we saw a big shift to the user experience with a turn towards what it was like to be a learner, versus the learning function presuming it knows best. I got a strong sense of the “voice of the customer” in that moment. David went on to discuss the ongoing expansion of the learning ecosystem through the variety of models used to facilitate learning, such as cohort learning and the 70/20/10 model. Today we are continuing to broaden the ecosystem by including talent marketplaces and talent intelligence platforms.
Having sped through this timeline of digital content's evolution, David shifted focus to the current situation of content overload. We are now in an era when anyone, and I mean anyone, can be a content creator. TikTok. Instagram. Facebook. LinkedIn. This level of crowdsourced content is the wild, wild west where almost anything goes when it comes to content. The good, the bad, the ugly… it’s out there. David did a great job of driving this point home that the intention versus impact. The intention of the creator is often good: someone has something useful to say on a topic and there is a platform available for them to do so. The result, though, is a vast sea of readily available content. It's chaos. But there's plenty of silver lining. For example, we learned that content can be built quickly and cheaply, debunking the myth that content must be perfectly polished to achieve its objective.
Having demonstrated that we have no choice but to "embrace the chaos", David borrowed a four-quadrant characterisation of content from RedThread as a part of his own "Four Elements of a Successful Content Strategy". You really need to listen to the recap to get the best of out this part!
It was a fantastic methodology for modern learning content strategy. I liked how he talked about using a clear, simple call to action to empower your audience. As a learning function, you need not do it all (which so many of us have been guilty of) - seek out those who can support your efforts. David pointed out the need for a skills taxonomy underpinning the strategy, and this became a big discussion topic afterwards. In particular, I loved David's call to developing an assessment strategy to test whether content is truly helping elevate skill performance. And also the Marie Kondo question: “does this spark joy” when cleaning your content house. Naturally, David had a very clear idea of the tech stack required: LXP for aggregation and skills activation, CI for metadata enrichment and improved search, LMS for learning administration. My dad always says “work smarter, not harder.” The same can be said with your use of tech. Get your tech to work hard, so you can work smart.
Managing the content curation journey through AI
Protima Talapatra, Head of Learning Delivery Transformation, took the keynote baton to walk us through her proof of concept using Filtered’s Content Intelligence tool with Degreed. She started us off by sharing four hard facts related to democratising content curation:
1) This is essential in ensuring a thriving employee-owned learning culture,
2) LXPs can play a fabulous role,
3) LXPs are often overloaded, and
4) Content cleanup on the LXP is a Sisyphean task.
Basically, everyone wants to come to the party and no one wants to stay to clean up. Knowing these hard facts, Protima and team set out to learn how well could Filtered’s Content Intelligence (CI) tag Degreed content using Ericsson’s critical skills and levels and how could it affect discovery through skills tagging.
First step was to cast the net and see how many fish they would catch. Protima focused on three critical skills for her example: software & component security, business storytelling, and design thinking. The haul from the initial casting was an astounding total of 143,013 assets. Next, the CI tool was given the skills’ definitions and criteria and then they threw the net out again. The CI-supported catch found only 1904 of those assets were relevant. Doing some quick math, that gets to about 1.3% of what they initially got back. Impressive, right? But wait, there’s more!
Next, the human SMEs got involved. They reviewed what the CI tool brought back and judged its relevancy. On average, they found about 60% of what CI found for them hit the mark. They noodled on why there was a difference between the human SMEs and the tech, then reconfigured the tagging. Hold onto your hats for this one. The updates increased accuracy by about 22% resulting in a mind-blowing average of 82% relevance!!!
The next test for CI was content levelling. Same song, different verse if you will. They applied the same approach to classification by level. Classifying by level was much harder because there isn't a human benchmark: SMEs agree between themselves only 50% of the time. This proved to be a valuable experiment - where humans can't go themselves (in this case: agreement on levels), AI struggles to tread.
Summarising how CI helped, Protima said:
In terms of tagging learning assets to skill, the Content Intelligence tool within Filtered does a marvelous job. It has almost human to human accuracy there.
Thanks Protima! I felt the success of the POC raised another interesting question: what better things could L&D be doing with the time that is currently spend in the weeds, searching out content?
In summary
As usual, the session expanded beyond the topic of connecting content to skills. We had a fascinating conversation about how you define a skill, how granular you need to go, and the endless pursuit of a common language for skills that is representative of reality. These are some of the most urgent, interesting and controversial conversations at CLO Coffee Club - and we'd love for you to join in some day.
But we all agreed there is a plethora of content out there, and it’s messy. Newer technology has allowed for anyone who has something to share a platform to do so. How does one make sense of this ocean of content? How do you find the fish in an ocean that is littered? Let technology do the heavy lifting! Protima shared what is possible when you blend Content Intelligence with human efforts. What else could your learning function be accomplishing based on that time savings?
Could we have kept peeling back the layers of the content-skills-data onion? Undoubtedly. That is why we will build on what we discussed for our next coffee chat. This will be a hybrid event on 13 September, including Don Taylor, on the heels of Learning Live. Details to come soon!
