If there’s one thing there is no shortage of at the moment, it’s information. We are all drowning in it. A similar scenario is playing out within many organizations and their learning content. Old, new, custom, off-the-shelf, learner-generated … it seems endless. But, as in most cases, there is no such thing as too much of a good thing. It can be a daunting task for admins, subject matter experts and learners to make sense of everything available to them. And it’s not just the learner experience that is affected.
Content overload means that the task of getting the right material to the right people when and where they need it can be challenging, time-consuming and costly. At a time when L&D teams need to stretch their budgets as far as possible, it makes sense to find areas to boost efficiencies and ways to prove ROI. The sea of available content is just such a place to look. According to Brandon Hall Group’s Learning Benchmarking Study, content accounts for about 25% of the L&D budget on average, reaching seven figures in many large organizations. Does your organization really have a handle on what you’re doing with that spend?
Thinking more intelligently about the right learning content rather than buying more of it solves two huge challenges for companies. Firstly, identifying duplicated content spend and finding more efficient libraries that better match your learner needs can generate huge cost savings from new efficiencies. Secondly, digging deep into skills definitions and metadata makes it easier to map content to the skills that the business needs to be successful and make tangible ROI an achievable reality.
Additionally, when content is not adequately mapped to the needs of the business, legacy content is hard to find and use, and often gets recreated or left to decay. This duplication of efforts presents another layer of inefficiencies. An intelligent content strategy filters out the noise and gets people moving more quickly toward reskilling and upskilling.
Speaking of filtering, I hosted a webinar Marc Zao-Sanders, the CEO of Filtered. We looked at some of the pitfalls and challenges that companies face when it comes to getting the best return on their learning content spend, as well as strategies and tools for digging out.
You can watch the recording below:
This blog originally appeared on brandonhall.com.
We have also added the full transcript below so you don't miss a thing!
Melissa: Hello everyone. Thank you for joining us on today's webinar maximizing L&D spend with a data-driven content strategy. Our presenters today are David Wentworth, Principal Learning Analyst at Brandon Hall Group, and Marc Zao-Sanders CEO at Filtered. I would like to extend a thank you to Filtered for sponsoring today's webinar. Filtered helps organizations maximize the impact of learning spend by deciding on the right skills and matching them to the right content.
They drive end-user engagement through a focused LXP and personalized emails backed by powerful curation.
Now, for those of you that aren't familiar with Brandon Hall Group, we are a research and analyst firm that empowers excellence in organizations around the world through our research and tools. A quick mention that our certification programs are currently open and we do offer programs to become a certified learning strategist, a certified diversity equity and inclusion leader or champion, and more. Please visit our main website for more information.
Without any further delay, I will turn it over to Dave so we can get started.
David: Great. Thanks, Melissa. As always I like to leave this up for a couple of seconds here. Just familiarize yourself with that control panel. That is the way you're going to ask Marc and I questions today or you just share any thoughts you might have any insights that you want to provide. That's the way you'll be able to communicate with us so we're going to keep an eye on that question box throughout and try to answer any relevant questions as they come in.
We'll also, have a little bit of time at the end to answer any questions we may have missed or put on hold or any new questions that come in at that time. Again, please feel free to use that. We'd love to hear from you, we love the questions. We want to make this interactive as possible in this webinar format. As Melissa had mentioned, Marc and I are going to be talking about an intelligent content strategy. One of the big reasons why we wanted to talk about that is, there's such a big shift currently going on within L&D and I think all of you are acutely aware of these things.
I wanted to use some of our most recent research to lay out the environment in which this conversation is taking place and why there's an impetus to think through these things. Maybe take another look at the way your organization either thinks about or approaches your learning content. First, we'd take a look at the way the learning budget is broken out. In our most recent state of learning practices study, we asked organizations to break down their L&D budget across three very basic big picture buckets.
You've got your headcount which makes up the biggest group portion of the budget, nearly half, and then the rest is split across learning technology. Specific tools and technologies that companies are using for learning, whether it's creating the content, delivering it, what have you. Then the rest is in that non-technical spending. You're spending content administration, maybe you're doing some outsourcing.
That 30% represents a lot of organizations, a large amount of money, a large amount of spending. It's also typically the spending that's the hardest to keep track of. There's so many moving parts, so many different things happening there that maybe don't have the same governance as something like technology or the headcount would have. It's really important in areas where we can spend a little time and really dig into, what is the spend? What's going on there? How do we maximize that?
I think 29%, that purple, is an important section of the budget that tends to get overlooked because it's not as clean as the others but that's what makes it so opportune for reimagining and streamlining. That's where we want to start today to think about these things. Obviously, the past 18 months have been pretty dramatic in what has happened to organizations, a lot of different changes in a lot of different directions. One thing we did see is for years L&D budgets have remained relatively stagnant not a lot of increase but also not much decrease either. There's been a long history of L&D trying to do more without any increase in their budget. Well, that's actually been completely exacerbated over the last 18 months or so. We asked companies if they've seen their per learner budgets, so how much you're spending per each learner per year, has that been decreased or increased during the pandemic. For nearly half of organizations, it's actually decreased. We're significant seeing a lot more decrease than we have in the past. In fact, only 10% of companies said that they've seen an increase in their per learner spending budget.
Then we asked for those organizations that say that they do have a budget cut of some kind, of how much? For most, 40%, the largest group anyway, it's less than a quarter of the budget but then you see even 15% have seen more than 75% cut in the budget. You add that to the 11% that have seen anywhere from a half to three-quarters of their budget cut, there's a significant cut in the resources that are available for L&D. This is all happening at a time when organizations are relying even more heavily on L&D to accomplish what needs to get done in this time, whether it's upskilling and reskilling the workforce because the business needs have changed. Keeping workers connected, because they're now a majority remote, they hadn't been before and learning has the tools and the infrastructure in place to reach out to those kinds of audiences or they're getting leaned on through that so well.
Learning strategic profile is increased their resources and their ability to do things has increased, which just puts the things that we're talking about today in much sharper focus, the idea of where can we maximize the spend that we've got seeing as how it's being strained to the limit as it is. Marc, at this point, this is where I would just love you to come in and just say hi, and let me know what you think when you see these things from the budget, whether it's just your reaction to the data or the things that you're hearing from companies that you work with or what you're hearing in the industry.
Marc: Hi, Dave. Thanks. Hi, everyone. Yes, we're definitely seeing that there's some pressure on budgets so just like your data is revealing. We're also hearing though, that it's not just less to spend, sometimes it's the same budget to spend but there's more scrutiny on the justification. The business case needs to be that much tighter, there needs to be some more data and we'll be talking about that over the course of this session. Yes, it's not just that it's less and you have less money to spend but also look if you're going to spend anywhere near the same amount or the same amount, then you need to use justify better, and data is obviously a big part of that.
David: Yes, absolutely. One of the other more specific pressures that organizations are feeling in this environment is clearly-- If you go back prior to the pandemic, there had been a lot of discussions and a lot of talk about the digital transformation of learning, of the organization as a whole. Progress has been relatively slow on that front, organizations have been mapping out what their plans were, what they thought they might do but had still been relying on mostly their traditional methods of delivering and measuring learning. Then when the pandemic hits, whatever plans they may have had, whatever thoughts they had have been just shifted into overdrive, and there had to be an immediate shift into a digital transformation.
We saw that go on, and we looked at our studies, and this-- We've been asking this, throughout the pandemic, starting in March of 2020. This data is more recent, as of the middle of this year. We asked, "As the pandemic eases, whenever and however that may occur, what do you see happening in your organization in terms of digital learning? What will be the trajectory of your use of digital learning?" What we have here is the 51% saying that it'll continue to increase, it's actually going to increase more than what we're doing right now in the current environment but if you look overall, only 8% of companies say that their use of digital learning will probably go back to what they were doing prior to the pandemic.
For everyone else, there's going to be more digital learning than there was in the past, whether that's significantly more like a continued increase after the pandemic, or maybe when the pandemic eases off, they roll back digital learning a bit but still not down below or to pre-pandemic level. At the end of the day, the result is whatever organizations have been doing from a digital learning standpoint, they will be doing more of that in the future, and that requires a whole lot of new ideas, strategy, rethinking. Of course, as we talked about the topic today, that has a huge impact on learning content, what that looks like, where it comes from, how it's managed. There's a lot more scrutiny on, do we have the contents available to us to exist and to be successful in a more digital environment for learning?
Marc: Yes, is it going to be good enough? The way I was reading that slide, Dave, just go back one, was just the middle two columns say that three-quarters of people will be doing more digital learning than the pre-COVID levels. It's not just the amount in-- If they're doing more than, obviously, we want to get them looking at the right stuff that's going to really make an impact and I think we've talked about that a little bit later on as well.
I think there's also more firms that are thinking about their role in society and this concept of the extended enterprise. It's not just about educating your workforce, but maybe the families of your workforce, or just wider society as a whole. That will further increase consumption of digital learnings and even more important that we know what's good and what's relevant and what's going to make an impact on people. Those middle two bars for me, three-quarters of people can be doing more than before the pandemic hit.
David: Yes. You're absolutely right because not only is it just that there's going to be more use of these tools because of the environment. You're 100% correct that the types of things that organizations will find themselves, the breadth of what they are trying to deliver from a learning standpoint will grow. We've seen an increase in the need for things like coping skills, resiliency, the idea of how do we make sure that employees feel that they are included and that they belong. Learning has a lot to do with a lot of these things. These are areas that may be the L&D department hadn't been focused on previously, they've increased in importance so that just broadens the spectrum of the things that companies need to pull together to operate in this environment.
One of the things that we found, we had asked organizations about their learning tools and technologies. What you see here is low impact and high impact. What that means is, the companies in green, those high impact companies are the companies that say, their learning approach is having a strong positive impact on a series of outcomes like employee engagement, individual performance, time to productivity, things like that.
In the grey, the lower impact companies say that their approach is not having that strong of an impact on those outcomes. We want to see what are those companies doing differently in terms of their approach. What we see is that these high-impact companies are far more likely to be using a wider variety of tools, technologies, and modalities, video for learning, the differences is massive. 79% of these high-impact companies say they use it regularly or frequently, as opposed to 21% of the low-impact companies. You see the same for coaching and mentoring opportunities, informal peer-to-peer learning, microlearning, it goes on and on.
As organizations recognize the effectiveness and the impact that these kinds of learning experiences have, again, it calls into mind, do we have the types of content, the material that can support this, that fits into this that works in these kinds of environments, because if we think back just a few short years, or even currently today, most of what organizations have available to them is what we'd call legacy content, the big SCORM packages, learning programs from the e-learning standpoint that don't necessarily lend themselves to any of these modalities that we're seeing on the screen.
Marc: Yes, there's so much nuance to it. If you look at any of those categories, there's such a thing as actually, this data is showing you that there's low impact video learning and there's high impact video learning, the same is true for any of these modalities. I was slightly surprised actually that coaching and mentoring doesn't have an even higher impact score because it's by far the most labour-intensive and costly of the modalities here. Really, the point I want to make about this slide is that, with each of these, there's some brilliant content if you find it and that really is, the curator, in particular, that's the nightmare and the opportunity is finding the really good stuff.
A slide like this is really good use data to start a conversation, but if you're thinking about your company and your content, the content you have access to that you want to bring to your workforce, it's thinking about the quality that lies beneath this and how do you get to the really good stuff. Like you were saying, Dave, is there enough given the increased demand for digital learning. Enough of the right stuff.
David: Yes, precisely. When we think about that question, is there enough of the right stuff? We take a look, we asked organizations, where is your content coming from? How are you creating it or where are you purchasing it? What is the source of the content? For the vast majority of companies, 79% say, it shows, "Yes, we use some PowerPoint type presentation creation tool." We're basically building slides, presentations, for our learning programs. That's how most of the learning programs exist.
Then you see things like a video capture tool or a desktop authoring tool, which would be a more formalized learning program created but still essentially the same click-through type of content. 55% say they get it right out of the internet. We're pulling content from the internet that we think is valuable or they're using a cloud-based author tool to allow multiple authors to work on the same content at the same time.
Maybe they're going to a third party to have something custom-built. You've got 41% say that they're doing that. Half of the companies say they're purchasing third-party off-the-shelf content. What this does is just eliminates the wide variety and complexity of the content that is available. It just showcases two things. One, is that there is a vast majority of legacy type content. Static large event-based type learning content available but there's also a very wide variety of places that it comes from and ways that it's put together. That can create a very big, broad, deep complex content environment for a lot of organizations to try to manage, trying to figure out.
Again, this goes back to why this is an important thing from that budget standpoint that spend because for all of these items that exist, there has been some investment in that content, but because of the nature of it, and then it's hard to manage, hard to find. A lot of times, it gets replicated and things get rebuilt because companies aren't able to manage all of this content. They're actually duplicating their efforts and they're actually recreating the spend and there's a lot of waste that goes on there because they don't have a great content strategy. That's why we wanted to talk to Marc a bit today about, well, how can you think about content in a better way. As Marc puts it, use Content Intelligence as you look at the content that's available to your organization. Marc, I was going to have to turn it over to you for that.
Marc: Oh, cool. I'll definitely talk about that in a second. If you could just go back one slide there. Seeing this, I don't really appreciate it how many different ways there are with making content. The way that we think of it is three buckets. There's proprietary content, which is created by the company or someone at the company.
Then there's the libraries and then there's the web, which has picked up in this. Actually, most of the items here are ways of creating content from within the company for that company. The interesting thing that we find with this is that that content as you would probably guess but our data bears out, is the most relevant to that firm and to their industry because they created it themselves, that makes sense, but it is the least well tagged. It has the least amount of metadata.
You can see that there was a big opportunity if you can lift a lot of that content, which is the most relevant created by your workforce, lift it out from obscurity, and surface it in the right way for the right people. Yes, we think of it in three buckets. There's a simplified version of this slide that we have. I think it's worth thinking about that the stuff that your colleagues create as really important. Not very well understood because of the lack of metadata because if you think about the internet that needs to be well tagged because otherwise, those web pages don't get found. With the library content, that needs to be well tagged because producing it, that's part of the system. Within a company, there's less process about the publication and creation of content.
David: Yes, absolutely.
Marc: Yes staying with me. Content Intelligence, is something that we've created. It presupposes that there is such an idea of something like the right content for an organization. What we mean by that is that it's aligned to the skills that your organization needs to develop and to the quality that you need it and in the modalities that your workforce enjoys and get something from. A foundational part of that, I've mentioned that already in this webinar is tagging, and tagging content at scale. Tagging it as well, as a human being can tag content and obviously a lot faster.
The reason it's important is that that's foundational. If you understand your content in terms of important tags, in terms of the high-value skills of your organization, you can just get a lot more from it. One, that data in and of itself is useful for making internal business cases and being persuasive, but then more tangibly it's curation. It's aggregating up the level of the library so you can say, "Well, this particular library," and I won't mention any on this call but they're less well aligned or more well-aligned to the high-value skills that you're thinking about or should be thinking about.
You can see the data to make some spend decisions, procurement decisions with that data-driven decision-making that people talk and talk about. Then there's discovery as well. If you've got well-tagged content, then people are going to find it because it's tagged, it's in the way pathway, it's searchable. Again, it's just about surfacing the most relevant content for a user, when it comes to discoverability so there's several benefits. Overall, like I say, I think one thing that's missing or been missing for a lot of people working in L&D for a long time, is having the data at their disposal to make the right decisions. This is part of what Content Intelligence is about.
David: Yes, I think that's so true that one thing that we hear when we work with organizations, is some of these decisions that they're trying to make are very difficult for them to make. Because they either, I don't want to say, don't have the data because typically, the data exists somewhere but they haven't gathered it correctly, and they haven't been able to analyze it, and then use it. What this does is by taking this kind of approach, that data-driven approach, makes these decisions far easier to make, and then you've got that as your foundation, that backup as to why the decisions get made. It just continues, the more that you go down this path, the smarter the process gets because it's continually built on that data and you're continually collecting and adding to that and analyzing it so it just gets better over time.
Marc: Yes, exactly. I wouldn't say it makes the decisions easy, but it does make them easier. Very often, I was going to actually talk about this in a few slides time, but I think it's come up now, Dave. Very often, there'll be a hunch from someone that works in L&D that such and such library is not that effective for our workforce but they know that there are some big fans of it still within the company. You have this inertia of like a three to a five-year deal that then gets extended and no one dares get out of that contract because someone at the company is going to get upset if it happens.
What data does is give you a way of comparing libraries so that you can make a decision, and then if someone's not happy with it, you've got the data to say, "Look, this is why we made the decision, this is what the data said." and you can scrutinize that but at least, it wasn't just on a whim, or just an opinion. That's the empowering thing that this little thing can do. There's loads of data at the moment, but it's very hard to compare like for like, that's the thing. If you get the skills and actually that's part of the next slide, Dave if you move on to that.
It's a cliche in L&D that skills are the common currency. We've talked about a lot in those slides but what we mean by that is that they're common about the role and to content. Fundamentally, if there's a job and it's about such and such skills, and there's content and it's about such and such skills, and you're trying to match the two as one of the means of developing the skills further. If you've got that common currency, then you can compare different libraries or different actual learning assets, learning objects for relevance to a particular set of skills that you want to develop at your workforce.
The first step for us and I think for a lot of vendors that are either selling an LXP, or even an LMS, or content, the conversation is should start, at least, with the business problem that you have, the priorities that you have in mind, and the skill that you need to develop to achieve those. It always starts with us, with the conversation about skills. We have our own skills framework, very often clients will have their own preconceived ideas, and there's some match between the two.
I would say though, that although every company is different, there's a huge commonality with skills. I would say something like 80% of the knowledge worker skills, sound and are very similar from large or medium-sized company to large and medium-sized companies. One of the things that we've noticed is that companies tie themselves in knots about what framework to use and spend years, literally years, and millions of dollars, pounds, euros on trying to decide on the perfect skills framework but one, no such thing exists.
Two, it's going to change because there are always new skills that come. You can make a lot of progress by just saying, "Look, here are some important skills that are important for us and they're important for a lot of other companies out there, get going with that and just make sure that the system that you have and whatever system that is that you use, whether you're working with us, Filtered, or you're working with anyone else, make sure it's extensible and flexible because there will be new things that happen and as we've seen lots over the last few years and last year and a half, in particular, the world can change suddenly.
David: Absolutely and that is reflected in some of the data that we've collected, we asked organizations what was most important to them on a scale of one to five, one being not at all important, five, being critically important to the business and you can see leadership development, leadership skills, the number one. Course, that obviously includes a vast array of sub-skills, things like emotional intelligence and design thinking and empathy and all those kinds of things.
Then also organizations are very concerned about this upskilling and reskilling for the organization's future needs, I think it really gets at what Marc, you were just saying about that framework, that desire to have this permanent list, this taxonomy that helps them map out what the skills are but the truth is, you can't really know that. It's really more about, like you said, creating an environment that allows the organization to map and build to those skills as they become apparent rather than having this existing list.
Being able to upskill and reskill as needed, that's important, but also training in the skills that we need now. Then you see some of the others, team development, D&I is very important. Soft skills which depending on how you look at them, some of them are quite critical but you see there's a list of different things and things that are on this list that maybe more honest was previously or maybe not as high on this list, things like coping skills. I mentioned those skills around D&I have really grown in importance in recent years.
As Marc was saying, you may have an idea of what is important to the organization but it does shift in priority over time and you need to be able to have a learning environment that adapts that. This mapping that Marc is talking about is what's going to allow you to be more flexible and agile as these things change over time. Anything here that sticks out to you Marc.
Marc: Yes, a couple of things. Leadership is top of every single list in L&D, where skills are mentioned. It's only by 3%, in this particular case but it's generally at the top. I think again if you're thinking about your company, you need to think beyond leadership and to break it down I think you were alluding to that, Dave, but what kind of leaders are especially going forwards in the future going to be most effective at your company. What are you hiring for? What are you trying to cultivate? Is it the visionary leader, compassionate leader, commercial, managerial, mindful, well networked? There's so much again, nuance underneath.
This is where if you're thinking about content, you're going to have on your system, a bunch of content that's tagged to leadership. That's not that useful, though because although people are searching for leadership, if they just searched for leadership, and they got 600 search results that are about leadership in such a big blanket way, it's quite hard to deal with that and process that as a user. If instead, you've encouraged the workforce, one, you've tagged your content, so it's much more nuanced. You've got some things that are like mindful leadership. Then you've also developed the culture of the company and the trust of the company and the systems that you have, that the search query is more likely to be mindful leadership or compassionate leadership, then the results are so much more relevant and the experience is much more like you get with Google. In Google, you can put in really specific search terms and get something meaningful back.
I guess what I'm saying is leadership is a big bucket. I mean, a number of these are quite big buckets and again, but the Brandon Hall Group research is great for starting the conversation, but as you're thinking about your firm, you need to drill down. The other thing, Dave, I was going to say was about coping, actually. Like you said, it's often talked about as being, I would describe it as with a different word, resilience.
Thinking about skills at the outset, it really matters. The naming matters, actually. There's a big difference between the meanings or the associations, the connotations of coping and resilience and what you would tag with those. With coping, just imagine if you tagged a load of internet content that was about coping, you're going to get some pretty raw stuff that comes up. Now, that might be fine for your context in your organization or it might not be. This is why being intentional about your skills at the start, which skills should we have, what should we name them, and what should we mean by each of those, it actually makes quite a big difference to the end-user experience which is often overlooked, in our experience.
David: Absolutely. Let's dig in a little bit into this framework that you'd started to mention.
Marc: Developing a skills framework, there's two models. This is the complex way. There's a complex way and a simple way. A simple way is just to say like I was actually advocating before, there are so many skills that are common amongst knowledge workers and workforces, generally. You just get started with them. We have the skills palette as we call it of 100 skills, each of which can be tweaked a bit. 100 can become quite a bit more than that quite quickly. That I think is the more, I wouldn't say sensible, but it's the fastest start to getting to a skills framework.
The more complex way which is on this slide is especially looking at the data that pertains to the skills that you want to develop. That's feature skills, it's any skills framework that you've already got in such queries that people put into extranets, and intranets, and SharePoint, and LXPs, and what have you, is what else is in job descriptions. It's just synthesizing a lot of disparate data sources to come up with this one skills framework that's going to be still flexible, but really taking in a lot of data in person. We've done this with a number of clients.
Like I said, there's another approach, which is maybe the next slide which is more take things as they are. What you've got there is some of the skills from our Skills Palette and a lot of these terms, data security, business intelligence, they'll have similar names, you might think of them with a different couple of words or single word. What we've got underneath each of these is a definition, which is partly numeric, actually, and partly words associated with that skill. If it's not quite right what we have which is, what we mean by business intelligence doesn't exactly match what you mean by business intelligence, then that can be tweaked.
Actually, the example I normally give, which you can't see on this slide is resilience actually. Resilience can or cannot include sleep management. You may want it to and you may not want it to and there's all sorts of arguments for and against. I think in our definitions of resilience, we do include some element of sleep management, but let's say you didn't want that at all or you just cut that out, but the point is to be intentional about it.
We thought about resilience. We considered sleep management as being something that our workforce should learn more about and sleep better, but we decided to cut it out for whatever reasons so that you end up with a skills framework which you really mean and is right for your workforce rather than just of taking a definition or a skills concept from a vendor. I think being flexible is important.
David: That is something that we see all the time as an advisory organization. Companies come to us and they ask for those lists and those taxonomies. That's the conversation that we have, is that they do exist in places and you can get them but does it make the most sense for your organization? Obviously, people like easy answers, simple answers. Getting a list that already exists makes that work simpler. The truth is, does that really apply to how your organization thinks about those things, the skills your organization really needs?
That's the other thing we find in these conversions, is that organizations are relatively unique from one to the next in what they think is important, what works for them, what their vision, their mission is, and do these things align. The ability to take maybe a foundationalist and rework it so that it matches better to your organization's DNA is critical. It isn't about just taking these lists and storing them away and using them moving forward. You really have to shape them to match the organization and revisit that and change it overtime if necessary.
Marc: If you don't have the granularity or the vendor you're talking to doesn't allow you access to be able to see what they really mean by some of these, then that'll be difficult. Another one that's on a lot of people's minds or has been the last two, three, maybe five years, is a growth mindset. Growth mindset has in the term growth mindset, mindset. For a lot of companies, it's become an important part of how they think about skills and development. Sometimes, there are some very strong opinions about what a mindset is as separate from a skillset.
If there are those strong opinions at your place, then it's even more important to be able to be intentional about what you mean exactly by growth mindsets. Include, for example, learning agility. Does it include resilience? Is it inclusive in management? All of these questions which don’t need to take that long to do, by the way. I don't want to say that tweaking your skills framework needs to take months or even weeks. This kind of work can be done in a week, but you need to sit down with that list and work through some definitions and take some stuff out, put some stuff in, be happy with it.
David: I think that's one of the other issues that we always hear about, is that when we talk about these ideas and these strategies, that the challenge with them is that they require some upfront work that is difficult just from a time standpoint, a head count standpoint to do but it pays off in the long run. It makes it easier to continue to do but yes, there is some work that needs to be done upfront.
Marc: I'd even say with that, Dave, yes, you're right. There's paying the price upfront makes a little sense. For most companies, even large companies, my advice would be just get going with something early on, make sure it's possible to tweak it, it's three months down the line, because you cannot forecast how things are going to go exactly. If you've got a pretty good idea of what growth mindset is, then just go with that, go with a skills framework, get it tagged and make some progress. You'll refine it as you go. That's exactly the agile mentality and methodology. I think it's even better suited as time elapses and we go further into the future, staying agile is essential now.
David: Talk to us a little bit about using the content to drive the skills.
Marc: Well, I think if you've understood content, if you thought hard about the skills and what you mean by the skills, so what we've just been discussing, then you've got something that you can tag. You can't really tag until you have and you've got to that stage because what are you going to tag with? You've got those terms, let's say there are 50 skills, so then, we need to know what content is really going to help move the needle for your workforce with those skills, and you might be facing 100,000 items of content to search results. There's just a lot of them on your learning system.
What are you going to do? Well, one of the things that you can do is work with us to tag the content. If you tag all of the content with all of the relevant skills, then you've gone a long way to work out how useful that content on mass in totality is for the skills that you want to address. From that point then, there's all sorts of things that you can do. You can say, "Well, these providers should be in and these ones should not be in." That's the procurement piece that I talked about.
Or, you can do curation. You can say, "Well, we really need to build some playlists or pathways or run some campaigns on such and such a topic. Now, we've got the content tagged with those topics, here's a ready-made list," which probably a human being is still going to need to go through and make cohesive and add some commentary to, and some company context, but it massively accelerates the process. The idea is to get the skills first, tag the content, and then you can line that content to those skills and deliver that in various ways, just for curators. It could be for the workforce through an email campaign, could be on an LXP, but it gives you the data and the power to be able to do that.
David: Coming back to some of the things that we've found as to why this can be so important and so impactful, we go back to those high impact learning organizations where learning approaches having a strong impact on those outcomes. Again, some of the things that occur that are different in those organizations, they're much more likely to have a better ability to search and explore and discover learning opportunities, and that experience can't exist without the things that Marc's been talking about, because you probably experienced it in your own life.
If you get overwhelmed with irrelevant search results, it's impossible to know what to look at, what you should not. You can spend the majority of your time doing the searching and not actually using the material that you're finding. We know that the way that people interact with most applications, web pages, what have you, is through search. You open a learning platform, you open a web page, anything that you do, you'll be presented with some sort of dashboard that will show you various bits of hopefully relevant information.
If you don't find what you need right away, the very next step will be search. That experience has to be really well done and without this kind of tagging, you can't expect learners to be getting the right, relevant returns in their search. The other thing that these companies do, and you can see the difference here, 57% versus 15%, is providing learning recommendations. Again, recommendations, very difficult to do without the proper tagging to set to know what the crossover is between one set of content and other where you know that this was effective, or this you found useful, let's find the other things that are tagged like that, and we can recommend those to you.
Also, a personalized learning plan where learners can track your progress. That personalization can't really occur without a proper tagging and alignment strategy. We find that this is a very impactful learning experience and it should have these elements and without things that we're talking about, that can't exist. These are some of the things that you see as outputs when you're working with clients, Marc?
Marc: We think about search and recommendations really a lot. Actually, the difference between the two is actually really important. I think it's interesting because a search implies that there's a very specific intent from the user and they've got a specific idea in mind of what they want to find. I key in a specific term like agile project management and I hope the system is going to pick up on that intent and return me useful results.
The problem is, and I would encourage everyone to gather some data on this on your own survey on your workforce, how happy are they with the results that come from search queries on your system? Very often, it's either too much or nothing at all that comes up and in many cases also just not relevant stuff. Maybe your system is working perfectly, but getting some data from end-users on search results is really interesting. It'll uncover whether you've got a problem or not, and whether you maybe don't need tagging whatsoever, but I would say in many cases, you will, that search where you got a really specific item in mind.
Recommendations come in when the user doesn't have a specific item in mind, they're browsing on the system and they're just there. On the basis of profile and probably historic usage of the system, the learning system will make some recommendations to that user, but they will be a bit hit and miss because it's not as specific an input as a search query, but both of them require nuance, tagging that make sense for the company and for the user. Not just a company set of skills that make sense for the company, but relate to search queries that the user actually will put in. Tagging is fundamental to either of those, but I think it's worth thinking about the different use cases and different user experience, search versus recommendations and browse.
David: Let's talk a little bit about improving discoverability.
Marc: Sure. I've touched on it already or a little bit anyway. I'm going to say something which sounds a bit like a tongue twister or a riddle. Anyway, you need to get your head around it, but tagging and curation are really two sides of the same coin. If you associate an asset like a learning object with a skill, that sounds a lot like tagging because you're taking, say, an article on HBR or whatever and you're saying this is about growth mindset, or resilience, or whatever. That sounds like tagging in a lot of people's heads.
Imagine the other way around now, that you associate a skill, say resilience, with an asset so you start with resilience and then you say this asset is relevant, and this asset is relevant, and this other asset is relevant. That sounds like curation but, really, you're doing the same thing. You're just associating the two, a skill and an asset, they're two sides of the same coin. I like to think that that's a useful way of thinking about this stuff for a start.
We're trying to improve discoverability in two ways. One is- first of all, is just by developing a methodology and technology which will do all of that tagging stuff, the tagging, curation. It's been described by some people as L&D voodoo, but actually, it's not that mysterious. If you give us a couple of skills and some content, we can run Content Intelligence on it and give you some results free of charge. For example, I'd say that actually in general, there's a lot of vendors that talk about, and many cases have, AI, but if they can't point to the benefits and give you some tangible output, I think it's less appealing. I would feel that that's less appealing as a buyer.
Once you're convinced that it works, you can see that curation is going to be done to a higher standard because you're uncovering those gems of content that are really good and pertain to that particular skill but we're being lost before. It'd also get done 5 to 10 times quicker because you're saying, "Curator, hey, here's a already made list. You need to just choose from this list rather than start from scratch," and then going down the line with end-user discoverability.
What I've said so far is more L&D teams to build campaigns and pathways and playlists and what have you. Further down the line, you want to improve the experience of the end-user, so that'll be where you enrich the metadata and then you re-upload that to the LXP like Degreed, for example, and then on that platform. It's not our platform at all, the end-user experience would be on whatever that LXP is. Then all of a sudden, the assets are just far more findable. They were searchable before but they're now findable because they are better tagged. That's where discoverability comes in.
To be honest with you, we find the greatest and most urgent pull from the market is when there's an RFP around content and so there's an urgent, "There's a specific thing that's coming up. We need to make some decisions about content. Can you run Content Intelligence with your content and give us some data to help with that decision?" Then, the more steady state use cases, curation and discoverability, but all of this stuff is important.
David: You have actually a case study of an organization you work with to walk through some of this stuff and some of the results they had. Let's talk about that, but I was just thinking about what you're saying. One thing we always need to remember about our learners is that there are a lot of cases where it's that difference between search and recommendation.
It makes me think of there are times when learners know what they don't know and so they go looking, but there are obviously many times where a learner doesn't know what they don't know. Without this kind of tagging, those questions never get answered because in the case of they don't think to search for that because they don't know that it's something that's important to them, don't know that it's something that's available to them or that they don't know about. That's why we need that stuff on the back end to be able to do that for them. Talk to us a little bit about this particular case study.
Marc: We want a graphic on exactly that date, by the way, on the known unknowns and maybe I should get that into the pack that gets shared off from this. This case study was a pharma company. We actually do a lot of work with them, with the large pharma companies, that are obviously very much in the spotlight at the moment. With this one, we've been working with them for, I think, 18 months. Initially, it was a skills framework design or helping them with that. Once we had that, as I've explained before, we ran the Content Intelligence algorithms through all of their content, built the belief with some early results that this actually works.
It's all very well a vendor saying, "Hey, we can do it to human standard. Well, give us a list then and get a human to do it as well and see how different those lists are. We'll get the human to mark the machines' work," but built the belief, and now we provide quarterly content refreshes down to the asset level, and we're very much in steady-state. The cost saving is well into six figures, and that comes from identifying content spend that is either totally redundant because there's just overlap with other content, or there's data that we can provide that can just help with the negotiation with a content vendor that it's totally redundant, but it's not as useful as the vendor's been saying, so how about a discount of whatever it is.
There's another case study I wish I was closer to, which was more about, do we have the right libraries? At big companies, as you will know, Dave, that there's very often a couple of very large incumbent content providers. Frankly, they're most often LinkedIn Learning and Skillsoft. I think everyone just knows that. With this other situation, we ran Content Intelligence in a huge number of assets, about 300,000, some providers that they had and some providers that they were considering, and helped them with the economic decision which they had a hunch, actually, they had an inclination towards that one of the two big ones they didn't need.
Now all of a sudden, there was the data that said that there's a million dollars a year or something like that that could be spent elsewhere or allocated to some different learning activities or some other activity, or just save the money, but having the data finally to be brave and make that decision. I talked about too that the first was more curation-focused and skills framework design, and then curation. The second one was more to do with making sure that spend is going in the right way, go in the right places.
David: Right, and to be able to have data to make those decisions is critical because a lot of organizations don't come with that. They don't have that available to them and so the decisions are being made on many other things that might not be the right reasons.
Marc: When you said before, Dave, like at the top of this webinar, the data existed, there's definitely some data. All these large companies will have some data through the LRS, or LXP, or LMS, or whatever it is on like usage by this provider, usage by that provider. Because there's much that's tangled up in that, like, "Well, we did this campaign for this provider," or, "This provider was sitting on this other system, or it was advertised this other way," you're not comparing like with like, and that leads to this decision paralysis because, well, you can't really compare A with B because there were all these differences so we better not make any decisions.
We just leave things how they are for another year, another year, another year, and that's not an inspiring outcome for the end users, for the workforces. We're coming along and saying, "Hey, look, here's a better way of actually comparing like with like, and that may well break you out of that measure and make some bold decisions that you weren't able to make before because you didn't have the data."
David: Absolutely. One of the questions that came in, you'd mentioned something at the top when we were talking about the content that companies are sitting on, like a kind of volume of material that they have, just to have you reiterate the three buckets that you guys think about them, was it proprietary, licensed, and web content, or?
Marc: Yes, basically that. Yes, proprietary, we mean, the stuff that you make yourselves, where there's web, of course. I mean, there's some nuance to that, but web is web, and then it's, yes, those libraries, a couple of the large ones I just mentioned. Of course, there are many, many, many, many libraries and yes, all three of them. We use those buckets because that kind of splits things. All three of those categories are important.
In general, we find that there's most opportunity, I think, with proprietary and web content. I think most companies feel they could make better use of the web, but they're worried about rights and restrictions and just in quality and cohesiveness. With proprietary content, like I said before, a lot of the data's not very-- The metadata is not very good. That's tricky, but there's big opportunity with both of those. Then with library spend, I think there's just big savings to be made, so there's opportunity in all three buckets, actually.
David: One of the things that comes to mind is, as you talked through this, we're talking about things that L&D typically hasn't tackled or done a great job of tackling. As we think about this, and obviously, you guys represent a tool to help do this, but what about on the L&D skill side, what's a skillset or something that L&D should be looking for internally to help them manage this kind of process or get this going?
Is there something that might be missing from the toolbox from an L&D skillset? That's where we get these questions a lot from organizations, is what kind of skills should we be looking to add to L&D? Is there something like that from some other part of a business or some other field of expertise that maybe L&D hasn't been thinking about?
Marc: I think they have been thinking about the example I'm going to give, but there's still work to be done. I would say it's data and data skills in order to inform decision-making. I think that that's just so important because otherwise, you just go with the flow. Not in a good way, in a bad way. You just go with the inertia. I think, yes, developing those data skills in order to specifically make better decisions, bolder but correct, and justified decisions. I think it's a key thing.
With data skills, it's partly technical, but I'm not talking about data science here. I'm not talking about even data analytics. I'm talking about better use of Microsoft Excel so that you can synthesize different data sources, make some comparisons on a spreadsheet. I'm talking about that. Then also, understanding and persuading and storytelling that goes along with that.
I actually wrote an article in Harvard Business Review with Josh Bersin about exactly this. There's something if you google Boost Your Team's Data Literacy Skills. Data literacy. In that article we're talking about both things. It's, first, some of the hard skills that we need in business to be able to understand data better, but also the soft skills that go with that, and storytelling and presentation and making a convincing argument is a lot of that. Maybe we can maybe add that in the references of the pack, but I think that's yes, data skills to support decision-making would be big, big, big one.
David: I think we've been seeing that too, that companies are looking to get better data analysis skills within the learning function, or at least available to the learning function. Like you said, it doesn't necessarily have to be full-blown data scientists but folks that are better able to understand how to ask the right questions and get those answers. One last thing. When you think about tagging and so, because a lot of times we've been talking about how this can get relatively personal to each organization, is part of the strategy for the organization to sit down like learning leaders, business leaders, to sit and come up with a unified language about how we're going to talk about these things?
You might have in one part of the organization might talk about a skill in a certain way, or there's another part of the organization might talk about it in another way. Do we need to have that come together, or is that something that's part of a process later on where L&D can just meld things together, or should they come up with a standardized way of talking about these things?
Marc: I think standardized is, of course, that sounds nice and want to get there. You can imagine that working across the organization, the whole employee life cycle, from hiring and job descriptions through to onboarding and then learning and development and even offboarding. You can see how that's the dream, but most companies are a long, long way from that.
I think for now, what makes sense is to have some sort of definitions. Whether they're like ours, which are partly numeric and partly word-based, or whether they're just in sentences, then at least you can say, "Well, we meant by resilience this. What did mean by other department?" Otherwise, you're just talking across purposes and it leads to friction at some point down the line, or inefficiency, or people just not delivering the thing that was asked of them.