Searches & Clicks: LXP Data Requirements for Learning Analytics

LXPs have hugely valuable data for telling you not just what learners are learning, but what they want to learn. So in this post, we’ll explore how you can get that data from your LXP and make use of it in analytics to inform your content offering and more.

What’s an LXP and how does it fit in a learning ecosystem?

Learning experience platforms, or LXPs, have emerged in the last decade in response to two key trends:

1. an emphasis on learner experience
2. an emphasis on self-directed learning​

While learning management systems (LMSs) are traditionally designed to empower the L&D team to organize and manage learning and learners, the LXP is designed to empower the learner. LXPs typically don’t host content, but instead provide search and social sharing functionality for learning content aggregated from many sources—of which the LMS is only one.

LXPs work especially well in contexts where there is a good learning culture and where the organization was to support learners to develop themselves, rather than where content chosen by the organization needs to be pushed to learners.

An LXP can stand alone in place of an LMS; but more often, organizations will place an LXP in front of the LMS as a user-centric portal that acts as a single point of entry and learning content aggregation for the rest of the ecosystem.

LXP data requirements and what to ask your vendor

Because the LXP’s primary focus is to serve learners, LXP data is particularly useful for understanding what learners want to learn. As such, learner search data is especially valuable because it tells you:

• what search topics are being used,
• which content is returned in results, and
• which content is accessed as a result of those searches.

This data, as shown in the example above, empowers the L&D team to watch in real time as learner demand changes and new topics are requested. As a result, the team can quickly curate and create content to respond to those demands to ensure that search always returns helpful content.

After search data, data about any learner interactions within the LXP is useful in analyzing both the content and the learners:

• For content, this data will highlight content and content sources that are rarely used, as well as popular content that you may wish to promote to other learners as particularly popular (and presumably helpful).
• For learners, understanding who is using the LXP and how they’re using it can be particularly useful for promoting the LXP—especially if it is a new platform. This data can inform targeted marketing and education campaigns to encourage use of the LXP by parts of the organization with lower adoption.

LXPs aggregate learning content and experiences for learners and, as such, LXP data is most useful for learner analytics and learning experience analytics (i.e. content analytics). Because content is not normally part of a formal program, LXP data is unlikely to contribute significantly to learning program analytics.

From a learner analytics perspective, the LXP is likely to provide high-level data about what online learning content each learner has accessed and what they are searching for. It may also provide data about skills a learner has developed. This data can form part of a learner transcript in the LAP, informing training and skills discussions with a manager or mentor.

At a higher level, this data can be used to identify which parts of the organization are most or least engaged with learning and use of the platform. If a particular part of the organization is not engaging with the platform, the L&D team can investigate the reasons and take appropriate action.

From a learning experience analytics perspective, LXP data can be incredibly insightful in terms of what content is being used and what isn’t. This is especially true if you’re paying for any content that isn’t being used—you know where you can save some money by removing or not renewing those licenses.

Conversely, the data (especially search data) may also reveal areas where more training content is required.

^{The example above shows that the Teleworking 101 course has been incredibly popular. This might suggest to the L&D team that sourcing more content around teleworking would be appreciated by learners.}

When talking to an LXP vendor about data extraction, some of the questions to ask will be similar to the questions to ask an LMS vendor:

• Does the data exist?
• Is it possible to get the data out?
• Is the data accurate?
• Is the data complete?

Specific to LXPs, I cannot emphasize enough the value of search data and, therefore, it’s particularly important that you ensure this data is available, complete, and accurate.

The search data should comprise a record for each search—including the time of the search, who made the search, the search term used, and how many results were returned. It’s equally important to include any search tags or content results (if available) in the data.

Compared to other kinds of platforms, LXPs tend to do a good job of generating detailed data. However, LXPs can vary when it comes to what data they make available and how that data is structured.

This differentiation between LXPs means it’s especially important to ask vendors for details about how they handle data availability and structure so you can compare that against your data requirements. If data is likely to be a deal breaker in choosing an LXP, ask for sample data to test in your LAP before you make your selection.

If you already have an LXP, your main data extraction decision will be around whether you extract the data via xAPI or CSV.

Many LXPs have an xAPI implementation, but they aren’t always as fully developed as we (and you) might like. The best approach is to test the xAPI implementation first.

This will help you avoid getting caught out by a bad implementation later, after you’ve already started collecting data. If it works and the data is sufficient, great. If not, either talk to the vendor about making improvements, or look into using a CSV import (e.g., Watershed’s Data Conversion Engine) instead.

The LXP and your learning ecosystem

While most LXPs typically include some reporting on content use and search activity, this can be enhanced by bringing that data into your LAP.

First, LXP data can be combined with HR data to analyze how search and usage varies in different segments of your organization. Second, it can be combined with data from the content sources aggregated by the LXP (the other platforms in your learning ecosystem).

Using learning ecosystem data alongside the data from the LXP empowers you to explore what learners do after they search for and access a piece of content. For example, you can see how long learners spend interacting with that content or if they access related resources.

The LXP data will typically provide high-level information about which content learners have accessed. This information can then be supplemented with additional data about more granular learner interactions within that content, sent directly to the LAP from that content.

A challenge here is the LXP’s identifier for content may not match the content provider’s identifier.

Where possible, this disparity should be fixed at the point of data extraction to the LAP; but where this is not possible, LAP tools (such as Watershed’s Activity Aliasing) may be used to link the identifiers from the two platforms so the two data sets can be mapped together in reporting.

High-level content analytics data will help you identify the most and least popular content for different audiences. And search data will inform you as to what topics you need to address in new content.

This data should be used to inform the design and curation of additional content. As a result, learners have new content that addresses the topics they are searching for and follows the style and approach of existing popular content.

For example, learners are suddenly searching for content about working remotely, and data shows your most popular content tends to be 3-minute explainer videos. The data would suggest that a series of 3-minute explainer videos about remote working might be popular and useful.

Another potential benefit of extracting the data to an LAP is the retention of the data. Some LXPs may only keep the data available for a short period of time, whereas you may be able to keep it longer in an LAP.

Having historical data is helpful, as it provides a baseline against which to compare current data. This will tell you, for instance, if searches for a particular topic are increasing or decreasing, or if content usage from a certain vendor is trending up or down.

Because the LXP is the front end “shop window” for the learner, adding, replacing, or removing tools that sit behind the LXP is unlikely to have any impact on the LXP data itself. However, reporting on the LXP is likely to include data directly from those tools themselves.

Adding, replacing, or removing a tool could mean a change in the data. (You may find the principles outlined in our 5 Steps to Good xAPI Governance eguide helpful in ensuring consistency of data as you transition between tools.)

The role of the LXP in skills and compliance

Typically, there are two kinds of compliance training:

1. One where the relevant authority specifies particular topics that must be trained on, or specific competencies that must be achieved. These topics are typically generic topics like health and safety, diversity and discrimination, data privacy, or bribery and corruption.
2. And another where the relevant authority specifies a particular amount of training to be completed (normally measured in hours of learning), but is less specific about exactly what content is covered. This is typically the case for professionals, such as doctors or lawyers, where it’s important for the professional body to ensure the professional is taking Continuing Education (CE) seriously, but trusts the professional’s own judgement as to what training is appropriate.

In the first scenario, an LMS will normally be much better suited than an LXP to push out and record completion of the required training. This is because LXPs are designed to give learners freedom to choose their own learning, and mandated compliance modules do not fit well into that philosophy.

In the second scenario, however, the LXP is much better suited to collect data about learning completed that contributes to CE hours since it is the access point to all the learners’ learning experiences, not just those housed in the LMS.

While the LXP is a good system for generating data relating to completed CE hours, it still isn’t a good system of record for this data.

Instead, importing this data into an LAP ensures that the data can be retained as long as it’s needed, and can be combined with data from learning activities not accessed via the LXP (e.g., in-person classroom training).

The LAP may also have access to more detailed tracking data from various learning experiences, allowing more accuracy in reporting on hours of training completed.

LXPs may have some of the skills functionality described in the previous post in the context of LMSs. This data can be reported on in an learning analytics platform in much the same way, combined with data from other platforms in your ecosystem.

In particular, many of the leading LXPs include functionality for learners to record their skills, or perhaps recommend others for skills—much like on LinkedIn.

This data might be initially generated when an individual joins the organization and updated as their skills develop, perhaps as part of an annual review or simply on an ongoing basis.

This is really valuable data, especially at times when external recruitment is challenging, as it can inform you about those already in your organization who may already have some or all of the skills needed to fulfill a vacancy.

That’s why it’s important to ensure that this skills data is included in the data you bring into your LAP. There are two key considerations when doing so:

1. Ensure that you’re collecting all of the skills data from the LXP. This should include ongoing changes to a person’s skills as well as historical skills data prior to setting up the integration, so you have a complete picture of each person’s current skills.
2. Standardization is important. If I list Learning Analytics as a skill, my colleague lists “Learning Evaluation,” and another colleague lists “Learning Data Reporting,” that’s going to be challenging to report on consistently. Make sure your LXP has functionality to define a skills taxonomy that specifies a list of skills that learners can select from that are relevant in your organization. Take the time to customize this taxonomy for your organization and keep it updated as new skills emerge or old skills become obsolete.

Up Next: Data requirements for video platforms

LXP data is particularly valuable for not only revealing what learners are looking for, but also for connecting the dots between the other platforms in your ecosystem that are launched from the LXP.

One such platform that might be launched from an LXP is your video platform—which contains valuable data about how learners interact with videos, the segments they watch, and where they pause or drop out. In the next post, we’ll explore how to get that data out of video platforms and use it to inform your video content strategy.