Transcript: #167 – Henrietta Palmer, Learning Solutions, TUI Group on creating learning content with AI
Sophie Bailey: Right. This week I’m in conversation with Henrietta Palmer, a strategic L and D professional and learning solutions manager at toury, a leading and innovative travel brand in the UK and an industry defined by the digital age and which companies have adapted to it. Henrietta is also passionate about the constraints corporate learning and development specialists is exist within.
Sophie Bailey: 18:13 Unlike the wild budget utopia our state system education folk might think of when we think about learning and the corporate world. In this episode, Henrietta talks about the use of artificial intelligence to curate learning across a large organization with keen goals. She also talks about going beyond Google searches and getting into the world of learning at work. I hope you enjoy this episode and if you’d like to be included in the next episode in our listener feature, just say hello, who you are and what you do in our 90 second voicemail platform at www.speakpipe.com/theedtechpodcast . Okay, here we go.
Sophie Bailey: 18:57 Yeah, absolutely delighted to have Henrietta Palmer, a strategic L and D professional and learning solutions manager on the line, so welcome Henrietta. Hello, so for those who don’t know Tui, formerly Thompson, is the UK is leading travel brand with 6 million holiday makers in the UK alone and with many more internationally and in the UK too. He has around 12 and a half thousand employees ranging from travel agents to cabin crew engineers and back office staff. Henrietta is responsible for ensuring Tui has the innovative development opportunities to drive the success of the Tui business. She has won numerous awards on behalf of her team, including the bronze awards in both learning technologies, team of the year and best learning technologies projects and in Henrietta’s his own words. I’m a creative problem solver with 20 years experience in the field of digital learning. Technology moves so quickly. No year is the same. I love it when the impossible suddenly becomes possible and I’m always developing to keep up with change and it should be noted that cross to E’s wider range of customer base and staff base that many of the innovations that come out from Henrietta’s team actually then become implemented at a global level. So Henrietta, just to kick off, perhaps you could explain to our listeners what it is that you do in a nutshell and then some of the kind of more recent projects that you’ve been working.
Henrietta P: 20:23 Okay. I just thought it’s actually about explaining how I ended up in, in this world and that was the, when I left school I um, I didn’t really know what direction I was going to go in. I think many people probably feel that way. Um, genuinely jealous that there’s people that have that focus. So I took a number of different roles working in a number of different areas, financial services, and also working in PR and communications, marketing and all of that sorts of thing. And then while working for a financial services organization, I was given a very bad objective. The company had just invested in a new CBT area. It was 1901 and they wanted to improve the return on investment and wanted people to go there. So with no direction, I was sent to the CBT room at least once a month as part of my annual review, the very first time I went, Julie, as expected, a light bulb came on and I found an approach and a way of learning that really worked for me.
Henrietta P: 21:23 It gave me the opportunity to reflect and dive into things that I was really interested in. I could return to things that I needed a bit more studying. I wasn’t going at the speed of everybody else in the room. It really engaged me. So I ended up going up there pretty much every lunchtime building my business skills and over that time I also identify some of the issues with digital learning. You sometimes got stuck in a loop. It didn’t give you the opportunity to dive deep enough, and I wanted to make that change. So nine months later I joined an organization that developed digital learning and that was in around 1999 so it really has been my passion from that point forward. So here at [inaudible], I started here about 12 years ago with a blank sheet of paper and we’ve really gone on a long journey from that point.
Henrietta P: 22:16 Part of that being, working with the technology that’s available in the business. I mean genuinely, when I started here 10 years ago, some of the e-learning that we were delivering was on the CD rom with a word document that people click them to get to the right piece of content. Um, now as a company we have around 300,000 completions a year and a R. E learning how to an operational level across the business. And my team’s focus is really thinking about where we need to be from a technology perspective to really drive the future of digital learning across the business so that we’re with future proof and ready for the new people that are coming into the business, but also keeping up with the technologies that are now available in the business and in the wider world.
Speaker 5: 23:03 And I mean that, that’s a great point too to kind of ask. So what technologies do you are leveraging in order to help deliver some of that learning as well?
Henrietta P: 23:12 Okay. We use quite a lot of different things now. We do, there’s a lot of things that, that people will know about and probably be using themselves that we deliver a whole range of different content. And this is sort of more traditional click next components, but we do that less and less. It seems, tends to be more when we’ve got something like a compliance or a legislative requirement that we deliver a lot of micro learning, which will be videos. We do virtual classrooms and we also curate content. So it’s not just a question of delivering content that’s being created within, but it’s
Speaker 8: 23:46 making sure that people are driven to great pieces of core quality content. And we found out there in places like the internet or even the internet.
Sophie Bailey: 23:55 And you know, in our previous chats you mentioned some of the work that you’ve been doing around using artificial intelligence, but also importantly in relation to some of the sort of budgetary constraints that and D professionals work within. And obviously it’s interesting coming from an EdTech perspective. I think a lot of people think, well, you know, corporate training, it’s got like a huge budget and there’s like a, the money’s there in a way that it isn’t perhaps across some of the sort of K12 or school or higher ed sector. But the reality sounds a little bit different. Perhaps,
Speaker 8: 24:32 yeah, I’m really glad you brought up budgets. Actually. Um, it makes me laugh because I think we all think, you know, the grass is greener isn’t it? And we all think everybody’s got it easier. And there is a belief that there’s a lot of budget in the corporate world that there really isn’t budget will be focused on legislative requirements. We have a lot in the travel industry and those are things that we need people to have completed in order for our business to operate because if people haven’t completed certain compliance training, they can’t fly or they can’t sell or they can’t do this. So there is a focus there on budget and for example, when the Thomson brand moved to to the, to E brand, that’s obviously a huge hearts and minds. Patients have budget will be set against that. But genuinely there are budgets there and it’s very difficult for us to prove the return on investment.
Speaker 8: 25:24 There may be an issue that there is an a business and there isn’t training there because of the cost of taking people out of their day jobs, have them traveling to the training. And you know, people can’t see how that training can be available and then you might be able to think, well if we did it digitally like this, then you wouldn’t have those issues at home at the time that people are need to take off work. They don’t need to do the traveling. That how can you prove the return on investment for it? Because currently the training’s not happening so you can’t say it’s going to be X amount cheaper because nothing’s X amount cheaper than nothing. You can’t show the difference it will make to the business because there’s no measurement at that point. You can only show the difference it’s made to the business after that training’s been developed. So proven return on investment is a real challenge.
Sophie Bailey: 26:16 So where you’ve collaborated with partners sort of to deliver some newer modes of this kind of training, how has that got around this sort of chicken and egg situation?
Henrietta P: 26:27 I have, I’ve got two really good scenarios. The first one was something we delivered for apprenticeship training. Whether it was a desperate need and actually the business case for the need was not difficult to prove in that one. I’ll talk you through that one in a second. The issue we had that was the amount of money and the time. The other one was an interesting one. It was something I could clearly see we needed in the business, but it was a completely new concept and I was never going to be able to prove that return on investment. So when I went into it, I had two possible free scenarios to kind of get buy in in the business. So I’ll talk you through both of those to give you an idea of how we’ve been able to get over that problem. So the first one was around apprenticeship training and it was around the time that the levy came in, which is an apprenticeship tax that the government, I’m not an expert on the levy, so please anybody out there that is saying, judge me.
Henrietta P: 27:25 By the way, I explain this just how it works in my brain is an apprenticeship tax that the government had put in in to encourage people to do more apprenticeships in a larger business environment. And I think it’s a great idea. At two E we already had some really established great quality apprenticeships when this came in. Um, but it gave us the opportunity to look at what else we were going to be delivering and we were working very hard on a lot of new apprenticeships that were coming in. One of the established apprenticeships that we had was on travel geography and that’s been running for over 10 years now very successfully. The problem was over that period, the content that had been first created for it by an external organization had come out of date, had the kind of date and also the technology that was used to deliver that was actually no longer functional.
Henrietta P: 28:19 You know, if you look at what technology was doing 10 years ago and what it’s doing now, it just wasn’t compatible. So the assessors had been delivering that training face to face. And if I take you back to what I just said about us in an opportunity where we were now suddenly able to broaden our apprenticeship offering, we couldn’t have our assessors focusing on face to face delivery. We need them to be doing, you know, what the core things that assessors do, you know, supporting people and driving them and getting them through that and they needed to be doing that on a much broader range of apprenticeships. We couldn’t move rapidly with looking at the content because of the legislative changes that were coming into apprenticeships at the time. And so we sat there and it was a bit of a waiting game. What we ended up with was a requirement to deliver 140 modules of learning about 70 hours of learning in eight weeks.
Henrietta P: 29:15 And this was digitally obviously. So we looked at how we could do it in house. We couldn’t, it was going to take us by eight months dedicated resource, obviously missing that target. We looked rapidly at whether we could do it externally and even throwing external resource at it really wasn’t going to speed the timescales up. And the cheapest budget that we quote that we got on that was 500,000 which was significantly more than we had to spend on such a piece of training. Now we’d seen earlier in the year at a conference, somebody talking about a tool they were developing that uses artificial intelligence to analyze source content and deliver an output that’s a high quality digital learning experience, so they won’t like the way that I’m going to explain this, but again, we’re back to how it works in my brain. A bit like a sausage machine.
Henrietta P: 30:05 You put your core content in word document, PowerPoint. With something like this, the AI analyzes that content and then structures it into what it knows will work into a great piece of content that will work for an adult learner. So what it did was exactly what I said. We gave it the cool content as PowerPoints, as word documents, and then it sent off back learnings that provided core knowledge to learn is to read. That was the first phase of it. They then went into the next section which reinforced what the learners had covered using a whole range of different questioning techniques and then when they completed that training, it recommends required revision based on the results of what they did when they were getting the questioning techniques and that is personalized to the individual user. The AI also looks at the core information that’s in that source content and then goes out to the internet to specific areas.
Henrietta P: 31:09 We used Wikipedia to pull in additional content. If people want to delve into a specific topic in more detail. All of that happened automatically using artificial intelligence. Here’s the really fantastic thing. This tool could deliver us two or more modules in a day, so it was obviously the way forward. At the time the tool was still at its beats phase, but we were really lucky. The tools called wildfire and the wildfire team agreed to work with a specific tool at that beta phase to help us get this project out. So we knew that we could get those 170 modules in the timescale, but we were ignorant at the time about how much effort it would require from us. So so far it sounds like we do nothing and it delivers everything that we need and I think it’s a realization from anyone that’s thinking about using AI and getting the benefits from it.
Henrietta P: 32:06 Let’s look at what we have to do. If you’re delivering a piece of digital learning, you obviously have to test everything and that testing is still done at a human level. Instead of us testing 140 modules over an eight month period, we were having to test 140 modules in that eight week period and we really, really struggled keeping up with the artificial intelligence. So we had to completely change the way that we worked. We had to work from an agile approach, both from a testing perspective and also a project management perspective and we were still working all hours of the day and night to keep up. But the amazing thing at the end was that we actually delivered 140 modules in six weeks and the budgets were eight times less than they would have been if we’d gone for a more traditional approach.
Sophie Bailey: 32:58 And how did you go about actually once, once we’ve got this project, which must have been a huge relief to, to deliver in such a tight timeframe, but, and then in terms of assessing the impact on the learner, you know, obviously you mentioned the testing. How did you go about doing the review and the assessing and all of the rest of it?
Henrietta P: 33:17 So obviously we were in on a very tight timescale, but we did want to include the user testing with it. So what we did was we got some of the people that were a good way through the travel geography apprenticeship, so they knew what it was all about and they worked with us and they advised us on changes. And we did make some very specific changes that one of the clever approaches with the questioning techniques was that basically you covered some content and then it questioned you on what you’d learned. And originally we just had open input that there was a change to the approach. So it would say to you something like, what’s the capital of, I don’t want to pick on this, I’m going to show my ignorance about travel geography. What’s the fat capital of England or UK? There’s one I can do and it would just be open input and you’d have to type in London.
Henrietta P: 34:04 But actually there was a feeling that it kind of did leave you a bit stuck at that point. So w the way that it changed was if you, if you got it wrong, it would give you the first letter and if you got it wrong again it would give you the second letter and then the third letter if you still got it wrong at that point, what it would do would it would give you the answer and then that would come back to you in the feedback to tell you that that was something that you needed to go back and study in more detail because you didn’t pick it up in the questioning techniques. And the other bit of AI that we added in was if there were words that were particularly difficult to spell because they were unusual in the English language, that the AI was able to recognize that you would nearly there and give that to you as the right tone.
Henrietta P: 34:50 It’s that, and then again in the feedback tell you that actually you do need to learn how to spell this word, but you did know what you were talking about. So those changes were made based on user feedback. And then after we did the delivery, we surveyed the apprentices and we got some fantastic results. It was completely new approach for them. 95% of them rated the design approach as good or very good and 62% could identify a specific sale that they’ve made based on the knowledge that they’ve gained in it. And soon the whole of the retailer state were asking us why they didn’t have access, why it was there needy travel geography, apprentices had access and there was such an interest in this. We have around 6,000 people in our retail estate. I know about, sorry, sorry I lied. We have about 6,005 in our retail estate, but the the 6,000 comes from is that within a year we’ve had 6,000 voluntary completions from those 6,500 people on these travel geography learning because they recognize it would help them day to day.
Sophie Bailey: 35:54 That’s amazing. And will you be continuing to use that sort of approach for other kind of learning objectives going forward do you think?
Henrietta P: 36:01 Yeah, the the right project haven’t come up for it just yet, but we will definitely use this approach for, for more things because it was so successful and I know that now the tool itself has moved beyond the Bita phase. They’ve been do anything extraordinary changes to it. You know, you can have three D menus and the AI that sits behind it is a lot more sophisticated. So actually because it was a Dita phase, if you were to look at what we delivered when we did this, it’s quite basic compared to what the tool can achieve now. Um, and so we would definitely use it again in the future. The right project and the budget was available
Sophie Bailey: 36:40 [inaudible] and then what was the other project that you, you had in mind as well.
Henrietta P: 36:45 Okay. This is completely different and a really interesting one. And it started off with leadership. Now we’ve delivered it, we’ve expanded it to induction, we’ve expanded it to other support and we’ve got big plans for Whit and moving forward it’s going to become a much broader training tool. But it originally started when we had a real training requirement for all leaders. We’ve got a really rich boss leadership approach. We call it vibe and there’s a lot of training that goes around this. But because of our work force, it’s quite difficult to get training to people. We have a fast paced business and our leadership workforce are continually traveling globally. So you can have a really robust training strategy, you know, to give people. But if you can’t get to them, how do you do it? The issue that we had was that people were circumnavigating the training that we had on offer and going to Google, so it was one as face to face training.
Henrietta P: 37:44 So we had a range of different content clip, next micro learning, curated content that was available to our LMS. But the LMS was clunky and it was difficult to get to what you needed. Going to Google is great, but the problem is then you’re learning the Google way, not the two-way. And we work in a matrix environment. So I could be working on a project today with one area of the business and in three or four months time I could be working with a completely different area of the business in a completely different area of the world if we’ve all picked up different ways of doing things because we’ve gone to Google rather than doing it in the greed, two-way, you lose really valuable time at the start of a project. So this was happening more and more. And another issue that you’ve got, if you’re just going to Google and anybody in the education sector, I’m hoping that this will immediately make sense to you.
Henrietta P: 38:39 If you’re just going to Google you only learning your known unknowns. What about the things that you don’t know? You don’t know? Um, and he isn’t sentence I’ve said a hundred times in the last year, which is a killer. If you don’t know what you don’t know, you’re not gonna feel that learning gap. So we needed to come up with a solution that would get people to that knowledge that was doing it. The two E-way quickly and easily recognizing that people felt like what they were getting, you know they were getting it right from Google and also make sure that we filled those knowledge gaps. So when we were talking about it as a team, we came up with the concept of a tutor bot worked like a chat bot. You would ask your knowledge questions and it would take you to that great piece of content.
Henrietta P: 39:27 We immediately knew that this was the right answer. So we were talking about the possibility on this and people were saying, well no, we’ve got all of these other ways of delivering training digitally. We can’t be looking at another platform and another approach. So we accept that and we started to think about how we could build a proof of concept without that kind of AI component that comes with a chat bot so that we can prove that proof of concept in house. And we came up with, at that time, a very basic approach was that we would create all of the knowledge as baseline content, create a sort of a site map that was available through the intranet with an Eater’s ed index and that would take people to the core content and then there’d be links within that core content that would branch them out to the knowledge, wherever that was, that was completely free and something we could create in house.
Henrietta P: 40:21 But we were really lucky because word that we were looking at this sort of filtered out and an organization that was looking to build a tutor, bulk got in touch with us and asked if they could work with us as a proof of concept. They would start with their blank sheets of paper. We would stop where we’d already felt and building the work that I just talked about. And then they would build their tutor book to work to our requirements and give us a beta that we could then go out to the business with and show them how the proof of concept worked. So the more we got into this build, slowly our business case grew and grew and grew. And I think when you’re looking at finding the budget, the lesson I learned here was yes, as educational professionals we can clearly see the benefit of delivering learning is going to be that there’s going to be that knowledge transfer and people are going to get better.
Henrietta P: 41:18 But sometimes it’s difficult to prove that we had a massive wind to the business using this tutor bolt. The tutor bot, which is actually very reasonably priced functionality, sits above a lot of platforms that are very expensive to the business. So our learning management system cost us an awful lot of money every single year to deliver out to the, to the business as a whole. Our intranet has a big annual license and the other places where the content that we directed people out to had a huge budget impact. So what we are able to do is put the chat bot or a chip on our chatbots called okay. Above all of these different platforms. It doesn’t impact any of them. But people can now go into the chat bot and say, how do I write an objective? And it comes back, it gives the chat bot says to them, you know, just about a hundred words of high level information and then direct them out to a number of different options wherever they are, whether that be on the LMS, whether that be on the internet, whether that be on the internet.
Henrietta P: 42:29 And so it’s driving traffic to those bits of the business where there has been big spend. And the other thing we’ve been able to do with that to fill that, you know no none known it. If someone wants to come in and say how do I write an objective also can then say to them after it’s direct them to that right piece of learning. And if you’re writing an objective, have you got a really clear five year strategy so you know what your objective is helping you to achieve. If you haven’t, he has had to set up the strategy. So we’ve been really able to prove the ROI there.
Sophie Bailey: 43:02 No, it’s, it’s, it’s really fascinating cause listen to you speak, I can hear from the listeners perspective to this podcast, how transferable some of what you’re saying is, so for example, when you were talking about the difficulties of teams and leaders, you know, traveling and how do you get that sort of learning to them? It made me think of also distributed teams. People often talk aboutL and D but perhaps not in the context of this increasing trend towards distributed teams and the challenges that that bears. So that was really interesting. And then again, this sort of culture of learning within an organization. So you know, bringing people back to the, the Tuohy, Brandon, the, you know, the information that is important and sort of business critical and also that, I mean I’ve talked to universities and how, you know, for them it’s also retaining that brand and that essence of what that university means. And so it kind of travels across as well. And then also we talked about implementation before. So dr Neelan Palmer on the podcast before his talks about, you know, ed tech or educational technology or learning technology can have these benefits. But like you said, you know people, they do need to realize that there is an implementation cost. And so it’s sort of picking and choosing your projects carefully and the time and the training that’s needed for them to bet in [inaudible]
Henrietta P: 44:28 absolutely it reduces the time in the long but actually often that set up time is much bigger than you can even comprehend to begin with. So it’s getting that balance. It’s that it’s almost like the agony of when somebody new comes in and your having to spend that time building their knowledge and capability so that in the long run you benefit from their skills and it is exactly the same with this. It requires an awful lot of up front work for the longterm benefit.
Sophie Bailey: 45:01 Absolutely. And you seem to have gone about it in a very savvy way in terms of sort of partnering and piloting with your sort of external partners and and sort of both kind of working out the process as you go. Which I really like
Speaker 8: 45:15 and I think there’s more and more of that is what I said right at the beginning actually. It’s having a clear vision of where you want to be and what you need to achieve and then keeping an eye on the business and seeing whether any of the issues in the business actually fitting to your, your journey and also obviously for new things you hadn’t even anticipated. But then keeping an eye out on how technology is moving and how you think you might achieve something today could actually be achieved in a completely a much different and much better way in just three weeks time because there’s something that happens and we had a great response to the tutor, but so far we did after the launch, it was launched a hard launched in just may this year and we did a user survey two months afterwards and 85% confirmed that the tutor mock bot met their learning need. 73% felt that the time they spent with it was really valuable. And the most exciting response to us was that only 2% of people so that the time would have been better spent in Google.
Sophie Bailey: 46:21 [inaudible] that’s the, that’s the big win. Yeah. Cause a, you know, there’s also the kind of rabbit hole effect of going into the internet isn’t there, you know, you go on there for to look at how to write a pitch deck and the next thing you know you’re looking at cat videos. So, um,
Speaker 8: 46:38 get back from a mental health perspective that could be very valuable. But you can do that rabbit hole thing, the food, the tutor bot. It’s just then that the tutor bots taking you to new and interesting things you may discover that are gonna make you better and better at work.
Sophie Bailey: 46:55 Valuable learning. Uh, very quickly. Cause I knew that you have to pop off soon. Who sort of inspires you in the L and D world or the wider world and how do you go about getting your own learning and inspiration in this space? Or whether that’s, you know, books that you’ve read that you love or podcasts or networking or whatever it might be.
Henrietta P: 47:17 I guess there were two things that inspire me. I wouldn’t say that there was a particular person that inspires me. What I spend a lot of time doing is watching videos from conferences and I spend a lot of on LinkedIn because I find that very, very useful. So it wouldn’t be one specific person that inspires me, but it’s just keeping an eye on what’s going on out there in the big world and what’s hot at the moment. And if it could be one completely different person one week and the, you know, then it would be on another week. And then not necessarily all people in the digital learning world or people in the learning world at all. It could be absolutely anybody. In fact
Speaker 8: 47:58 put an 11 year old son and he spends a ridiculous amount of time on YouTube and I find con some of the YouTube stars doing gaming and things like that fascinate me because I’m thinking, how is this engaging my son for such a lengthy period of time? Because every time I create a video, I test it on him and I’ve now got his attention for one, one minute, 11 seconds,
Henrietta P: 48:24 and that’s how long I try and make my videos, how long they will [inaudible], you know, keep my son engaged. But these people can keep my son engaged for hours. So that inspiration can come from all over the place at the moment. And from a technology perspective, it’s also really, really valuable to keep an eye on what’s going on out there. And I’m going to mention the dreadful Google word, which I, I do love Google, we all love Google, but actually going onto Google news on a weekly basis.
Speaker 8: 48:54 Yeah, and you’d be amazed how many trends you can pick up. I mean, whether it’s a press release that somebody released or just some new knowledge and information and facts. Those are kind of the ways I really keep up to date with things. I’m also lucky that we work with some fantastic suppliers when we do have budget and they’re very, very good at the thought leadership piece and pointing you into the right direction.
Sophie Bailey: 49:17 Thank you so much for your time today. It’s been really super interesting and also quite astonishing what you achieved with all these usual constraints that we work within. But yes, so really thank you for sharing that with our listeners. If people are kind of keen to follow your own work, how should they go about doing that?
Henrietta P: 49:36 Well, I’m just experimenting at the moment with LinkedIn. I’m probably a bit backward on it because the rest of the world’s been using it forever, so I’m all over LinkedIn at the moment and do feel free to meet me there.
Sophie Bailey: 49:50 Wonderful. Well thanks very much, Henrietta. Okay, thank you for your time.
Sophie Bailey: 50:09 That’s all for this week’s episode. Thanks so much for listening in and I do hope you enjoy the fun, some gems of inspirations take away with you. Don’t forget that for events you might be interested in around the world, you can go see the ed tech post of comb food, such events, the battle of ideas, re-imagine education and times higher education live are all coming up. That’s all for now. Thanks for subscribing and listening.
Speaker 4: 50:31 Bye bye. [inaudible].
