We were delighted to sit down with Kirsten Campbell-Howes, our learning design mentor, for the first deep dive in a series designed to help founders and their teams approach learning design projects in the most effective ways.
Kirsten was Busuu's Chief Learning Officer prior to their exit to Chegg for $436M in 2021 and is now Chief Learning Officer at Visibly, a Brighteye portfolio company.
You can watch the video in full below or read the transcript. We hope you find it helpful!
The transcript:
Introduction
Hello! I'm Kirsten Campbell-Howes. I've been working in tech for about 22 years, working with businesses, large and small, to lead learning teams, teacher teams and qualifications teams. I've been working as a mentor for Brighteye for a few months now, and the ways I can generally help is if your business is focused on learning, I can help you talk about how to structure your team and how to come up with a learning design plan and methodology. expertise at the moment.
I’ve done a lot of work around product market fit and getting learning teams to work successfully together with product teams towards the same goals.
Module 001: How to build a curriculum and course outline for any topic
What needs to be in place before starting a course-building project?
What really needs to be in place before you embark is a solid sense of what the business is trying to achieve for its customers. You've probably got a hypothesis that some kind of learning or training or tool is missing in the market, or is there, but is not right for the market. Having a solid sense of where you're trying to go and good research from potential customers or existing customers that backs that up is only going to help your business overall. It's really going to help when you’re establishing a learning team and start recruiting into that team, because that team needs to be aligned with the rest of the business. You can have the best learning team in the world but if they're trying to create X and your business needs to create Y, you're going to have a problem!
So, the first thing to nail is that strategic alignment.
And then I'd say, the second thing is a sense of what's needed in the team. Quite often I work with Edtech start-ups where the founders come from a product or an engineering or data science or a pure business background and they don't necessarily have knowledge of learning expertise in the team. That’s okay, but you shouldn't wait too long before you bring it in.
A lot of founders I speak to underestimate the importance of that expertise, or make the quite common assumption that “we've all been to school, we can do this on our own”, and that is rarely the case in my experience. Learning design and methodology is a specific expertise and it can really help you accelerate your business in the early stages if you get some of that expertise on board.
How much detail should one go into when defining learning outcomes at the beginning?
At the beginning, you can start quite high level, depending on what you're trying to do. If you are creating learning yourself- if there's a gap in the market where this learning doesn't exist- you are probably going to create the learning outcomes yourself, and they are going to be highly aligned with what your customers are trying to achieve.
Here, I recommend using a research methodology like the Jobs to be Done framework to really get a sense of what your customers are trying to achieve because your learning outcomes need to align with that. I was the Chief Learning Officer at Busuu for a long time, and our customers were adult language learners. Now, casual, adult language learners are generally not trying to pass a high stakes exam. They’re trying to pick up a bit of French, a bit of German and have fun while they're doing it. So, the learning outcomes for a customer in that scenario are quite different - not totally different, but quite different than they are for a 16-year-old, who's trying to pass a high-stakes exam in French or if you're working in a regulated sector in which specficicity matters. Quite often the learning outcome provide for your students are already specified by a regulatory authority or an exam board. So, you need to be aware of those case, and you need to be able to align them with what you're trying to build in your product.
How do you approach getting to know the target audience and their needs?
Getting to know your customer, your target audience, is probably the most important thing that you can do, and it's something that you need to do consistently throughout the life cycle of your business. It's not sort of a bit of research at the beginning, and then you're done. It's something you need to do constantly. The way I like to do it depends a bit on the size of the team that I'm working with.
With a bigger team, there might be a dedicated qualitative researcher in the team whose entire job is to make sure that research happens, collect the results and share them around. For a very early stage start up, you might not have the luxury of a single dedicated person to do that role, in which case you need to identify one or 2 core people in your team and make sure that they are interacting with the customer regularly. You can use a framework like the Jobs to be Done framework, or you can devise your own framework for doing research. Again, it will depend a little bit on what your business is trying to achieve and the extent to which your sector is regulated.
Let's say you've got a list of hypotheses, a list of questions, and/or a list of things you need to know from your customer. It's really important that you are having interviews or surveys with your customers as often as possible. The way that you do this again really depends on what you're doing as a business. If you're working with adults, it's quite a lot easier to do surveys and interviews. If you're working with kids, then it can be very challenging for a range of obvious and less obvious reasons.
But let's say you're working with people that you can access through surveys or regular interviews. The next super important thing that you need to do is to always be collating information and sharing it with relevant people in the team. You can gather loads of research, but if you're not using that to create more hypotheses or learnings, and you're not sharing it properly across the team, it's a waste of time. So, you'll need a framework for gathering that information and then potentially codifying it into actionable insights.
For an early-stage startup. This might sound very complicated, but it could be as simple as generating a list of features that your customers want or a list of topics that they want to cover as part of their learning. Or if you're working with B2B clients, it could be: What are the managers really missing? And what do they need their teams to be able to do?
It could be that you just start with a list that you then stack rank according to the number of times that your clients, or your customers, or your interviewees are mentioning it to you. When I was working at Busuu, this was how we generated our product pipeline in the beginning. What are our customers asking for? Who's asking for it most often? How complicated is it to build? We'd stack rank a list and that became our product pipeline for the first couple of years.
What common mistakes are made when designing the structure and flow of a curriculum?
I think one of the most common mistakes that are made when you are designing a curriculum is to put too much in the curriculum. And this is a common mistake in product, design and learning design. We generally like to think and overthink things to the extent they become complicated. There's a real tendency to think ‘I need to give customers absolutely everything that they could possibly need to know’ which is a natural thought for the type of people trying to design the perfect product…
But it's usually a mistake, because if you overwhelm people with too much, if you're trying to cover every single learning outcome, if you're trying to fill your lessons or your product with as much information or as many features as possible, this quickly becomes really overwhelming.
I like to apply the principle of simplicity and simplifying to everything that I build. That doesn't mean that you don't create something robust, or that doesn't meet your customer's needs, but it needs to be simple. It needs to be easy to use. It needs to be pleasant to use, and it should not be overwhelming. So, keeping it simple is the first thing.
I think another mistake that gets made is gathering loads of research, stacking surveys full of 35 questions instead of 3 really good questions. First of all, that means that no one's gonna answer your survey because they're gonna take one look at it and say… ‘No, thanks.’
But it also means that someone’s going to have to go through all of that and codify all of that information which is going to take a lot of your team's time. It’s much better to design 3 really targeted, well designed questions and then collate the results. If you've got too much information, it can lead to sort of analysis paralysis which is really common for product teams or for more junior product managers or product designers.
Too much information can make it so hard to make a decision.
Can you give an example of a learning product that delivers content in a way that caters to different learning styles?
This is a really interesting question, and you've touched on a very hot topic in education and depending on who you talk to, some people will literally spit and hiss at you if you mention learning styles. Because it's a theory that's incredibly popular and incredibly widespread across education. But it's largely been debunked by research.
However, the core of learning styles is this idea that people learn differently and have different preferences when they're learning, right? And that's true. Some people do prefer to learn by reading, other people do prefer to learn by watching, some people are more predisposed to doing physical, active things, and they're less interested in reading or watching. Some people like all of these things - I know I like to learn in multiple different ways.
And I think what the research does show is that learning in multiple ways, encountering the same information in different contexts and different approaches does help you to retain information and develop skills. So that's the kind of principle I like to take with learning styles and a product where all you're doing is reading. For example, if all you're doing is listening to audio, it is probably going to bore your audience, and it's not necessarily going to appeal to all of them. In fact, it almost certainly won't.
At Busuu, we took the approach of trying to keep people engaged and interested, and present new language to them in different contexts and different ways. So, we had a mixture of video content, pure audio content, written content and real-world content, in which you're having conversations with real people. And this mimics what happens in the real world, right? The goal of learning is generally to prepare people to do something in the real world, whether it's in their jobs at school, out with friends, whatever that goal.
In the real world, information comes to us in multiple forms. So your product should help to prepare your learners for what they're trying to do in the real world and the learning styles or approaches that you use should mimic that to some extent.
How would you approach accessibility?
One of the goals of your research should be to uncover the personas of your customers, and also the needs of your customers and just generally to understand who your customers are. Whatever your population of customers, they're going to have accessibility needs.
You should try and uncover what those are likely to be, and you should design a product that meets those accessibility needs to the extent that it's possible for you to do that for early stage start-ups. It can be challenging when you're trying to find product market fit to think about accessibility, but the earlier you start thinking about it, the more likely it is that you are going to design a product that is accessible to your customers and meets their needs.
Diversity is another really important consideration. So, for us at Busuu we had customers in 190 countries of all ages, diverse races, gender, sexuality and we really, really wanted to make sure that all our customers felt represented. For us, this fed into our learning design, principles, and methodology. One of our principles was to be representative of the customers who were using our product. We were very careful when we chose our images, when we created writing exercises or reading exercises, we talked about situations that our learners might find themselves in, or issues that were going on in the real world that reflected what they were experiencing.
And this was quite challenging. Some of our customers reacted quite angrily to this. They, for example, didn't want to hear about transgender issues in some of the reading passages that we had, but we kept going with it. We got really positive feedback from our customers, saying that they felt like the product was for them and was representative of the world they were experiencing and in which they were speaking the language.
This is one example of diversity in a product. Every situation is different and the needs for different businesses and different products are very different, particularly depending on the age group of the learner. But the important thing is to find out who your customers are and what they need and try as best as you can design a product that meets those needs
Other than variety, what makes an engaging early part of a learning experience?
Well, interestingly, some of the research I've done has actually challenged an assumption I had that the very beginning of a learning journey in a product needs to have as much variety and interest packed into it as possible. Actually, I've worked on products where subsequent customer interviews and survey data have shown that too much variety can turn people off because they're still getting used to the UX of the product. They have many questions when they start using it. What is this product? How am I going to learn with this company, with this tool, with this app?
If you throw everything at them in the very first experience, it can be overwhelming, and they can disappear. Variety is super important as the journey goes on, particularly when you are considering longer term retention. But at the beginning, I'll return to my core principle: simplicity.
You need to introduce a little bit of what your business is as a brand, a bit of your methodology, and give the learner an understanding of what this experience is going to be like. And most importantly, I would want to make them feel like they will achieve their goals through using your tool, your product, or your course.
When I worked at a company called EduMe, which is a really big workplace training company, one of the principles that we were designing into the early stages was bitesize, learning. You might think that you want to have a long first lesson that's going to achieve multiple learning outcomes and give that learner a sense that “oh, I can do absolutely everything as a result of this”, but actually, it's really important to keep that first experience light but be sure to make sure there's a feeling of success at the end of it.
Maybe you pick one core learning outcome (at Busuu it was to learn ~2 sentences connected to the topic in an experience of ~5 mins), but at the end of the short segment, you've achieved something and think “Okay, I've learned a word or a phrase, or I've understood a little bit more about this topic. I enjoyed the experience. It's manageable for me. Let's move on to the second experience.”
But the tendency in some early stage start ups is to cram everything. I totally understand where that desire comes from. You're really excited about your product and you want to share it with customers. But that can backfire, so simplicity, a sense of achievement, and a sense of fun, I would say, are some of the principles I would design in that early use stage.
What can be built into a course outline to ensure learners remain motivated throughout the course?
I think this very much depends on the type of learning and what you're trying to achieve for your customers. If you look at a product like Duolingo, for example, they build in a lot of gamification like streaks and leaderboards- the streak is often cited by Duolingo's customers as one of the core motivating factors. The learning experience doesn't actually change that much month to month. And yet, you get people with year-long or longer streaks on Duolingo and that's because they're able to come day after day because they're not being asked to do very much. They're just doing a little bit each day, gradually building up their knowledge.
And that works in the sort of young adult casual learner market where you're not aiming to pass a big scary test, where you're not trying to get a visa to move to another country or get another job, for example. So those mechanics and that style of learning works really well in that specific context.
Now, if you're trying to get someone to pass a certificate so that they can be an electrician, for example, you're going to have to take quite a different approach. And a lot of the motivation that that person has is self-generated. They need to get a new job, they need this certificate, and what you need to do is get out of the way a little bit and give them exactly what they need. And probably no more than what they need to get where they want to go, because they're spending valuable time and money in order to achieve a career result for themselves.
They’re maybe not going to appreciate it so much if you're throwing in loads of fancy features or making them compete in leaderboards if they don't really want to and want to get on with learning the core skills.
So, here is where I come back to needing to understand your customer and your personas, and the various accessibility and diversity needs through good, solid research. And if you are confident in those things, you should be able to design something that works for your particular context and your particular customer.
Who is responsible for driving that research?
The responsibility really depends on what's going on in your business, but the most important thing is that someone is responsible for it. If everyone is responsible for it, no one is responsible for it. It should be the person who is most capable of doing it. And in some organizations that will be someone in the product team or the engineering team. It could be a designer. It could be a UX researcher. It could be the head of product if you were quite small.
You have a dedicated learning team, it could be someone like me, or it could be a learning designer. I have seen all of these things working at different stages of a business and in different types of business. But the key thing is that someone has the ability to do it, the support from leadership to do it, the tools that they need to do it and is able to share the results and see those having an effect. There's nothing more demotivating for someone doing research than not having support and having their research ignored.
You should know who is the most appropriate person to do it within your team. You'll need to discuss it with them and agree, and if it's not in their core skill set yet, which it may not be If you're very early stage, you're gonna have to build up that that skill set, which is where someone like me can help!
What should you consider other than the audience, when integrating assessment in a course?
When you're thinking about assessment, there are 2 or 3 main types.
There’s formative assessment, which is quite light touch. As you're going through a course, you might do a little quiz that tells you your score and gives you a sense of how well you've mastered the material. But it's not going to lead to you getting a certificate. It's not high stakes. It's just light touch, and it should be spread throughout the learning, so that the learner gets a sense of achievement and an understanding of how close they are to meeting learning outcomes.
It gives you as a course designer information about how well your course is performing as well, because if everyone's scoring a hundred percent or no one's scoring 100 and everyone's scoring 50, you know that there's a problem.
Then there's summative assessment, which comes at the end of a period of learning, and that could be covering a chapter, a module, or a course. It's more formal. It usually comes in the form of a test which could be also be digital, of course. It could be administered with a person with lots of ways of doing these summative assessments, but they're usually more tied to you getting some kind of certification. There's usually a score that's retained and recorded somewhere and you may not be able to take that assessment again immediately and overwrite that score.
It depends a bit on the level of formality of your course and the level of regulation. If you are delivering a university course, the way you build your assessments is going to be highly prescribed. Indeed, it may actually be completely out of your hands and you may have to follow a very detailed set of regulatory steps in order to build and design the assessment, and you may have to demonstrate the validity of that assessment.
in some big learning organisations, there are teams of hundreds of people who are working on assessment because it’s a science, essentially - really good assessment design is a science.
I would say the third type of assessment is like a big, very formal assessment like a GCSE (England’s age 16 exams)- in language teaching, it would be the ILS. The results of these tests are extremely important for the test taker, right? They determine whether you can go to university, or whether you can get a visa, whether you can get a job. And again, the design of those assessments is highly prescribed, and it's a science - it needs very experienced people building those assessments.
For most early-stage start-ups, I think you need to focus more on the formative and summative assessments.
If you're working in a regulated area, there will be guidance that you need to follow. If you're working in an area like Busuu was, in which it's for casual language learners who don't have particularly formal goals, you can actually build and design your own assessments. And again, the principle really should be to go back to the research and go back to your personas and go back to understanding your customers and their needs. Your assessments should be helping them to learn their skills, their competencies, their weaknesses, and therefore what they need to do next.
How does one facilitate community and collaborative learning at its most basic level? And is it relevant to all subjects?
I'm going to say that community is relevant to all subjects. But it may not be relevant to all learners in all contexts. So, in the real world, we can't survive without community, right? It's absolutely essential to our survival and our ability to thrive and develop as humans. We interact with other people and that's really core to being human. It’s also really core to learning as well, because actually, most learning takes place in communities. Most of what we learn we learn from interacting with other people. You and I are learning right now, while we're having this conversation.
But when you're designing a learning product, the extent to which you include community again, is really going to depend on who your customer is and what they are trying to do. In language teaching, which is my background, community is actually really important because if you are going to speak a new language, you are going to speak it with other people, and you are going to be in the real world. The more you can bring an element of community into your learning design, the closer you are going to get to preparing your learners for the real world.
However, speaking another language to people is quite intimidating, and it can be quite demotivating when it doesn’t go well. I think we've all had this experience of trying out our language skills on native speakers and them not understanding and it being quite overwhelming. At Busuu and in other language learning products, where there is an element of community, that has to be brought in very gradually and very carefully, because I have seen language learning products where the community is a bit of a wildwest, in which you start typing to someone, or you have a video chat with someone and you just don't know what's gonna come back, and that can be intimidating and off-putting!
You've got to design guard rails around that kind of experience, so that people don't become overwhelmed and they don't become scared.
Moving out of language learning into other topics: I gave an example of someone learning to become an electrician earlier. Now it may be that there's a specific part of learning to be an electrician where you need to study a lot of theory, and you need to do a lot of practical work to understand how to wire up a meter safely, for example. It may be that you don't need an awful lot of interaction with someone in the theory part, and maybe only a limited amount of interaction with someone during the practical part. Maybe you need to work with a mentor just to see them doing it and get a sense of how to do it yourself, but possibly you don't need to access a forum of other electricians. It really depends what people are trying to do. And quite often having a forum can be a really useful source of motivation and interest and other expertise.
Coming back to the principles of understanding your customer, their specific needs, how much time they have, whether they like learning in community or not is another aspect for you to consider... Some people like it, others can take it or leave it, and some people hate it. Often, good products will be able to facilitate a little bit of whatever people need. Maybe it’s an aspect of your product but not a core aspect…it’s self-serve.
It depends on the subject and it depends what you're trying to achieve, as ever. In some cases, community might be absolutely core to the learning experience, interacting with other people, developing your communication skills, developing your diplomacy might be part of the learning experience, and therefore community is essential.
But yeah, understand the customer, understand what you're trying to do and design for that.
What makes a good feedback loop?
The most important thing when you are designing feedback loops in learning is that the people doing the research are regularly observing customers using their product. I have never seen anything that shortcuts improvements in course design. More than that, even. The absolute best, most experienced learning designers, course developers, curriculum developers will make assumptions and they will tend to put in too much information, add too much complexity when they are confronted with a learner using what they've designed.
They learn very, very rapidly what works and what doesn't and that needs to be ongoing throughout their experience as a course designer or curriculum developer.
Everyone in every organization in the world should be in learning mode all the time. Having a form of continuous professional development is really important, even in an early-stage start-up. It doesn't need to be super formal, but the team should be talking and reading and listening to podcasts about what's going on in their regulatory environment, what's going on in educational psychology, what's going on in in in the government… They need to have a forum. They need to feel comfortable spending time at work doing this learning and have forums to discuss their learning.
The best teams are the ones that are engaged with the world and are learning and sharing with each other. That's a really important feedback loop.
A third feedback loop is (assuming that the learning team is not the senior leadership team in an Edtech business) there needs to be feedback from senior leadership into Learning and from other parts of the organization as well so that everyone is aligned, and everyone knows the business goals.
I've seen learning teams get quite siloed from what the business is doing and not understand the financial goals of the business or the financial constraints of the business and then you may end up with learning designers who are spending loads of time designing lovely learning that isn't meeting the needs of the business. It's taking too long to develop, it's not exactly what the customers need, so they need to be getting feedback, from sales, from the customer service team, from senior leadership, so they understand the context in which they're building this and can adapt as required.