Please give us a brief explanation of the term "transfer effectiveness". How is it determined or measured today?

Dr. Ina Weinbauer-Heidel: According to Baldwin and Ford, transfer means the extent to which participants effectively apply the knowledge, skills, and attitudes acquired in the training context in the context of their work. That is, how much of what has been learned in a training context - face-to-face sessions, blended learning, eLearning, via VR, etc. - is actually applied and used on the job.

I can answer the second question from two perspectives. From the transfer-research perspective, there are numerous methods and models for measuring and evaluating transfer success, the most popular being Kirkpatrick's four levels and Brinkerhoff's Success Case Method. The pivotal element of these models is that they do not begin and end with the participants’ satisfaction (i.e. questions like "How much did you like the program?") or with the learning achievement (for example, measured by quizzes or tests). The starting point is the impact on business, and this is placed at the centre.

Several questions can be posed here. What has to change in the company (e.g. the KPIs)? To which strategic goals should the program make a contribution (e.g. digitisation, development of new markets, etc.)?  Which "pain" or problem needs to be overcome (e.g. inadequate speed to react properly to constantly changing customer needs)?

Based on the goal or goals identified, the focus is placed on the participants’ activity that is critical for success. In concrete terms, what should the participants do on the job - not merely KNOW, but DO? How should employees act so that the desired business impact can be achieved? Transfer research calls this "success critical behaviour" or transfer goals. THIS is at the heart of what the evaluation should aim at.

Frome the second perspective, we shouldn’t only measure WHETHER the participants actually act differently but also ask "WHY". What are the reasons that the participants have actually changed their actions - or haven’t. Here it becomes clear that it CANNOT be due to the training programme alone but can be traced back to many other factors, often organisational, such as management or the organisational opportunities to apply what has been learned - or the lack thereof. These factors are just as decisive for the success of the transfer, and are often even more critical than the content, the technology, or the methodology used in the intervention.

If I understand the question differently, that is I don’t ask "How do companies measure transfer effectiveness today?" but rather “Do they actually even measure it?”, the answer is quite different. In the latter case, the reply, unfortunately, is usually no, they don’t measure it at all! The reality is that almost 90% of companies primarily evaluate their programmes on the level of satisfaction. These are what researchers affectionately call the classic Happy Sheets (or their digital counterpart) that are gathered at the seminar’s conclusion. Learning success might also be included in the form of an assessment of the participants’ knowledge before and after the programme.

However, research shows very clearly that "being satisfied with a programme" or "having accumulated a lot of knowledge" relates nothing about how much knowledge is actually implemented.

Do you identify the proper instrument to achieve transfer effectiveness on the basis of different types of training?

Dr. Ina Weinbauer-Heidel: Actually, the instruments aren’t as different as one might assume. If transfer in the sense described above is the goal, i.e. a change in performance at the workplace, then everything that leads to this new performance or prevents it is decisive. The knowledge I need to perform successfully or the way the knowledge is acquired are, of course, part of the process. However, other factors are also quite crucial - and often more decisive than how the knowledge is acquired. These include personal factors such as my motivation in performing the task; how well I succeed in performing the new routines in the long term; or the organisational factors I’ve already mentioned, such as what my manager says about it or my fellow employees.

I notice nowadays that we are once again very much concerned about the "right" form of intervention - which technology, which software, and which method is particularly effective. However, at the end of the day, it is always a question of the people who, in the best case, manage to change their performance for the better and maintain it in the long run. Technology can certainly help here, but discussing only the “right" technology often obscures the actual barriers, such as a lack of corporate support for the new ideas, the corporate culture that is not ready for it, etc. These transfer barriers are usually much more important than "the right intervention" and, in the course of digitalisation, receive far too little attention.

This being the case, which interventions have proven effective - or ineffective - in the digital learning context?

Dr. Ina Weinbauer-Heidel: We work with the twelve levers of transfer effectiveness. In digital learning, there are potential barriers in addition to the classic levers of transfer effectiveness. These include individual abilities, attitudes, and previous experience in dealing with these media, as well as practical issues such as the technical possibility of using these media at the learning venue and the availability of the appropriate equipment.

Furthermore, digital learning often requires more self-control and volition - willpower - because in classical classroom training, it’s less likely that the learning process will constantly be interrupted by daily work tasks or the irresistible attention traps we all know. In self-directed eLearning, an undisturbed cocoon has to be created.

At the same time, digital learning environments afford ingenious, novel possibilities that can promote transfer in unprecedented ways. Among them are as self-tracking, which has proven to be immensely helpful in transforming new behaviours into long-term routines, as well as  for diverse forms of supervisor or peer support that are so crucial for transfer - without prohibitive expenditures in terms of travel and time.

There is also the lever called "active practice", the ability to practice the behaviour critical for success in a real life context, which was often extremely difficult to practice in the training setting or which required enormous simulation devices. Now, with VR, participants are able to gain real experience in practicing the new behaviour and thus achieve much greater transfer effectiveness.

As mentioned above, the goal essential remains the same: to make people successful in what they do. The factors or levers that contribute to the success of behavioural are the same; how, though, we have a broader spectrum of possibilities to achieve them.

What should training developers and deployers of training pay attention to in general?

Dr. Ina Weinbauer-Heidel: In my view, five points are particularly critical:

1. Don’t consider the training design alone. It is also necessary to regard the factors in the organisation and among the participants that will be decisive in the transfer process. As Brinkerhoff and Gill so nicely stated it, "The workplace can untrain people far more efficiently than even the best training department can train people."

2. Don’t make the instruments (the methods, the technologies) the goal. Just because there are new learning technologies that are really great doesn't mean they are suitable for every training endeavour. And just because we've "always used face-to-face training sessions" doesn't mean there aren't more effective approaches. More energy should be expended talking about the goals: the business impact and the critical behaviours. Only when this has been done - without hype or obstinance - should the instruments become the subject of discussion.

3. Don’t automatically think about deploying training as a standard solution for every organisational problem. Rather, look to training to help eliminate the causes of problems. In casual terms, if there are too many cases of burnout in the team, the umpteenth resilience and burnout-prevention training course isn’t going to help. It’s better to invest the money in more efficient processes or in an additional employee.

4. Don’t only measure satisfaction, knowledge, and completion rates, but also what the training programme actually sought to achieve: successful individual performance on the job and a positive impact on business.

5. Don’t only hope for success in the transfer process; steer it in a way to ensure it hits the target. This can be achieved, for example, by asking whether all the levers that are decisive in transfer success have been considered in the transfer concept. Research and practice show that transfer success is controllable. Recognising this makes the work more meaningful - and it’s fun to design transfer effectiveness in a purposeful way!