The learning profit chain

The Learning Profit Chain “Connecting Learning Investments to Financial Performance” by Scott Mumma and Cindy Thatcher, Corporate University Xchange “An investment in knowledge pays the best interest” Benjamin Franklin In good economic times or bad, business leaders must make capital allocation decisions, deciding for example, whether to increase or decrease investments in developing talent. In bad economic times, which the world is currently experiencing, these decisions become even more important. However, most executives lack rigorous systems and methods to help them make better talent development investment decisions. The long-term cost of making uninformed decisions is substantial. Further, once made, executives are left “checking their gut” when it comes to determining the value of talent development investments. We believe there is a better way. Our research shows that too many organizations are ineffective at measuring the impact of training investments on business performance, resulting in ambiguous commitment from executives. We believe this is in part due to the de-facto standard measurement method used, which is over 50 years old. corpu.com Corporate University Xchange The Learning Profit Chain Spring 2009 1 At Corporate University Xchange, we have been benchmarking the Learning Function in major corporations for over ten years. Our method, which measures the Learning Function broadly across 12-Dimensions, shows a positive correlation between higher benchmark scores and company profitability. Further, through statistical analysis, our data shows that organizations who implement Learning Best Practices, as evidenced by benchmarking scores, tend to have a higher profit per employee. Learning Executives need new tools and methods to predict the business impact of training investments. Corporate University Xchange continues to develop and refine its method called, The Learning Profit Chain, which uses data and regression analysis to prove correlation and even causation between investments in talent and business performance outcomes. By benchmarking the Learning Function across the 12-Dimensions and following The Learning Profit Chain Method, Learning Executives will have the facts, data, and proof needed to better inform senior business executives when deciding how much and where to make training investments. Profit Per Employee Performace Based on the Effective & Efficient Index – EEI Index Scores Effective & Efficient Index – EEI Index Scores Source: Corporate University Xchange Measuring Training: The Impact of Training Effectiveness and Efficiency on Financial Performance “Mediocre performers set goals that are general and often focused on simply achieving a good outcome… the best performers set goals that are not about set highly specific technique based goals and strategies for themselves.” the outcome but about the process of reaching the outcome…best performers Geoff Colvin Talent is Overrated “Much of the work done in calculating the return on training investments is all just conduct training and retire as millionaires.” guilty of misrepresentation….If all courses had a [high] ROI, then we should In 1959, Donald Kirkpatrick wrote the book Evaluating Training Programs in which he unveiled his now widely adopted model for the measurement of training that focuses on “achieving a good outcome” for singular training events (which is the goal of mediocre performers according to Geoff Colvin’s research at Fortune Magazine). The Corporate University Xchange 12-Dimension Learning Benchmarking Model measures the best processes for reaching the outcome – the goal of the best performers. Below we explain the important difference and make the case for an alternative way to assess The Learning Function’s performance, measure it’s impact, and ultimately, make better business decisions. Donald Vanthrounout, et al, Return on Learning quoting Tad Waddington, PH.D. 2 corpu.com Corporate University Xchange The Learning Profit Chain Spring 2009 Existing Measurement Models and Their Structural Issues The Kirkpatrick model established four discrete levels of measurement normally referred to as Level I-IV: reaction (I), learning (II), application (III) and business impact (IV). In a business context, success at the first two levels is mildly interesting to the employee in that he/she had a good experience and added to his/her personal body of knowledge. However, achieving Level I and Level II success is arguably irrelevant to the enterprise absent use of the knowledge (Level III) to drive business impact (Level IV). Reaction Level I Learning Level II Application Level III Impact Level IV Level V ROI First, it is nearly impossible to draw a direct causal connection between training and business impact. One can only ascertain the impact by allocating a portion of the improvement in “derivative metrics” to the training investment. These derivative metrics include such things as employee engagement, employee retention, hiring rates, error rates, sales, etc. Even if, as some argue, such causal connection can ultimately be derived, the expenditure of resources required to accomplish that task is largely impractical in most business contexts. Second, the cost of determining the ROI, if viewed as a component of the cost of the training, may cause an otherwise viable program to become impractical based on ROI. Third, from the perspective of the Learning Function, measuring the ROI on many projects may result in a compounding error since, as mentioned above, it is nearly impossible to draw a direct causal connection between training and business impact. If a series of fundamental assumptions regarding the allocation of the contribution made by training on a specific derivative metric is in error, one may substantially over invest in training. As Tad Waddington, a PH.D. in measurement, evaluation, and statistical analysis from the University of Chicago, puts it “Much of the work done in calculating the return on training investments is guilty of misrepresentation.” Fourth, ROI does not look at the quality of the training function as a whole, but rather on the potential result of individual discrete events. Any attempt to measure the actual result suffers from the issue addressed above (cost, difficulty, and feasibility). Further, even the most rigorous measurement organizations have concluded that it is impractical to apply ROI to all activities such that there never is a true summation of the department’s impact on the business. Many conclude that the sheer resource required to accurately calculate ROI results in less than 18% of training programs actually being eligible for this sort of rigor. Levels I – IV (Kirkpatrick) Level V (Phillips) An argument can likewise be made that evidence of Level III is irrelevant absent calculable evidence of Level IV, in that such a condition connotes random activity within the workforce not tailored to a specific outcome. Courses such as diversity training are designed to drive a business impact (development of a work environment that increases productivity) rather than to simply influence activity. Kirkpatrick, however, stops short of defining business impact in a measurable way. “The linkage between individual-level training outcomes and organizational outcomes is at best complex and difficult to map and measure even when training is purposely designed to address organizational objectives.” The foregoing logic likely led Jack Phillips to the conclusion that if we must measure the business impact of training, the tried and true business mechanism of return on investment (ROI) is the most applicable approach. While this, in fact, seems quite logical, there are four fundamental problems with the Kirkpatrick to Phillips approach. Four Problems With Kirkpatrick Model 1 2 3 4 Hard to prove Expensive to measure Compound errors mulitply mistakes Measure’s training events, not L&D Function corpu.com Corporate University Xchange The Learning Profit Chain Spring 2009 3 This list of structural weaknesses in the application of ROI to training is not exhaustive. Others point to additional weaknesses including an inability to truly calculate costs, an uncertain understanding of targeted ROI for learning, etc. As Waddington put it, “If all courses had a [high] ROI, then we should all just conduct training and retire as millionaires.” In sum, the entire notion of the Kirkpatrick/Phillips model may not truly measure the impact of the Learning Function on the organization, even under the most optimistic scenarios. It measures the possible impact of isolated training events. The Corporate University Xchange Model and Hypothesis The ultimate question then becomes: of working diligently toward greatness. “Mediocre performers set goals that are general and often focused on simply achieving a good outcome…the best performers set goals that are not about the outcome but about the process of reaching the outcome...best performers set highly specific technique based goals and strategies for themselves.” Therefore, in the absence of financial measurements to determine the business impact of a function, perhaps the execution and measurement of the processes and methods is what leads to exceptional performance. It is further logical to conclude that, if the proper processes are purposefully prosecuted, superior financial performance will follow. More importantly, since these processes are not subject to the inherent limitations of training ROI described above, it is easier to measure training processes of a given company against best practice processes that are informed by data, research, and clearly established business requirements. Having concluded that the best approach to measuring training is by measuring the processes used to achieve outcomes, we must now determine how to measure the processes and methods. It is hardly controversial to contend that any executed business process or method should be effective in furthering the accomplishment of a business objective. Equally as well settled is the notion that any executed business process or method should be efficient in accomplishing its purpose. Arguing against effectiveness and efficiency is like arguing against motherhood and apple pie. The real question is what constitutes effectiveness and efficiency in the context of training? How does one objectively assess learning investments against business performance? One recurring theme we have heard from Learning Executives over the past 10 years is that they rely heavily on the beliefs senior executives hold regarding the value of training programs in impacting business performance. For example, one Learning Executive referred to a CEO speech that included the following: “We had a really good year exceeding our business goals – and we thank the training organization for helping to be a contributor to our success.” Many Learning Executives consider that “CEO Thank You” as a good enough indicator of value. This conclusion, however, is a lagging indicator that fails to guide current activity. Therefore, we are left to find an answer to the question: How does one objectively assess business performance for a function that is difficult to measure? In his book Talent is Overrated, Geoff Colvin, senior editor at Fortune uses a wide array of scientific research to argue that exceptional performance is the result of deliberate practice: a process 4 corpu.com Mediocre Performers Best Performers Set goals about the PROCESS of reaching the outcome Set general goals Goals focused on achieving a good outcome Set highly speci c, technique-based goals Source: Talent Is Overrated by Geoff Colvin, Senior Editor at Fortune and co-anchor on Wall Street Week. Corporate University Xchange The Learning Profit Chain Spring 2009 For the Learning Function to be effective, it must be aligned to the core strategy of the business and help drive a positive, measureable change in business performance. While some may argue that this is akin to the “CEO Thank You” measurement system set forth above, there is a glaring distinction. The “CEO Thank You” system assumes that measurement is irrelevant as long as the business leader acknowledges that training was instrumental to the success of the organization; in essence a laissez faire retrospective approach. The Ultimate Test – An Initial Examination If, as we propose, the rigorous adherence to best practice processes and methods measurement approach is a better indicator of training value than the traditional outcome based approach of Kirkpatrick/ Phillips, then companies that adopted best practice approaches should exhibit better corporate performance than companies that do not. Corporate University Xchange has tested and refined this hypothesis. The first step was to analyze a set of training processes and methods of companies as compared to the 12-Dimensions best practice approach. Nearly 200 companies completed the 2008 Benchmark Survey. Benchmarking participants were scored across the 12-Dimensions, which further categorized under two axes: efficiency and effectiveness. That distribution, shown below, indicates a fairly robust distribution: Our position is that the Learning Function should use processes to ensure alignment with business objectives and to measure the impact of training on the business. The former (alignment) should be a proactive approach rather than the “CEO Thank You” reactive approach and the latter (measurement) should not be the summation of the impact of isolated events the underlying measurement of which is premised solely on subjective, non-causative assumptions, but rather a comprehensive set of measurements that address the broader context of the Learning Function. For the Learning Function to be efficient, it must organize its governance structure, physical structure and infrastructure to rapidly respond to ever changing business requirements, and it must execute its programs and services in a cost effective manner using both internal and external resources. Based on our ten years of best practice research, awards, case studies, and conclusions of independent thought leaders, we have established a set of best practice processes and methods that, if adopted, achieve the two critical business impacts: effectiveness (alignment and measurement) and efficiency (organization and execution). The best practice processes are further subdivided into the critical 12-Dimensions of training set forth in the framework below: E ective e ur Al i Me as gn Cultural Outcomes Business Outcomes Personal Outcomes Capability Planning Strategy Integration Performance Consulting Program Design and Delivery Operations Partnerships Technology and Infrastructure Governance and Structure Branding u ec Ex te ga Or E cient corpu.com Corporate University Xchange The Learning Profit Chain Spring 2009 5 ni ze Distribution of Effective and Efficiency Scores Proving the Learning Profit Chain Clearly, the foregoing is not meant to be conclusive, but rather is designed to represent just one of the insights regarding our data and 12-Dimension Model. We will be extending this research in late spring 2009 with the launch of our 10th Annual Benchmark Survey, which builds on our longitudinal data set. As mentioned, stock price is just one proxy for financial performance. Subsequent research will collect and analyze additional private company data and will also control for many other measures of financial performance such as operating margins, net income, revenue growth, market share, etc. This data will also be further analyzed in the context of industry, size of organization, stage of corporate development, etc. In that process, we will also be teasing out other unique results of the data. For example, while the Aggregate Indices of public companies translated into the results on the previous chart, is there a subset of processes that create a tighter correlation? Are there trends within industries or other demographic subsets of the data (size, industry, etc)? We have many hypotheses and much more analysis to pursue in order to refine our Learning Profit Chain Theory, which seeks to uncover which training interventions best predict improved business performance. It will be a lengthy journey, more marathon than sprint, but we believe it is critical to the competitiveness of businesses and success of Learning Executives. We hope that you join us in this worthwhile endeavor. Source: Corporate University Xchange The effectiveness and efficiency scores on each axis were then combined to create an “Effective & Efficient Index – EEI” score for each participant. For public companies, the Effective & Efficient Index score was then measured against the change in stock price. If there is a correlation between effective training processes and methods and financial performance, then those companies with a higher EEI should perform better financially. Using public companies as a subset and stock price as a proxy for financial performance, a statistical analysis of the change in stock price over the preceding year of those companies compared to the EEI reveals the following results, including the trend line: Financial Performance vs. Learning Performance Source: Corporate University Xchange Companies that score higher on the Effective & Efficient Index – EEI, have stock market returns that outperform those who score lower. 6 corpu.com Corporate University Xchange The Learning Profit Chain Spring 2009