Evaluating Training ROI with a Learning Intelligence System

April 9, 2007 The eLearning Guild’s SM Practical Applications of Technology for Learning THIS WEEK: Management Strategies For many years, industrial organi- Evaluating Training ROI with a Learning Intelligence System By Mark Place zations have used statistical methods and disciplines such as Six Sigma to identify the root causes of problems and to achieve continuous improvement goals. Training departments can adapt these methods into a model for learning improvement, if they also implement a learning intelligence system. Such a system expands the traditional reporting and data management strategy beyond the LMS and into a cross-functional corporate reporting and data management strategy. Read this week’s article to find out how! A publication of provement. Yet, training is an area where the actual return-on-investment (ROI) is uncertain. Given the large expenditures for training in many organizations, it is important to develop tools that will help companies answer the following questions and improve the measurement of training effectiveness. These tools need to provide a methodology to measure, evaluate, and continuously improve training, as well as the organizational and technical infrastructure (systems) to implement the methodology. We would like to know: • Is the training program effective? • How can we improve the program? • Did the program achieve the desired results at the lowest possible cost? The emerging body of knowledge on transfer of training suggests a number of important propositions and conclusions. For example, the transfer “climate” can have a powerful impact on the extent to which people use newly acquired competencies back on the job. Delays between training and actual use on the job directly relate to skill decay. Social, peer, subordinate, and supervisor support all play a central role in transfer. And finally, it is possible to design intervention strategies to improve the probability of transfer. All four of these intervening factors affect the results of any given training program, and there are potentially other factors. T raining is a critical component in any organization’s strategy for innovation and continuous im- LEARNING SOLUTIONS | April 9, 2007 2 Management Strategies One might ask, “Why bother evaluating learning and knowledge transfer? If you evaluate organizational results, you will know if the training was ultimately successful.” The problem with this assertion is that you don’t really know if those results are due to training or to an intervening factor. Additionally, training does not always produce the desired organizational results. It is possible to gain important organizational knowledge by finding the causes of failed training. In this article, I discuss learning intelligence systems and an approach to learning improvement. A learning intelligence system is an important connection between the measures of learning effectiveness that an LMS can provide and the larger enterprise metrics that indicate whether learning transferred. But this is not enough to ensure improvement of results at the enterprise level. For that, we must borrow some ideas from industry. level may indicate other results. Unless a training program exists simply for the sake of training, measurements should include actual performance data, not just data about performance during the training. Selected metrics, such as sales, customer satisfaction, workplace safety, productivity, and others, should help demonstrate where training has increased revenue or decreased costs. For example, corporate health and safety programs train personnel in the hope that they will reduce workplace accidents, whether it is OSHA regulations or the company that mandates the program. Reduction of costs, both tangible and intangible, is implicit in the goal of reducing workplace accidents. While the tangible costs of workplace accidents are fairly straightforward, the company also incurs other less tangible costs in lost productivity that are even more difficult to measure and account for. An accident in the workplace can adversely affect the behavior of uninjured workers in other ways. They may approach a piece of machinery more tentatively, or otherwise work with extreme care to avoid an accident themselves. Rough ROI measurements that consider performance improvements can provide a benchmark for training effectiveness. After implementing a training initia- Measuring organizational results Calculation of training ROI requires measuring organizational results. Training operation metrics, such as course enrollments and completions, assessment scores, and the results of feedback forms and surveys, may directly relate to those results. Performance data on an individual, department, or business unit One might ask, “Why bother evaluating learning and knowledge transfer? If you evaluate organizational results, you will know if the training was ultimately successful.” The problem with this assertion is that you don’t really know if those results are due to training or to an intervening factor. Additionally, training does not always produce the desired organizational results. Publisher David Holcombe Editorial Director Heidi Fisk Editor Bill Brandon Copy Editor Charles Holcombe Design Director Nancy Marland Wolinski The eLearning Guild™ Advisory Board Ruth Clark, Lance Dublin, Conrad Gottfredson, Bill Horton, Bob Mosher, Eric Parks, Brenda Pfaus, Marc Rosenberg, Allison Rossett Copyright 2002 to 2007. Learning Solutions e-Magazine™ (formerly The eLearning Developers’ Journal™). Compilation copyright by The eLearning Guild. All rights reserved. Please contact The eLearning Guild for reprint permission. Learning Solutions e-Magazine™ is published weekly for members of The eLearning Guild, 375 E Street, Suite 200, Santa Rosa, CA 95404. Phone: +1.707.566.8990. www.eLearningGuild.com Learning Solutions e-Magazine™ is designed to serve as a catalyst for innovation and as a vehicle for the dissemination of new and practical strategies, techniques, and best practices for e-Learning design, development and management professionals. It is not intended to be THE definitive authority ... rather, it is intended to be a medium through which e-Learning professionals can share their knowledge, expertise, and experience. As in any profession, there are many different ways to accomplish a specific objective. Learning Solutions will share many different perspectives and does not position any one as “the right way,” but rather we position each article as “one of the right ways” for accomplishing an objective. We assume that readers will evaluate the merits of each article and use the ideas they contain in a manner appropriate for their specific situation. The articles in Learning Solutions are all written by people who are actively engaged in this profession — not by journalists or freelance writers. Submissions are always welcome, as are suggestions for future topics. To learn more about how to submit articles and/or ideas, please visit our Web site at www.eLearningGuild.com. LEARNING SOLUTIONS | April 9, 2007 3 Management Strategies tive or changing an existing program, an organization can observe and record a change in performance. Reduced incidents of workplace accidents would be one example of such a change. The organization could compare the value of the performance change (X dollars saved per reduced incident) to the program’s cost, in order to arrive at an ROI measurement. While this approach appears to answer the question of training program effectiveness, it provides no insight into improving training from either a results or cost perspective. It also does not provide a means to calculate the ROI that may occur as a result of routine training programs. (How often should workers take the workplace safety program to ensure that accidents remain at the lower level of incidence?) May 17 & 18, 2007 Evaluating training and performance Many learning operations evaluate success based on the ubiquitous Kirkpatrick model, or some variation of it. Most organizations that follow this model are unable to evaluate their programs beyond the first two Kirkpatrick levels. (See the article by Tita Beal in the March 26, 2007 issue of Learning Solutions for an explanation of the Kirkpatrick model.) In part, the learning management systems (LMS) that many organizations use make lower-level evaluations easy but don’t provide any mechanism for higher level evaluation. Most learning management systems will automatically track and report information required for Level One and Level Two analyses. They include assessment tools that can capture each learner’s reaction to the course, and templates that can create reports. For online and blended learning, the Level One assessment (the learner’s reaction) can be completely integral to the course. Likewise, training programs can inexpensively and easily administer pre- and post-tests that evaluate learning results (Level Two). Levels Three and Four evaluations become more difficult and costly to implement and administer. What makes these Levels so difficult to evaluate? In the case of the first two Levels, data collection occurred during course delivery. When evaluating changes in student behavior and training influence on business results, data collection requirements extend beyond course delivery. Different evaluation methods can help answer whether students’ behavior actually changed after completing a course or attaining a certification. Data can come from many different sources, including customer satisfaction survey data or performance evaluations by supervisors. Trainers or other designated staff can also observe students and record behavior as the students perform their jobs. To evaluate retention rates, there should be a delay between the training New Techniques for Designing and Developing e-Learning Interactions • LEARN how organizations — whether corporate, academic, or government — are using different types of interactivity • EXPLORE using different tools and processes for developing interactivity • DISCOVER new e-Learning interactivity and user engagement approaches and theories • EXAMINE how to design various interaction elements • ENSURE that your interactivity addresses all types of user and content needs Register Today! +1.707.566.8990 www.eLearningGuild.com Hosted by: Technology Sponsor: LEARNING SOLUTIONS | April 9, 2007 4 Management Strategies and these behavior measurements. Organizations rarely evaluate the business impact of a program. This is much more difficult and costly. Large organizations can use a control group to isolate training from other intervening variables. One group does not get training, while another group does. Comparing the business results from each group reveals whether the trained group performed better. Clearly, control groups present a few challenges. Performance measurements must take place over the same time periods. Both the control group and the group receiving training must be nearly identical in make-up, performing similar job functions. Sales groups operating in different sales regions or manufacturing groups working in different shifts will confound the results. With these challenges, structuring a valid control group requires great commitment on the part of the organization. But to perform a good ROI analysis, an organization must really be evaluating its training at Levels Three and Four. Without these levels, ROI analysis becomes merely an exercise of cost justification for expenditures. Implementing blended learning, for example, may appear to have a high ROI. After all, it’s less expensive than instructor-led, and Level Two assessment scores improve. However, without an understanding of the impact on individual and organizational performance, one must ask what value is the training really delivering to the organization? Actually, much of the data needed to bridge the gap between training and performance exists in many organizations. Individual performance data exists in performance management systems. Organizational data exists in marketing, sales, and financial systems. Bridging this gap requires a technical infrastructure that minimizes the administrative effort needed to collect and analyze the training and performance data together. However, learning management systems, both the most common repository for training data and most common mechanism to deliver training, cannot easily bridge the gap. From a functional standpoint, each new LMS release adds more robust reporting and data analysis capability, as well as human resource system integration. In addition, many LMS vendors have added talent and performance management features to their human capital solution suites. To some degree, these evolutionary changes address the gap between training and its impact on individual performance. But they don’t even begin to address the gap with organizational performance. Why are organizations still unlikely to evaluate training at Kirkpatrick’s Level Three? What path can lead to Level Four evaluations? System integration, one common point of failure, is critical. Many LMS vendors with a history as product companies have limited expertise in system integration that extends beyond learning systems and databases. Successfully managing performance-based training evaluation, however, requires expertise in data management and warehousing, a variety of corporate systems and databases, analytics, and Web-based application development. Evaluating training ROI most effectively requires the right technical infrastructure and a model of learning improvement. Learning intelligence system A reporting and data management strategy that focuses on the LMS as the foundation, only compounds the system integration challenges that make performance-based training evaluation unmanageable. Instead, the organization should adopt a cross-functional corporate reporting and data management strategy. The technical foundation for this strategy is not the LMS, but a learning intelligence system that acts as a broker between an LMS and other corporate systems. The features of a learning intelligence system (see Figure 1) include: • Independence from a LMS • Cross-functional system integration • Alignment to individual and organizational performance • Reporting and analytical tools Figure 1 Learning intelligence system: The functional parts and characteristics of a learning intelligence system LEARNING SOLUTIONS | April 9, 2007 5 Management Strategies LMS independence You should not lock a learning intelligence system into a single LMS platform. By utilizing a generic framework, common LMS data should map to variables in the learning intelligence system. LMS independence ensures more stability over time. It minimizes the extent of required system integration in case of LMS upgrade or replacement by another system. Typically, an organization will feed performance, job code, certification, and other corporate data into the LMS reporting system. By adding a learning intelligence system between the LMS and other corporate systems, the organization only needs to update one data connection if the LMS changes. the data, a learning management system would need to send and receive data through many connections. To manage these different data sources dynamically, a learning intelligence system can receive data from these disparate sources and present it through a common, cross-functional reporting and analytical system. Although integrating multiple data sources can require significant system integration effort, the organization gains greater control over its learning and business data. Automating the collection of the training data and consolidating it with business data reduces sources of error and ensures accurate and up-to-date information, which users can share more extensively with a minimal degree of administrative effort. Cross-functional system integration As a broker for business intelligence throughout the organization, a learning intelligence system needs to aggregate the data from multiple corporate systems. If assembling information is too cumbersome and time consuming, and the data is outdated or not even correct, the system cannot enhance ROI evaluations by combining training with other business data. Cross-functional system integration allows the organization to leverage training and business data together in a context-sensitive manner. Technical or political requirements may dictate that decision-makers in different corporate domains access data through different systems. Cross-functional system integration allows the learning intelligence system to push data to the portal or reporting system used by a particular decision-maker. One of the primary challenges when implementing cross-functional system integration is the migration of a diverse range of existing data sources. Different systems, including those of third-party vendors, may process the data feeds, often in a flat file format. It can be difficult to create reports and correct mistakes, especially when the work involves many people exchanging flat files. If the organization has a corporate data warehouse, the learning management system can push the learning management data into this consolidated data source. Any corporate reporting system can then access this learning data, combine it with other business data, and make more advanced ROI calculations. Different data owners maintain data integrity in the consolidated data source, which provides a unified data access point. If the organization does not have a centralized data warehouse, the learning intelligence system becomes critical to cross-functional reporting that includes training data. When many different locations contain Alignment to individual and organizational performance What differentiates a cross-functional learning intelligence system from most LMSs is the ability to align training with performance objectives for the entire extended enterprise, including individuals, the organization, and its business partners. The learning intelligence system can combine the course completion, certification, and assessment scores of the LMS with the evaluation and competency data in a performance management system. Likewise, the system can combine LMS data with business results from other corporate systems. Historical training and business data provides a good starting point for developing a statistical training model that will identify the training programs that had the greatest impact on individual and organizational performance. After developing this initial model, the organization can apply the model to current data to quantify how training affected performance. Clearly, control groups present a few challenges. Performance measurements must take place over the same time periods. Both the control group and the group receiving training must be nearly identical in make-up, performing similar job functions. Sales groups operating in different sales regions or manufacturing groups working in different shifts will confound the results. ... [S]tructuring a valid control group requires great commitment on the part of the organization. Reporting and analytical tools Flexible reporting and intuitive user interfaces are also keys to making the data available and easily accessible. Portal applications can provide reporting tools that training professionals can use to leverage their statistical training models. This allows for more informed decisions when designing and implementing a training program. Some possible tools include: • A training scorecard that evaluates training programs on ROI and other performance metrics • Sales, manufacturing, distribution, customer service, and other scorecards that provide performance metrics specific to each domain, including training • Ad hoc reporting for sophisticated and quick information retrieval to meet pressing business requirements LEARNING SOLUTIONS | April 9, 2007 6 Management Strategies • A predictive analytical tool that allows organizations to allocate training resources to achieve desirable organizational performance For example, a training scorecard application can track ROI metrics, such as sales increases as a result of course completions, assessment scores, or certifications, and it can maintain performance accountability. The training scorecard becomes a much more powerful tool to manage interdependent activities and performance if it has the following features: • Drills down to supporting data detail for ROI scores • Provides a breakdown of performance scores for a wide range of training attributes, including curriculum, location, instructor, delivery method, and others • Accounts for regional economic differences, and other environmental factors beyond the organization’s direct control With an easy-to-use “drill-down” capability, training professionals can identify how cost and performance results contribute to a training program’s ROI score. Providing this type of analysis, however, depends on a statistically validated model of cause-and-effect. ods used in industry to improve quality, such as Six Sigma and structural equation modeling. I believe this model and these industrial methods are applicable to training. Evaluating effectiveness Tracking training and business performance results is critical to achieving an expected ROI. However, when an organization simply measures without an understanding of interdependent cause-and-effect relationships, it does not accurately evaluate training effectiveness. Often, people make inferences about simple causal relationships, focusing on a single cause and effect. For example, good or bad sales may be the result of general economic conditions and other factors. A company may achieve better sales numbers following a sales training initiative even if the training itself was deficient. Tracking results does not necessarily evaluate how training modified sales staff behavior or ability. There can be a distressing disparity between training metrics and the information that actually helps an organization leverage training to improve business results. Multivariate analyses can provide a tool to help organizations evaluate and quantify training effectiveness. Specialized techniques, such as structural equation modeling (SEM), can enhance a decision maker’s intuitive understanding of the world with more precise multivariate analyses. In this way, the decision maker is better equipped to propose and test broader models of the world. (See the References at the end of this article for some details and examples.) This statistical understanding can become a predictive model that identifies how to invest training allocations and expenditures in order to have the greatest influence on training results or business results. [B]y selecting those measurements that can support valid inferences about the effectiveness of programs, learning and training professionals can know where to improve and how to allocate resources and effort. Developing a model for learning improvement Physicians study the signs and symptoms revealed by medical tests and prescribe medicines, diets, surgical procedures, or exercise programs to their patients. Taking the medicines in the prescribed dosages, following the recommended diet, and doing the exercises are the critical behaviors that help patients improve their health. If the medical tests were faulty, the signs and symptoms would be wrong, and the doctors would be unable to make proper recommendations. In medicine, physicians can properly read the signs and symptoms uncovered by tests and other diagnostic tools. The tests and diagnostic tools work because they embody scientific theories of human anatomy and physiology that explain biological interdependencies. To develop the tools that will help achieve higher ROI, training professionals need a model of training for their organization similar to the diagnostic tools used by physicians. A model of training would identify the critical learning variables in all areas of interest and show how those variables affect business results. A training model based on multivariate statistical analysis provides the necessary — and often missing — basis to reduce variation and improve training processes. (See Figure 2.) Multivariate statistical analysis is a collection of procedures which allow measurement and analysis of multiple variables simultaneously. This is one of the disciplines behind meth- Figure 2 Multivariate statistical analysis: Given a set of data, statistical analysis can identify patterns in the data. These patterns can quantify how one or more variable(s) in the data set impact another variable in the set. LEARNING SOLUTIONS | April 9, 2007 7 Management Strategies Training results could equate to competency demonstrated by assessment scores or job performance evaluations. Reductions in assembly line rework and increased customer service satisfaction might be appropriate business results. Such metrics facilitate ROI calculations. Training does not always produce the desired organizational results. Improving outcomes depends on recognizing failed training and its causes. The mathematical method I propose here not only helps ROI measurements, but it can also identify remedies for lower than expected ROI. By identifying the intervening factors that disrupt training effectiveness, it becomes possible to address the root causes for failed training. types of training. The training model both provides a justification for service manager training and it identifies an area for improvement. The entire organization, from the manufacturer down to the individual service manager, develops a greater confidence in how training will impact performance. Example: Analysis of dealership evaluation One automotive dealership study contained a detailed analysis of an evaluation system that an automotive manufacturer used to measure dealer compliance to the manufacturer’s standards. The evaluation system measured more than a hundred items, including cleanliness, size of signs, whether the waiting room had fresh coffee, and a wide range of other factors. Independent of the compliance scores, the study obtained eight dealer performance measures, including unit sales, part sales, market share for two different vehicles, and customer satisfaction with the sales, parts, and service departments. After applying statistical analysis to this data, only seventeen of the measured items in the dealer evaluations system showed consistent relationships with business results. Of those items, certified training was an item that had a positive impact on the dealer performance measures; however, non-certified training did not appear to affect dealership performance. In this case, certification could demonstrate a measurable ROI based on the cost of the certified training programs, and on the business result improvement, as quantified by the statistical analysis. Simplify interdependencies by reducing complexity A complete ROI analysis depends on the causal connections between training and non-training data. Developing a robust training model that can make these connections may require significant data collection and analysis effort. Many training and non-training activities contribute to performance. Which activities does the organization attempt to manage and to what degree? Identifying which factors actually affect desired performance narrows the scope of activities that the enterprise must manage. This reduces both the complexity of the relationship and the administrative effort required to influence performance. Statistical analysis of performance and training data and other information can help map the interdependencies of an enterprise’s training programs with other factors. The analysis can also identify the impact of those interdependencies on business performance. It is possible, for example, to produce a causal mapping that quantifies how the many-to-many relationships of a set of activities will impact overall organizational performance. Such a map or model can then help predict how modifying the training in a particular way will impact desired behavior, such as key performance indicators. For example, suppose that an automotive manufacturer finds that different types of training yield different customer satisfaction results. One approach would be to create incentive systems and certification programs that encourage all dealerships to allocate resources and effort in those areas that will positively affect customer service. By offering the incentives and certifications to all dealerships, the manufacturer reduces the amount of variability across the entire dealer channel. For example, the manufacturer may mandate that dealership service managers complete specialized training that maximizes customer satisfaction more than other A reporting and data management strategy that focuses on the LMS as the foundation, only compounds the system integration challenges that make performance-based training evaluation unmanageable. Instead, the organization should adopt a cross-functional corporate reporting and data management strategy. The technical foundation for this strategy is not the LMS, but a learning intelligence system that acts as a broker between an LMS and other corporate systems. Structural equation modeling Structural equation modeling (SEM) provides a statistical method to develop a causal network of exceptional service elements and to quantify how each relevant variable affects service. This advanced statistical technique studies the simultaneous impact of several independent variables on a specific outcome variable. Each outcome variable of significant interest would have its own model. For example, if the goal of training is improving customer service, there would be a model for the outcome variable that captures the effects of the improvement, the customer satisfaction index. Structural modeling involves four steps: 1. Set initial model according to expert judgment. 2. Collect data for all variables. Data may already exist in company databases or may be commercially available. 3. Run the statistical software to estimate the model parameters. 4. Review and revise model according to data results. The result of this structural modeling process is a LEARNING SOLUTIONS | April 9, 2007 8 Management Strategies path diagram (see Figure 3) showing which variables cause changes in other variables. This diagram could represent the causal network of interrelated factors in the dealer channel that affect customer service. Not only do training professionals get an idea of the variables that affect customer service directly or indirectly, they also get coefficient values that quantify how the variables affect one another. For example, an initial model for automotive customer satisfaction (defined in the first step of the SEP process) may include a range of potentially relevant data. A large amount of data is available from thirdparty dealer management systems, and from the evaluation systems that an automotive manufacturer’s field organization uses to score dealerships on a wide range of metrics. Statistical software tests the initial model. Some variables will have an effect on the customer satisfaction outcome and others will not. The next step is to retain the variables that appear relevant, and then to run the model parameters again. The emerging body of knowledge on transfer of training (see the References at the end of this article) suggests a number of factors that can affect training effectiveness. For example, the transfer “climate” can have a powerful impact on the extent to which learners use their newly acquired competencies back on the job. Delays between training and actual use on the job directly relate to skill decay. In addition, social, peer, subordinate, and supervisor support all play a central role in transfer. Finally, appropriately designed intervention strategies can improve the probability of transfer. These factors would be good starting points for inclusion in the initial iteration of a structural equation model for training programs. dures in order to continually improve manufacturing processes. Six Sigma methodologies significantly improve quality and reduce variability from one unit to the next. In the same way, by selecting those measurements that can support valid inferences about the effectiveness of programs, learning and training professionals can know where to improve and how to allocate resources and effort. Adopting a Six Sigma-like method for training would improve every program’s influence on business results. References Kline, R. B. (2005). The principles and practice of structural equation modeling (Second ed.). New York, NY: Guilford Press. MacCallum, Robert C. and Austin, James T. (2000). Applications of Structural Equation Modeling in Psychological Research. Annual Review of Psychology, Vol. 51: Page 201 — 226. Salas, E., and Cannon-Bowers, J. A. (2001) The science of training: a decade of progress. Annual Review of Psychology, Volume 52: Page 471-499. Multivariate statistical analysis is a collection of procedures which allow measurement and analysis of multiple variables simultaneously. This is one of the disciplines behind methods used in industry to improve quality, such as Six Sigma and structural equation modeling. I believe this model and these industrial methods are applicable to training. About the Author Mark Place is a senior consultant and founding principal of the Latitude Consulting Group, an e-Business consultancy. He is responsible for analyzing business needs, developing project requirements and design specifications, and ultimately for solving his client’s business challenges through the creative use of technical and human resources. Mark began his career at Latitude in 1999 as a Senior Architect. Since then he has played a key role Figure 3 Dealership Service Training: How specialized service manager and technical infrastructure training can affect customer satisfaction. Conclusion: A proposal for continuous improvement In the education and training field, it is not unusual to hear criticisms of ROI measurement as being an instrument of justification, rather than of performance improvement. However, by using a robust learning management platform and applying statistical methodologies, ROI calculations can support continuous improvement of instruction just as they do in other activities. The question is, “How?” In manufacturing, final inspection was the dominant model for quality control until management understood that it makes more sense to focus on the processes. They learned that it is easier to locate the true cause of a failure, or potential failure, in the process rather than in the final product. Manufacturers found a method to improve product quality. You may have heard of it. Six Sigma methodologies measure process quality with statistical proce- LEARNING SOLUTIONS | April 9, 2007 9 Management Strategies in developing solutions for a diverse group of Fortune 500 clients from a variety of industries, including automotive, banking, oil and gas, pharmaceuticals, software, and supply chain automation. Mark has over fifteen years of commercial software development experience, and has been involved in the creation of numerous software products used by millions. Prior to joining Latitiude, Mark was a founder, Vice President of Development, and Chief Technology Officer of Computer Support Technologies, where he conceived, designed, and developed an award-winning technical support system combining Internet and artificial intelligence technologies. Mark is a graduate of Eastern Michigan University, where he received a degree in Computer Science and Mathematics. He is married and lives in Hamburg, Michigan. Mark is an active private pilot and enjoys traveling, cooking, and woodworking. Contact Mark by e-mail to mark.place@latitudecg.com. Discuss this article in the “Talk Back to the Authors” Forum of Community Connections (http://www. elearningguild.com/community_connections/forum/ categories. cfm? catid= 17& entercat=y). You can address your comments to the author(s) of each week’s article, or you can make a general comment to other readers. Additional information on the topics covered in this article is also listed in the Guild Resource Directory. DO YOU HAVE AN INTERESTING STRATEGY OR TECHNIQUE TO SHARE? Get It Published in... This publication is by the people, for the people. That means it’s written by YOU the readers and members of The eLearning Guild! We encourage you to submit articles for publication in Learning Solutions e-Magazine. Even if you have not been published before, we encourage you to submit a query if you have a great idea, technique, case study, or practice to share with your peers in the e-Learning community. If your topic idea for an article is selected by the editors, you will be asked to submit a complete article on that topic. Don’t worry if you have limited experience writing for publication. Our team of editors will work with you to polish your article and get it ready for publication in Learning Solutions. By sharing your expertise with the readers of Learning Solutions, you not only add to the collective knowledge of the e-Learning community, you also gain the recognition of your peers in the industry and your organization. In the Archives This is the first article by Mark Place for Learning Solutions. The eLearning Guild has previously published articles whose topics also relate to this week’s. These are available to Members in the Learning Solutions Archive online. Members must log in to download them. Here are the authors, the article topics, and the publication dates. (Not a Guild Member? Join today for immediate access to these articles and over two hundred others!) How to Submit a Query If you have an idea for an article, send a plain-text e-mail to our editor, Bill Brandon, at bbrandon@eLearningGuild.com, with the following information in the body of the e-mail: • A draft of the first paragraph, written to grab the reader’s attention and identify the problem or issue that will be addressed. • A short outline of your main points addressing the problem or resolving the issue. This could be another paragraph or it could be a bulleted list. • One paragraph on your background or current position that makes you the one to tell this story. • A working title for the article. • Your contact information: name, job title, company, phone, e-mail. This information is to be for the writer of the article. We are unable to accept queries from agents, public relations firms, or other third parties. All of this information should fit on one page. If the topic fits our editorial plan, Bill will contact you to schedule the manuscript deadline and the publication date, and to work out any other details. Refer to www.eLearningGuild.com for Author Guidelines. Articles on related topics Tita Beal: ADDIE and the Kirkpatrick 4 (March 26, 2007) Ty Johnson: Nine too-often-neglected principles (May 23, 2005) Patti Shank: Designing for transfer of learning (September 7, 2004) Bill Brandon: Connecting e-Learning to business objectives (April 12, 2004) LEARNING SOLUTIONS | April 9, 2007 10 About the Guild A Worldwide Community of Practice for e-Learning Professionals offered for each membership level. To learn more about Group The eLearning Guild is a Community of Practice for e-Learning design, Membership and pricing, go to www.eLearningGuild.com. development, and management professionals. Through this memberdriven community we provide high-quality learning opportunities, networking services, resources, and publications. Members represent a Member+ Premium Associate Member Guild Benefits $695 US $1,695 US Free $99 US diverse group of managers, directors, and executives focused on eLearning Insider     training and learning services, as Past Conference Handouts     well as e-Learning instructional Resource Directory — Access & Post designers, content developers,     Web developers, project managers, Community Connections — Access & Post     contractors, and consultants. Guild Job Board — Access Jobs & Resumes     members work in a variety of setJob Board — Post Resumes tings including corporate, govern    ment, and academic organizations. Job Board — Post Jobs     Guild membership is an investLearning Solutions e-Magazine     ment in your professional developGuild Research — Standard Interactive Reports ment and in your organization’s * * * * future success with its e-Learning Guild Research — Online Briefings     efforts. Your membership provides Guild Research — Archives     you with learning opportunities and Guild Research 360˚ Report Purchase Discounts resources so that you can increase  10%* 20%*  your knowledge and skills. That’s Online Forums — Live Events $   $ what the Guild is all about ... putOnline Forums — Archive     ting the resources and information Annual Gathering or DevLearn Registration you need at your fingertips so you $ $ 1 year free $ can produce more successful Learning Management Colloquium Upgrade $ Upgrade $ Upgrade $ Upgrade $ e-Learning. Optional Workshop OR Colloquium Upgrade 1 year free $ $ $ The eLearning Guild offers four Event Fee Discounts levels of membership. Each level 20% 20% 20%  provides members with benefits Online Event Site License Discounts  20% 20%  commensurate with your invest*See www.eLearningGuild.com for details ment. In the table you will find a comprehensive summary of benefits = Included in Membership = Not available $ = Separate fee required The eLearning Guild organizes a variety of important industry events... April 10 - 13, 2007 BOSTON April 11 & 12, 2007 BOSTON CHECK ONLINE for topics and dates! Fall 2007 Dates TBD WEST COAST, USA