April 9, 2007 For many years, industrial organizattion have used statistical methood and disciplines such as Six Sigma to identify the root causes of problems and to achieve continnuou improvement goals. Training departments can adapt these methods into a model for learning improvement, if they also implement a learning intelligence system. Such a system expands the traditional reporting and data management strategy beyond the LMS and into a cross-functional corporate reporting and data management strategy. Read this week’s article to find out how! Evaluating Training ROI with a Learning Intelligence System By Mark Place Training is a critical component in any organizatioon’ strategy for innovation and continuous improveement Yet, training is an area where the actual retuurnon-investment (ROI) is uncertain. Given the large expenditures for training in many organizaations it is important to develop tools that will help companies answer the following questions and improve the measurement of training effectiveness. These tools need to provide a methodollog to measure, evaluate, and continuously improve training, as well as the organizational and technical infrastructure (systems) to implement the methodoloogy We would like to know: • Is the training program effective? • How can we improve the program? • Did the program achieve the desired results at the lowest possible cost? The emerging body of knowledge on transfer of training suggests a number of important propositions and conclusions. For example, the transfer “climate” can have a powerful impact on the extent to which people use newly acquired competencies back on the job. Delays between training and actual use on the job directly relate to skill decay. Social, peer, subordinate, and supervisor suppoor all play a central role in transfer. And finally, it is possible to design intervenntio strategies to improve the probability of transfer. All four of these intervennin factors affect the results of any given training program, and there are potentially other factors. A publication of THIS WEEK: Management Strategies The eLearning Guild’s Practical Applications of Technology for Learning SM Management Strategies2 LEARNING SOLUTIONS | April 9, 2007 level may indicate other results. Unless a training progrra exists simply for the sake of training, measuremeent should include actual performance data, not just data about performance during the training. Selected metrics, such as sales, customer satisfaction, workpllac safety, productivity, and others, should help demonsstrat where training has increased revenue or decreased costs. For example, corporate health and safety programs train personnel in the hope that they will reduce workpllac accidents, whether it is OSHA regulations or the company that mandates the program. Reduction of costs, both tangible and intangible, is implicit in the goal of reducing workplace accidents. While the tangibbl costs of workplace accidents are fairly straightforwaard the company also incurs other less tangible costs in lost productivity that are even more difficult to measuur and account for. An accident in the workplace can adversely affect the behavior of uninjured workers in other ways. They may approach a piece of machinery more tentatively, or otherwise work with extreme care to avoid an accident themselves. Rough ROI measurements that consider performannc improvements can provide a benchmark for trainiin effectiveness. After implementing a training initia-Learning Solutions e-Magazine™ is designed to serve as a catalyst for innovation and as a vehicle for the dissemination of new and practical strategies, techniques, and best practices for e-Learning design, developmmen and management professionals. It is not intended to be THE definitive authority ... rather, it is intended to be a medium through which e-Learning professionals can share their knowledge, expertise, and experieence As in any profession, there are many different ways to accomplish a specific objective. Learning Solutions will share many different perspecctive and does not position any one as “the right way,” but rather we position each article as “one of the right ways” for accomplishing an objective. We assume that readers will evaluate the merits of each article and use the ideas they contain in a manner appropriate for their specific situation. The articles in Learning Solutions are all written by people who are actively engaged in this profession — not by journalists or freelance writerrs Submissions are always welcome, as are suggestions for future topiccs To learn more about how to submit articles and/or ideas, please visit our Web site at www.eLearningGuild.com. Publisher David Holcombe Editorial Director Heidi Fisk Editor Bill Brandon Copy Editor Charles Holcombe Design Director Nancy Marland Wolinski The eLearning Guild™ Advisory Board Ruth Clark, Lance Dublin, Conrad Gottfredson, Bill Horton, Bob Mosher, Eric Parks, Brenda Pfaus, Marc Rosenberg, Allison Rossett Copyright 2002 to 2007. Learning Solutions e-Magazine™ (formerly The eLearning Developers’ Journal™). Compilation copyriigh by The eLearning Guild. All rights reserved. Please contact The eLearning Guild for reprint permission. Learning Solutions e-Magazine™ is published weekly for members of The eLearning Guild, 375 E Street, Suite 200, Santa Rosa, CA 95404. Phone: +1.707.566.8990. www.eLearningGuild.com One might ask, “Why bother evaluating learning and knowledge transfer? If you evaluate organizational results, you will know if the training was ultimately successful.” The problem with this assertion is that you don’t really know if those results are due to trainiin or to an intervening factor. Additionally, training does not always produce the desired organizational results. It is possible to gain important organizational knowledge by finding the causes of failed training. In this article, I discuss learning intelligence systeem and an approach to learning improvement. A learning intelligence system is an important connectiio between the measures of learning effectiveness that an LMS can provide and the larger enterprise metrics that indicate whether learning transferred. But this is not enough to ensure improvement of results at the enterprise level. For that, we must borrow some ideas from industry. Measuring organizational results Calculation of training ROI requires measuring organizaationa results. Training operation metrics, such as course enrollments and completions, assessment scores, and the results of feedback forms and surveeys may directly relate to those results. Performance data on an individual, department, or business unit One might ask, “Why bother evaluating learning and knowleddg transfer? If you evaluate organizationaa results, you will know if the training was ultimately successsful. The problem with this assertion is that you don’t really know if those results are due to training or to an intervening factoor Additionally, trainiin does not always produce the desired organizational results.tive or changing an existing program, an organization can observe and record a change in performance. Reduced incidents of workplace accidents would be one example of such a change. The organization could compare the value of the performance change (X dollaar saved per reduced incident) to the program’s cost, in order to arrive at an ROI measurement. While this approach appears to answer the question of trainiin program effectiveness, it provides no insight into improving training from either a results or cost perspecctive It also does not provide a means to calculate the ROI that may occur as a result of routine training prograams (How often should workers take the workplace safety program to ensure that accidents remain at the lower level of incidence?) Evaluating training and performance Many learning operations evaluate success based on the ubiquitous Kirkpatrick model, or some variation of it. Most organizations that follow this model are unabbl to evaluate their programs beyond the first two Kirkpatrick levels. (See the article by Tita Beal in the March 26, 2007 issue of Learning Solutions for an explanation of the Kirkpatrick model.) In part, the learniin management systems (LMS) that many organizatiion use make lower-level evaluations easy but don’t provide any mechanism for higher level evaluation. Most learning management systems will automaticaall track and report information required for Level One and Level Two analyses. They include assessmeen tools that can capture each learner’s reaction to the course, and templates that can create reports. For online and blended learning, the Level One assessmeen (the learner’s reaction) can be completely integrra to the course. Likewise, training programs can inexpensively and easily administer pre-and post-tests that evaluate learning results (Level Two). Levels Three and Four evaluations become more difficult and costly to implement and administer. What makes these Levels so difficult to evaluate? In the case of the first two Levels, data collection occurred during course delivery. When evaluating changes in student behavior and training influence on business results, data collection requirements extend beyond course delivery. Different evaluation methods can help answer whethhe students’ behavior actually changed after compleetin a course or attaining a certification. Data can come from many different sources, including custoome satisfaction survey data or performance evaluatiion by supervisors. Trainers or other designated staff can also observe students and record behavior as the students perform their jobs. To evaluate retention rates, there should be a delay between the training • LEARN how organizations — whether corporrate academic, or government — are using different types of interactivity • EXPLORE using different tools and processes for developing interactivity • DISCOVER new e-Learning interactivity and user engagement approaches and theories • EXAMINE how to design various interactiio elements • ENSURE that your interactivity addresses all types of user and content needs New Techniques for Designing and Developing e-Learning Interactions May 17 & 18, 2007 Register Today! +1.707.566.8990 www.eLearningGuild.com Hosted by: Technology Sponsor: 3 LEARNING SOLUTIONS | April 9, 2007 Management StrategiesManagement Strategies4 LEARNING SOLUTIONS | April 9, 2007 and these behavior measurements. Organizations rarely evaluate the business impact of a program. This is much more difficult and costly. Large organizations can use a control group to isolate training from other intervening variables. One group does not get training, while another group does. Comparing the business results from each group reveals whether the trained group performed better. Clearly, control groups present a few challenges. Performance measurements must take place over the same time periods. Both the control group and the group receiving training must be nearly identical in make-up, performing similar job functions. Sales groups operating in different sales regions or manufacturing groups working in different shifts will confound the results. With these challenges, structuring a valid contrro group requires great commitment on the part of the organization. But to perform a good ROI analysis, an organizatiio must really be evaluating its training at Levels Three and Four. Without these levels, ROI analysis becomes merely an exercise of cost justification for expenditures. Implementing blended learning, for example, may appear to have a high ROI. After all, it’s less expensive than instructor-led, and Level Two assesssmen scores improve. However, without an understtandin of the impact on individual and organizatioona performance, one must ask what value is the training really delivering to the organization? Actually, much of the data needed to bridge the gap between training and performance exists in many organizations. Individual performance data exists in performance management systems. Organizational data exists in marketing, sales, and financial systems. Bridging this gap requires a technical infrastructure that minimizes the administrative effort needed to colleec and analyze the training and performance data together. However, learning management systems, both the most common repository for training data and most common mechanism to deliver training, cannno easily bridge the gap. From a functional standpoint, each new LMS releeas adds more robust reporting and data analysis capability, as well as human resource system integratiion In addition, many LMS vendors have added taleen and performance management features to their human capital solution suites. To some degree, these evolutionary changes address the gap between trainiin and its impact on individual performance. But they don’t even begin to address the gap with organizatioona performance. Why are organizations still unlikely to evaluate trainiin at Kirkpatrick’s Level Three? What path can lead to Level Four evaluations? System integration, one commmo point of failure, is critical. Many LMS vendors with a history as product companies have limited expertise in system integration that extends beyond learning systeem and databases. Successfully managing performanncebased training evaluation, however, requires experrtis in data management and warehousing, a varieet of corporate systems and databases, analytics, and Web-based application development. Evaluating training ROI most effectively requires the right technical infrastructure and a model of learniin improvement. Learning intelligence system A reporting and data management strategy that focuuse on the LMS as the foundation, only compounds the system integration challenges that make performanncebased training evaluation unmanageable. Insteead the organization should adopt a cross-functionaa corporate reporting and data management strategy. The technical foundation for this strategy is not the LMS, but a learning intelligence system that acts as a broker between an LMS and other corporate systems. The features of a learning intelligence system (see Figure 1) include: • Independence from a LMS • Cross-functional system integration • Alignment to individual and organizational performmanc • Reporting and analytical tools îFigure 1 Learning intelligence system: The functional parts and characteristics of a learning intelligence systemManagement Strategies5 LEARNING SOLUTIONS | April 9, 2007 LMS independence You should not lock a learning intelligence system into a single LMS platform. By utilizing a generic framework, common LMS data should map to variabble in the learning intelligence system. LMS independdenc ensures more stability over time. It minimiize the extent of required system integration in case of LMS upgrade or replacement by another system. Typically, an organization will feed performance, job code, certification, and other corporate data into the LMS reporting system. By adding a learning intelligeenc system between the LMS and other corporate systems, the organization only needs to update one data connection if the LMS changes. Cross-functional system integration As a broker for business intelligence throughout the organization, a learning intelligence system needs to aggregate the data from multiple corporate systeems If assembling information is too cumbersome and time consuming, and the data is outdated or not even correct, the system cannot enhance ROI evaluatiion by combining training with other business data. Cross-functional system integration allows the organization to leverage training and business data together in a context-sensitive manner. Technical or political requirements may dictate that decision-makeer in different corporate domains access data through different systems. Cross-functional system integration allows the learning intelligence system to push data to the portal or reporting system used by a particular decision-maker. One of the primary challenges when implementing cross-functional system integration is the migration of a diverse range of existing data sources. Different systeems including those of third-party vendors, may procees the data feeds, often in a flat file format. It can be difficult to create reports and correct mistakes, especially when the work involves many people exchanngin flat files. If the organization has a corporate data warehouse, the learning management system can push the learniin management data into this consolidated data source. Any corporate reporting system can then access this learning data, combine it with other businees data, and make more advanced ROI calculatioons Different data owners maintain data integrity in the consolidated data source, which provides a unifiie data access point. If the organization does not have a centralized data warehouse, the learning intelligence system becomes critical to cross-functional reporting that includes training data. When many different locations contain Clearly, control groups present a few challennges Performance measurements must take place over the same time periods. Both the control group and the group receiviin training must be nearly identical in make-up, performing similar job functions. Sales groups operatiin in different sales regions or manufacturrin groups workiin in different shifts will confound the results. ... [S]tructuriin a valid control group requires great commitment on the part of the organizatiion the data, a learning management system would need to send and receive data through many connections. To manage these different data sources dynamically, a learning intelligence system can receive data from these disparate sources and present it through a common, cross-functional reporting and analytical systeem Although integrating multiple data sources can requuir significant system integration effort, the organizattio gains greater control over its learning and businees data. Automating the collection of the training data and consolidating it with business data reduces sources of error and ensures accurate and up-to-date information, which users can share more extensively with a minimal degree of administrative effort. Alignment to individual and organizational performance What differentiates a cross-functional learning intelliggenc system from most LMSs is the ability to align training with performance objectives for the entire extended enterprise, including individuals, the organizattion and its business partners. The learning intelligeenc system can combine the course completion, certification, and assessment scores of the LMS with the evaluation and competency data in a performance management system. Likewise, the system can combiin LMS data with business results from other corporrat systems. Historical training and business data provides a good starting point for developing a statistical training model that will identify the training programs that had the greatest impact on individual and organizational performance. After developing this initial model, the organization can apply the model to current data to quantify how training affected performance. Reporting and analytical tools Flexible reporting and intuitive user interfaces are also keys to making the data available and easily accesssible Portal applications can provide reporting tools that training professionals can use to leverage their statistical training models. This allows for more informed decisions when designing and implementing a training program. Some possible tools include: • A training scorecard that evaluates training progrram on ROI and other performance metrics • Sales, manufacturing, distribution, customer servicce and other scorecards that provide performannc metrics specific to each domain, including training • Ad hoc reporting for sophisticated and quick information retrieval to meet pressing business requirementsManagement Strategies6 LEARNING SOLUTIONS | April 9, 2007 • A predictive analytical tool that allows organizatiion to allocate training resources to achieve desirable organizational performance For example, a training scorecard application can track ROI metrics, such as sales increases as a result of course completions, assessment scores, or certificatiions and it can maintain performance accountabilitty The training scorecard becomes a much more powerful tool to manage interdependent activities and performance if it has the following features: • Drills down to supporting data detail for ROI scores • Provides a breakdown of performance scores for a wide range of training attributes, including curricuulum location, instructor, delivery method, and others • Accounts for regional economic differences, and other environmental factors beyond the organizatioon’ direct control With an easy-to-use “drill-down” capability, training professionals can identify how cost and performance results contribute to a training program’s ROI score. Providing this type of analysis, however, depends on a statistically validated model of cause-and-effect. Developing a model for learning improvement Physicians study the signs and symptoms revealed by medical tests and prescribe medicines, diets, surgiica procedures, or exercise programs to their patieents Taking the medicines in the prescribed dosagees following the recommended diet, and doing the exercises are the critical behaviors that help patients improve their health. If the medical tests were faulty, the signs and symptoms would be wrong, and the doctors would be unable to make proper recommendatiions In medicine, physicians can properly read the signs and symptoms uncovered by tests and other diagnostti tools. The tests and diagnostic tools work because they embody scientific theories of human anatomy and physiology that explain biological interdependenciies To develop the tools that will help achieve higher ROI, training professionals need a model of training for their organization similar to the diagnostic tools used by physicians. A model of training would identify the critical learning variables in all areas of interest and show how those variables affect business results. A training model based on multivariate statistical analysis provides the necessary — and often missing — basis to reduce variation and improve training processes. (See Figure 2.) Multivariate statistical analysis is a collection of procedures which allow measurement and analysis of multiple variables simultaneoously This is one of the disciplines behind methood used in industry to improve quality, such as Six Sigma and structural equation modeling. I believe this model and these industrial methods are applicable to training. Evaluating effectiveness Tracking training and business performance results is critical to achieving an expected ROI. However, when an organization simply measures without an understanding of interdependent cause-and-effect relationships, it does not accurately evaluate training effectiveness. Often, people make inferences about simple causal relationships, focusing on a single cause and effect. For example, good or bad sales may be the result of general economic conditions and other factors. A company may achieve better sales numbers following a sales training initiative even if the training itself was deficient. Tracking results does not necessarily evaluaat how training modified sales staff behavior or abilitty There can be a distressing disparity between trainiin metrics and the information that actually helps an organization leverage training to improve business results. Multivariate analyses can provide a tool to help organizzation evaluate and quantify training effectiveneess Specialized techniques, such as structural equatiio modeling (SEM), can enhance a decision maker’s intuitive understanding of the world with more precise multivariate analyses. In this way, the decision maker is better equipped to propose and test broader modeel of the world. (See the References at the end of this article for some details and examples.) This statistical understanding can become a predicctiv model that identifies how to invest training allocattion and expenditures in order to have the greatest influence on training results or business results. [B]y selecting those measurements that can support valid inferennce about the effectiveenes of programs, learning and training professionals can know where to improve and how to allocate resources and effort. îFigure 2 Multivariate statistical analysis: Given a set of data, statistical analyssi can identify patterns in the data. These patteern can quantify how one or more variable(s) in the data set impact another variable in the set.Training results could equate to competency demonstrrate by assessment scores or job performance evaluations. Reductions in assembly line rework and increased customer service satisfaction might be appropriate business results. Such metrics facilitate ROI calculations. Training does not always produce the desired organizattiona results. Improving outcomes depends on recognizing failed training and its causes. The mathemattica method I propose here not only helps ROI measurements, but it can also identify remedies for lower than expected ROI. By identifying the interveniin factors that disrupt training effectiveness, it becoome possible to address the root causes for failed training. Simplify interdependencies by reducing complexity A complete ROI analysis depends on the causal connections between training and non-training data. Developing a robust training model that can make these connections may require significant data collectiio and analysis effort. Many training and non-training activities contribute to performance. Which activities does the organization attempt to manage and to what degree? Identifying which factors actually affect desiire performance narrows the scope of activities that the enterprise must manage. This reduces both the complexity of the relationship and the administrative effort required to influence performance. Statistical analysis of performance and training data and other information can help map the interdependeencie of an enterprise’s training programs with other factors. The analysis can also identify the impact of those interdependencies on business performance. It is possible, for example, to produce a causal mappiin that quantifies how the many-to-many relationshhip of a set of activities will impact overall organizatioona performance. Such a map or model can then help predict how modifying the training in a particular way will impact desired behavior, such as key performmanc indicators. For example, suppose that an automotive manufactuure finds that different types of training yield different customer satisfaction results. One approach would be to create incentive systems and certification programs that encourage all dealerships to allocate resources and effort in those areas that will positively affect custoome service. By offering the incentives and certifications to all dealerships, the manufacturer reduces the amount of variability across the entire dealer channel. For examplle the manufacturer may mandate that dealership service managers complete specialized training that maximizes customer satisfaction more than other Management Strategies7 LEARNING SOLUTIONS | April 9, 2007 A reporting and data management strategy that focuses on the LMS as the foundation, only compounds the system integration challenges that make performance-based training evaluation unmanaggeable Instead, the organization should adopt a cross-functionaa corporate reporting and data management strategy. The technical foundation for this strateeg is not the LMS, but a learning intelligence system that acts as a broker between an LMS and other corporaat systems. types of training. The training model both provides a justification for service manager training and it identifiie an area for improvement. The entire organization, from the manufacturer down to the individual service manager, develops a greater confidence in how trainiin will impact performance. Example: Analysis of dealership evaluation One automotive dealership study contained a detaiile analysis of an evaluation system that an automottiv manufacturer used to measure dealer compliannc to the manufacturer’s standards. The evaluation system measured more than a hundred items, includiin cleanliness, size of signs, whether the waiting room had fresh coffee, and a wide range of other factors. Independent of the compliance scores, the study obtained eight dealer performance measures, includiin unit sales, part sales, market share for two differeen vehicles, and customer satisfaction with the sales, parts, and service departments. After applying statistical analysis to this data, only seventeen of the measured items in the dealer evaluatiion system showed consistent relationships with business results. Of those items, certified training was an item that had a positive impact on the dealer performmanc measures; however, non-certified training did not appear to affect dealership performance. In this case, certification could demonstrate a measurabbl ROI based on the cost of the certified training programs, and on the business result improvement, as quantified by the statistical analysis. Structural equation modeling Structural equation modeling (SEM) provides a statisttica method to develop a causal network of exceptioona service elements and to quantify how each relevaan variable affects service. This advanced statistical technique studies the simultaneous impact of several independent variables on a specific outcome variable. Each outcome variable of significant interest would have its own model. For example, if the goal of trainiin is improving customer service, there would be a model for the outcome variable that captures the effeect of the improvement, the customer satisfaction index. Structural modeling involves four steps: 1. Set initial model according to expert judgment. 2. Collect data for all variables. Data may already exist in company databases or may be commerciaall available. 3. Run the statistical software to estimate the model parameters. 4. Review and revise model according to data results. The result of this structural modeling process is apath diagram (see Figure 3) showing which variables cause changes in other variables. This diagram could represent the causal network of interrelated factors in the dealer channel that affect customer service. Not only do training professionals get an idea of the variabble that affect customer service directly or indirectly, they also get coefficient values that quantify how the variables affect one another. For example, an initial model for automotive custoome satisfaction (defined in the first step of the SEP process) may include a range of potentially relevant data. A large amount of data is available from thirdpaart dealer management systems, and from the evaluattio systems that an automotive manufacturer’s field organization uses to score dealerships on a wide range of metrics. Statistical software tests the initial model. Some variables will have an effect on the custoome satisfaction outcome and others will not. The next step is to retain the variables that appear relevaant and then to run the model parameters again. The emerging body of knowledge on transfer of training (see the References at the end of this article) suggests a number of factors that can affect training effectiveness. For example, the transfer “climate” can have a powerful impact on the extent to which learneer use their newly acquired competencies back on the job. Delays between training and actual use on the job directly relate to skill decay. In addition, social, peer, subordinate, and supervisor support all play a central role in transfer. Finally, appropriately designed intervention strategies can improve the probability of transfer. These factors would be good starting points for inclusion in the initial iteration of a structural equatiio model for training programs. Conclusion: A proposal for continuous improvement In the education and training field, it is not unusual to hear criticisms of ROI measurement as being an instrument of justification, rather than of performance improvement. However, by using a robust learning management platform and applying statistical methodologgies ROI calculations can support continuous improvement of instruction just as they do in other activities. The question is, “How?” In manufacturing, final inspection was the dominant model for quality control until management understtoo that it makes more sense to focus on the processes. They learned that it is easier to locate the true cause of a failure, or potential failure, in the process rather than in the final product. Manufacturers found a method to improve product quality. You may have heard of it. Six Sigma methodoloogie measure process quality with statistical proceduure in order to continually improve manufacturing processes. Six Sigma methodologies significantly improve quality and reduce variability from one unit to the next. In the same way, by selecting those measurements that can support valid inferences about the effectivenees of programs, learning and training professionals can know where to improve and how to allocate resources and effort. Adopting a Six Sigma-like method for training would improve every program’s influence on business results. References Kline, R. B. (2005). The principles and practice of structural equation modeling (Second ed.). New York, NY: Guilford Press. MacCallum, Robert C. and Austin, James T. (2000). Applications of Structural Equation Modeling in Psychological Research. Annual Review of Psychology, Vol. 51: Page 201 — 226. Salas, E., and Cannon-Bowers, J. A. (2001) The sciennc of training: a decade of progress. Annual Review of Psychology, Volume 52: Page 471-499. About the Author Mark Place is a senior consultant and founding principal of the Latitude Consulting Group, an e-Businnes consultancy. He is responsible for analyzing business needs, developing project requirements and design specifications, and ultimately for solving his client’s business challenges through the creative use of technical and human resources. Mark began his career at Latitude in 1999 as a Senior Architect. Since then he has played a key role Management Strategies8 LEARNING SOLUTIONS | April 9, 2007 Multivariate statistical analysis is a collection of procedures which allow measurement and analysis of multippl variables simultaneouusly This is one of the disciplines behiin methods used in industry to improve quality, such as Six Sigma and structural equation modeling. I believe this model and these industrial methods are applicabbl to training. îFigure 3 Dealership Service Training: How specializze service manager and technical infrastruuctur training can affect customer satisfacttionManagement Strategies9 LEARNING SOLUTIONS | April 9, 2007 in developing solutions for a diverse group of Fortune 500 clients from a variety of industries, including automottive banking, oil and gas, pharmaceuticals, softwaare and supply chain automation. Mark has over fifteee years of commercial software development experieence and has been involved in the creation of numerrou software products used by millions. Prior to joining Latitiude, Mark was a founder, Vice President of Development, and Chief Technology Offiice of Computer Support Technologies, where he conceived, designed, and developed an award-winniin technical support system combining Internet and artificial intelligence technologies. Mark is a graduate of Eastern Michigan University, where he received a degree in Computer Science and Mathematics. He is married and lives in Hamburg, Michigan. Mark is an active private pilot and enjoys traveling, cooking, and woodworking. Contact Mark by e-mail to mark.place@latitudecg.com. Discuss this article in the “Talk Back to the Authorrs Forum of Community Connections (http://www. elearningguild.com/community_connections/forum/categories. cfm? catid= 17& entercat=y). You can address your comments to the author(s) of each week’s article, or you can make a general comment to other readers. Additional information on the topics covered in this article is also listed in the Guild Resource Directory. This is the first article by Mark Place for Learning Solutions. The eLearning Guild has previoousl published articles whose topics also relate to this week’s. These are available to Members in the Learning Solutions Archive online. Members must log in to download them. Here are the authors, the article topics, and the publication dates. (Not a Guild Member? Join today for immediate access to these articles and over two hundred others!) Articles on related topics Tita Beal: ADDIE and the Kirkpatrick 4 (March 26, 2007) Ty Johnson: Nine too-often-neglected principles (May 23, 2005) Patti Shank: Designing for transfer of learning (September 7, 2004) Bill Brandon: Connecting e-Learning to business objectives (April 12, 2004) In the Archives This publication is by the people, for the people. That means it’s written by YOU the readers and members of The eLearning Guild! We encourage you to submit articles for publication in Learning Solutions e-Magaziine Even if you have not been published before, we encourage you to submit a query if you have a great idea, technique, case study, or practice to share with your peers in the e-Learning community. If your topic idea for an article is selected by the editors, you will be asked to submit a complete article on that topic. Don’t worry if you have limited experience writing for publication. Our team of editors will work with you to polish your article and get it ready for publication in Learning Solutions. By sharing your expertise with the readers of Learning Solutions, you not only add to the collective knowledge of the e-Learning community, you also gain the recognnitio of your peers in the industry and your organization. How to Submit a Query If you have an idea for an article, send a plain-text e-mail to our editor, Bill Brandon, at bbrandon@eLearningGuild.com, with the following information in the body of the e-mail: • A draft of the first paragraph, written to grab the reader’s attention and identify the problem or issue that will be addressed. • A short outline of your main points addressing the problem or resolving the issue. This could be another paragraph or it could be a bulleted list. • One paragraph on your background or current position that makes you the one to tell this story. • A working title for the article. • Your contact information: name, job title, company, phone, e-mail. This informatiio is to be for the writer of the article. We are unable to accept queries from agents, public relations firms, or other third parties. All of this information should fit on one page. If the topic fits our editorial plan, Bill will contact you to schedule the manuscript deadline and the publication date, and to work out any other details. Refer to www.eLearningGuild.com for Author Guidelines. Get It Published in... DO YOU HAVE AN INTERESTING STRATEGY OR TECHNIQUE TO SHARE?The eLearning Guild is a Community of Practice for e-Learning design, development, and management professionals. Through this memberdriive community we provide high-quality learning opportunities, networrkin services, resources, and publications. Members represent a diverse group of managers, directoors and executives focused on training and learning services, as well as e-Learning instructional designers, content developers, Web developers, project managers, contractors, and consultants. Guild members work in a variety of settiing including corporate, governmeent and academic organizations. Guild membership is an investmeen in your professional developmeen and in your organization’s future success with its e-Learning efforts. Your membership provides you with learning opportunities and resources so that you can increase your knowledge and skills. That’s what the Guild is all about ... puttiin the resources and information you need at your fingertips so you can produce more successful e-Learning. The eLearning Guild offers four levels of membership. Each level provides members with benefits commensurate with your investmeent In the table you will find a comprehensive summary of benefits offered for each membership level. To learn more about Group Membership and pricing, go to www.eLearningGuild.com. About the Guild 10 LEARNING SOLUTIONS | April 9, 2007 The eLearning Guild organizes a variety of important industry events... A Worldwide Community of Practice for e-Learning Professionals Guild Benefits Associate Free 33333388* 388$ 8$ Upgrade $ $ 88 Member $99 US 33333333* 338$ 8$ Upgrade $ $ 20% 8 Member+ $695 US 33333333* 33 10%* 33$ Upgrade $ $ 20% 20% Premium $1,695 US 33333333* 33 20%* 33 1 year free Upgrade $ 1 year free 20% 20% 3= Included in Membership 8= Not available $ = Separate fee required *See www.eLearningGuild.com for details CHECK ONLINE for topics and dates! Fall 2007 Dates TBD WEST COAST, USA April 10 -13, 2007 BOSTON April 11 & 12, 2007 BOSTON eLearning Insider Past Conference Handouts Resource Directory — Access & Post Community Connections — Access & Post Job Board — Access Jobs & Resumes Job Board — Post Resumes Job Board — Post Jobs Learning Solutions e-Magazine Guild Research — Standard Interactive Reports Guild Research — Online Briefings Guild Research — Archives Guild Research 360° Report Purchase Discounts Online Forums — Live Events Online Forums — Archive Annual Gathering or DevLearn Registration Learning Management Colloquium Optional Workshop OR Colloquium Upgrade Event Fee Discounts Online Event Site License Discounts