Docstoc

E-Learning and Return on Investment

Document Sample
E-Learning and Return on Investment Powered By Docstoc
					           OCSLD




Designing (e)learning to meet
the needs of the learner, the
manager and deliver return on
investment

Chartered Insurance Institute
10 May 2007
Oxford Centre for Staff and
Learning Development


  Who, me?
  Education
  •   1968 - DEC PDP8 & FORTRAN
  •   1972 - BA (English Lit)
  •   1986 - MPhil (Historical & Comparative Linguistics)
  •   2001 - MA (Education w/Open and Distance Education)
  •   2004 - PhD study at U of Southampton
       • The extent to which beliefs about learning and teaching
          are embedded in the artefacts of learning technology

  Work & Related
  •   1987 PEP Preparatory Education Project
  •   1988 ACE adult community continuing education tutor
  •   1989 College of Petroleum and Energy Studies
  •   2000 Brookes
  •   2003 Open University

  •   ALT
  •   Writing

 George Roberts
 Educational Developer
 groberts@brookes.ac.uk
Oxford Centre for Staff and
Learning Development


   Outline
   • Issues

   • Objectives

   • ROI

   • Learning Design
Oxford Centre for Staff and
Learning Development


   Exploring the issues
   Consider the following images

   1. What are the issues, wide and narrow?

   2. What are the training issues?
Oxford Centre for Staff and
Learning Development          The issues?
Oxford Centre for Staff and
Learning Development          The issues?
Oxford Centre for Staff and
Learning Development                       The issues?
    •   1970 Equal Pay
    •   1974 Rehabilitation of Offenders
    •   1975 Sex Discrimination
    •   1976 Race Discrimination
    •   1995 Disability Discrimination
    •   2000 Part-time Workers
    •   2002 Fixed term workers
    •   SENDA 2001 (prevents
        discrimination in education &
        training)
    •   Race Relations (Amendment)
        Act 2001
    •   Employment Act 2002
    •   Sexual Orientation, Religion &
        Belief 2003
    •   Gender Recognition Act 2004
    •   Civil Partnership Act 2005
    •   Disability Discrimination Act
        2006
    •   Age 2006
    •   Equality Act 2006
Oxford Centre for Staff and
Learning Development               Exploring the issues?



       Consider the preceding images: with your neighbour, in
       pairs or small groups

   1. What are the issues, wide and narrow?

   2. What are the training and development issues?
Oxford Centre for Staff and
Learning Development                   Learning objectives

   At the end of this session you should be able to explain

   • the role of continuing professional development in a
     knowledge-based enterprise
   • a balanced approach to ROI
   • a multi-modal approach to CPD
   • basic principles of Designing for Learning

   • and bust some sad myths
Oxford Centre for Staff and
Learning Development                              Sad myths?
   • One firm training manager surmises that it will take
     between 50 and 200 development hours to produce one
     instructional hour

   • … industry insiders say that initial development costs
     can run from $5,000 - $35,000

   • … lack of human contact can create a static learning
     environment

   Activity

   • In pairs or small groups, identify at least one counter
     example to each of these statements.
Oxford Centre for Staff and
Learning Development                  Busting the myths


   Learning is not just content delivery

       • Communication        =>   Channel

       • Community            =>   Relationship

       • Content              =>   Topic
Oxford Centre for Staff and
Learning Development                       The role of CPD:
                                       Managing change in…
   1. Socio-economic-political environment
      • stakeholders, shareholders, principals and agents
      • legislation and regulation
      • governance and accountability
      • assurance

   2. Cultural environment
      • working practices
      • family patterns

   3. Natural environment

   4. Personal environment
      • your own aims, ambitions
Oxford Centre for Staff and
Learning Development



   Compliance

   • Compliance is practically a full-time job by itself, not just
     because of the constant changes, but because of the
     mandatory training requirements (Cormier 2006)
Oxford Centre for Staff and
Learning Development             ROI: two perspectives

   1. Too hard
   • Research by Ashridge Business School has found that it
     is almost impossible to quantify ROI around
     management development because there are so many
     variables involve

   • net programme benefits cannot be accurately predicted
     and require complex assumptions concerning the future
     effectiveness of a programme
Oxford Centre for Staff and
Learning Development                 ROI: two perspectives

   2. Has to be done

   •   Measuring the impact of training comes down to asking
       the right questions to find the data that can be linked to
       changed behavior
Oxford Centre for Staff and
Learning Development                                ROI: Activity



   In pairs/small groups produce a short list of:

       1. What you need to know: what data will be useful

       2. What questions will elicit that data
Oxford Centre for Staff and
Learning Development                               ROI: has to be done
   •   Measuring the impact of training comes down to asking the right
       questions to find the data that can be linked to changed behavior

   Superficial
       • ask learners to rate how happy they were
       • list the “take homes”
       • Use psychometrics (Myers Briggs, Belbin) to relate multiple motivation
            to departmental objectives
       • establish Learning Transfer Climate, e.g. making sure someone's boss
            had been on a course before them
       • Apprenticeships are cost-effective and low-risk methods of identifying
            people with high potential: trainees AND mentors
             • treat entry-level jobs as development opportunities.
             • hire college students to do entry level jobs part time with a view to
                 talent spotting
       • Formal coaching demonstrates high return-on-investment
   Significant
Oxford Centre for Staff and
Learning Development          Philips evaluation methodology
   • Reaction and planned action (near term)
      • e.g. 90% of participants indicate intention to apply new
         knowledge within 1 month
      • e.g. 80% of participants indicated the training “represented good
         value for the company”
   • Learning (near to mid term)
      • compliance testing
   • Application (mid term)
      • formal and informal diagnosis of learning and planned action
         through observation and conversation
   • Business impact (mid to long term)
      • align departmental aims with corporate and individual aims and
         structure training interventions around departmental aims
   • Value of Investment
      • growth, development, sustainability
      • shareholder value
   • Intangibles
Oxford Centre for Staff and
Learning Development          Philips evaluation instruments
   Activity
   • In groups, suggest appropriate instruments for each
     stage of the Philips methodology:

       •   Reaction and planned action
       •   Learning
       •   Application
       •   Business impact
       •   Value of Investment
       •   Intangibles
Oxford Centre for Staff and
Learning Development


   Communites of Practice

                              The community is
                              the expert system
Oxford Centre for Staff and
Learning Development          The knowledge is in the network



   • Communities of Practice
   • Benchmarking clubs
   • ROI networks
   • Reframe conceptions of learning and teaching in
     the networked environment
   • Learner-centred experiential learning
Oxford Centre for Staff and
Learning Development                 Discourse technologies

   • Development is continuous: a conversation with the
     world, life, and everything
      • punctuated waves
      • pressure boosting
   • Use instruments to measure
      • Matrices
      • Checklists
      • Semi-structured interviews
      • 360 degree review
      • HRD Learning Management Systems (LMS)
   • Value the informal along with the formal
      • Pareto rules
      • Counting everything is not efficient or effective
Oxford Centre for Staff and
Learning Development


   Instruments as ways of seeing
   From individual to social/organisational learning

   •   Spiral IRF
   •   Kolb learning cycle and learning styles
   •   Activity modelling
   •   Pedagogical pragmatism
   •   Five stages of learning socialisation
   •   Learning Technology Support Architecture
Oxford Centre for Staff and
Learning Development          Spiral IRF
Oxford Centre for Staff and
Learning Development                           Kolb learning cycle



                                  Experience




          Experimentation                           Reflection




                              Conceptualisation
Oxford Centre for Staff and
Learning Development          Kolb learning cycle
Oxford Centre for Staff and
Learning Development          Activity model
Oxford Centre for Staff and
Learning Development          Pedagogical pragmatism:
                                       the Borg cube?
Oxford Centre for Staff and
Learning Development          Salmon’s 5-stage model
Oxford Centre for Staff and
Learning Development          Learning Technology
                              Support Architecture
Oxford Centre for Staff and
Learning Development


   Activity
   • For each visualisation, develop an instrument to
     elicit evidence for the value of an intervention
Oxford Centre for Staff and
Learning Development            Designing for learning (D4L)

   The systematic approach
   1. Write aims and outcomes
   2. Identify topics
   3. Structure the programme
   4. Design teaching and learning strategies taking into
      account student characteristics
   5. Identify, select and design learning resources
   6. Plan and design assessments
   7. Evaluate
Oxford Centre for Staff and
Learning Development                         Learning outcomes

  Learning outcomes describe:
  • the kinds of knowledge the learner is expected to be
    able to employ
  • the kinds of problems learners may be expected to solve
  • the kinds of skills (intellectual, interpersonal, practical)
    which the student is expected to be able to use
  • the conditions under which the student is expected to be
    able to demonstrate their attainments
  • some indication of the level of performance that will be
    expected
Oxford Centre for Staff and
Learning Development                      Learning outcomes

   Why an outcomes-based approach?
   • Outcomes define students knowledge, understanding,
     intellectual and subject specific skills at each level.
   • Outcomes clarify the purpose of the course – for you and
     your students
   • Outcomes help you decide and prioritise which topics to
     teach, and in what depth
   • Outcomes help define appropriate teaching and learning
     strategies
   • Thinking about how students demonstrate their learning
     leads naturally to purposeful assessment tasks
Oxford Centre for Staff and
Learning Development                 Learning outcomes

   But,
   • Outcomes have been criticised for being
     reductionist – especially when written for lower
     cognitive levels
   • Outcomes usually focus on knowledge,
     neglecting attitudes, values, motivations and
     interests
   • May limit the unexpected outcomes of learning
Oxford Centre for Staff and
Learning Development                   Learning outcomes

   By the end of this course, students will be able to…

     verb                     object   context
Oxford Centre for Staff and
Learning Development                              Learning outcomes

   Levels of learning: Bloom

                                                          evaluation
                                              synthesis
                                       analysis

                              application

          comprehension
    knowledge
Oxford Centre for Staff and
Learning Development                         Learning outcomes

   Hierarchies of learning: Biggs
      theorising         Deep learning
       applying
         relating
     explaining
     describing
                                                    Surface learning
    note-taking
   memorising

                       passive                      active
                                  Teaching method
Oxford Centre for Staff and
Learning Development


   Mix modes
   • on-job coaching

   • classroom training

   • self-directed education
Oxford Centre for Staff and
Learning Development             Blended learning
                               hi collocation
                               hi collaboration   traditional laboratory
                               lo computerisation


                               hi collocation     whiteboards in classrooms
                               hi collaboration   virtual field trips
                               hi computerisation


                               lo collocation    CACL, online forums
                               hi collaboration “Learning to teach online”
                               hi computerisation



                               hi collocation
                               lo collaboration video link lecture
                               hi computerisation


                               lo collocation
                               lo collaboration “traditional” DL
                               lo computerisation

                              lo collocation
                              lo collaboration CBT training
                              hi computerisation
Oxford Centre for Staff and
Learning Development          10 Dimensions of
                               blended learning
           •   delivery
           •   technology
           •   locus
           •   pedagogy
           •   chronology
           •   roles
           •   focus
           •   learning
           •   knowledge
           •   politics
Oxford Centre for Staff and
Learning Development
                                      Online programs provide:
                                                Myth or magic?
   •   flexibility
   •   accessibility
   •   convenience
   •   a consistent message
   •   ease with updating
   •   integration of learning and work

   • increased impact of the money invested in training programs,
   • significantly reduced employee travel cost and time
   • the ability to train more people, more frequently and in shorter
     sessions that are easier to coordinate and schedule
   • online training is scaleable because it offers the ability to add
     instructors and students as needed, with fewer changes
Oxford Centre for Staff and
Learning Development                  A continuum of learning

   • Learning, change, and innovation are part of the same continuum
     and are not discrete activities
      • Communication, Coaching, Consulting

   •   Value time
   •   Plan ahead
   •   Structure the programme
   •   Make it mandatory
   •   Keep it flexible
   •   Build in reflection
   •   Assess and evaluate every intervention

                                                (Bates 2006, Howe & Kornfeind 2006)
Oxford Centre for Staff and
Learning Development                                Human capital:
                                        ROI = Return on Intelligence
   • Long-term organizational ROI in knowledge-based industries means
     making routine investments in human capital
   • Value of the investment in training goes beyond the compliance
     requirements
   • When combined with macro-level analysis… ROI studies supply the
     proof of the impact of learning and development


            •   Commit to your investment
            •   Know your goal
            •   Create a positive culture
            •   Measure progress
            •   Offer feedback
            •   Structure applied learning
Oxford Centre for Staff and
Learning Development                                     Indicative list of providers
   •   Brainshark, a provider of on-demand rich media communications solutions
                http://presentation.brainshark.com/powerpoint-presentations-index.asp
   •   Desire2Learn, providing innovative eLearning solutions
                http://www.desire2learn.com/
   •   Eedo Knowledgeware, a provider of real time learning and knowledge system software
                http://www.eedo.com/
   •   Epic: the UK market leader in e-learning, blended learning and knowledge solutions.
                http://www.epic.co.uk/
   •   eVergance, a provider of consulting services for customer relationship management optimization
                http://www.evergance.com/
   •   HyperWave, collaborative Information Management (CIM) with a fully integrated e-learning and web conferencing
       environment.
                http://www.hyperwave.com/e/
   •   NIIT Global, a learning and knowledge solutions provide
                http://www.niit.com/
   •   OutStart, a provider of software solutions for learning and knowledge-sharing
                http://www.outstart.com/
   •   Saba, human capital management
                http://www.saba.com/index.htm
   •   Tribal: one of the largest providers of training and CPD in the UK, clients include government, the NHS, schools,
       colleges, and the private sector
                http://www.tribalgroup.co.uk/index.php?ob=1&id=16
   •   WBT Systems, a provider of enterprise e-learning solutions
                http://www.wbtsystems.com/
   •   Zeroed-in Technologies: provides software and services that help organizations visualize, measure, and execute
       effective learning and business strategies.
                http://www.getzeroedin.com/
Oxford Centre for Staff and
Learning Development                                                       Reference
 •   anon (2005) HR urged to use ROI for training, People Management, 21 April 2005.
 •   anon (2006) Can Customized Online Training Boost Firm ROI? Compensation and
     benefits for law offices, May 2006, 1, 11-13.
 •   Baraban, R. and Kovaleski, D. (2006) Maximising ROI: Setting objectives for sales
     training, Financial and Insurance Meetings, May/June 2006, 9-10
 •   Bartley, S. J. and Golek, J. H. (2004) Evaluating the Cost Effectiveness of Online and
     Face-to-Face Instruction, Journal of Educational Technology & Society, 7, 167-175.
 •   Brannick, M. (2005) Reporting Results: Managing Measurement, Chief Learning
     Officer, June, 2005, 28-33.
 •   Bates, S. (2006) You can measure training ROI, Official Board Markets, November
     2006, 18-19
 •   Howe, P. E. and Kornfeind, J. F. (2006) The New ROI: Return on Intelligence,
     Pennsylvania CPA Journal, Fall 2006
 •   Paton, Nic, Time to shift the focus from ROI? Training Magazine, Nov 2005, p4
 •   Phillips, J. J. (2005) The value of human capital: a micro-level approach, Chief
     Learning Officer, December, 2005, 52-54.
 •   “Training today”, Training; Sep 2006, Vol. 43 Issue 9, p12-13
 •   Weinstein, Margery, “Measuring ROI”, Training, Jan 2006, Vol. 43, Issue 1