Social_value_tool_slides_Warsaw_15.03.11 by qingyunliuliu


									Network: The better future for social economy
Seminar on 15-16 March, Warsaw

Evaluation of socio-economic value of social enterprises
A model for measuring of social added value

Daniela Gatti
Lombardy Region

q   The general goal of the evaluation model is to measure the social value
    produced by a social enterprise as a whole or by a specific project.

q   The primary goal of the system is to give funding entities, particularly
    public bodies, a tool for evaluating the social value created by social

     allowing institutions to make                 using parameters that
        informed decisions and                      are as objective as
       better monitor the use of                  possible and applicable
               resources                               on large scale

q   Public bodies: public bodies play a key role in the process. Their job is to monitor
    and verify the results and social- environmental impacts produced by the allocated
q   Social enterprises: social enterprises are the object of the evaluation. They needs
    to be aware of the generated social value and to develop monitoring, evaluating
    and accounting tools.

q   Associations of social enterprises: The associations of social enterprises are
    responsible for ensuring the quality of their enterprises and developing their
    entrepreneurial skills.
q   Banks and financial institutions: credit institutions, according to the new BASEL II
    rules, can also evaluate enterprises according to their social performance.
q   Private providers (e.g. foundations): foundations for territorial social policies have
    become interesting providers of financing.
Starting point

q   Co-design the research with project partners > choosing the reasearch

    During the meeting held in Warsaw last year it was decided to create a
    tool for evaluating the social value produced by social enterprises by
    using two main instruments:

        Social Accounting         SROI–Social Return on Investment
q   The research outline is based on two principles:

     the importance of the
                                             the participatory approach
       experience aspect

    collecting as many                       involving all stakeholders:
    experiences as possible on               project partners, subjects
    local, national and                      who in the future will use the
    international levels                     evaluation tool, but also
                                             social enterprises that will be
                                             the object of evaluation

    It should lead to the implementation of a tool that is as close as possible to
    the actual needs of the players involved and applicable to/in the everyday
    life of organisations.
Work phases

The research work has implemented the following specific interlinked phases
   which are being carried out partially in parallel:

q   Desk research

q   Questionnaire for project partners

q   Consultation of relevant stakeholders

q   Editing the model for measuring social added value

q   Pilot project

q   Final steps
1. Desk research (1)

q   Survey of documents and site-bibliography: to identify sensitive areas
    and aspects to be considered, approaches and models for quality and
    social responsibility were examined.

q   Subsequently, the research has focused the analysis on:
    4   International systems of quality evaluation (and particularly the ISO, SA8000
        and EFQM rules)
    4   Evaluation systems promoted by EU projects within the last few years (e.g.
        several projects under EQUAL Initiative)
    4   Experiences of social accounting developed in Italy and Europe during the last
        twenty years.
    4   SROI models and tools
1. Desk research (2)
                             Public Administration             Social enterprise

Goals                        -System governance                -Management control
                             -Management control               -Quality
                             -Objectivity and comparability    -Social legitimacy and
                             of choices                        affirmation of identity
                                                               -Strengthening of relationships
Prevalent criteria           -Economic quantitative criteria   -Qualitative and economic
                             defined ex ante                   criteria defined ex ante and ex
Open questions               -Definition of output and         -Definition of output and
                             performance standards             performance standards
                             -Measurement of production        -Finding quality criteria
                             factors and results in economic   -Measurement of outcomes
                             dimensions                        -Stakeholder engagement
                             -Simplicity in the application
                             to ensure homogeneous
Procedures for determining   -Under the government control     -Negotiated
evaluation models
1. Desk research (3)

q   Links between evaluation and Social accounting
     4   they are closely linked by a common thread
     4   they share certain characteristics of social research, such as repeatability,
         transparency, controllability
     4   they are both processes of search for meaning, i.e. a pluralistic process of
         meaning construction that allows room for discrepancies and different points
         of view
q   Links between SROI and Social accounting
     4   they are both approaches to help organisations understand and account for
         the impact of their work.
     4   they recognise that the true social, economic and environmental value
         achieved by organisations is not fully understood or adequately reported.
     4   Both are founded on similar principles.
2. Questionnaire for project partners (1)

The questionnaire tackles the following items :
q   Context analysis: describing the main programmes or funding lines dedicated to
    social enterprises in each country)
q   Currently used evaluation systems: in this part the partners were asked to
    describe in depth the currently used evaluation systems
q   Objectives and perspectives for a new evaluation tool: the aim is to understand
    the main objectives that each partner has in relation to evaluation of social
    enterprises accessing public funding. Moreover the partners were asked to
    describe the features which should characterize a new evaluation system.
q   Questions concerning a possible evaluation tool integrating Social Accounting
    and SROI. In this sections there are questions concerning Social Accounting and
    SROI, in relation to their main strengths and weakness, their features, the need for
    indipendent evaluation and the applicability of the two instruments within the
    social enterprises context of each country.
2. Questionnaire for project partners (2)
q   The partners agree on the following items:
     4   The system has to consider economic, social and environmental results
     4   The system should be relatively easy to carry out (no experts needed)
     4   The system should be clear enough for those who are supposed to use the
         results of the evaluation
     4   The system should take into account small social enterprises, which lack
         human and economic resources
     4   The results of social evaluation should be comparable between organizations
     4   The systems should support social enterprises’ empowerment.
     4   Agreement on a system which uses the methodology of both social accounting
         and SROI, but the partners generally ask for a simplification of the two tools
q   The partners have different ideas on:
     4   indipendent assessment /evaluation : it is a guarantee for process and data
         correctness; on the other hand it is an additonal cost.
3. Consultation of relevant stakeholders (1)

q   focus group with selected local stakeholders:
    4   representatives of the Lombardy Region, of Italian Cooperatives’
        National Agencies and of Finlombarda. Aim: identifying relevant and
        significant dimensions of the social added value analysis

q   one-to-one interviews :
    4    Interviews with representatives of banks working closely with the
        Third Sector both directly and through initiatives and agreements with
        the Lombardy Region and other public investors (3 banks).
    4   Interviews with representatives of social cooperatives (i.e. Legacoop
        and Confcooperative)
    4   Interview with a representative of a guarantee funds Institution:
3. Consultation of relevant stakeholders (2)

The one-to-one interviews have tackled the following themes:

   4   Definition of social enterprise and of social value
   4   Main elements which could be considered indicators of a virtuous
       social enterprises
   4   Main elements which could be considered indicators of problematic
       aspects within social enterprises
   4   Evaluation tools already used to take decisions concerning the
       allocation of funding or loans
   4   Main characteristics of a system which could help public investors in
       evaluating the social added value of social enterprises
   4   Analyse a selection of indicators to provide comments, changes and
3. Consultation of relevant stakeholders (3)

q   Main results form one-to-one interviews:
    4   necessity to elaborate simple tools which could be easily and
        effectively used within daily activities.
    4   the importance of the relation between the evaluator and the
        evaluated enterprises, which is the necessary basis for a coherent and
        adequate evaluation.
    4   Evaluation systems should represent a first step of analysis but not the
        only one.
    4   the importance of training and competence acquisition for evaluators
    4   a the widespread introduction of evaluation systems could be very
        useful in promoting the organizational empowerment of social
4. Editing the model for measuring social added value

The evaluation system consists of two parts

q   The first evaluation system is designed to "measure" the social value
    produced by the social enterprise as a whole

q   The second system is intended for evaluating mainly single projects ore
    interventions (even if it can be applied to the whole organisation). It is
    based on SROI analysis

Each part could be applied at different levels of depth analysis, based on the
   context (i.e. amount of the financial grant, size of the social enterprise,
Evaluation of the overall social added value produced by the enterprise (1)

q   The model has been built and elaborated largely on the basis of the
    accounting reporting guidelines and other experiences concerning the
    generation and distribution of “value” developed in recent years.

q   Main goal: to evaluate social, economic and environmental value
    pruduced by the social enterprise in its whole.
Evaluation of the overall social added value produced by the enterprise (2)

q   Principles

     4   Connection: analysis of elements which could be attributed directly to the
         social enterprise
     4   Completeness: information presenting a complete picture (triple bottom line)
         are requested.
     4   Significance and relevance: the relevance needs to be evaluated on the basis
         of two main factors: 1-the importance given by the stakeholders involved (i.e.
         Public Administrations and social enterprises); 2-the nature, scale and size of
         the elements involved (i.e. amount of funding involved, size of the social
         enterprise, relevance of the project etc.)
     4   Applicability: the evaluation tool and the results from its implementation
         need to be understandable by a sizeable range of stakeholders
     4   Comparability: the results obtained from the implementation of the tool need
         to be comparable
     4   Credibility: the information is reliable and credible only when it is unbiased
         and error-free.
     4   Verification: Data and information (or at least the process of implementation
         and the compliance with the basic principles) will need to be verified by a
         third independent party. Currently, the issue of verification concerning the
         tool is still open: there are difficulties related to costs and timing as well as to
         a lack (in several countries) of professionals accredited to this end
Evaluation of the overall social added value produced by the enterprise (3)

q   The system takes into considerations the following dimensions:
     4   Financial and economic soundness
     4   Democracy and governance
     4   Organizational functioning
     4   Professional resources
     4   Equal opportunities
     4   Socio-occupational integration
     4   Clients
     4   Networks and partners
     4   Project design and innovation abilities
     4   Environmental sustainability
     4   Services and interventions

q   Each of the above dimensions is made up of further sub-dimensions.
Evaluation of the overall social added value produced by the enterprise (4)

q   To simplify the collection and analysis of data; to facilitate comparison and
    make it possible to build a concise index for each area of evaluation:
     4   Numeric indicators are always indexes.
     4   Qualitative indicators are presented as logical indicators, with closed
         questions requiring the answer 'yes' or 'no'.
     4   It will be possible to identify two types of indicators, core and added

q   Regarding the scoring model and calculation methods:
     4    a relative weight will be given to the various dimensions of the analysis in
         order to ensure the efficacy and compliance of the evaluation system.
     4   Scores will need to be determined for each index. The objective is to achieve
         an overall score for each dimension in order to build a dashboard for instant
     4   The (Italian) scores will be defined through the method of percentiles based
         on the weighted average number of responses received from various social
Evaluation of social value produced in projects: calculation of
the SROI (1)

q   SROI is an analytic tool developed recently in the United States for
    measuring and accounting for the social, economic and environmental
    impact created by organisations.

q   This approach aims to reveal the value of changes experienced by
    stakeholders whether those changes have been priced in market
    transaction or not and helps identify which changes are significant. It uses
    financial proxies to help reveal the value and as part of the process of
    deciding which changes are significant.

q   Thanks to funding from the Cabinet Office, a consortium led by SROI
    Network prepared a new Guide to SROI. The SROI Network is an
    international membership body supporting the consistent use and
    development of SROI. The model proposed is based on this Guide
Evaluation of social value produced in projects: calculation of
the SROI (2)

q   Principles
1- Involve stakeholders: Stakeholders are those people or organisations that
    experience change as a result of the activity and they will be best placed to
    describe the change. This principle means that stakeholders need to be identified
    and then involved in consultation throughout the analysis,
2- Understand what changes:. This principle requires the theory of how these changes
    are created to be stated and supported by evidence. These changes are the
    outcomes of the activity, made possible by the contributions of stakeholders, and
    often thought of as social, economic or environmental outcomes. It is these
    outcomes that should be measured in order to provide evidence that the change
    has taken place.
3- Value the things that matter: Many outcomes are not traded in markets and as a
    result their value is not recognised. Financial proxies should be used in order to
    recognise the value of these outcomes and to give a voice to those excluded from
    markets but who are affected by activities.
Evaluation of social value produced in projects: calculation of
the SROI (2)
4- Only include what is relevant and appropriate:. The relevance of an activity or of a
    change will depend from the organisation and its stakeholders: they are the actors
    who identify what is relevant so as to provide a reasonable picture of the impacts
5- Do not over-claim: Only claim the value that organisations are responsible for
    creating. This principle requires reference to trends and benchmarks to help assess
    the change caused by the activity, as opposed to other factors, and to take
    account of what would have happened anyway or happened with the help of
    other organisations.
6- Be transparent: This principle requires that each decision relating to stakeholders,
    outcomes, indicators and benchmarks; the sources and methods of information
    collection; the difference scenarios considered should be explained and
7- Verify the result: Although a SROI analysis provides the opportunity for a more
    complete understanding of the value being created by an activity, it inevitably
    involves subjectivity. Appropriate independent assurance is required to help
    stakeholders assess whether or not the decisions made by those responsible for
    the analysis were reasonable.
Evaluation of social value produced in projects: calculation of
the SROI (3)

q   There are 5 work phases:
    4   Identifying aims and scope of the SROI and identifying the stakeholders. Its
        important to identify aims, subjects, targets, resources of the analysis. it is
        necessary to identify the key stakeholders. The stakeholders are those
        affected by the changes and the impacts produced by the activities analysed.
    q   Identifying the impacts of the project/intervention/service. We can develop
        the “map of impacts”, i.e. describe what changes and impacts the project
        activities might cause through its activities, starting with the incoming
        resources. The following factors must be taken into account: input, activities,
        output, outcome, attributions.
    q    Identifying indicators, economic values and proxies. Meaningful indicators
        must be identified in order to measure the outcomes achieved. Once the
        outcomes, the significant indicators and the data have been identified, it is
        necessary to express the outcomes in economic terms. When the outcomes
        dont’have an economic value, it is necessary to develop proxies.
Evaluation of social value produced in projects: calculation of
the SROI (3)

   4.   Defining Impacts. In this workphase it will be necessary to verify whether the
        outcomes produced are indeed directly linked to the activities carried out by
        the social enterprise. This way, it is possible to avoid an over-estimation of
        the project value and hence increase the credibility and the reliability of the
        calculations carried out. We have to consider: deadweight, attribution,
        negative effects, drop off.

   5.   Calculating the SROI: The SROI measures the value created in relation to the
        costs sustained, thereby acknowledging the social return on investment.

                                         Social value
                            SROI =
                                     Investments (input)
To carry on…

q   Pilot Projects
     4   identifying the key indicators (both compulsory and voluntary)
     4   identifying a system of weights to establish the relative scoring of each area of
     4   Testing the evaluation system and modifying it if necessary

q   Within the Italian pilot project:
     4   the first part of the evaluation tool will be applied to an adequate number of
         social enterprises within the Lombardy Region. This application will allow for
         the identification of values which will be used to calculate a weighted average
     4   the Italian tool (first and second part), will be applied and tested so as to be
         able to verify the actual functioning of the tool
     4   The results from this final testing phase will be used to introduce final
         modifications to the model (if necessary)

To top