background data are gathered by questionnaire and

Document Sample
background data are gathered by questionnaire and Powered By Docstoc
					Achievement in Relation to
  Background Factors
       Background Data Gathering

   During every cycle of assessment, background data are
    gathered by questionnaire and analysis of fieldworkers’
    reports.

   Questionnaires are given to students (including a section
    for parents), head teachers and teachers.

   Conditions, facilities etc. in schools are observed and
    recorded by test administrators.

   Background data are correlated with learning
    achievement to identify needs for improvement and
    factors associated with achievement.

                                                                2
        Background Data Gathering Framework

   Background data are classified into five areas following
    the Craig-Heneveld Framework:

    •   The Students
    •   Supporting Inputs from Outside the School
    •   Teaching-Learning Process
    •   Enabling Conditions
    •   School Climate

   Results are presented in the next section



                                                               3
       The Students

Data are gathered from students about:

   Socio-economic background (e.g., family size,
    father’s/mother’s education and occupation)

   Factors in the home environment related to achievement
    (e.g., help w/ homework, space for reading, need to work
    outside)

   Attitudes towards education

                                                               4
        For Example:

Do you work outside the home to earn money?

                                    Scaled Scores
Variable %
                   Maths         Urdu        Science       Social Studies

Yes     23          383           370           455             481
No      77          403           389           473             503

Difference       Significant   Significant   Significant     Significant


Conclusion: about 23% of Grade 4 students reported having
to work outside the home. Their achievement is significantly
lower than others’.
                                                                            5
       Supporting Inputs from Outside

Information is gathered from different sources about:

   Parent and community support (e.g., frequent school-
    parent meetings, financial and other support, parental
    involvement in homework)

   Effective support from the education system (e.g.,
    supervision, availability of teaching materials and
    supplementary materials)

   Adequate material support (e.g., facilities, furniture in
    school, school budget for supplies)

                                                                6
       For Example (1):

    How many of the following six activities in the school
    does the head-teacher invite parents to participate in?

   Attend special events (e.g., functions, sports events)
   Raise funds for the school
   Participate in development planning
   Volunteer for school projects, programs, and trips
   Ensure that their child completes his/her homework
   Serve on school committees (e.g. personnel, finance)




                                                              7
       For Example (2):

Number of activities:
 Activities                      Maths          Language
 Low (0-2 Activities)             360               373
 High (3-6 Activities)            385               401
 Difference                    Significant       Significant

Conclusion: most head teachers reported inviting parents
to participate in a significant number of school-related
activities. This participation is positively associated with
student achievement.

                                                               8
       Teaching - Learning Process

Information is gathered from different sources about:

   Learning time (no. of teaching days, length of period,
    time on task)

   Variety of teaching strategies used in class (e.g.,
    questioning techniques, investigation, practical work)

   Homework (amount given, how it is assessed and what
    feedback is given)

   Student assessment and what feedback on their work
    students receive

                                                             9
     For Example:

How often does your teacher check your Maths homework?

         Responses                      Maths
         Twice or More Weekly            401
         Once a Week or Less             347
         Difference                   Significant


Conclusion: the great majority (95%) of students reported
that they are taught by teachers who check their
mathematics homework at least twice weekly. These
students achieve significantly better than students who
report their homework is checked less frequently.
                                                            10
       Enabling Conditions

Information is gathered from different sources about:

   Leadership (e.g., whether the head-teacher is primarily
    an administrative or an instructional leader in the school)

   Capable teaching force (e.g., teacher education,
    qualifications, experience, style)

   Flexibility and autonomy (e.g., decisions and discretion
    of head teacher, teacher)

   High level of time in school (starting/finishing time of
    periods, presence/absence of teachers)

                                                                  11
       For Example (1):

    How many of the following six indicators of academic
    leadership does the head teacher have explicit policy
    for?
   Corporal punishment
   Homework
   Recording and reporting student progress
   Explaining students’ mistakes
   Dealing with weak students
   Parental involvement



                                                            12
     For Example (2):


   Responses                Maths         Language
   None                      393             375
   One                       388             370
   Two or More               404             403
   Difference             Significant     Significant


Conclusion: about two-thirds (63%) of all students are in
schools with no explicit policy for academic matters (i.e.,
most head teachers see themselves as administrative
rather than instructional leaders). Policy for academic
matters is associated with student achievement.
                                                              13
       School Climate

Information is gathered from different sources about:
 High expectations for students (school expects and
   rewards success)

   Teacher Attitudes (attitudes to students and school
    goals, perception of school climate)

   Order and Discipline (e.g., punishment policy and
    methods, feeling safe at school)

   Organized Curriculum available to all

   Classroom Climate (e.g., student participation, teacher-
    student interaction)
                                                               14
       For Example (1):

Students’ Perception of School Climate

   I like being in school

   I think that students in my school try to do their best

   I think that teachers in my school care about the
    students

   I think that teachers in my school want students to do
    their best

                                                              15
    For Example (2):


    Responses                Maths       Language
    Yes to all four           401           387
    Yes to 3 or less          372           368
    Difference             Significant   Significant


Conclusion: an overwhelming majority (92%) of students
responded “yes” to all four statements. This was
associated with higher achievement in both language and
mathematics

                                                          16
           Use of Blackboard and Achievement


    Use of                           Scaled Scores
    Blackboard   %      Maths       Urdu       Science    S. Studies
    Yes          97      399         384         469          497
    No           03      350         341         431          457

    Difference        Significant Significant Significant Significant

    Teachers’ use of black board significantly increased
     students’ achievement in all the four subjects
    Ninety-seven percent teachers reported they daily use
     black boards
                                                                        17
         Use of Textbooks and Additional
         Resources

   Majority of the subject   100
    teachers used textbooks as 90                                                  Urdu Teacher
                                                                                   Maths Teacher
    major source of knowledge. 80                                                  Science Teacher
                                          70                                       Social Studies
                                                                                   Teacher
                                          60
   Around 10% teachers
    reported consulting         Percent   50

                                          40
    additional resources for
                                          30
    teaching
                                          20

                                          10
   Science teachers were                  0
                                               Textbook   Additional   No use of
    more likely to consult                                 sources      textbook

    additional resources than
    other subject teachers
                                                          Back to Slide 15
                                                                                                    18
           Multi-grade Teaching and
           Achievement


Multi-grade           Students’ Scaled Scores       Teachers’ Scaled Scores
Teaching        %         Maths       Urdu            Maths         Urdu

Yes             21    442             495               616           644
No              78    466             516               644           661

Difference              Significant   Significant    Significant   Significant


     Lower frequency of Multi-grade Teaching is linked with increased
      performance for both students and teachers
     Twenty- one percent teachers reported they have to teach more than on
      classes in one period
                                                                              19
             Rewards and Punishment in School

                                                 Scaled Scores
    Reward           %
                            Maths         Urdu           Science     S. Studies

Yes                 85       403           390             474          504
No                  14       372           354             440          464

Difference                Significant   Significant    Significant   Significant
Punishment           %
Frequently          .03      351           345             444          452
Occasionally         38      388           376             458          490
Never                58      405           391             476          505
Difference                Significant   Significant    Significant   Significant
    Student who Never got punished by their teachers performed significantly
     better Fifty-eight percent students reported they never got punished.
    Only .03 % students reported frequent punishment
    Rewarding students’ performance increased achievement all the subjects
                                                                                   20
         PTA/ SMC in School: Head Teachers’ Report


• Around 80% head
teachers reported                      100
presence of PTA/ SMCs’ in               90
School                                  80
•Presence, need, funding,               70
or role of PTA/SMC did not              60
                             Percent

increase students                                                                              Yes
                                        50
performance in any of the                                                                      No
                                        40
NEAS tested school
                                        30
•Only 39% head teachers                 20
reported getting funds                  10
from Govt.
                                         0
•Ninety-seven percent                        PTA/SMC    Need For    Funding of   Copperative
                                              School   PTA/SMC in    PTA/SMC       Role of
head teachers reported
                                                        Schools     from Govt.    PTA/SMC
need for SMCs in Schools


                                                                                                21
                Teaching Resources and Achievement

                                                                   Scaled Scores
    Teaching Resources
                                           %            Maths          Urdu        Science    Social Studies
    Curriculum Document
                                  Yes      42            395           384           476           382
                                   No      56            392           380           459           397
                           Difference                    n. s          n. s          n. s          n.s
    Teaching Kit
                                  Yes      44            396           369           465           483
                                   No      56            393           390           467           503
                           Difference                    n. s       Significant      n. s          n.s
    Teachers Guide
                                  Yes      65            394           372           465           485
                                   No      34            395           400           469           513
                           Difference                    n. s       Significant      n. s       Significant
    Textbooks
                                  Yes       90           396           380           468           496
                                   No      .09           373           389           448           495
                           Difference                    n. s          n. s          n. s          n. s
    AV Aids
                                  Yes      58            397           382           468           477
                                   No      41            396           379           465           483
                           Difference                    n. s          n. s          n. s          n. s

      Only 46% Head teachers reported they have the curriculum document in their school
      Only 37% teachers reported using the curriculum document; use of curriculum document and other
       teaching resources did not significantly increase students achievement
                                                                                                               22
         Teachers General Education in the
         NEAS Sample
                                      Rural %                     Urban %
General Education
                             Female             Male       Female        Male
Matriculation                   47              23           33             23
Intermediate                    25              23           34             18
Bachelors in Arts               18              38           23             40
Bachelors in Science            .07             16           10             20
Masters                         .02              -            -             -
Higher than Masters              -               -            -             -

   Male teachers were better qualified than female teachers in both rural and
    urban areas
   There were more matriculate female than male teachers
   In the NEAS selected schools there were no teachers with higher than
    masters qualification
                                                                                 23
        Head Teachers’ and Teachers’
        Professional Education

   About 57% teachers                60

    were PTC qualified
                                      50


   There were more M. Ed             40
                                                                      PTC
    head teachers than
                            Percent
                                                                      CT
    teachers                          30                              B.Ed
                                                                      M.Ed
                                                                      Higher
                                      20
                                                                      than M.Ed
   There were less than
    1% B. Ed Teachers                 10

    and head teachers
                                       0
                                           Teachers   Head Teachers




                                                                            24
        Teachers Professional Training

                                    60
   Less than 50%
    subject teachers
                                    50
    secured any kind of
    training                                                                Subject Content
                                    40                                      Methodology
                                                                            Subject Curriculum
                          Percent
                                                                            Analytic thinking

   Subject curriculum              30                                      Problem solving


    and problem solving
    were highest                    20

    reported areas of
    training obtained               10


                                    0
                                         Maths   Urdu   Science   Social
                                                                  Studies




                                                                                          25
            Teachers’ Scaled Scores

                                   Scaled Mean
    Subject Teacher                                           Difference
                            Teachers         Students
    Maths                      620              404             Highly
    Urdu                       564              382           Significant
    Difference                   Highly Significant
    Science                    660              467             Highly
    Social Studies             784              496           Significant
    Difference                   Highly Significant

    Teachers scaled scores were significantly higher than students in all
     four subjects
    Teachers scores were well above the set mean of 500 (SD 100)
                                                                             26
          Assessment of Students’
          Performance in School
                                                   Scaled Scores
Method of Assessment
                               Urdu         Maths           Science      Social Studies
Verbal
                    Yes        398           416              476             513
                     No        380           426              479             510
                    Diff.      n.s           n.s              n.s             n.s
Written
                    Yes        395           419              480            513
                     No        452           390              463            531
                    Diff.   Significant   Significant      Significant    Significant
Home work
                    Yes        400           419              481            518
                     No        335           382              418            460
                    Diff.   Significant   Significant      Significant    Significant
Class Performance
                    Yes        399           417              478             514
                     No        372           406              466             502
                    Diff.   Significant      n.s              n.s             n.s
                                                                                          27
          Parents Educational Profiles in the NEAS
          Sample


   Urban fathers were                           Father %        Mother %
    relatively more educated    Education
                                               Rural   Urban   Rural   Urban
    than rural fathers
                                Illiterate      32      25      72      53
   Only .02% urban fathers
    were graduates              Primary         29      27      17      26

                                Matriculates    17      23      .04     13
   Seventy-two percent rural
    and 53 % urban mothers
                                Intermediate    04      .08     .09     .04
    were illiterate
                                Bachelors        -      .02      -       -
   There were no graduate
    or postgraduate mothers     Masters         01      .01      -       -
    in NEAS sample

                                                                               28
           Fathers’ Education and Student
           Achievement

   The table shows a                       Scaled Scores     Scaled Scores
    trend that children of   Education
    fathers with post-                      Maths   Urdu    Science S. Studies
    primary education
                             Illiterate      389    376       462      491
    performed better on
    different subjects       Primary         399    385       469      497
                             Matriculates    408    394       476      506
                             Intermediate    391    407       472      512

                             Bachelors       431    414       493      569

                             Masters         422    399       474      536
                             Higher than
                                             398    357       488      478
                                Masters

                                                                                 29
           Fathers’ Occupation in the NEAS
           Sample

   Majority of rural fathers                                 Percent
    were agriculture            Occupation
                                                           Rural   Urban
    landowners

                                Agricultural Wage Earner    .06     .03
   Majority of urban fathers
    were small business
                                Agriculture Land Owner      35      .08
    owners and skilled
    workers
                                Skilled Workers             13      23

   Fifteen percent urban       Small Business Owner        17      30
    and twelve percent rural
    fathers in govt. jobs       Government Job              12      15

                                Private Job                 03      .07

                                                                           30
           Fathers’ Occupation and Students
           Achievement

                                                    Scaled Scores
    Fathers’ occupation
                                       Maths     Urdu     Science     S. Studies
    Agriculture wage earner             408       367        474         507
    Agriculture land owner              392       375        460         491
    Skilled workers                     393       389        472         498
    Small business owner                401       397        477         510
    Government job                      404       402        473         513
    Private job                         384       374        471         476

    Fathers’ occupation did not significantly affect students achievement on Maths
     and Science tests
    Children of Agricultural wage earner however, scored significantly lower as
     compared to govt. servants on language test
    Children of private employed parents scored significantly lower on social
     studies test
    All other mean differences were non-significant                                  31
       Strengthening Links between NEAS
       and the Educational Planning Process

   NEAS can support educational planning by:
    • providing data about the actual conditions in schools; and
    • indicating the possible impact of policy actions on student
      achievement.

   Educational planners can make NEAS more effective by:
    • proposing topics to be added to background data gathering; and
    • suggesting relationships to be explored.




                                                                       32
Conclusion and Next Steps




                            33
       The presentation in summary:

   NEAS has assessed the achievement of Class IV
    students in 4 subjects: Urdu, Mathematics, Science and
    Social studies.

   The assessment reveals large differences in average
    performance among provinces and between students
    from differing backgrounds.

   These differences in average performance can be traced
    back to differences in student and family characteristics,
    school processes and inputs.

   As illustrated in the next slide the factors that influence
    performance can be divided into four types:
                                                                  34
       Additional work:

   The lack of performance standards limits the
    interpretability of the assessment results.

   Additional analysis is needed to identify causal
    relationships and hence where policy might yield
    improvement.

   The next slides illustrate how multi-level multivariate
    analyses can be used to reflect on the impact of different
    policy interventions.



                                                                 35
       Expected changes in achievement
       over time:
   All other things being equal one would not expect to see
    large changes in achievement levels from year to year.
    Evidence suggests that education reforms take at least 5
    to 10 years to bear fruit.

   For Pakistan one would actually expect test scores to fall
    over time as rising enrollment rates draw more children
    from relatively disadvantaged backgrounds into the
    system.




                                                                 36
     Future Vision of National Assessment
     in Pakistan


Proposed Recommendations by National Select Committee:

•   NEAS future planning should be till 2020 at Grade 4, 5, 8, 10
    and 12 levels

•   Private schools should be included in NEAS sample on Pilot
    basis in 2008 National Assessment

•   Listening and speaking competencies (skills) of student
    assessment would be part of NEAS future activities

•   Pakistan can explore the possibility to participate in the next
    TIMSS in 2012

                                                                      37
Thank You…




             38

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:7
posted:6/8/2012
language:English
pages:38