DIFFERENCES IN SCIENCE TEACHING AND LEARNING AMONG AUSTRALIAN STATES

Document Sample
DIFFERENCES IN SCIENCE TEACHING AND LEARNING AMONG AUSTRALIAN STATES Powered By Docstoc
					       DIFFERENCES IN SCIENCE TEACHING AND LEARNING
                          AMONG AUSTRALIAN STATES

                                 John Ainley and Sue Thomson
                         Australian Council for Educational Research

This paper examines the differences among and within the Australian States in science
teaching and learning based on the analysis of data from TIMSS. It focuses on science
achievement at Grade 8 in 2002. The paper begins with a consideration of the differences
among states in science achievement at Grade 4 and Grade 8 and the way in which
patterns have changed between 1994 and 2002. It then examines the influence of factors
operating at state, school and student level on science achievement at Grade 8 in the
national picture and the way those influences differ among states. It concludes with a
discussion of the factors influencing Grade 8 science achievement.

CONTEXT

Australia’s national goals for schooling assert that when students leave school they
should have attained high standards of knowledge, skills and understanding in eight key
learning areas: the arts; English; health and physical education; languages other than
English; mathematics; science; studies of society and environment; and technology. A
Performance Measurement and Reporting Taskforce (PMRT)1 administers a National
Assessment Programme (NAP) that defines key performance measures and monitors
progress towards the achievement of the national goals (MCEETYA, 2005).

The Trends in International Mathematics and Science Studies (TIMSS) is defined as part
of the NAP. For each cycle of TIMSS extensive national reports are produced that detail
the pattern of results for Australia (Thomson & Fleming, 2004a;Thomson & Fleming,
2004b; Lokan et al, 1996; Lokan et al, 1997). In addition the NAP incorporates annual
assessments of literacy and numeracy using the full population of students at Grades 3, 5
and 7. In civics and citizenship and ICT literacy assessments are conducted using sample



1
    Established by the Ministerial Council for Education. Employment Training and Youth Affairs
    (MCEETYA).
Science learning in Australia                       2                                Ainley & Thomson


surveys of students in Year 6 and Year 10 every three years and in science there is a
sample survey at Grade 6 every three years.

Australia has a federal system of government with States having the major responsibility
for education.          There are differences among States in educational organisation and
curriculum in many fields, including science education and there is increasing interest in
examining the differences among States in fields such as science and mathematics. There
are also differences among the states in the age of students at any given Grade and in the
demographic characteristics of the population. Table 1 contains an indication of some of
these variations.

Table 1        Population Characteristics of Australian States.

                                               Average         IRSED(b)           %                %
                                               Age (a)                         LBOTE(c)         Metro(d).

    New South Wales                               14.0           1000               15              69
    Victoria                                      14.1           1016               16              72
    Queensland                                    13.4            989                7              62
    South Australia                               13.8            994                7              65
    Western Australia                             13.4            996                8              65
    Tasmania                                      14.2            969                1              49
    Northern Territory                            13.8            903               12               0
    Australian Capital Territory                  14.1           1076               10              99
    Australia                                     13.9           1000               11              66

       (a)     Based on data recorded in the TIMSS Australia Report for Science (Thomson & Fleming,
               2004b)

       (b)     Based on the Socioeconomic Indexes for Areas: Index of relative socioeconomic disadvantage
               (IRSED) (ABS, 2004)2. The national mean for collection districts is 1000 and the standard
               deviation is 100.
       (c)     Percentage of Year 9 students for whom the main language spoken at home is other than
               English. Data are from the longitudinal surveys of Australian Youth (LSAY).
       (d)     Percentage of Year 9 students living in a metropolitan zone according to the MCEETYA
               three-category classification of geolocation.   Data are from the longitudinal surveys of
               Australian Youth (LSAY).


2
      The Index of relative socioeconomic disadvantage (IRSED) is one of the Socioeconomic Indexes for
      Areas (SEIFA) computed from census data for collection districts and published by the Australian
      Bureau of Statistics (ABS, 2004). For all SEIFA indexes the mean is 1000 and the standard deviation is
      100. A higher score indicates greater average advantage.
Science learning in Australia              3                           Ainley & Thomson


Boosting science learning has become a priority of the federal government. A national
review of the quality and status of science education in Australian schools concluded that
there was a gap between the ideal and reality especially in secondary school and
particularly in relation to the teaching of science as scientific literacy (Goodrum,
Hackling & Rennie, 2001). The TIMSS 1999 Video Study reported that Australian
lessons were characterised by a core pedagogical approach that involved analysing data
gathered through independent practical activity and focussing on connections between
ideas and real-life experiences (Lokan, Hollingsworth & Hackling, 2006).

There is no common school curriculum in science across the country although there is a
non-mandatory national statement of learning in science that outlines the learning
opportunities that should be provided at each stage of schooling from Grade 1 to Grade
10 (Curriculum Corporation, 2006). A national online science assessment resource bank
(SEAR) has been developed for use by schools to support science teaching. Within states
the pattern is that central authorities specify broad curriculum frameworks and schools
have considerable autonomy in deciding curriculum detail, text-books and teaching
methodology. Learning materials and tests are prepared by a variety of agents including
the curriculum sections of state education departments, academics, commercial
publishers, and teachers’ subject associations.    As a consequence what is taught in
science varies between states and between schools within states.           There are also
variations between states and between schools within states in the amount of time
allocated to science in the junior secondary Grades and specifically in Grade 8.

Differences in measures of science achievement need to be interpreted in relation to
curriculum and policy differences at state level, differences in educational practices at
school and classroom level and differences in student characteristics. This paper presents
an analysis of the ways in which various aspects of science teaching impact on student
learning.

DATA

The international sample design for TIMSS is a two-stage stratified cluster sample design
(Martin, Mullis, Gonzales & Chrostowski, 2004).
Science learning in Australia                           4                                Ainley & Thomson


The first stage consists of a sample of schools, which in Australia is stratified by State
and by sector (with disproportionate sampling across strata followed by weighting).
Nationally, non-government schools enrol 33 per cent of students (29% of elementary
students and 38% of secondary students).

Table 2         Australia’s designed and achieved sample in TIMSS 2002

                                         Grade 4                                         Grade 8
           Design
State      School
           Sample         N           N                                   N           N
                                               Weighted     Weighted                           Weighted    Weighted
                        Schools    students                             Schools    students
                                                  N         Percent                               N        Percent

NSW           40          35         912        90781        35.3         34         880        84456        32.8

VIC           35          32         675        62852        24.4         34         860        65435        25.4

QLD           35          31         759        43597        16.9         33         881        48270        18.8

SA            30          27         600        20901         8.1         28         703        18902        7.3

WA            30          27         661        26123        10.2         26         702        27616        10.7

TAS           30          25         501        6444          2.5         26         625        6424         2.5

NT            15          13         251        2300          0.9         14         321        1578         0.6

ACT           15          14         316        4224          1.6         15         383        4727         1.8

Total        230         204        4675       257222        100.0       210        5355       257408       100.0




The second stage consists of a random sample of one classroom from the target grade in
each sampled school. The numbers of students in TIMSS 2002 for Population 1 and
Population 2, along with the number of schools are shown in Table 23. In the achieved
sample for Grade 8 there were 5,335 students from 210 schools and for Grade 4 there
were there were 4,675 students from 204 schools.




3
        Australia achieved the required participation rate of 85% of sampled schools and 85% of sampled
        students (or a combined schools and students participation rate of 75%) for Grade 8 but just fell short
        of the minimum requirements for Grade 4. Sampling weights were calculated by Statistics Canada to
        ensure that the population at each year level was appropriately represented by the students participating
        in TIMSS.
Science learning in Australia                   5                               Ainley & Thomson


ANALYSIS

Results from two types of analyses are reported in this paper.

The first type of analysis is a comparison of means. The comparison of means is based
on weighted data so that the distribution of students accurately reflects the population and
jack-knife replication techniques are used to properly estimate standard errors from
complex samples4. Because the estimation of errors is based on rigorous procedures and
because our focus is on implications for policy and practice (rather than establishing
laws) we have commented on results significant at the 10% level.

The second type of analysis is based on multi-level regression analysis5 that examines the
patterns of association between science achievement measured on the TIMSS Science
Scale and predictors measured at the level of the student, the school (or classroom) and
the state. Two main forms of Hierarchical Linear Modelling are reported. The first form
was a three-level national analysis with predictors considered at state, school/classroom
and student level.       This analysis provided information about national patterns of
influences on science achievement. The second form was a series of replicated within-
state two-level models with predictors considered at the school/classroom and student
level. Those analyses established whether the effects of the predictors were similar or
different across different contexts.

The analyses refer to the school/classroom level because of the nature of the sampling for
TIMSS in Australia. Within-school sampling is based on the random selection of one
mathematics class per school. Grade 8 students from that class may either be in the same
class for science or be dispersed among different classes for science (with some of those
classes containing very few sampled students). Many of the data of interest to these
analyses are based on information provided by teachers. We chose to aggregate those to
school level so as to ensure stability in the level-two unit.




4
    Using the program AM developed by the American Institutes for Research.
5
    Using the program Hierarchical Linear Modelling Version 6.03 (Raudenbush et al, 2004).
Science learning in Australia                     6                                Ainley & Thomson


The following predictor variables were included at the student level in the final model6.

        •    Gender was coded with males as zero and females as one. Fifty-one percent of
             respondents were female.

        •    Age expressed in months. The mean age was 166 months (13 years 10 months)
             with a standard deviation of six months.

        •    Indigenous status was coded with non-Indigenous students as zero and
             Indigenous (Aboriginal or Torres Strait Islander) students as one. Three per cent
             of the sample was Indigenous.

        •    Language spoken at home was coded so that where a language other than
             English was the main language the code was zero and where English was the
             main language the code was one. Eight per cent of students spoke a language
             other than English at home.

        •    Parental education was coded with those whose parents had not reached
             university level being coded as zero and those whose parents had participated in
             university education being coded as one. Twenty-one per cent of the sample had
             parents who had attained university level.

The following predictor variables were included at the school/classroom level in the final
model7

    •       Participation in professional development on science assessment was recorded as
            the average proportion of science teachers in the school who had participated in
            such programs over the past two years. Teacher responses were initially coded as
            zero for non-participation and one for participation. The mean school level of
            participation was 0.51.

    •       The extent to which teachers reported that students were required to “write
            explanations about what was observed and why it happened” when doing science
            investigations in half their lessons or more was coded as zero for not reporting

6
    Location coded as metropolitan or non-metropolitan was included in the initial analysis but dropped
    from the final model because there was no significant effect.
7
    A large number of variable relating to teacher qualifications and other aspects of teaching were
    included in the initial analyses but were dropped from the final model.
Science learning in Australia               7                          Ainley & Thomson


       that and one for reporting it. The between school mean proportion of teachers
       recording this emphasis was 0.62 with a standard deviation of 0.39.

   •   The percentage time spent teaching physics was recorded as the within-school
       average for responding science teachers. The between-school mean was 21.8%
       with a standard deviation of 7.4%.

   •   Homework emphasis was represented as a variable indicating the proportion of
       teachers at the school who recorded a high or medium emphasis on homework
       (assigning it in more than half the lessons) in science. The mean proportion was
       0.93.

The following predictor variables were included at the state level in the final model

   •   The average time in minutes allocated to science each week for the state based on
       data provided on the teacher questionnaire. The national mean was 198 minutes
       but the values ranged from 169 to 22 minutes per week.

   •   The average age in months for students in Grade 8 in each state.

RESULTS

Differences among states

Table 3 records the mean achievement scores in science for all states of Australia. At
Grade 4 level, there are essentially no differences in TIMSS science achievement among
the states. The difference in the means for Victoria and New South Wales, the two most
populous states, and those which are most comparable in terms of student age and
demographic characteristics was only two scale points. The only difference that was
statistically significant at Grade 4 was between the Australian Capital Territory and
Western Australia. At Grade 8, the significant differences in TIMSS science achievement
were between New South Wales and Victoria (31 scale points), New South Wales and the
Northern Territory (65 scale points) and between the Australian Capital Territory and the
Northern Territory (56 scale points).
Science learning in Australia                             8                                      Ainley & Thomson


On the national science assessment at Grade 6 students from the Australian Capital
Territory achieved a significantly higher mean score than those from all the other States
and Territories except New South Wales and Tasmania. Students from New South Wales
achieved a significantly higher mean score than those from all the other States and
Territories except the Australian Capital Territory, Tasmania and Victoria. There was no
significant difference in the performance of students from Victoria, Western Australia,
South Australia, Queensland and the Northern Territory.

Table 3         TIMSS 2002 Science Scores and National Science Assessment Scores
                for Australian States

                                             TIMSS 02                    TIMSS 02                  National Test 03
                                              Grade 4                     Grade 8                      Grade 6

                                         Mean          SE             Mean          SE             Mean       SE

 New South Wales                          526         10.1             547          9.6             411       4.1
 Victoria                                 528          6.8             516          5.3             399       4.2
 Queensland                               513          7.7             516          6.0             392       3.8
 South Australia                          515          8.5             524        10.9              393       4.1
 Western Australia                        502          7.3             520          6.9             390       4.8
 Tasmania                                 517         11.6             504        11.7              407       6.1
 Northern Territory                       503         13.8             482        13.7              379       9.9
 Australian Capital Territory             547          9.7             538          9.2             430       6.2

 Australia                                521          4.2             527          3.8             400       1.9

Note

       The international mean for Grade 4 was 489 with a standard deviation of approximately 100
       The international mean for Grade 8 was 474 with a standard deviation of approximately 100
       The national test was calibrated to have a mean of 400 and a standard deviation of 100.


Changes between 1994 and 2002

Between 1994 and 2002 there was a small increase in the average scale score in TIMSS
science for Grade 8 students in Australia; from 514 to 527. This increase was significant
at the 5% level. For Grade 4 students in Australia the average science scores did not
change at all (being 521 on both occasions).
Science learning in Australia                       9                                 Ainley & Thomson


As shown in Table 4 there were differences between states in the extent of the change
from 1994 to 2002. In New South Wales there was an improvement in the average Grade
8 score of 30 scale points and in Victoria there was an improvement of 19 scale points8.
Although neither gain was statistically significant at the 5% level both were significant at
the 10% level.

Table 4         TIMSS Science Scores in 1994 and 2002 for Australian States

    Grade 4
                                             TIMSS 1994                           TIMSS 2002
                                          Mean      SE (Mean)                  Mean      SE (Mean)

    New South Wales                        522               6.1                526              10.1
    Victoria                               529              10.7                528               6.8
    Queensland                             503               7.6                513               7.7
    South Australia                        519               7.1                514               8.5
    Western Australia                      527               6.1                502               7.3
    Tasmania                               523               8.7                517              11.6
    Northern Territory                     512              11.2                503              13.8
    Australian Capital Territory           557               6.0                547               9.6
    Australia                              514               3.9                527               3.1

    Grade 8

    New South Wales                        517               8.2                547               9.6
    Victoria                               497               6.2                516               5.3
    Queensland                             510               8.4                516               6.0
    South Australia                        510               5.9                524              10.9
    Western Australia                      531               6.6                520               6.9
    Tasmania                               496              10.7                504              11.7
    Northern Territory                     466              16.8                482              13.7
    Australian Capital Territory           529              12.7                537               9.2
    Australia                              521               3.8                521               4.2




There were declines in student science achievement in Western Australia of 25 points at
Grade 4 (just failing to reach the 5% level) and ten points at Grade 8 (but not statistically
significant).

8
       New South Wales and Victoria are the two most populous states and together enrol just fewer than
       60% of the student population in Australia.
Science learning in Australia                 10                              Ainley & Thomson


Multi-level regression analysis of Grade 8 Science Scores in 2002

The results of a three level regression analysis for the Australian Grade 8 TIMSS Science
data are recorded in Table 59. In that analysis the dependent variable was the TIMSS
Science scale score. The results show all the predictor variables that were statistically
significant at the 10% level. A number of potential predictors that were examined and
found to be not statistically significant at the 10% level have not been shown.

State level predictors

Although state-level variables contributed little to the percentage of variance in student
scores (there were only eight units at this level) there were some moderately strong
effects on state means.        Two state level factors were related to student science
achievement.

        •   The average time allocated to science for the state was related to science
            achievement. Each additional 10 minutes of time (the national average was
            195 minutes per week) was associated with an increment of just under six
            scale points, other factors being equal.

        •   Average age for the state was related to science achievement with each
            additional six months being associated with approximately 14 scale points,
            other things being equal.

School/class level predictors

School or classroom level influences accounted for ten per cent of the variance in student
science scores. Four of the school/class level predictors were significantly related to
science achievement.

    •   The percentage of science classroom time allocated to physics was related to
        science achievement (the average was 21% with a standard deviation of 7%). For




9
    Using Hierarchical Linear Modelling Version 6 (Raudenbush, Bryk, Cheong, Congdon & du Toit,
    2004).
Science learning in Australia                   11                               Ainley & Thomson


          each additional 20 percent of time allocated to physics within science there was a
          net gain of 26 scale points.

    •     Where science homework had a moderate or high emphasis from most teachers
          there was a gain in science achievement. For each additional 20 percent of
          teachers who had a moderate or high emphasis on home work the net gain was 13
          scale points. The difference between schools where all teachers had this emphasis
          on homework and those where none did was 67 scale points.

Table 5       Results of a Three-Level Regression Analysis of Grade 8 TIMSS Science
              Achievement in Australia

                                                                 Coefficient   Std Error   p-value

Student Level
Gender (male = 0, female =1)                                       -13.35        1.87       0.00
Age (months)                                                       -1.04         0.19       0.00
Indigenous (non-Indigenous = 0, Indigenous =1)                     -39.26        5.07       0.00
Language at home (LOTE = 0, English = 1)                           23.97         3.49       0.00
Parental education (non-university = 0, university =1)             23.16         2.22       0.00
School/Class Level
Teacher participation in PD on science assessment                  -14.14        8.23       0.09
Explanations about observations in science investigations          27.29         9.75       0.01
Percentage of science time on physical science                      1.29         0.44       0.00
Teacher emphasis on science homework                               67.07        17.12       0.00
State level
Average time allocated to science (minutes)                         0.56         0.16       0.02
Average age in Grade 8 (months)                                     2.44         0.90       0.04
Initial variance
    Level 1                                                         37%
    Level 2                                                         62%
    Level 3                                                         1%
Variance explained by the model                                     15%
    % of level 1 variance                                                        6%
    % of level 2 variance                                                        27%
    % of level 3 variance                                                       100%
N = 4737 level 1 units, 205 level 2 units and 8 level 3 units.
Science learning in Australia               12                          Ainley & Thomson


     •   Students from schools where a higher proportion of teachers indicated half or
         more science lessons involved formulating hypotheses performed better than their
         peers from other schools. The net gain was six scale points for every additional
         20 per cent of teachers indicating this emphasis. The difference between schools
         where all teachers indicated this and those where none did was 27 scale points.

     •   Students from schools where more teachers participated in professional
         development focussed on assessment. For each additional 20 per cent of teachers
         who participated in this form of professional development the decrement was
         three scale points. The difference between schools where all teachers did this and
         those where none did was 14 scale points.

Student level predictors

Student background factors contributed to more than one third of the variance in student
science scores. Five student characteristics were related to science achievement at Grade
8.

     •   Male students performed better than female students (51% of students) on the
         TIMSS Science assessment in Grade 8 with the net difference being 13 scale
         points.

     •   Non-Indigenous students performed better than Indigenous students (3% of
         students) with the net difference being 39 scale points.

     •   Students for whom English was the main language spoken at home (93% of
         students) performed better than students for whom a language other than English
         was the main language at home. The net difference was 24 scale points.

     •   Students whose parents had experienced a university education (18% of students)
         performed better than those whose parents had not proceeded to university with
         the net difference being 23 scale points.
Science learning in Australia              13                           Ainley & Thomson


   •   Students who were younger in Grade 8 performed better than older students with
       each additional month being associated with one scale point difference in
       achievement.

Differences among states in influences on science achievement

The multi-level analysis conducted for the Australian sample indicates that gender,
parental education, Indigenous status, language at home and age were related to
achievement on the TIMSS Science scale for Grade 8. There are differences among
States and Territories in student background as reflected in statistical indicators of
population characteristics reported by the Australian Bureau of Statistics. The extent to
which student background influences the pattern of differences in achievement among
States and Territories depends on:

   •   The magnitude of the relationship between a characteristic and student science
       achievement;

   •   The extent to which there are differences among States in the distribution of that
       characteristic (e.g. there are differences among States in the distribution of
       parental occupation but not of gender).

   •   The difference in effect between states and territories of these background
       variables on student achievement in civics and citizenship.

Regression analysis was used to examine the effect on the means after controlling for
specified student background characteristics. The student background characteristics
included in the analysis were: age, parental education, Indigenous status, gender, and
home language. All of the background characteristics are measured at the student level
but the analysis was conducted using two-level HLM to make allowance for the clustered
sample design in estimating standard errors. The regression analyses were conducted
separately for each State so that the adjustment takes account of the effects of the variable
in each State. The intercepts from the regression provide an indication of the adjusted
means for each state. Results are recorded in Table 6. In interpreting the results the
focus is on the magnitude of the effects because the significance level is dependent on the
Science learning in Australia                    14                               Ainley & Thomson


numbers in the sample and more especially the numbers of students with a given
characteristic in the state.

Table 6       Results of Two-Level Regression Analyses of Grade 8 TIMSS Science
              Achievement in each Australian State

                   Intercept     Gender   Indigenous      Parent         Home       Age      %
                                                         education     language           variance
                                                                                          explained
                                                                                          by model

New South          516.8         -13.6       -13.7          17.6           25.6    -0.5      6%
Wales
Victoria           494.4          -8.4       -15.9          31.7           21.4    -1.4      5%
Queensland         481.3         -19.3       -20.7          21.5           43.0    -1.7     10%
South              510.2          -0.2       -19.4          28.2            5.5    -1.2     11%
Australia
Western            510.4         -20.4       -89.9          17.5           24.6    -0.5      8%
Australia
Tasmania           484.3          0.3        -44.6          45.2           11.7     0.5     11%
Northern           439.0         -14.2       -34.8          46.6           52.2    -1.5     11%
Territory
ACT                486.4         -13.4       -49.1          21.0           52.8    -1.5      9%
Note: Coefficients in bold are statistically significant at the 5% level


The results in Table 6 indicate that:

    •      The influence of gender (males performing better than females) is evident in five
           of the eight states. It is non-existent in South Australia and Tasmania, not
           significant in the Australian Capital Territory and strongest in Western Australia
           and Queensland.

    •      The influence of Indigenous status is largest in Western Australia and moderately
           large in Tasmania and the Northern Territory. The influence is relatively weaker
           in South Australia. It can be noted that the numbers of Indigenous students are
           very small in Victoria and the Australian Capital Territory and therefore the effect
           is not significant.      In Queensland the effect is not significant because the
           dispersion is wide.
Science learning in Australia              15                           Ainley & Thomson


   •   Parental education is related to science achievement in all states but is stronger in
       Tasmania and the Northern Territory and less strong in New South Wales,
       Western Australia and Queensland. This can be taken as an indication of equality
       in the social distribution of science achievement.

   •   Being from a home where a language other than English is the main language
       spoken has no effect in South Australia and Tasmania (where there are fewer
       students in this category) but a stronger effect in Queensland and the two
       territories. The effect is similar in Victoria and New South Wales where there are
       the highest proportions of students from a non-English speaking background.

   •   Age within Grade has effects in Victoria, Queensland and the Australian Capital
       Territory (with older students performing a little less well) but not elsewhere.

The intercepts from these analyses can be taken as related to the adjusted means for each
state, based on the effects of each factor on achievement for that state. It can be seen
from Table 6, compared with Table 2, that the adjustment process does not alter the
relative order of the states greatly (the correlation coefficient between adjusted and
unadjusted scores is 0.81).

DISCUSSION

The discussion of the results from the analyses conducted for this paper focus on the
junior secondary years for that is where most of the variation at state level is evident.
These analyses of Australian Grade 8 TIMSS science data provide perspectives on three
sets of issues. The first of these sets of issues concerns the factors that influence science
achievement in the junior secondary years in Australia. The second set of issues focus on
the extent to which there are differences among states in the influence of various factors
on science achievement. The third set of issues concerns the extent to which there are
differences between states, as the formal locus of authority for decisions about policy
organisation and curriculum, in science achievement in secondary school.

In general secondary school science achievement appears to be influenced by factors at
state, school and classroom and student level. From the perspective of state influences
Science learning in Australia                      16                                Ainley & Thomson


science achievement is influenced by the average age of students and the average amount
of time allocated to the teaching of science. Even though these variables account for a
minute percentage of the variance in student scores the effect sizes are not trivial. In
systems where the average age of students is greater then achievement is higher. The gap
between states with the youngest and the oldest Grade 8 students respectively is nine
months which would correspond to an effect of 22 scale points. The average time
allocated to science is also related to science achievement: the range in allocated time is
more than 50 minutes per week which would correspond to approximately 31 scale
points. The average age of students in Grade 8 reflects the age at which students
commence school and is only alterable by a major change to the structure of schooling.
Allocated time is more readily susceptible to policy changes and increasing the time
allocated to science on a state-wide basis could result in improved science learning. It is
interesting that time was not a significant influence at school level possibly because
variations within states in allocated time are much less (even where time is not prescribed
there are state patterns of allocating time between learning areas that have been evident
for several decades). It could be hypothesised that through allocating more time at state
level there is greater breadth and depth of what students are expected to learn and this is
reflected in what they do learn. Variations among schools within states are less likely to
reflect variations in the breadth and depth of the curriculum coverage.

School or classroom influences included in this analysis accounted for approximately ten
per cent of the variance in student science scores. There are three pedagogical factors
that relate to science achievement: the percentage of time allocated to physics within
science teaching10, the extent to which students are required to provide written
explanations about observations made during science investigation and the emphasis on
homework. Taken together these can be seen as manifestations of an emphasis on
science learning that requires a relatively high level of cognitive engagement with science
content. The result that participation in assessment-related professional development has
a negative influence possibly reflects a similar orientation: that professional development



10
     It can be noted that compared to other countries Australian Grade 8 students performed relatively
     better in life, earth and environmental sciences and relatively worse in chemistry and physics (Martin
     et al, 2004).
Science learning in Australia                   17                               Ainley & Thomson


focus is less orientated to the deeper understanding of science than other forms of
professional development.

At a national level the student level influences on science achievement follow a pattern
similar to that found in many Australian studies of science. Male students achieved
higher scores than their female counterparts and by a larger margin than in most other
countries. Students whose home language was not English performed less well than
those whose home language was English (the net effect was 24 points) and Indigenous
students performed less well than non-Indigenous students (the net effect was 39 points).
Students whose parents had completed a post-secondary qualification performed better
than other students with the net effect being 23 scale points.

There is a tendency to regard the influence of student background characteristics on
achievement as almost immutable. However, one of the benefits of international studies
is to demonstrate that these effects are not immutable and that the strength of the
influence varies between countries that are comparable in other ways.                     The results
presented in this paper indicate that there are variations among the Australian states in the
influence of various background characteristics on science achievement. The difference
in achievement between males and females is non-existent in South Australia and
Tasmania but substantial in Western Australia and Tasmania. The effect of parental
education is less in New South Wales and Western Australia than elsewhere. Home
language has nearly twice the effect in Queensland as it does in Victoria. Importantly the
achievement gap between Indigenous and non-Indigenous students is much greater in
Western Australia than in New South Wales. Differences such as these invite further
inquiry into the associated differences in demographics, social policy and educational
practice that are associated with these disparities.

It was observed near the start of this paper that there were significant differences in
TIMSS science achievement at Grade 8 between New South Wales and Victoria11. The
relatively strong performance of students from New South Wales is also reflected, but
less strongly, in results from the national sample survey of science literacy at Grade 6. At
Grade 4 there were almost no differences between Victoria and New South Wales. In
11
     As well as between New South Wales and the Northern Territory and between the Australian Capital
     Territory and the Northern Territory
Science learning in Australia             18                           Ainley & Thomson


addition it was observed that the relatively high achievement of students from New South
Wales had emerged over the period from 1994 to 2002. Although both Victoria and New
South Wales had improved over that time New South Wales had improved to a greater
extent. The differences between these two states do not appear to arise from differences
in the age of students (the average age is similar and the effect of age on achievement is
similar). Nor do they arise from gender (although the gender gap is smaller in Victoria
than New South Wales), or language background (the composition of the population in
the two states and the effects of language are almost identical). There is a difference in
the effect of parental education with New South Wales exhibiting less of an effect of
parental education (which may reflect the smaller percentage of students in non-
government schools). However, this difference has very little influence on the state
average score.

The difference between New South Wales and Victoria in the average time allocated to
science would account for a substantial (possibly 23 of the 31 point difference between
the states) part of the difference between the states. The school and classroom level
influences identified in these analyses do not differ substantially between the two states.
The remaining differences possibly reside in factors not captured in these data such as the
extent to which there is a strong curriculum framework that shapes the teaching in
schools.
Science learning in Australia            19                          Ainley & Thomson


REFERENCES

Australian Bureau of Statistics (ABS) (2004). Information Paper: Census of Population
       and Housing -- Socio-Economic Indexes for Areas, Australia (Catalogue 2039.0).
       Canberra: Australian Bureau of Statistics.
Australian Bureau of Statistics (ABS). (2003). Schools Australia 2002 (Catalogue
       4221.0). Canberra: Australian Bureau of Statistics.
Curriculum Corporation (2006). National Consistency in Curriculum Outcomes: Draft
       Statements of Learning and Professional Elaborations for Science. Melbourne:
       Author.
Goodrum, D., Hackling, M., & Rennie, L. (2001). The status and quality of teaching and
       learning of science in Australian schools. Canberra: Department of Education,
       Training and Youth Affairs.
Lokan, J., Hollingsworth, H. & Hackling, M (2006). Teaching science in Australia
(TIMSS Australia Monograph No. 8). Melbourne: ACER.
Lokan, J., Ford, P. & Greenwood, L. (1996). Maths & Science on the Line: Australian
       Junior Secondary Students' Performance in the Third International Mathematics
       and Science Study (TIMSS Australia Monograph No. 1). Melbourne: ACER.
Lokan, J., Ford, P. & Greenwood, L. (1997) Maths & Science on the Line: Australian
       Middle Primary Students' Performance in the Third International Mathematics
       and Science Study (TIMSS Australia Monograph No. 2). Melbourne: ACER.
Martin, M. O., Mullis, I. V. S., Gonzalez, E. J. & Chrostowski, S. J. (2004). TIMSS 2003
       international science report: Findings from IEA's Trends in International
       Mathematics and Science Study at the fourth and eighth grade. Boston College:
       Chestnut Hill, MA.
Ministerial Council for Education, Employment, Training and Youth Affairs
       (MCEETYA) (2005). A Measurement Framework for National Key performance
       Measures. Melbourne: Author.
Raudenbush, S., Bryk, A., Cheong, Y. F., Congdon, R., & du Toit, M. (2005), HLM6:
       Hierarchical Linear and Nonlinear Modelling, Lincolnwood, IL: Scientific
       Software International.
Science learning in Australia        20                        Ainley & Thomson


Thomson, S. & Fleming, N. (2004a). Summing it up: Mathematics achievement in
       Australian schools in TIMSS 2002 (TIMSS Australia Monograph no. 6).
       Melbourne: ACER.
Thomson, S. & Fleming, N. (2004b). Examining the evidence: Science achievement in
       Australian schools in TIMSS 2002 (TIMSS Australia Monograph no. 7).
       Melbourne: ACER.