Emergence and accuracy by wuyunyi


									  Methodological issues in
testing Processability theory
   on languages with rich
  inflectional morphology

         Gabriele Pallotti
         Sassari University
I. Identifying test structures

   a) All structures or some?
b) Choosing the right structures
Processing      L2 process               Italian morphosyntax
5. S'-          inter-clausal information subordinate clauses with subjunctive verb marking

4. S-procedure inter-phrasal information (Topic) Clitic clusters
                                         Topic-object agreement
                                         NPTopic(pl)-object(pl)+Auxiliary(Ave)+Vto(pl) (NPsubj)
                                         NPTopic(pl)-object(pl)+V+ (NPsubj)

                                         Subject(pl)-AuxEsse(pl)+Predicative adjective(pl)

                                         Subject-Verb agreement
3. Phrasal      phrasal information      VP (and V') constructions
procedure       VP                       AuxEsse(pl)+V-to(pl)
                V'                       Copula (pl)+Adjective(pl)

                                         AuxAve+ V-to
                phrasal information      NP agreement (plural -i )

                NP                       modifier(-i )+noun(-i )+adjective(-i )
                                         noun(-i )+adjective(-i )
                                         modifier(-i )+noun(-i )
                                         numeral+noun(-i )
                                         verb form alternation
2. Category     lexical morphemes        Vbase V-to (past/non-past) ~V-re
                                         noun form alternation
                                         plural marking on nouns (-i )
1. word/ lemma ‘words’                   invariant forms
       All structures or some?

• Select some structures at different
  stages and see if they emerge in the
  predicted order (e.g. Di Biase-
  Kawaguchi 2002, Glahn et al. 2001)
• Inspect all/several structures for each
  stage to see if any of them emerges
  (e.g. Johnston 1985, Kersten 2004,
  Pallotti et al. 2004)
                      1,2,3 = procedures
time             3c
                 3b   a,b,c,d = structures
            2b                  3d
            2a               2d 3c
       1d                 1d 2c 3b
       1c                 1c 2b 3a
       1b                 1b 2a       3a2c1z
       1a                 1a
 II. Organizing data

    a) Factorization
b) Structure aggregation

Languages with fusional morphology:

  – one form  many functions
  – one function  many forms
  – many forms  many functions
          Factorization: an example
Standard Swedish adjectival suffixes varying for gender, number,
definitess and syntactic role
             uter, plural, indefinite, attributive
-a=          neuter, plural, indefinite, attributive
             uter, singular, definite, attributive, etc.

-Ø=          uter, singular, indefinite, predicative + attributive
             uter, singular, definite, predicative
             neuter, singular, indefinite, predicative + attributive
-t=          neuter, singular, definite, predicative
          Factorization: an example
A learner‟s Swedish interlanguage:
- a = plural
- Ø = singular
    Problems with factorization

• The learner may follow IL rules in some
  cases and TL rules in others, i.e. „rule
  application‟ may not be categorical in
  either case.
 how are applications of the two „rules‟
  to be scored?
 Factorizing gender and number in
              L2 Italian
Standard Italian
1st class (masc) plural    -i
2nd class (fem) plural     -e

Interlanguage options for expressing plural
unique inflectional class with one    -i

unique inflectional class with two   -i / -e
allomorphic exponents
 What emerges? One form-one
     function association
Present tense in Italian: first person plural

               V-iamo         other forms


      What emerges? Set of forms – set
          of functions association
Present tense in Italian: the whole paradigm
        V-o   V-i   V-a   V-e   V-amo   V-te   V-no   other   ?







  What emerges? A procedure
 manifested by several structures
For example, in English and Italian…
Category procedure: plural marking on
 nouns (cat-s); tense/aspect marking on
 verbs (go-ing, arrive-d)
For example, in Italian only…
S-procedure: subj-V agreement; subj-
 predicative adj agreement; Top-clit
Data aggregation at the procedure level

1) Evidence for S-procedure:
  – Weak evidence from subj-V agreement
  – Weak evidence from subj-predic. Adj agreement
  – Weak evidence from Top-clit agreement
2) Evidence for S-procedure:
  – Weak evidence from subj-V agreement
  – No evidence from subj-predic. Adj agreement
  – No evidence from Top-clit agreement
III. Interpreting data
     Some acquisition criteria

A structure is acquired when it is
• 60% (Vainikka & Young Sholten 1994)
• 75% (Ellis 1989)
• 80% (Andersen 1978)
• 90% (Dulay & Burt 1974; Bahns 1983)
… of the times it is required
       Acquisition orders based on
       emergence and other criteria

                     c       a        b

                 c       b       a

         a   b       c
      Emergence as first use

• Hammarberg (1996)
• Bayley (1999)
• Glahn et al. (2001)
Emergence as first systematic use

“[Emergence] is the first systematic use of
  a structure, so that the point in time can
  be located when a learner has – in
  principle – grasped the learning task”
(Pienemann 1984:191)
Emergence = first
systematic use

                               No longer „first‟

                    First systematic use
    Not yet „systematic‟
   Operationalizing emergence:
         possible errors
• Premature attribution. Type 1 error:
  see emergence when it isn‟t there

• Delayed attribution. Type 2 error: deny
  emergence when it is actually there
      Type 1 errors: mistaking
         emergence for…
… formulas
… random hits

What are the minimal requirements for
  excluding type 1 errors? I.e., what
  evidence is needed to demonstrate that
  use is
1) productive 2) not random?
   Evidence for productive use

• Morphological minimal pairs:
  dog/dogs, book/books, year/years
• Creative constructions: two childs,
  two wakers [= two alarm-clocks]
• Lexical and morphological variety: a
  dog/three books, a cat/many girls, a
  teacher/two chairs, a window/many
  trees, a radio/three pictures
      Evidence for non-random use
How can you tell incipient application from

5 hits out of 20 trials (25%)?

10 hits out of 20 trials (50%)?

15 hits out of 20 trials (75%)?

‘Hit’ may be relative to TL or IL
            From random hits to
          systematicity: an example

Interview 3        5       7      9      11     13      15     17      19

ich V-e    0.78    0.80    0.63   0.81   0.63   0.82    0.67   0.56    0.74
x V-e      0.67    0.59    0.51   0.73   0.80   0.79    0.69   0.73    0.47

ich V-Ø    /       /       /      /      /      (0.5)   /      (0.5)   1.
x V-Ø      /       /       /      /      /      (0.5)   /      (0.5)   0.

From Pienemann 1998: 126

 “A distribution of 60 to 0 would have been just as much grounds for
 rejecting the null hypothesis” (Pienemann 1998: 144-5)
     Random use: the null hypothesis


-i                            -no        -o
    Is the learner doing any better?

Some criteria for non-random use
• Suppliance in Obligatory Contexts (SOC):
  number of applications / number of contexts.
• Target-Like Use (TLU): number of
  applications + number of over-uses /number
  of contexts.
• Statistics for calculating probability of
  distributions (Chi square, Retrocumulated
  binomial distribution, others…)
              SOC and emergence

The structure „plural –s‟ is provided 80 times out of 100:
has it emerged?
Maybe yes, but maybe it is just produced all over the

         - s ~ -s                  - s ~ -s
         80    20         pl.      80   20
         0     n          ~ pl.    80   n
 ~ pl.

   Emergence                    Random hits
             SOC and emergence

The structure „plural –s‟ is provided 10 times out
of 100: has it emerged?
Maybe yes, although its use is not very consistent

        - s ~ -s                 - s ~ -s
        10    90        pl.      10   90
        0     n         ~ pl.    10   n
~ pl.

  Emergence                   Random hits
         Consistency vs. selectivity

            - s ~ -s
                                            This line indicates consistency,
   pl.      10      90                    consolidation, spread of a structure
            10      n                        (10, 20, 50, 100 uses in 100
   ~ pl.                                          relevant contexts)

This column indicates selectivity,
specificity, goal-orientedness (10, 15,
20 appropriate, targeted uses of –s out
of 20 applications)
             - s ~ -s   SOC: 88%

             15    2    TLU: 47%
                        χ2 two ways: p < .001
 ~ pl.       15    25
                        χ2 one way: p < .90
                        Retroc. binomial: p < .57
 Random hits

         - s ~ -s       SOC: 50%

         15       15    TLU: 47%
                        χ2 two ways: p < .001
~ pl.    2        25
                        χ2 one way: p < .005
                        Retroc. binomial: p < .001
Incipient application
                    Binomial formula

           P (x;n;p) = n!/[x!(n-x)!] * px * (1-p)n-x

x = number of successes
n = number of trials
p = probability of success
Binomial formula If the probability of hitting the target is 0.5, and you try
17 times, what is the probability of obtaining exactly 10 hits? 14,84 %
Retrocumulated binomial formula. If the probability of hitting the target
is 0.5, and you try 17 times, what is the probability of obtaining 10 or
more hits? 31.45 %
        Some open questions

• Can an emergence criterion formulated for
  one structure of one language be generalized
  to other structures and/or other languages?
• Can emergence criteria be validated?
• Can the +/- dichotomy be refined?
                    Data           Variability     Random hits

English pl. -s      5 or more      5 types or 3   <25% over-
(Kersten)           contexts       types + 1 min. extensions

Italian pl. -i/-e   3 or more      15 types or 3   <20%
(Pallotti et al.)   types per cell min.            probability of
                                   pairs/creat.    random
                                   constructions   application
Validating the emergence criterion

And so on…

Different ways of eliciting data
Different statistical formulas for excluding
random application,
Different criterion levels
               rule    c
                               Real acquisition order: a – b - c
       rule    b

rule   a

                rule       c
                                 Wrong acquisition order (c – b – a)
        rule       b                due to too loose criteria for
 rule      a
                                 emergence of rule c and too strict
                                  criteria for emergence of rule a
                   Emergence or not?

Emergence as dichotomy              Emergence as continuum
+ = all requirements are met        + = all requirements are met
that satisfy the criterion
                                    (+) = most requirements are met
- = no / not all requirements are
                                    (-) = few requirements are met
                                    - = no requirements are met / clear
/ = no or insufficient evidence
                                    evidence against emergence
                                    / = no or insufficient evidence

                                    N.B.: robustness (# of
                                    contexts) is just one
         One learner, three conclusions

John, interview 1

S- proc.         +       S- proc.     +             S- proc.      +

Phrase           -       Phrase       +             Phrase        (+)
proc.                    proc.                      proc.
Cat. proc.       -       Cat. proc.   +             Cat. proc.    (+)

1) Strict criteria to   2) Loose criteria to      3) Same criteria as 1): one
exclude formulaic       exclude formulaic use +   can‟t exclude that all uses
use (+ a certain        a certain way of          of structures at level 1 and
interview format)       aggregating data          2 are formulaic, but this
                                                  can‟t be claimed for sure
  Methodological issues in
    testing Processability
  theory on languages with
rich inflectional morphology
         Gabriele Pallotti
        Sassari University

To top