Operation Research Taha Solution Manual by jwa52607

VIEWS: 1,912 PAGES: 76

More Info
									Recent advances in multi-document
         summarization

                      Dragomir Radev
             University of Michigan, Ann Arbor
                    radev@umich.edu

     Presentation at UC Berkeley SIMS, November 10, 2004
 WWW as a textual database
• Large: 1010 pages, 200 TB [Lyman&Varian 03] cf. brain
  (1011 neurons)
• Multilingual: English 56.4% of sites, German 7.7%,
  French 5.6%, Japanese 4.9%, Chinese 2.4%
• Evolving: 22% of sites change every day, another 31%
  change every month [Cho&Garcia-Molina 00]
• Uneven importance: at different levels
• Adequate representations are needed for user-friendly
  access
                         Outline
•   Introduction
•   Random walks and social networks
•   LexRank
•   Projects in language modeling and machine learning
                         Outline
•   Introduction
•   Random walks and social networks
•   LexRank
•   Projects in language modeling and machine learning
Natural Language Processing
           (NLP)
Typical NLP problems            •   NLP is very hard!
    Entity extraction                –   The pen is in the box.
    Relation extraction              –   Every American has a mother.
    Text classification              –   Boston called.
    Summarization                    –   I saw Zoe. The poor girl looked
    Information retrieval                tired.
    Machine translation              –   Mary and Sue bought each
    Question answering                   other a book.
    Text understanding               –   The spirit is willing but the flesh
                                         is weak.
    Parsing
                                     –   Children make delicious
    Word sense disambiguation            snacks.
    Lexical acquisition              –   Army head seeks arms.
    Paraphrasing                     –   Czech President and playwright
                                         Havel to receive honors
            Recent trends in NLP
•   Multidisciplinary
•   Statistical
•   Well founded
•   Scaleable

                           Linguistics      E-commerce
                         Lin. Algebra            Bioinformatics

                        Graph theory               Info. Retrieval
                                          NLP
                  Bioinformatics                  Intelligence
                        Stat. Mechanics         User interfaces
                          Sociology          Translation
                 Finding structure
•   Language doesn’t have a            •   Finding structure
    regular structure (like a               – Across sentences
    database)                               – Across sites/sources/documents
•   Sentences are very unlike each          – Over time
    other                              •   Representations
•   Linguistic analysis: parse trees        – Graphs everywhere!
•   Hard to generalize
                     NewsInEssence
•   MEAD: salience-based extractive
    summarization
•   Centroid-based summarization
    (single and multi document)
•   Vector space model
•   Additional features: position, length,
    lexrank

•   (1000+ downloads)
•   Cross-document structure theory
    (CST)
•   NIE: first robust news
    summarization system (2001)
                         Outline
•   Introduction
•   Random walks and social networks
•   LexRank
•   Projects in language modeling and machine learning
             Social networks
• Induced by a relation
• Symmetric or not
• Examples:
  –   Friendship networks
  –   Board membership
  –   Citations
  –   Power grid of the US
  –   WWW
Krebs 2004
Graph-based representations
            Graph G (V,E)               Square connectivity
                                        (incidence) matrix P
                                         1 2 3 4 5 6 7 8
    1             6         8
                                    1                         1 1
                                    2                         1
                                    3       1           1
2
                  7                 4           1
                                    5               1       1 1 1
                                5
                                    6                         1 1
                                    7
        3
                       4            8
              Markov chains
• A homogeneous Markov chain is defined by an initial
  distribution x and a Markov kernel P.
• Path = sequence (x0, x1, …, xn).
• The probability of a path can be computed as a
  product of probabilities for each step i.
                  Random walks
• Access time Hij = expected number of steps to go from i to j.
• Example [Lovász 1993]. What is Hij on a path with nodes 0, 1, n-
  1?
   H(k-1,k) = 2k-1
   H(i,k) = H(i,k-1) + 2k-1
   H(i,k) = (2i+1) + (2i+3) + … + (2k-1) = k2 – i2
   H(0,k) = k2
   (Brownian motion: travel distance sqrt(t) in time t)


• Electrical networks
   – Rst is the resistance between two nodes s and t. The round-trip
     travel time between s and t is exactly 2mRst, where m is the
     number of edges.
              Stationary solutions
•   The fundamental Ergodic Theorem for Markov chains [Grimmett and
    Stirzaker 1989] says that the Markov chain with kernel E has a
    stationary distribution p under three conditions:
     – E is stochastic
     – E is irreducible
     – E is aperiodic
•   To make these conditions true:
     – All rows of E add up to 1 (and no value is negative)
     – Make sure that E is strongly connected
     – Make sure that E is not bipartite
•   Example: PageRank [Brin and Page 1998]: use “teleportation”
               Example
     1               6                8
                                                          1
                                                         0.9
                                                         0.8
                                                         0.7
                                                                                       t=10




                                              PageRank
                                                         0.6

 2                                                       0.5
                                                         0.4
                      7                                  0.3
                                                         0.2
                                                         0.1
                                                          0
                                          5                    1   2   3   4   5   6    7   8




         3
                              4

This graph E has a second graph E’
superimposed on it:
E’ is the uniform transition graph.
                 Eigenvectors
• An eigenvector is an implicit “direction” for a matrix.
   Ev = λv, where v is non-zero, though λ can be any complex
     number in principle.
• The largest eigenvalue of a stochastic matrix E is λ1 =
  1.
• For λ1, the left (principal) eigenvector is p, the right
  eigenvector = 1
• In other words, ETp = p.
         Prestige and centrality
• Degree centrality: how many neighbors each node has.
• Closeness centrality: how close an actor is to all of the other
  nodes
• Betweenness centrality: based on the role that a node plays by
  virtue of being on the path between two other nodes
• Eigenvector centrality: the paths in the random walk are
  weighted by the centrality of the nodes that the path connects.
• Prestige = same as centrality but for directed graphs.
      Computing the stationary
           distribution
Solution for the          function PowerStatDist (E):
stationary distribution   begin
                            p(0) = u;
 pE p   T                  i=1;
                            repeat
 (I  ET ) p  0              p(i) = ETp(i-1)
                              L = ||p(i)-p(i-1)||1;
                              i = i + 1;
                            until L < 
                          end
    Example                             1
                                       0.9
                                       0.8
                                       0.7
                                                                      t=0




                            PageRank
                                       0.6
                                       0.5
                                       0.4
                                       0.3
                                       0.2
    1       6       8                  0.1
                                        0
                                             1   2   3   4   5   6    7   8



                                        1
                                       0.9
                                       0.8
                                       0.7
                                                                      t=1
2




                            PageRank
                                       0.6
                                       0.5
            7                          0.4
                                       0.3
                                       0.2
                                       0.1
                        5               0
                                             1   2   3   4   5   6    7   8



                                        1

        3                              0.9

                                                                     t=10
                4                      0.8
                                       0.7




                            PageRank
                                       0.6
                                       0.5
                                       0.4
                                       0.3
                                       0.2
                                       0.1
                                        0
                                             1   2   3   4   5   6    7   8
                   Outline
• Introduction
• Random walks and social networks
• LexRank
   Centrality in summarization
• Motivation: capture the most central words in a
  document or cluster
• Centroid score [Radev & al. 2000, 2004a]
• Alternative methods for computing centrality?
    Sample multidocument cluster
                                                              (DUC cluster d1003t)
1 (d1s1) Iraqi Vice President Taha Yassin Ramadan announced today, Sunday, that Iraq refuses to back down from its decision to stop cooperating with
disarmament inspectors before its demands are met.
2 (d2s1) Iraqi Vice president Taha Yassin Ramadan announced today, Thursday, that Iraq rejects cooperating with the United Nations except on the issue of
lifting the blockade imposed upon it since the year 1990.
3 (d2s2) Ramadan told reporters in Baghdad that "Iraq cannot deal positively with whoever represents the Security Council unless there was a clear stance on
the issue of lifting the blockade off of it.
4 (d2s3) Baghdad had decided late last October to completely cease cooperating with the inspectors of the United Nations Special Commission (UNSCOM), in
charge of disarming Iraq's weapons, and whose work became very limited since the fifth of August, and announced it will not resume its cooperation with the
Commission even if it were subjected to a military operation.
5 (d3s1) The Russian Foreign Minister, Igor Ivanov, warned today, Wednesday against using force against Iraq, which will destroy, according to him, seven
years of difficult diplomatic work and will complicate the regional situation in the area.
6 (d3s2) Ivanov contended that carrying out air strikes against Iraq, who refuses to cooperate with the United Nations inspectors, ``will end the tremendous work
achieved by the international group during the past seven years and will complicate the situation in the region.''
7 (d3s3) Nevertheless, Ivanov stressed that Baghdad must resume working with the Special Commission in charge of disarming the Iraqi weapons of mass
destruction (UNSCOM).
8 (d4s1) The Special Representative of the United Nations Secretary-General in Baghdad, Prakash Shah, announced today, Wednesday, after meeting with the
Iraqi Deputy Prime Minister Tariq Aziz, that Iraq refuses to back down from its decision to cut off cooperation with the disarmament inspectors.
9 (d5s1) British Prime Minister Tony Blair said today, Sunday, that the crisis between the international community and Iraq ``did not end'' and that Britain is still
``ready, prepared, and able to strike Iraq.''
10 (d5s2) In a gathering with the press held at the Prime Minister's office, Blair contended that the crisis with Iraq ``will not end until Iraq has absolutely and
unconditionally respected its commitments'' towards the United Nations.
11 (d5s3) A spokesman for Tony Blair had indicated that the British Prime Minister gave permission to British Air Force Tornado planes stationed in Kuwait to
join the aerial bombardment against Iraq.
    Cosine between sentences
•   Let s1 and s2 be two sentences.
•   Let x and y be their

                                                   x y
    representations in an n-
    dimensional vector space                            i    i
•   The cosine between is then
    computed based on the inner
                                      cos(x, y )  i 1,n
    product of the two.
                                                       x y


•   The cosine ranges from 0 to 1.
 LexRank (Cosine centrality)
            1          2          3          4          5          6          7          8          9      10      11

 1   1.00       0.45       0.02       0.17       0.03       0.22       0.03       0.28       0.06       0.06    0.00

 2   0.45       1.00       0.16       0.27       0.03       0.19       0.03       0.21       0.03       0.15    0.00

 3   0.02       0.16       1.00       0.03       0.00       0.01       0.03       0.04       0.00       0.01    0.00

 4   0.17       0.27       0.03       1.00       0.01       0.16       0.28       0.17       0.00       0.09    0.01

 5   0.03       0.03       0.00       0.01       1.00       0.29       0.05       0.15       0.20       0.04    0.18

 6   0.22       0.19       0.01       0.16       0.29       1.00       0.05       0.29       0.04       0.20    0.03

 7   0.03       0.03       0.03       0.28       0.05       0.05       1.00       0.06       0.00       0.00    0.01

 8   0.28       0.21       0.04       0.17       0.15       0.29       0.06       1.00       0.25       0.20    0.17

 9   0.06       0.03       0.00       0.00       0.20       0.04       0.00       0.25       1.00       0.26    0.38

10   0.06       0.15       0.01       0.09       0.04       0.20       0.00       0.20       0.26       1.00    0.12

11   0.00       0.00       0.00       0.01       0.18       0.03       0.01       0.17       0.38       0.12    1.00
       Cosine centrality (t=0.3)
d3s3           d2s3

                             d3s2

                                                         d3s1

        d1s1

                                           d4s1

                      d2s1                                      d5s1




                                    d5s2
                                                  d5s3
                      d2s2
       Cosine centrality (t=0.2)
d3s3           d2s3

                             d3s2

                                                         d3s1

        d1s1

                                           d4s1

                      d2s1                                      d5s1




                                    d5s2
                                                  d5s3
                      d2s2
       Cosine centrality (t=0.1)
d3s3           d2s3

                             d3s2

                                                         d3s1

        d1s1

                                           d4s1

                      d2s1                                      d5s1




                                    d5s2
                                                  d5s3
                      d2s2


          Sentences vote for the most central sentence!
                               LexRank
              1 d      p (T 1)         p(Tn)
      p( A)        d(          ...        )
               N        c(T 1)          c(Tn)
•   T1…Tn are pages that link to A, c(Ti) is the outdegree of pageTi, and N is the total
    number of pages.

•   d is the “damping factor”, or the probability that we “jump” to a far-away node
    during the random walk. It accounts for disconnected components or periodic
    graphs.

•   When d = 0, we have a strict uniform distribution.
    When d = 1, the method is not guaranteed to converge to a unique solution.

•   Typical value for d is between [0.1,0.2] (Brin and Page, 1998).
Cosine centrality vs. centroid
         centrality
 ID     LPR (0.1)   LPR (0.2)   LPR (0.3)   Centroid
 d1s1      0.6007      0.6944      1.0000     0.7209
 d2s1     0.8466      0.7317      1.0000     0.7249
 d2s2     0.3491      0.6773      1.0000     0.1356
 d2s3     0.7520      0.6550      1.0000     0.5694
 d3s1      0.5907      0.4344      1.0000     0.6331
 d3s2     0.7993      0.8718      1.0000     0.7972
 d3s3     0.3548      0.4993      1.0000     0.3328
 d4s1     1.0000      1.0000      1.0000     0.9414
 d5s1     0.5921      0.7399      1.0000     0.9580
 d5s2     0.6910      0.6967      1.0000     1.0000
 d5s3     0.5921      0.4501      1.0000     0.7902
              Evaluation metrics
• Difficult to evaluate summaries
    – Intrinsic vs. extrinsic evaluations
    – Extractive vs. non-extractive evaluations
    – Manual vs. automatic evaluations
• ROUGE = mixture of n-gram recall for different values of n.
• Example:
    – Reference = “The cat in the hat”
    – System = “The cat wears a top hat”
    – 1-gram recall = 3/5; 2-gram recall = 1/4;
      3,4-gram recall = 0
• ROUGE-W = longest common subsequence
• Example above: 3/5
                  Evaluation results
 Centroid: C0.5, C10, C1.5, C1, C2.5, C2
 Degree: D0.5T0.1, D0.5T0.2, D0.5T0.3, D1.5T0.1, D1.5T0.2,
   D1.5T0.3, D1T0.1, D1T0.2, D1T0.3
 LexRank: Lr0.5T0.1, Lr0.5T0.2, Lr0.5t0.3, Lr1.5t0.1, Lr1.5t0.2,
   Lr1.5t0.3, Lr1T0.1, Lr1T0.2, Lr1T0.3



Rouge-1                   Rouge-2                     Rouge-4
Lr1.5t0.1 0.400           Lr1.5t0.2 0.115             Lr1.5t0.1 0.124
Lr1.5t0.2 0.400           D1.5T0.2 0.114              Lr1.5t0.2 0.124
Lr1T0.2           0.396   D1T0.2              0.113   Lr1T0.2           0.124
…                         …                           …
C1                0.382   C1.5           0.099        C2                0.118
                               DUC results
Peer code   Task     ROUGE-1    ROUGE-2     ROUGE-3   ROUGE-4   ROUGE-L    ROUGE-W


141         3        5          2           1         1         2          2
142         3        5          1           1         1         4          3

143                4 1          2           1         1         6          6
144                4 3          1           1         1         7          7
145                4 1          2           2         2         4          4




                                          Recall                          LCS
     Results and applications
• DUC results (MU recall,   • applications:
  ROUGE):                      – Web page summarization
                                 (WIE)
   – 1st place 2003
                               – Topical crawling
     (duc.nist.gov)
                               – Answer focused
   – 1-2 place 2004            – wireless access
                               – Cross-lingual
                               – IR-based evaluation
                               – Knowledge based
                            • Beyond summarization:
                               – Classification
                               – WSD
                               – Spam recognition
1
2
3
    4
         5
    6
    7
8
    9
              10
         11
         12
         13
              14
                   15
                   16
                   17
              18
              19
    20
    21
              22
         23
              24
              25
         26
              27
         28
                        Outline
•   Introduction
•   Random walks and social networks
•   LexRank
•   Projects in language modeling and machine learning
    Syntax in Statistical Machine
             Translation
•   Noisy channel model: assume     •   Obvious problems can be fixed
    that a source sentence has to       with syntax (?)
    be translated into a target     •   JHU 02 and 03 projects
    language sentence               •   (Franz Och, Jan Hajic, Dan
•   Goal: find                          Gildea + others)


    e  argmax{P(e | f )}
    ˆ
•   Solution using log-linear
    combination of features

    e  argmax{ mhm (e, f )}
    ˆ
                             Setup
•   Given: a Chinese sentence+       •   Example:
•   The top 1000 candidate                – Is the number of constituents
    translations in English                 across languages the same?
•   Parse all of these                    – Is the english tree grammatical?
                                          – Are the two sentences of
•   Compute features: monolingual,          comparable length?
    bilingual, syntax-free, and
                                     •   Feature combination
    syntactic
                                          – Use a greedy maxbleu
•   Evaluation using BLEU                   algorithm
    (BiLingual Evaluation
    Understudy)
             Chinese parse tree
                                                  IP




                   NP



             QP



NP                CLP             NP                        NP               VP



NR      CD        M     NN        NN    NN         NN       NN      NN       VV




中国 十四 个 边境 开放 城市 经济 建设 成就 显著
China        14         border   open   cities   economic    achievements   marked
                              Multiple references
1. fourteen chinese open border cities make significant achievements in economic construction
2. xinhua news agency report of february 12 from beijing - the fourteen chinese border cities that have been opened to foreigners achieved satisfactory results
in their economic construction in 1995 .
3. according to statistics , the cities achieved a combined gross domestic product of rmb 19 billion last year , an increase of more than 90 % over 1991 before their
opening .
4. the state council successively approved the opening of fourteen border cities to foreigners in 1992 , including heihe , pingxiang , hunchun , yining and ruili ,
and permitted them to set up 14 border economic cooperation zones .

1. significant accomplishment achieved in the economic construction of the fourteen open border cities in china
2. xinhua news agency , beijing , feb. 12 - exciting accomplishment has been achieved in 1995 in the economic construction of china 's fourteen border cities open
to foreigners .
3. statistics have indicated that these cities produced a combined gdp of over 19 billion yuan last year , an increase of more than 90 % , compared with that in
1991 before the cities were open to foreigners .
4. in 1992 , the state council successively opened fourteen border cities to foreigners . these included heihe , pingxiang , huichun , yining , and ruili . meanwhile ,
the state council also gave its approval to these cities to establish fourteen border zones for economic cooperation .

1. in china , fourteen cities along the border opened to foreigners achieved remarkable economic development
2. xinhua news agency , beijing , february 12 - the economic development in china 's fourteen cities along the border opened to foreigners achieved gratifying
results in 1995 .
3. according to statistics , these cities completed a gross domestic product in excess of rmb 19 billion in last year , an increase of more than 90 % over 1991 ( the
year before they were opened ) .
4. in 1992 , the state council successively approved fourteen cities along the border to be opened to foreigners , which included hei he , pingxiang , hunchun ,
yining and ruili etc. at the same time , these cities were also given approvals to set up fourteen border @-@ economic @-@ cooperation zones .

1. economic construction achievement is prominent in china 's fourteen border opening up cities .
2. xinhua news agency , beijing , february 12 - delightful economic construction result was achieved in china 's fourteen border opening up cities in 1995 .
3. according to statistics , gdp registered over 19 billion yuan last year in those cities , over 90 % higher than those of year 1991 before opening up .
4. fourteen border cities like heihe , pingxiang , huichun , yinin , and ruili etc were approved successively by the state council in 1992 as the cities opening to the
outside world , setting up of fourteen border economic cooperation zones in these cities were also approved simultaneously .

1. china 's 14 open border cities marked economic achievements
2. xinhua news agency , beijing , february 12 chinese 14 border an open city 1995 economic development to achieve good results
3. according to statistics , the city last year 's gross domestic product ( gdp ) over 19 billion yuan , and opening up of more than 90 % growth in 1991 .
4. the state council in 1992 has approved the heihe , pingxiang , huichun , yining and ruili , 14 border cities as an open city , and the city also approved a total of
14 border economic cooperation .
                      Syntactic features
                                                           (S1 (S (PP (IN in)
                                                                      (NP (NNP china)))
                    (S1 (NP (NP (JJ significant)                  (, ,)
                                 (NN accomplishment))             (NP (NP (CD fourteen)
                            (VP (VBN achieved)                            (NNS cities))
                                 (PP (IN in)                          (PP (IN along)
(S1 (S (NP (CD fourteen)
                                     (NP (NP (DT the)                     (NP (DT the)
           (ADJP (JJ chinese)
                                             (JJ economic)                     (NN border))))
                 (JJ open))
                                             (NN construction)) (VP (VBN opened)
           (NN border)
                                         (PP (IN of)                  (PP (TO to)
           (NNS cities))
                                             (NP (NP (DT the)             (NP (NP (NNS foreigners))
       (VP (VBP make)
                                                      (CD fourteen)            (VP (VBN achieved)
           (NP (JJ significant)
                                                      (JJ open)                    (NP (JJ remarkable)
               (NNS achievements))
                                                      (NN border)                      (JJ economic)
           (PP (IN in)
                                                      (NNS cities))                    (NN development))))))))
               (NP (JJ economic)
                                                 (PP (IN in)
                   (NN construction))))))
                                                      (NP (NNP china))))))))))



           (S1 (S (NP (JJ economic)
                      (NN construction)
                                                                    (S1 (S (NP (NP (NNP china)
                      (NN achievement))
                                                                                   (POS 's))
                  (VP (AUX is)
                                                                               (CD 14)
                      (ADJP (JJ prominent)
                                                                               (ADJP (JJ open))
                            (PP (IN in)
                                                                               (NN border)
                                (S (NP (NP (NNP china)
                                                                               (NNS cities))
                                            (POS 's))
                                                                           (VP (VBD marked)
                                        (NP (CD fourteen)
                                                                               (NP (JJ economic)
                                            (NN border)))
                                                                                   (NNS achievements)))))
                                    (VP (VBG opening)
                                        (PRT (RP up))
                                        (NP (NNS cities)))))))))
                                    Flipdeps




                    p ( w1  w2 )
d ( w1 , w2 )  log
                    p ( w2  w1 )
                                                                                   PRED
      TR                                                                           say

                                                        APPS                                              PAT
                                                        ,                                                 increase



                                               ACT              ACT                       ACT            EXT          TWHEN
                                               Spoon            name                      rate           pct          January




                                        RSTR       TWHEN               PAT                        RSTR
                                        Alan       recently            president   APP                         RSTR
                                                                                   Newsweek       ad           5




                                                               ACT            RSTR
                                                               &Gen;          Newsweek




      FUF
                                                                             PARTIC


                                 AFFECTED                                                        AGENT                                          CIRCUM



                                                              PARTIC
                     CAT      PROCESS
      PROCESS        clause



                                            CREATED                     AGENT


LEX    TENSE OBJECT-CLAUSE
say    past  that

                                                         CAT            HEAD       CLASSIFIER       POSSESSOR                   CAT     PREP             NP
                                 CAT
                                                                                                                                pp



                                                                                                    LEX                               LEX      LEX       DETERMINER
                                                                                                    Newsweek                          in       January   none
                                 Results
•   BLEU baseline:                   •   Results in [Och&al.04]
     – 31.6%
•   Most features:
     – 30.0%-31.8%
•   Flipdeps:
     – 31.8%
•   Best single feature:
     – 32.5%
•   Best combination
     – 32.9%
•   (statistically significant
    improvement)
       Phylogenetic Text Modeling
Machine translation identification
           其他党政及司法部门也必须从明年年初开始采取类似行动。
1. Other Party, governmental and law enforcement authorities must take similar actions beginning from the start of next year.
2. Other Party and government agencies and judicial departments must also take similar actions early next year.
3. All other Party, Government and Judicial Departments must start similar actions at the beginning of next year.
4. Other Party, government, and judicatory departments must take similar action at the beginning of next year.
5. Other party and government departments as well as judicial departments must take similar action from the beginning of next year.
6. All other party government and judicial departments must also take similar measures from the beginning of next year.
7. Other party and judicial authorities should take similar actions from the beginning of next year.
8. Other departments of the Party, the government and the judicial departments must also take similar actions early next year.
9. Other Party and Government departments as well as judicial departments must also take similar measures from the beginning of
next year.
10. The other law enforcement agencies and departments will also take part in similar proceedings from the beginning of next year.
11. Other party, governmental and judicial departments will have to take similar action from the beginning of next year.

12. Other party politics and judicial department also will have to start from next year beginning of the year to adopt similar motion.
13. Other party and judicial section must start from the beginning of year of next year taking similar action also
14. The beginning of a year for and res judiciaria as welling must from next year of other party commences assuming
is similar toing the proceeding.
15. At the beginning of next year politics and judicial department other parties must also start to pick to take similar action.
16. Other party politics and the judicial department also will have to start from at the beginning of next year to take the similar
action.
17. Other party policies and judicial department must also begin from early next year to take similar action.
t-test: p<0.05
Chinese: Levenshtein 50/50, BLEU 50/50
Arabic: Levenshtein 50/50, BLEU 48/50
                  Chronological ordering
S1: Italian TV says the crash put a hole in the
25th floor of the Pirelli building, and that
smoke is pouring from the opening. (04/18/02
12:22)
S2: Italian TV showed a hole in the side of the
Pirelli building with smoke pouring from the
opening. (04/18/02 12:32)
S3: Italian state television said the crash put a
hole in the 25th floor of the Pirelli building.
(04/18/02 12:42)
S4: Italian state television said the crash put a
hole in the 25th floor of the 30-story building.
(04/18/02 12:44)

          S1      S2 S3 S4
  S1       0      10 12 13
  S2      10       0 15 16
  S3      12      15 0 1
  S4      13      16 1 0
            S1 (d=0)                                         S1




                       1 (d=3.5)




 S2(d=10)                                                    S2


                                              2 (d=12)
                                   S3(d=12)                  S3
                                                  S4(d=13)   S4


                                                             time t
Best representation: stop words removed
A small plane has hit a skyscraper in central Milan, setting the top floors of the 30-story building on fire, an Italian journalist told CNN. The crash by the Piper
tourist plane into the 26th floor occurred at 5:50 p.m. (1450 GMT) on Thursday, said journalist Desideria Cavina. The building houses government offices and is
next to the city's central train station. Several storeys of the building were engulfed in fire, she said. Italian TV says the crash put a hole in the 25th floor of the
Pirelli building, and that smoke is pouring from the opening. Police and ambulances are at the scene. Many people were on the streets as they left work for the
evening at the time of the crash. Police were trying to keep people away, and many ambulances were on the scene. There is no word yet on casualties.
A small plane has hit a skyscraper in central Milan, setting the top floors of the 30-story building on fire, an Italian journalist told CNN. The crash by the Piper
tourist plane into the 26th floor occurred at 5:50 p.m. (1450 GMT) on Thursday, said journalist Desideria Cavina. The building houses government offices and is
next to the city's central train station. Several storeys of the building were engulfed in fire, she said. Italian TV showed a hole in the side of the Pirelli building
with smoke pouring from the opening. RAI state TV reported that the plane had apparently radioed an SOS because of engine trouble. Earlier though, in Rome,
the senate's president, Marcello Pera, said it "very probably" appeared to be a terrorist attack. Police and ambulances are at the scene. Many people were on the
streets as they left work for the evening at the time of the crash. Police were trying to keep people away, and many ambulances were on the scene. There is no
word yet on casualties. TV pictures from the scene evoked horrific memories of the September 11 attacks on the World Trade Center in New York and the
collapse of the building's twin towers. "I heard a strange bang so I went to the window and outside I saw the windows of the Pirelli building blown out and then I
saw smoke coming from them," said Gianluca Liberto, an engineer who was working in the area told Reuters. The building is known as the Pirelli skyscraper but
the Italian tyre and cable company does not operate out of the building. It is one of the symbols of Italy's financial capital and is one of the world's tallest
concrete buildings, designed between 1955 and 1960.
A small plane crashed into a skyscraper in downtown Milan today, setting several floors of the 30-story building on fire. The plane crashed into the 25th floor of
the Pirelli building in downtown Milan. The weather was clear at the time of the crash. Smoke poured from the opening as police and ambulances rushed to the
area. The president of the Italian Senate, Marcello Pera, told Italian television it "very probably" appeared to be a terrorist attack but soon afterwards his
spokesman said it was probably an accident. A transport official told Reuters the plane had reported problems with its undercarriage and was circling the city
ahead of trying to land at a local airport. The Pirelli building houses the administrative offices of the local Lombardy region and sits next to the city's central train
station. It is constructed of concrete and glass. The crash happened just before rush hour, as office workers were closing their day.
A small airplane crashed into a government building in heart of Milan, setting the top floors on fire, Italian police reported. There were no immediate reports on
casualties as rescue workers attempted to clear the area in the city’s financial district. Few details of the crash were available, but news reports about it
immediately set off fears that it might be a terrorist act akin to the Sept. 11 attacks in the United States. Those fears sent U.S. stocks tumbling to session lows in
late morning trading. Witnesses reported hearing a loud explosion from the 30-story office building, which houses the administrative off ices of the local
Lombardy region and sits next to the city s central train station. Italian state television said the crash put a hole in the 25th floor of the Pirelli building. News
reports said smoke poured from the opening. Police and ambulances rushed to the building in downtown Milan. No further details were immediately available.
Un aereo da turismo, un Piper si è schiantato questo pomeriggio a Milano, poco prima delle 18, contro il grattacielo Pirelli, sede anche della Regione Lombardia
(il presidente della Regione, Roberto Formigoni, è in missione ufficiale in India con una delegazione della regione). Lo si è appreso in ambienti investigativi. L'
impatto sarebbe avvenuto attorno al 25/o piano dei 30 del grattacielo. Almeno sei piani alla vista risultano sventrati. I detriti sono stati lanciati dal'esplosione a
una quarantina di metri intorno all'edificio. In tutta l'area attorno al grattacielo Pirelli lecomunicazioni telefoniche anche via cellulare sono interrotte o quasi
impossibili. La Borsa ha sospeso la seduta serale a Piazza Affari dopo lo schianto dell'aereo da turismo, anche il presidente Bush è stato subito avvertito
dell'espolosione al Pirellone.«Con molta probabilità si tratta di un attentato». Lo ha detto Marcello Pera aprendo la seduta a Palazzo Madama. Ma secondo quanto
si è appreso, l'aereo da turismo era probabilmente in avaria: il pilota, infatti, avrebbe lanciato l'SOS, raccolto dalla torre di controllo di Linate.


 CNN 4/18/02 12:22pm; CNN 4/18/02 12:32pm; ABCNews 4/18/02 1:00pm;
 MSNBC 4/18/02 1:00pm; La Stampa 4/18/02 12:45pm
                             Fact tracking
04/18/02 13:17 (CNN)
The plane, en route from Locarno in Switzerland, to Rome, Italy, smashed into the Pirelli
    building's 26th floor at 5:50 p.m. (1450 GMT) on Thursday.

04/18/02 13:42 (ABCNews)
The plane was destined for Italy's capital Rome, but there were conflicting reports as to whether it
    had come from Locarno, Switzerland or Sofia, Bulgaria.

04/18/02 13:42 (CNN)
The plane, en route from Locarno in Switzerland, to Rome, Italy, smashed into the Pirelli
    building's 26th floor at 5:50 p.m. (1450 GMT) on Thursday.

04/18/02 13:42 (FoxNews)
The plane had taken off from Locarno, Switzerland, and was heading to Milan's Linate airport, De
    Simone said.
    Questions from Milan corpus
1. How many people were injured?
2. How many people were killed? (age, number, gender, description)
3. Was the pilot killed?
4. Where was the plane coming from?
5. Was it an accident (technical problem, illness, terrorist act)?
6. Who was the pilot? (age, number, gender, description)
7. When did the plane crash?
8. How tall is the Pirelli building?
9. Who was on the plane with the pilot?
10. Did the plane catch fire before hitting the building?
11. What was the weather like at the time of the crash?
12. When was the building built?
13. What direction was the plane flying?
14. How many people work in the building?
15. How many people were in the building at the time of the crash?
16. How many people were taken to the hospital?
17. What kind of aircraft was used?
           Relative order, time to stabilize and number of incorrect
               or partially correct answers before stabilization
•   Changing answers:
    – How many people were injured?: 40 different answers!
      ``no word yet on casualties/injuries'', ``20 people were taken to a nearby
      hospital'', ``20 to 30 people were hospitalized with iinjuries'', ``many people
      were injured'', ``there was no official word on the number of people injured in
      the building'', ``at least 20 injured were taken to hospital from the scene dozens
      of people had been taken to the hospital'', ``injuring dozens'', ``injuring at least
      30'', ``injuring 60'', ``dozens were injured'', ``60 others were injured'', ``the
      number of injured, originally at 60, was revised downward Friday to 36''.
    Only 24 hours after the crash do agencies settle on the accurate number, namely
      ``36 people''.
Source


                  one dead                    at least two                                                    four people                  four dead
ABCNews


           no word yet
           on casualties                 two deaths               at least three                           at least four
    CNN

                                                                                                                                                 incorrect

                                                                                                                                                 partial
                                           three people
                   no immediate two deaths killed       three people dead                                                                        correct
FoxNews            reports



                                                                                                                    five people                  at least
                                   at least two                       at least three                                killed                           five
  MSNBC
                                                                                                                                                        Fasulo
                                                                                                                                                       and two
                                                                                                                                                        others
                                                                                            at least             five reported                          killed
USAToday                                                                                    three                         dead




                                                                                                                                                  Next Day
            9:50 12:47 12:49 12:51 12:51 13:01 13:17 13:42 13:46 14:13 14:21 14:29 14:32 14:52 15:02 15:22 15:31 15:36 17:52 18:13 18:35 18:40               9:31 18:02


                                                                                                                                                  Time(EST)
              Syntactic Alignment
•    Sequence alignment for (near)    •   Dynamic programming
     paraphrasing [Barzilay&Lee 03]   •   Different penalties for alignment
•    No syntax used                       depending on the syntactic
                                          similarity



                           talked


    John                                               with           Mary

               had            a           chat
                         Syntactic Alignment
A police official said it was a Piper tourist plane and that the crash had set the top floors on fire.
According to ABCNEWS aviation expert John Nance, Piper planes have no history of mechanical troubles or other problems that would
lead a pilot to lose control.
April 18, 2002 8212; A small Piper aircraft crashes into the 417-foot-tall Pirelli skyscraper in Milan, setting the top floors of the 32-
story building on fire.
Authorities said the pilot of a small Piper plane called in a problem with the landing gear to the Milan's Linate airport at 5:54 p.m., the
smaller airport that has a landing strip for private planes.
Initial reports described the plane as a Piper, but did not note the specific model.
Italian rescue officials reported that at least two people were killed after the Piper aircraft struck the 32-story Pirelli building, which is in
the heart of the city s financial district.
A small piper plane with only the pilot on board crashed Thursday into a 30-story landmark skyscraper, killing at least two people and
injuring at least 30.
Police officer Celerissimo De Simone said the pilot of the Piper Air Commander plane had sent out a distress call at 5:50 p.m. just before
the crash near Milan's main train station.
Police officer Celerissimo De Simone said the pilot of the Piper aircraft had sent out a distress call at 5:50 p.m. (11:50 a.m.)
Police officer Celerissimo De Simone said the pilot of the Piper aircraft had sent out a distress call at 5:50 p.m. just before the crash near
Milan's main train station.
Police officer Celerissimo De Simone said the pilot of the Piper aircraft sent out a distress call at 5:50 p.m. just before the crash near
Milan's main train station.
Police officer Celerissimo De Simone told The AP the pilot of the Piper aircraft had sent out a distress call at 5:50 p.m. just before
crashing.
Police say the aircraft was a Piper tourism plane with only the pilot on board.
Police say the plane was an Air Commando 8212; a small plane similar to a Piper.
Rescue officials said that at least three people were killed, including the pilot, while dozens were injured after the Piper aircraft struck the
Pirelli high-rise in the heart of the city s financial district.
The crash by the Piper tourist plane into the 26th floor occurred at 5:50 p.m. (1450 GMT) on Thursday, said journalist Desideria Cavina.
Police officer Celerissimo De Simone said the pilot of the Piper aircraft, en route from Switzerland, sent out a distress call at 5:54 p.m.
just before the crash near Milan's main train station.
Algorithm and results
            •   Three lexical methods
            •   Two syntactic methods
            •   Generate new sentences
            •   method 4 (syntactic alignment
                except for stop words):
                 –   Grammaticality 3.74
                 –   Fidelity 3.77
                 –   on a scale from 1 to 4
            •   Best lexical method:
                 –   Grammaticality 3.12
                 –   Fidelity 3.07
                  Web-based QA
•   TREC questions
    – Where is Inoco based?
    – When was London's Docklands
      Light Railway constructed?
    – Who followed Willy Brandt as
      chancellor of the Federal
      Republic of Germany?
    – What is Grenada's main
      commodity export?
•   TREC evaluation
    – Earliest conference papers
      (Radev & al. ANLP’2000,
      Prager & al. SIGIR’2000)
•   Reranking models
              Question Modulation
•   TREC question set                         •   What country is the biggest
•   Start with initial formulation                producer of tungsten? 0.44
•   TRDR = Total Reciprocal                   •   What country “biggest producer”
    Document Rank (range: 0 to                    of tungsten? 1.11
    2.92)
                                              •   country “biggest producer of
•   Evolutionary operators:                       tungsten”? 1.98
    mutation, permutation,
    crossover, drop, insert, phrase



•   Query modulation results                  •   Web results using Google as the
     –   42% increase in TRDR (from 0.79 to       backend search engine
         1.12)                                     –   0.4 MRR (mean reciprocal rank)
                   Models of the Web
•   Evolving networks: fundamental object of statistical physics, social networks,
    mathematical biology, and epidemiology
•   Erdös/Rényi 59, 60


•   Barabási/Albert 99

•   Watts/Strogatz 98

•   Kleinberg 98
                                                                                a
•   Menczer 02
                                                    A
•   Radev 03
                                                                                        e  k   k  k
                                                                               P(k ) 
                                                    B                             P(k ) 
                                                                                            kk 
                                                                                                 !
                                                                                 b
                                                                                k   Np  ( )
        Self-triggerability across
                hyperlinks
•   Document closures for                            pi            pj
    information retrieval
•   Self-triggerability                 p ' p ( w  p j | pi  p j  w  pi )
    [Mosteller&Wallace 84]        r      
    Poisson distribution                p                    p
•   Two-Poisson
    [Bookstein&Swanson 74]              p’
•   Negative Binomial, K-mixture
    [Church&Gale 95]
•   Triggerability across hyperlinks?




                                               by with from path
                                               photo dream
                                                                           p
     Evolving Word-based Web
•   Observations:                       •   Model (cont’d)
     – Links are made based on topics        – Pick words in decreasing order
     – Topics are expressed with               of r.
       words                                 – Generate hyperlinks with
     – Words are distributed very              random directionality
       unevenly (Zipf, Benford, self-   •   Outcome
       triggerability laws)                  – Generates power-law degree
•   Model                                      distributions
     – Pick n                                – Generates topical communities
     – Generate n lengths according          – Natural variation of PageRank:
       to a power-law distribution             LexRank
     – Generate n documents using a
       trigram model

                                                           p  ET p
                                                                   '
                                            PageRank
                                                    a E h    h  Ea
                                                     '  T      '
                                            Hits
                Tripartite updating
•   Modeling classification           •   Tripartite updating
    problems using bipartite graphs   •   Matrix representation
•   Weakly supervised learning –      •   Iterative power method
    why?
     – bootstrapping, co-training,
       active learning
•   Spectral partitioning                              T1
     – Fiedler vector
•   Singular value decomposition                                   L
•   Random walks
                                                F


                                                                   U
                                                       T2
                  Tripartite updating
                                        •   Four-way or three-way classification
•   Tasks:                              •   For the same accuracy of SP and
     –    Spam detection                    TU, TU handles twice as many
                                            labeled examples with ten times as
     –    Named entity classification       many unlabeled examples
     –    PP attachment
     –    Number classification
•   Features:
     – Number classification: 5
                                                           T1
       classes based on context and
       hobbs class                                                      L

                            ( t 1)
                                                   F
                T LF
         (t )     T
    F            1

    U ( t )  T2 F ( t )  U ( t 1)
    F ( t )  T2 U ( t )  F ( t )
                  T                                                     U
                                                           T2
                 Relation extraction
•   User gives examples of entity E1
    and entity E2.                         •   Weakly supervised learning
                                               based on graphs is used.
•   Example: song = “Let it Be”, singer
    = “the Beatles”.

•   System finds other songs and
    singers with a very minimal number
    of training examples.

•   The relation may be quite different,
    e.g., protein-protein, organization-
    leader, book-author, drug-disease.
     Protein Regulatory Network
             Recognition
•   Wnt signaling
•   Glycogen synthase kinase-3 (GSK-
    3) and CK1 (casein kinase 1) alpha
    phosphorylate Arm (Armadillo, -
    catenin) and cause it to degrade.
•   Axin also binds to the phosphatase
    PP2A
•   PP2A activity inhibits Wnt signaling




Hsu 1999, Li 2001, Yanagawa 2002, Liu
2002, Nusse 2003
              Method and Results
•   Medline:
     – “signal transduction” as MeSH
         major topic and “Wnt” or “AKT”
         or “Beta-catenin” as words
•   3300 papers extracted by Carlos
    Santos
•   441 putative proteins (“X is a
    protein”, “the X protein” “X verbs”)
•   Verbs: Bind associate interact
    activate repress inhibit upregulate
    regulate downregulate complex
    dimerize localize bound regulate
    stabilize control translocate
    antagonize amplify transduce trigger
                                                          Uneven importance




                                                                                                                                                      Manual evaluation
                                                                                          Graph structure




                                                                                                                       unstructured


                                                                                                                                      Hard to train
                                            multisource
                             multilingual




                                                                              redundant




                                                                                                            evolving
Number classification                         X                               X            X                                            X                 X

Summarization MEAD/CST/NIE   X                X           X                   X            X                 X           X              X                 X

Lexical Web models                            X           X                                X                 X           X

Statistical MT               X                X                                            X                             X              X

Protein networks                              X           X                   X            X                             X                                X

Relation extraction                           X           X                   X            X                             X              X                 X

Phylogenetic alignment       X                X                               X            X                 X           X              X                 X

QA/NSIR                                       X           X                   X                                          X              X

Topical crawling                              X           X                   X            X                 X           X

XML retrieval                                 X           X                                                                             X

Fact tracking                                 X           X                   X                              X           X
          A grabbag of research
                problems
•   Finding adequate               •   Relation extraction
    representations for dynamic    •   Syntax-based machine
    texts
                                       translation and summarization
•   Integrating user models
                                   •   Automatic knowledge
•   Using self-triggering for
    information retrieval              acquisition from the Web
•   Weakly supervised and active
    learning
•   Robust semantic analysis
•   Adequate models of the Web
                               Conclusion
•   New approaches to natural language processing and information retrieval using
    graph-based techniques such as random walks
•   Applications beyond NLP
•   Highest ranked system at DUC
•   Promising results in semi-supervised machine learning
•   Acknowledgments:
     – CLAIR (Güneş Erkan, Jahna Otterbacher, Siwei Shen, Zhu Zhang)
     –   UROP program
     –   NSF and NIH
     –   Mark Newman
•   To read more:
     –   http://tangra.si.umich.edu/clair
     –   http://www.summarization.com
     –   http://www.newsinessence.com
•   Papers: CACM 2005; JAIR 2004; EMNLP 2004; IP&M 2004; JASIST 2002, 2004, 2005; WWW 2002;
    AAAI 2002; SIGIR 1995, 2000; ACL 1998, 2003; HLT 2001; HLT-NAACL 2004; CIKM 2001, 2003; ANLP
    1997, 2000; LREC 2002, 2004; IJCNLP 2004; CL 1998, 2002; COLING 2000, 2004
      20                                  05


                      ACL 2005
                       www.aclweb.org
                       June 25-30, 2005
                        Ann Arbor, MI

                General chair: Kevin Knight, ISI
Program co-chairs: Kemal Öflazer, Sabanci U.; Hwee Tou Ng, NUS
           Local chair: Dragomir Radev, U. Michigan
                Submission deadline: January 14
            S

         VP

 VB      NP          PP

      PRP       IN         NP

                     PRP$          NN


Thank you for your attention !
       tangra.si.umich.edu/clair

								
To top