Summarization of Online Conversations by nehalwan

VIEWS: 21 PAGES: 14

									SUMMARIZATION OF ONLIN
    CONVERSATIONS

                      Group No : 41
                     Mridul Mundara
                      Snehal Shinde
                      Rishika Modi

  International Institute of Information Technology(IIIT),
                         Hyderabad
             CONTENTS
 Introduction
 Approach
 Features
 System Design
 Evaluation
 Future Work
 Conclusion
 References
           INTRODUCTION
 This project focuses on online conversations
  from email threads which aim to discuss and
  resolve user-related issues.
 It can be used to analyze the impact of virtual
  social interactions and virtual organizational
  culture on software/product development.
 Dataset: BC3 corpus (xml file).
 BC3 corpus contains around 40 email threads.
STRUCTURE OF BC3 CORPUS
<Thread>
   <listno>
   <name>
   <doc>
       <received >
       <from>
       <to>
       <subject>
       <text>
           <sentence 1>
           <sentence 2>
           ……………..
           ……………..
                 APPROACH
We have used three phase approach
 First Phase:
   Parsing of xml threads using SAX parser into various
    sections- To, From, Subject , List of Sentences.
   Implemented TF-IDF as a feature.
 Second Phase:
   Implemented TF-ISF, Sentiment Score, Length of sentence
    w.r.t the document, Is a Question, Similarity with subject
    and final score of the sentence.
 Third Phase:
   Comparison of results with ROUGE.
SYSTEM DESIGN
           FORMULAE USED
 If Sentence is a not a URL and length of
  sentence is greater than four words
   If sentence is not a question
      Score=(TF-IDF+TF-ISF + Length + Sentimental Score +
      5*Similarity Score)/4.5
   If sentence is a question
      Score=(0.7*(TF-IDF+TF-ISF + Length + Sentimental
      Score +5* Similarity Score))/4.5
 Else
  Score=(0.7*(TF-IDF+TF-ISF + Length + Sentimental
    Score + 5*Similarity Score))/0.5
           EVALUATION
Procedure:
1. Created manual summaries for threads.
2. Created system summaries for threads
   through our system.
3. Created xml input for ROUGE evaluation by
   providing above summarized files.
4. Compared the results for ROUGE-1, ROUGE-2
   and ROUGE-L.
          COMPARISON TABLE
---------------------------------------------
--- When Top 20% sentences Retrieved ---------------------
---------------------------------------------
SUMMARY ROUGE-1 Average_R: 0.36829 (95%-conf.int. 0.36829 - 0.36829)
SUMMARY ROUGE-1 Average_P: 0.77893 (95%-conf.int. 0.77893 - 0.77893)
SUMMARY ROUGE-1 Average_F: 0.50012 (95%-conf.int. 0.50012 - 0.50012)
---------------------------------------------
SUMMARY ROUGE-2 Average_R: 0.17177 (95%-conf.int. 0.17177 - 0.17177)
SUMMARY ROUGE-2 Average_P: 0.36331 (95%-conf.int. 0.36331 - 0.36331)
SUMMARY ROUGE-2 Average_F: 0.23326 (95%-conf.int. 0.23326 - 0.23326)
---------------------------------------------
SUMMARY ROUGE-L Average_R: 0.36327 (95%-conf.int. 0.36327 - 0.36327)
SUMMARY ROUGE-L Average_P: 0.76830 (95%-conf.int. 0.76830 - 0.76830)
SUMMARY ROUGE-L Average_F: 0.49330 (95%-conf.int. 0.49330 - 0.49330)
--------------------------------------------------------------
--- When Top 30% sentences Retrieved ---------------------
---------------------------------------------
SUMMARY ROUGE-1 Average_R: 0.49431 (95%-conf.int. 0.49431 - 0.49431)
SUMMARY ROUGE-1 Average_P: 0.73249 (95%-conf.int. 0.73249 - 0.73249)
SUMMARY ROUGE-1 Average_F: 0.59028 (95%-conf.int. 0.59028 - 0.59028)
---------------------------------------------
SUMMARY ROUGE-2 Average_R: 0.22447 (95%-conf.int. 0.22447 - 0.22447)
SUMMARY ROUGE-2 Average_P: 0.33264 (95%-conf.int. 0.33264 - 0.33264)
SUMMARY ROUGE-2 Average_F: 0.26805 (95%-conf.int. 0.26805 - 0.26805)
---------------------------------------------
SUMMARY ROUGE-L Average_R: 0.48942 (95%-conf.int. 0.48942 - 0.48942)
SUMMARY ROUGE-L Average_P: 0.72524 (95%-conf.int. 0.72524 - 0.72524)
SUMMARY ROUGE-L Average_F: 0.58444 (95%-conf.int. 0.58444 - 0.58444)
           FUTURE WORK
 Use of machine learning to improvise the
  summaries.
 A large size corpus with more than 40 email
  threads.
 Implementation of Discourse Marker feature to
  get more precise summary statements.
            CONCLUSION
 Results were found to be improved over the
  baselines scores.
             REFERENCES
 Arpit Sood, Thanvir P Mohamed, Vasudev
  Verma, “Summarizing Online Conversations:
  A Machine Learning Approach”, 24th
  International Conference on Computational
  Linguistics - (Coling-2012), Mumbai, India,
  Report No: IIIT/TR/2012/-1
 Jan Ulrich, “Supervised Machine Learning for
  Email Thread Summarization”, the University
  of British Columbia, September 2008
THANK YOU!

								
To top