Preliminary by Mtri58q

VIEWS: 0 PAGES: 64

									                     Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




Lecture 9: Rank Aggregation in
MetaSearch

   MetaSearch Engine
   Social Choice Rules
   Rank Aggregation
                     Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




    Choices of Search Engines
   Many search engines exist to compete for
    users
       The results are not necessarily the same
       Different users prefer different search engines
       Search results may, in the future, be biased
        towards paid advertisements.
                                        Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




      GOOGLE: City University
1.   City University London - the University for business and the ...
     ... The University for business and the professions. Contact Us | About City University
     | Maps & Directions, AZ Index | Site Map | Help. Prospective Students. ...
     Description: Official site with information about courses, research, schools, and departments. Includes details...
2.   City University Home
     A University which believes in forward-thinking business and leadership skills, exposing students to the latest
     technology in all courses of study.
     Description: A private, nonprofit institution founded to serve working adults wanting to pursue educational
     opportunit...
3.   Welcome to Dublin City University
     ... here... Dublin City University, Dublin 9, Ireland. Tel. +353 (0) 1
     700 5000, Fax. +353 (0) 1 836 0830. Page updated: 02/09/03 legal. ...
     Description: Information on facilities, services, degree courses, research, the campus, student life, the library...
4.   The City University of New York
     Description: The University's Main Website.
5.   City University of Hong Kong
     AD Working Group lends staff, students an ear. Several issues still
     evoked strong emotions as staff members and students voiced their ...
     Description: Formerly the City Polytechnic of Hong Kong. Includes information on university, links to learning...
6.   Welcome to Oklahoma City University
     ... At Oklahoma City University, our students come first. Our ... Oklahoma City
     University offers a quality, values-centered education. United ...
     Description: Admissions, academic programs and alumni relations, sports, services, news, calendar of events
     and...
                                      Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




      YAHOO
1.   City University London - the University for business and the ...
     ... The University for business and the professions. Contact Us | About City University | Maps & Directions, AZ
     Index | Site Map | Help. Prospective Students. ...
2.   City University
     A University which believes in forward-thinking business and leadership skills, exposing students to the latest
     technology in all courses of study.
     www.cityu.edu/ - 3k - Cached - More pages from this site
3.   Dublin City University
     ... here... Dublin City University, Dublin 9, Ireland. Tel. +353 (0) 1 700 5000, Fax. +353 (0) 1 836 0830. Page
     updated: 02/09/03 legal. ...
     www.dcu.ie/ - 6k - Cached - More pages from this site
4.   City University of New York
     The University's Main Website.
     www.cuny.edu/ - More pages from this site
5.   City University of Hong Kong
     AD Working Group lends staff, students an ear. Several issues still evoked strong emotions as staff members and
     students voiced their ...
     www.cityu.edu.hk/ - 26k - Cached - More pages from this site
6.   Oklahoma City University
     ... At Oklahoma City University, our students come first. Our ... Oklahoma City University offers a quality,
     values-centered education. United ...
     www.okcu.edu/ - 20k - Cached - More pages from this site
                                       Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




        AllTheWeb
1.   Apartment Listings in University City (sponsored)
     St. Louis, Missouri area apartment listings at Apartments.com. Free nationwide apartment search with
     visual rental listings online.
     http://www.apartments.com
2.   City University - Washington (sponsored)
     Contact information and resources such as yellow page information, phone number, address, maps and
     directions as provided by QwestDex.
     http://service.bfast.com
3.   City University Apartments - Rent.com (sponsored)
     Rent.com has millions of free apartment listings nationwide. Get $100 when you sign a lease near your
     school - it's easy.
     http://www.rent.com
4.   City University London - the University for business and the professions
     ... Contact Us | About City University | Maps & Directions A-Z Index | Site Map | Help ...
     Description: Official site with information about courses, research, schools, and departments. Includes
     details of news and events.
     more hits from: http://www.city.ac.uk/ - 13 KB
5.   Flash Upgrade
     Description: A University which believes in forward-thinking business and leadership skills, exposing
     students to the latest technology in all courses of study.
     http://www.cityu.edu/ - 27 KB
6.   Welcome to Dublin City University
     ... KnowledgeWorks, wins DCU Mallin-invent award Full text you can go anywhere in the world from here...
     Dublin City University, Dublin 9, Ireland. Tel. +353 (0) 1 700 5000, Fax. +353 (0) 1 836 0830. Page updated:
     02/09/03 legal search ...
     Description: Information on facilities, services, degree courses, research, the campus, student life, the
     library and DCU news.
                                  Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




        MSN Search
1.   City University of New York Details the CUNY system's campuses and admissions
     requirements. Eye photos of notable graduates such as actor Judd Hirsch.
     www.cuny.edu
2.   City University Offers prospectus, admissions, academic calendar, courses and degrees, and
     news.
     www.cityuniversity.net
3.   City University Private, nonprofit institution located in Belleview, Washington, offers higher
     education opportunities to working professionals and lifelong learners.
     www.cityu.edu
4.   Valley City University Admissions Site covers financial aid and tuition and offers a virtual
     campus tour. Site also provides access to an online application.
     www.vcsu.nodak.edu/admissions
5.   Oklahoma City University The Oklahoma City University site contains information about
     admissions, academic departments and student life. The site also provides library resources,
     faculty information and news.
     frodo.okcu.edu
6.   Dublin City University, Dublin - TripAdvisor Browse reviews of the Dublin City University in
     Dublin, County Dublin, Ireland. Check consumer reviews, articles and opinions about the
     Dublin City University.
     www.tripadvisor.com/Hotel_Review-g186605-d209093-Reviews-Dublin_City_U...
                                  Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




       AOL Search
1.   City University London - the University for business and the ... - ... The University for
     business and the professions. Contact Us | About City University | Maps & Directions, AZ
     Index | Site Map | Help. Prospective Students. ...
     http://www.city.ac.uk/
2.   City University Home - A University which believes in forward-thinking business and
     leadership skills, exposing students to the latest technology in all courses of study.
     http://www.cityu.edu/
3.   Welcome to Dublin City University - ... here... Dublin City University, Dublin 9, Ireland. Tel.
     +353 (0) 1 700 5000, Fax. +353 (0) 1 836 0830. Page updated: 02/09/03 legal. ...
     http://www.dcu.ie/
4.   http://www.cuny.edu/ - The University's Main Website.
     http://www.cuny.edu/
5.   City University of Hong Kong - AD Working Group lends staff, students an ear. Several
     issues still evoked strong emotions as staff members and students voiced their ...
     http://www.cityu.edu.hk/
6.   Welcome to Oklahoma City University - ... At Oklahoma City University, our students come
     first. Our ... Oklahoma City University offers a quality, values-centered education. United ...
     http://www.okcu.edu/
                             Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




       Ask Jeeves
1.   City University of Hong Kong Formerly the City Polytechnic of Hong Kong.
     Includes information on university, links to learning resources, research and
     departmental information.....From:www.cityu.edu.hk/
2.   City University Text Version. The University for business and the professions
     ... Contact Us | About City University | Maps & Directions A-Z Index | Site
     Map | Help...From:www.city.ac.uk/
3.   New Jersey City University Official site includes graduate and undergraduate
     information, news, events, online learning, calendars, and
     athletics.From:www.njcu.edu/
4.   Elizabeth City State University Admission Application Introduction 2004-
     2005 APPLICATION FOR ADMISSION. Welcome to the Elizabeth City State
     University Online Application. New
     Freshmen...From:www.ncmentor.org/applications/unc/apply/elizabeth_city
     _state_univ...
5.   City University A University which believes in forward-thinking business and
     leadership skills, exposing students to the latest technology in all courses of
     study.From:www.cityu.edu/
6.   The City College of New York City College of New York (CUNY) A senior
     college of The City University of New York. Located in Manhattan. 138th
     Street at Convent Avenue. A Harlem masterpiece in the neo-
     Gothic...From:www.ccny.cuny.edu/
                                    Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




          HotBot
1. City University of New York
        Details the CUNY system's campuses and admissions requirements. Eye photos of notable
        graduates such as actor Judd Hirsch.
        www.cuny.edu/ - August 10, 2003 - 25 KB
2. City University
        Private, nonprofit institution located in Belleview, Washington, offers higher education
        opportunities to working professionals and lifelong learners.
        www.cityu.edu/ - August 23, 2003 - 27 KB
3. New Jersey City University
        Explore the course requirements for this liberal arts institution. Link to admissions and
        financial aid information. ... New Jersey City University. 2039 Kennedy Boulevard Jersey
        City, New Jersey 07305-1597 ...
        www.njcu.edu/ - November 1, 2003 - 20 KB
4. City University London - the University for business and the...
        ... Contact Us | About City University | Maps & Directions. A-Z Index | Site Map | Help ...
        www.city.ac.uk/ - September 28, 2003 - 14 KB
5. Oklahoma City University
        Take a tour of the campus, and explore a roster of academic programs. ... At Oklahoma
        City University, our students come first. ...
        www.okcu.edu/ - October 26, 2003 - 20 KB
6. University of Missouri, Kansas City
        University based in Kansas City, Missouri, presents and overview of its academic
        programs, and offers campus news. ... UNIVERSITY OF MISSOURI-KANSAS CITY. Bulletin.
        Winter 2004 Fee Update, more... ...
        www.umkc.edu/ - October 23, 2003 - 16 KB
                                   Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




         Lycos
1. City University London - the University for business and the...
        … Contact Us | About City University | Maps & Directions A-Z Index | Site Map | Help …
        More results from: www.city.ac.uk September 16, 2003 - 14 KB
2. Flash Upgrade
        A University which believes in forward-thinking business and leadership skills, exposing
        students to the latest technology in all courses of study.
        www.cityu.edu November 1, 2003 - 27 KB
3. The City University of New York
        The University's Main Website.
        www.cuny.edu December 31, 1969 - 121 B
4. Welcome to Dublin City University
        … KnowledgeWorks, wins DCU Mallin-invent award Full text you can go anywhere in the
        world from here... Dublin City University , Dublin 9, Ireland. Tel. +353 (0) 1 700 5000,
        Fax. +353 (0) 1 836...
        More results from: www.dcu.ie October 29, 2003 - 5 KB
5. City University of Hong Kong
        Formerly the City Polytechnic of Hong Kong. Includes information on university, links to
        learning resources, research and departmental information and student information.
        More results from: www.cityu.edu.hk October 29, 2003 - 26 KB
6. HCU HomePage -English-
        … contact us For any suggestion and requests to this web site, www-admin@hiroshima-
        cu.ac.jp Feel free to link this web site. Hiroshima City University
        More results from: www.hiroshima-cu.ac.jp May 11, 2003 - 26 KB
                                 Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




       Teoma
1.   City University of Hong Kong Formerly the City Polytechnic of Hong Kong. Includes
     information on university, links to learning resources, research and departmental
     information...www.cityu.edu.hk/
     [Related Pages]
     [More Results from www.cityu.edu.hk]

2.   City University Text Version. The University for business and the professions ... Contact Us
     | About City University | Maps & Directions A-Z Index | Site Map | Help...www.city.ac.uk/
     [More Results from www.city.ac.uk]

3.   New Jersey City University Official site includes graduate and undergraduate information,
     news, events, online learning, calendars, and athletics.www.njcu.edu/
     [Related Pages]
     [More Results from www.njcu.edu]

4.   Elizabeth City State University Admission Application Introduction 2004-2005
     APPLICATION FOR ADMISSION. Welcome to the Elizabeth City State University Online
     Application. New Freshmen...www.ncmentor.org/applications/unc/apply/el...

5.   City University A University which believes in forward-thinking business and leadership
     skills, exposing students to the latest technology in all courses of study.www.cityu.edu/

6.   The City College of New York City College of New York (CUNY) A senior college of The City
     University of New York. Located in Manhattan. 138th Street at Convent Avenue. A Harlem
     masterpiece in the neo-Gothic...www.ccny.cuny.edu/
     [More Results from www.ccny.cuny.edu]
                     Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




        MetaSearch Engine
   Metasearch Engines are designed to
    increase the coverage of web by
    forwarding users’ queries to multiple search
    engines
       Users’ requests are sent to multiple search
        engines such as AlltheWeb, Google, MSN.
   Then the results from the individual search
    engine are combined into a single result set
    to present to users.
                  Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




     Different Forms of MetaSearch
   Submit different representations of the
    same query to the same search engine,
    then combine the results.

   Submit the same query to several search
    engine adopting different information
    retrieval models, then combine the results.
               Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng

    Issues
   How to combine the results retrieved by
    different source search engines is
    crucial for the success of a metasearch
    engine.
   And this is the problem that social
    choice theory has been trying to
    answer.
                      Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




         Search Engine Watch

   Interesting metat search engines are listed at
       http://www.searchenginewatch.com/links/article.p
        hp/2156241
Award Winners                 Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng



Source: www.searchenginewatch.com
     Vivisimo
      Enter a search term, and Vivismo will not only pull back matching responses from
      major search engines but also automatically organize the pages into categories.
      Slick and easy to use. Winner of Best Meta Search Engine award from Search
      Engine Watch in 2002.
     EZ2WWW
      This meta search engine provides results from AllTheWeb, AltaVista, Google, Open
      Directory, Teoma, Wisenut and Yahoo. Advanced search offers access to more
      than 1,000 specialized resources, searchable by category. Honorable mention for
      Best Meta Search Engine award from Search Engine Watch in 2002.
     Kartoo
      If you like the idea of seeing your web results visually, this meta search site shows
      the results with sites being interconnected by keywords. Honorable mention for
      Best Meta Search Engine award from Search Engine Watch in 2002.
     SurfWax
      Searches against major engines or provides those who open free accounts the
      ability to chose from a list of hundreds. Using the "SiteSnaps" feature, you can
      preview any page in the results and see where your terms appear in the
      document. Allows results or documents to be saved for future use. Honorable
      mention for Best Meta Search Engine award from Search Engine Watch in 2002.
     Fazzle
      Fazzle offers a highly flexible and customizable interface to a wide variety of
      information sources, ranging from general web results to specialized search
      resources in a number of subject specific categories. Formerly called SearchOnline.
Other Top Choices                 Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng



Source: www.searchenginewatch.com
     InfoGrid
      In a compact format, InfoGrid provides direct links to major search sites and topical web sites in
      different categories. Meta search and news searching is also offered.
     Infonetware RealTerm Search
      This site is primarily designed to demonstrate classification technology from Infogistics. It's a
      meta search engine, and it does topical classification of results, like Vivisimo. However, it is
      unique in that you can select several different topics, then "drill down" to see results from all of
      them, rather than being restricted to the results from only one topic.
     Ithaki
      Ithaki is probably the most "global" of all meta search engines, available in 14 languages and
      offering more than 35 different categories for limiting your search. In addition, Ithaki offers
      country specific search, querying only local search engines rather than the regional versions of
      the major search engines.
     Ixquick
      Meta search engine that ranks results based on the number of "top 10" rankings a site receives
      from the various search engines.
     ProFusion
      Brings back listings from several major search engines as well as "Invisible Web" resources.
      Formerly based at the University of Kansas, the site was purchased by search company
      Intelliseek in April 2000.
     qbSearch
      Want to get multiple pages of results from a search engine combined into one single page? QB-
      Search will quickly join up to 200 pages of listings from major search engines.
     Query Server
      Search against major web-wide search engines, as well as major news, health, money and
      government search services.
     Turbo10
      Turbo10 is a metasearch Engine accesses both traditional web search engines and some invisible
      web databases, with a very speedy interface.
Popular Services                  Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng



Source: www.searchenginewatch.com
     Dogpile
      Popular metasearch site owned by InfoSpace that sends a search to a customizable list of search
      engines, directories and specialty search sites, then displays results from each search engine
      individually.
     Excite
      Formerly a crawled-based search engine, Excite was acquired by InfoSpace in 2002 and uses
      the same underlying technology as the other InfoSpace meta search engines, but maintains its
      own portal features.
     MetaCrawler
      One of the oldest meta search services, MetaCrawler began in July 1995 at the University of
      Washington. MetaCrawler was purchased by InfoSpace, an online content provider, in Feb. 97.
     Webcrawler
      Formerly a crawled-based search engine owned by Excite, Webcrawler was acquired by
      InfoSpace in 2002 and uses the same underlying technology as the other InfoSpace meta search
      engines, but offers a fast and clean, ad-free interface.
     Mamma
      Founded in 1996, Mamma.com is one of the oldest meta search engines on the web. Mamma
      searches against a variety of major crawlers, directories and specialty search sites. The service
      also provides a paid listings option for advertisers, Mamma Classifieds.
     Search.com
      Search.com is a meta search engine operated by CNET. It offers both web-wide search and a
      wide variety of specialty search options. Search.com uses technology from SavvySearch, which
      was acquired by CNET in October 1999. The SavvySearch site itself no longer operates.
      SavvySearch was one of the older metasearch services,
Special Services            Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng



Source: www.searchenginewatch.com


     Family Friendly Search
      Meta search service that queries major kid-friendly search engines.
     Searchy.co.uk
      Searches 15 U.K. engines. The advanced search form allows you to change the
      order that results are presented, either by speed or manually to suit your own
      preferences.
     TeRespondo
      Spanish metacrawler that searches on the more popular search engines. Motor de
      busqueda que busca tu consulta en los buscadores mas populares en espanol.
     Watson for the Macintosh
      Watson is a "Swiss Army Knife" with nineteen interfaces to web content and
      services -- an improvement on Sherlock, with nearly twice as many tools,
      including Google Searching.
               Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




    Social Choice Theory

   Studies on protocols that help a group
    of people make collective decisions,
    such as vote.
                 Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




A Fundamental problem
   Given a collection of agents (voters)
       with preferences over different alternatives
        (allocations, outcomes),
   how should society evaluate these
    alternatives and make a decision for all
       that may be for the will of some voters but
        against that of others.
              Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




Applications
   Voters elect president from several
    candidates.
   National polls for economic or political
    policy of the government
   The procedure or rule of election
   The rank of metasearch engine
    obtained from those of search engines
                Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




Group Descisions
How do we make decisions
 Flip a coin?
 Dictatorship?

 Democracy (Majority rule)?

in a group with
       Heterogeneity of individual preference
               Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




    Group Decision Rules
   Majority rule ,
   Condorcet paradox (voting cycle)
   Borda rule
                      Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




    Mathematical model
   A set of voters V={v1,v2,v3,…,Vn}

   A set of alternatives or outcomes
    S={s1,s2,s3,…Sm}, with |S|=m; and

   A set of preference relation P={R1,R2,R3…Rn},
    called a preference profile,
       the preference relation Ri for each voter i is a
        permutation (order) of elements in S.
                         Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




    Example 1 Majority Rule
   3 rational people have rational preferences
    over 2 alternatives {x,y}
                  Person
              1      2        3

        1st   X      Y        X                 1 : X>Y
Pref.                              i.e.Person 2 : Y>X
        2nd   Y      X       Y                  3 : X>Y


How to Aggregate their preferences? How to choose?
              Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




   Using majority rule.

   Since more than ½ people (two out of
    three) prefer x to y.
   Then the group prefers x to y
                  Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




    Example 2 Condorcet Paradox
   3 rational people have rational
    preferences over 3 alternatives {x,y,z}
                 Person
             1        2         3
       1st   X        Y         Z                     1 : X>Y>Z
Pref. 2nd    Y        Z         X i.e. Person 2 : Y>Z>X
       3rd   Z        X          Y                    3 : Z>X>Y
                     Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng


    Binary/paired Comparison With Majority rule
   Person
             1   2   3
     1st     X Y      Z             1 : X>Y
Pref. 2nd Y Z         X   for (x,y) 2 : Y>X X>Y
      3rd    Z X       Y            3 : X>Y
 Similarly, for (Y,Z) we can get Y>Z; for (Z,X) we

  can get Z>X.
 Then X>Y>Z>X (cycling) , Intransitive  Not

  rational
                Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




   It was noted by Condorcet in the 18 century
    that no alternative can win a majority against
    all other alternatives.

   Pairwise majority is not satisfactory in all
    cases.
                Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




Example 3 Borda Rule
   For each voter,
       associate the number 1 with the most
        preferred alternative,
       2 with the second and so on,
   Assign to each alternative the number
    equal to
       the sum of the numbers the individual
        voters assigned to the alternative.
                    Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




                   Person
               1       2           3
     1st    X(1)   Y(1)       X(1)            X(4)            X
Pref. 2nd   Y(2)   X(2) W(2)  Y(7)  Y
     3rd    Z(3)   W(3) Z(3)   Z(10)  W
     4th    W(4)   Z(4) Y(4)   W(9)    Z

  Then We get choice X>Y>W>Z
               Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




   For above example, if we use binary/paired
    comparison With majority rule . We can get

    X>Y in 2 out of 3, Y>W in 2 out of 3,
    W>Z in 2 out of 3, X>W in 3 out of 3,
    X>Z in 3 out of 3, Y>Z in 2 out of 3

    Then we can achieve same choice
          X>Y>W>Z
                 Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




   For the previous example we
    had trouble with majority rule
    via binary/paired comparison,
    we get a tie between all
    three alternatives with the
    Borda’s rule:
       All three alternatives get a sum
        of 6.
                 Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




   Some variations
1 with relevant scores available
  allotting each input system a point p to be
  distributed according to relevance scores of the
  documents.

2 Weighted Borda-rule
  Each voter may not have equal effectiveness to
  the final result. We may set more weight to good
  quality input systems.
               Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




   Condorcet winner algorithm


    It also comes from social choice theory. The
    Condorcet algorithm says that any candidate
    that can beat all other candidates in a head-
    to-head contest (pair-wise comparison)
    should win the election.
                  Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




   Step 1, Construct Condorcet Graph.
    For each candidate pair (x,y), there exists an edge
    from x to y if x would receive at least as many votes
    as y in a head-to-head contest.

    In Condorcet graph, there is at least one directed
     edge between every pair of candidates. ( we call the
     graph is semi-complete)

    It may contains cycles in the graph. This is due to
     voting paradox of the condorcet voting.
                Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




   Step 2, We form a new acyclic graph from an
    old cyclic one by contracting all of the nodes
    in a cycle into one. It is a strongly connected
    component graph (SCCG).

Definition of Strongly connected component(SCC):
 A strongly connected subgraph, S, of a directed
  graph, D, such that no vertex or subset of vertices of
  D can be added to S such that the new subgraph is
  still strongly connected.
                  Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




  The graph is totally orderable at the level of the
  SCC’s and each SCC is a “pocket” of cycles,
  within which each candidate is tied.

Step 3, The condorcet-consistent Hamiltonian
  path is any Hamiltonian path through Condorcet
  graph.

Definition Hamiltonian path: A path between two vertices
  of a graph that visits each vertex exactly once.
                    Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




   Theorem 1. Every semi-complete graph
    contains a Hamiltonian path.

   Theorem 2. Suppose x and y are nodes in a
    graph g, and that X and Y are nodes of the
    associated SCCG G such that x  X and y Y.
    If there exists a path from X to Y in G, then
    every Condorcet path of g has x before y.

    Refer to [Javed A. Aslam, Mark Montague 2001] for
    proof.
                         Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




    Rank Aggregation in MetaSearch
    Here we discussed two cases which using
    algorithm rooted at social choice theory for
    MetaSearch rank aggregation.
   Data fusion track in TREC
[Javed A. Aslam, Mark Montague 2001] Models for Metasearch
 in SIGIR2001


   Rank aggregation for web search engine
    [Cynthia Dwork, Ravi Kumar, Moni Naor, D.Sivakumar 2001]
Rank Aggregation Methods for the Web in WWW10
               Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




Data fusion track in TREC
   TREC (Text Retrieval Conference ,see
    http://trec.nist.gov/) maintains about 6Gb of
    SGML tagged text, queries and respective
    answers for evaluation purposes.
   The TREC organizers distribute data sets in
    advance and 50 new queries each year.
   The competing teams then submit ranked
    lists of documents that their system gave in
    response to each query. And these retrieval
    systems will be evaluated.
                 Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




   These ranked lists are available for
    metasearch researchers to download and
    fuse.
   For each query, every retrieval system will
    return top 1000 documents and relevant
    score is available.
   Then given these results retrieved by many
    different retrieval systems, how to
    aggregate them for better performance?
                 Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




Previous algorithms
   Min, Max and Average Models
    [Fox and Shaw,1995]
   Linear Combination Model
    [Bartell 1995]


   Logistic Regression Model
                   Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng


     Example
   Min, Max and Average model
    The final score of each document d is based on
    the scores given to d by each input systems
    (voters).
Algorithm                      Final score
CombMin         minimum of individual relevance scores
CombMed         median of individual relevance scores
CombMax         maximum of individual relevance scores
CombSum        sum of individual relevance scores
CombANZ        CombSum / num non-zero relevance scores
CombMNZ        CombSum * num non-zero relevance scores
                Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




   Linear Combination Model

        S LC (d )   ai si (d )
                        i

    The final score of document d is a simply
     linearly (each weighted differently)
     combining the normalized relevance
     scores given to each document.
                  Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




Experiment result on TREC Model
      The performance of rank aggregation is
       evaluated by average precision over the
       queries
      Score-based borda-fuse is usually the best
       method among several borda variant
       algorithms.
      It is better than best input system over most
       of data collection. Such as TREC3, TREC5
                       Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




       Experiment result II
   The performance of rank aggregation is evaluated by
    average precision over the queries.

   Condorcet-fusion is the only algorithm that , without
    training data, ever matches the performance of the best
    input system over TREC 9.

   Condorcet-fusion seems particularly sensitive to the
    dependence of input systems. If the input systems (voters)
    are too similar, the performance will decrease.
                    Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




Rank aggregation methods for web
 New Challenges: Different from the case in
       TREC data fusion,
        The coverage of various search engine is
         different
        Thus some highly relevant web pages may not
         be ranked by some search engines.
        Therefore, each voter ranks a partial
         candidate list
                         Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




        Preliminaries
   Given a universe U, an ordered list  with
    respect to U is an ordering of a subset S  U,
    i.e.,  [ x1  x2  ...  xd ] ,with each xi  S , and  is
    some ordering relation on S.
   If  contains
       all the elements in U, then it is said to be a full list,
       otherwise it is called partial list.
                     Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




   Distance measures between two full lists with
    respect to a set S
      The Kendall tau distance
       It counts the number of pairwise disagreements between
        two lists.
       The distance is given by

    K ( , )  {(i, j) | i  j,  (i)   ( j), but  (i)   ( j)}
       Normalize it by dividing the maximum possible value
        2
     S /2
                         Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




   Spearman footrule distance

   Given two full lists  and  , the distance is given by

             F ( ,  )  i 1  (i)   (i)
                              s




    Normalize it by dividing the maximum value
                                                                                  2
                                                                              S /2
                     Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




   Distance measures for more than 2 list

    Given several full lists  ,  1,  2 ,...,  k , for instance, the
    normalized Footrule distance of  to  1 , 2 ,...,  k is
    given by
              F ( , 1,  2 ,..., k )  (1 / k )i 1 F ( ,  i )
                                                   k




    If  1 , 2 ,...,  k are partial lists, let U denote the union of
     elements in  1 , 2 ,...,  k and let  be a full list with
     respect to U. Considering the distance between  i
     and the projection of  with respect to  i , we have
     the induced footrule distance

             F ( ,  1 ,..., k )  i 1 F ( | i ,  i )
                                             k
                     Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




   Given one full list and a partial list, the scaled
    footrule distance is given by
          F ' ( ,  )  i  (i) /    (i) / 

   In a similar manner, Kendall tau distance can be
    extended for partial lists and more than two lists.


Example   (1,2,3),   (1,2), 
                             1             2    (2,3),  3   4   5  (3,1)

  then K ( ,  ,  ,...,  )  3
                 1   2     5
                           Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




       Optimal rank aggregation
The question is
 Given (full or partial) lists  1 , 2 ,...,  k , find a  such that 
 is a full list with respect to the union of the elements of
   , ,...,  and  minimizes K ( , , ,...,  )
   1   2     k                                     1   2      k




 The aggregation obtained by optimizing Kendall distance is
  called Kemeny optimal aggregation.
                 Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




   When k>=4,computing the Kemeny
    optimal aggregation is NP-hard.

    (please refer to [Cynthia Dwork, Ravi Kumar, Moni
     Naor, D.Sivakumar 2001] for detailed proof )



    We can use Spearman footrule distance to
    approximate the Kendall distance.
                    Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




   Proposition 1 For any two full lists  ,  ,

       K ( , )  F ( , )  2 K ( , )

   Proposition 2 if  is the Kemeny optimal
    aggregation of full lists  1 ,  2 ,...,  k , and  
    optimizes the footrule aggregation, then

       K ( , 1 ,...,  k )  2 K ( , 1 ,...,  k )
                       Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




   Proposition 3 Given full lists  1 ,  2 ,...,  k if the
    median positions of the candidates in the lists
    form a permutation, then this permutation is a
    footrule optimal aggregation.

   Proposition 4 Footrule optimal aggregation of full
    lists can be computed in polynomial time,
    specifically, the time to find a minimum cost
    perfect matching in a bipartite graph.
                  Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




   The footrule optimal aggregation for full lists can
    be recast as a minimum cost bipartite matching
    problem.

   For partial list, we define the bipartite graph as
    before, except that the weights are given by
    scaled footrule distance .

   The weight W(c,p) of a ranking that places
    element c at position p is

                 W (c, p)  i 1  i (c) /  i  p / n
                                    k
                    Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




      Scaled Footrule aggregation
   As before , we can solve the minimum cost
    maximum matching problem on this bipartite
    graph to obtain the footrule aggregation algorithm
    for partial lists.

   We called this method the scaled footrule
    aggregation (SFO)
               Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng

Locally Kemeny optimal
aggregation
   Since Kemeny optimal aggregation is NP-hard
    for k>=4, Here we introduce locally Kemeny
    optimal aggregation.


   It ensures satisfaction of the extended
    Condorcet principle and yet remains
    computationally tractable.
                      Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




   A full list  is locally kemeny optimal
    aggregation of partial lists  1 , 2 ,...,  k if there
    is no full list  that can be obatined from  by
    a single transposition of an adjacent pair of
    elements and for which
         K ( , 1 ,...,  k )  K ( , 1 ,...,  k )
                 Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




   A local Kemenization of full list  with
    respect to  1 ,  2 ,...,  k is a procedure that
    computes a locally Kemeny optimal
    aggregation of  1 ,  2 ,...,  k that is maximally
    consistent with  .

   This approach also preserves the strengths of
    the initial aggregation 
                     Social Choice and Rank Aggregation in MetaSearch/ Xiaotie Deng




It has following characteristics.
1.   The Condorcet losers receive low rank, while the
     Condorcet winners receive high rank.

2.   The result disagrees with  on the order of any given
     pair (i,j) of elements only if a majority of those  ‘s
     expressing opinions disagree with  on (i,j)

3.   For every 1  d   ,the length d prefix of the output is
     a local Kemenization of the top d elements in  .

								
To top