Docstoc

simon

Document Sample
simon Powered By Docstoc
					         On the Epistemic Value of Reputation:
The Place of Ratings and Reputational Tools in Knowledge
                      Organization

               11th International ISKO Conference
                  Rome, February 23-26 2010



                Gloria Origgi & Judith Simon
                        Institut Jean Nicod
                       ENS-EHESS-CNRS
                          Paris, France
REPUTATION: Overview


    • Background
    • Introduction
    • Reputation as Evaluative Social Information
    • A Rational Model for the Epistemic Use of
      Reputation
    • Reputational Tools on the Web
    • Problems with the Epistemic Use of Reputation
    • Conclusions
REPUTATION: Overview


   • Guiding Questions

      – How to use reputation for epistemic purpose?
      – What‟s the epistemic value of reputation?
      – Is this a good thing or a bad thing?
REPUTATION: Background: LiquidPub


                                    Different methods of
                                    quantifying, assessing &
                                    propagating reputation




 Further Information on:
 http://project.liquidpub.org
 http://liquidpub.wordpress.com
REPUTATION: Background


                               Thesis:
                              Types of
                         Epistemic Sociality
REPUTATION: Overview


    • Background
    • Introduction
    • Reputation as Evaluative Social Information
    • A Rational Model for the Epistemic Use of
      Reputation
    • Reputational Tools on the Web
    • Problems with the Epistemic Use of Reputation
    • Conclusions
REPUTATION: Introduction

                                                                            Hero

                                                                 Sinner
                                                                               Drunkard




                                                                Reputation as Heuristic:

                                                                Reputation as a way to classify social types
                                                                within the community that will allow its
                                                                member to manage their relations with
                                                                others, to make inferences and predictions
                                                                about their behavior, i.e. to construct a basic
Source:
http://en.wikipedia.org/wiki/File:Matteson_Scarlet_Letter.jpg   "social map" that will help them orient in their
http://en.wikipedia.org/wiki/The_Scarlet_Letter                 society.
REPUTATION: Introduction

                                                                            Hero

                                                                 Sinner
                                                                               Drunkard




                                                                Reputation as Heuristic:

                                                                Reputation as a way to classify social types
                                                                within the community that will allow its
                                                                member to manage their relations with
                                                                others, to make inferences and predictions
                                                                about their behavior, i.e. to construct a basic
Source:
http://en.wikipedia.org/wiki/File:Matteson_Scarlet_Letter.jpg   "social map" that will help them orient in their
http://en.wikipedia.org/wiki/The_Scarlet_Letter                 society.
REPUTATION: and Knowledge Organization


    • Reputation as social information about the value of
      people, systems and processes that release
      information.
    • Focus: relationship between this special form of
      social information and the processes of knowledge
      organization and evaluation.
    • More precisely, we argue not only that
       – (1) we make use of other people's reputations to evaluate
         information in various ways
       – (2) within systems, like the Web, that make possible the
         easy and dynamic organization and re-organization of
         knowledge, we also use our own rankings to determine new
         content and generate new categories.
REPUTATION: Overview


    • Background
    • Introduction
    • Reputation as Evaluative Social Information
    • A Rational Model for the Epistemic Use of
      Reputation
    • Reputational Tools on the Web
    • Problems with the Epistemic Use of Reputation
    • Conclusions
REPUTATION: as Evaluative Social Information


    • Reputation is the informational track of our past
      actions, it is the credibility that an agent or an item
      earn through repeated interactions.
    • Reputational Cues are indicators/proxies of
      reputation where quality of objects or agents cannot
      be directly assessed
    • Relying on reputational cues is an efficient way of
      shaping the too rich informational landscape around
      us by creating new relevant categories.
REPUTATION: as Evaluative Social Information


    • In an information-dense environment, where sources
      are in constant competition to get attention and the
      option of the direct verification of the information is
      often simply not available at reasonable costs,
      evaluation and rankings are epistemic tools and
      cognitive practices that provide an inevitable shortcut
      to information

    • Modest Prediction: The higher the uncertainty on the
      content of information, the stronger is the weight of
      the opinions of others in order to establish the quality
      of this content.
REPUTATION: Overview


    • Background
    • Introduction
    • Reputation as Evaluative Social Information
    • A Rational Model for the Epistemic Use of
      Reputation
    • Reputational Tools on the Web
    • Problems with the Epistemic Use of Reputation
    • Conclusions
REPUTATION: A Model for the Epistemic Use of Reputation

      Lehrer & Wagner (1981) “Rational Consensus in
      Science and Society”

    • Proposes a model for rational decision making
      processes in science, society and the arts that
      makes epistemic use of reputation

    • It rests upon the employment of consensual
      probabilities, utilities and weights

    • For decision making processes to be rational, it is
      central that all evidence or empirical information
      available for the topic of concern has to be used

    • Experimental + Social Information
REPUTATION: A Model for the Epistemic Use of Reputation


    • Social information = information about the expertise of
      other experts on issues at hand = Reputation

    • Example: Expert Dilemma: Do we need to vaccinate
      large parts of the population to prevent a pandemie?
       – Step 1: each expert gives a weights other experts‟
         competency
       – Step 2: weights are laid open
       – Step 3: revision of own weights taking the others„ assessment
         into account depending on the weights assigned to them
       – Repeat cycle till consensus is achieved…

    • Once these consensual weights are achieved, they
      can be applied to answering the question of concern
      by weighting each member‟s votes on the issue with
      their consensual personal weight.
REPUTATION: A Model for the Epistemic Use of Reputation


    • Lehrer & Wagner propose a model of how to
      rationally reach consensus that rests upon the
      epistemic use of reputation

    • This implies that reputational information, i.e. social
      information about other people that is evaluative, is
      epistemically useful.

    • Do we need such formal models?

    • Epistemic use of reputational cues does not have to
      follow such a formal method. But on the Web,
      models similar to this one are embedded and hidden
      within different applications.
REPUTATION: Overview


    • Background
    • Introduction
    • Reputation as Evaluative Social Information
    • A Rational Model for the Epistemic Use of
      Reputation
    • Reputational Tools on the Web
    • Problems with the Epistemic Use of Reputation
    • Conclusions
REPUTATION: Reputational Tools on the Web




  http://www.briansolis.com/2008/08/introducing-conversation-prism/
REPUTATION: Reputational Tools on the Web


    • What the Web makes possible today is an algorithmic treatment
      of methods of gathering social information to extract knowledge.
      Ratings and rankings on the Web are the result of collective
      human registered activities with artificial devices.

    • However, the control of the heuristics and techniques that
      underlie this dynamics of information may be out of sight or
      incomprehensible for the users who find themselves in the very
      vulnerable position of relying on external sources of information
      through a dynamic, machine-based channel of communication
      whose heuristics and biases are not under their control.

    • Thus, the reputational tools that are proliferating on the Web
      should be scrutinized by epistemically responsible users who do
      not want to accept too naïvely the outcome of a process they do
      not control.
REPUTATION: Reputational Tools on the Web
REPUTATION: Reputational Tools on the Web
REPUTATION: Reputational Tools on the Web
REPUTATION: Reputational Tools on the Web




                                       Interestingness!
                           “There are lots of elements that make
                          something 'interesting' (or not) on Flickr.
                       Where the clickthroughs are coming from; who
                       comments on it and when; who marks it as a
                       favorite; its tags and many more things which
                          are constantly changing. Interestingness
                           changes over time, as more and more
                         fantastic content and stories are added to
                                            Flickr.”

                                http://www.flickr.com/explore/interesting/
REPUTATION: Reputational Tools on the Web




                                                Interestingness!
                                  “There are lots of elements that make
                                 something 'interesting' (or not) on Flickr.
                              Where the clickthroughs are coming from; who
                               comments on it and when; who marks it as a
                              favorite; its tags and many more things which
                                 are constantly changing. Interestingness
   Interestingness is a new       changes over time, as more and more
category based on reputational fantastic content and stories are added to
                                                   Flickr.”
  mechanisms, making use of
 different proxies whose weight          http://www.flickr.com/explore/interesting/
and combination is not obvious!
REPUTATION: Reputational Tools on the Web


    • Reputational tools get more and more central on the Web
    • Rankings and Ratings provide new arrangements of information
    • Early years of 2000: focus on personalized information (My-
      Features)
    • Now: trend towards systems of shared preferences, were
      people can rely on others‟ preferences and rankings to construct
      there own access to and categorization of information
    • Examples:
       – Flickr‟s Interestingess
       – Twitter-Logic of Followers and Leaders
       – LiquidJournal: people or groups create their own journals by selecting
         (existing) content and making it available via their selection
REPUTATION: Problems for the Epistemic Use of Reputation




                 So, all is well, or?
REPUTATION: Problems for the Epistemic Use of Reputation




                 So, all is well, or?
                 Well, not quite…
REPUTATION: Overview


    • Background
    • Introduction
    • Reputation as Evaluative Social Information
    • A Rational Model for the Epistemic Use of
      Reputation
    • Reputational Tools on the Web
    • Problems with the Epistemic Use of Reputation
    • Conclusions
REPUTATION: Problems for the Epistemic Use of Reputation


    1. The danger of misuse of reputation: danger of
       epistemic injustice (Fricker 2007), judging epistemic
       credibility and social identity (Alcoff 2001)

       –   Using proxies that are not valid to assess the reputation
           and epistemic credibility of epistemic agents (gender, race,
           nationality, institutional background,…)

           “testimonial injustice occurs when prejudice causes a
           hearer to give a deflated level of credibility to a speaker‟s
           word” ((Fricker 2007) 1)
REPUTATION: Problems for the Epistemic Use of Reputation


    2. Limits of the epistemic usefulness of reputation itself
       –   How to calculate reputational values in the first place?
       –   What are the pros and cons of different methods: e.g. peer
           review versus Amazon-type ratings?
       –   Which proxies should be used and how should they be
           combined?
       –   Stability of reputation over time?
       –   Transferability of reputation over domains?

    3. Lack of transparency of reputational algorithms and
        metrics
       –   How should users be responsible knowers if they do not
           understand the functioning, the strengths and weaknesses
           of different mechanisms?
       –   How to detect biases, if the mechanisms are not laid open?
       –   Need to make these mechanisms visible and
           understandable
REPUTATION: Overview


    • Background
    • Introduction
    • Reputation as Evaluative Social Information
    • A Rational Model for the Epistemic Use of
      Reputation
    • Reputational Tools on the Web
    • Problems with the Epistemic Use of Reputation
    • Conclusions
REPUTATION: Conclusions

    • Ratings and reputational tools in knowledge
      organization have epistemological, practical as well
      as ethical implications.
       – Epistemological questions: How epistemically warranted is the use
         of these tools?

       – Practical questions: How to develop these mechanisms? Which
         proxies to use, how to combine and weigh them? What‟s the status
         of these new types of classes, such as interestingness? Can
         ratings and ranking serve as middle-ground categorizations?

       – Ethical and political question: Epistemic injustices & lack of
         transparency: Once reputation mechanisms become formalized
         and are embedded within tools, there is a clear danger that
         epistemic injustices are inscribed in and reinforced by technology.
REPUTATION: Conclusions

    • What is the epistemic values of reputation? Is it useful? Or
      rather dangerous?
REPUTATION: Conclusions

    • What is the epistemic values of reputation? Is it useful? Or
      rather dangerous?

    • Both - it is useful and dangerous. But either way, reputational
      information, different reputational proxies and methods of
      quantifying and combining them are being used extensively on
      the Web and elsewhere.
REPUTATION: Conclusions

    • What is the epistemic values of reputation? Is it useful? Or
      rather dangerous?

    • Both - it is useful and dangerous. But either way, reputational
      information, different reputational proxies and methods of
      quantifying and combining them are being used extensively on
      the Web and elsewhere.

    • An additional problem on the Web concerns the lack of visibility:
      for the users the metrics and algorithms behind different
      reputation tools are often unknown.
REPUTATION: Conclusions

    • What is the epistemic values of reputation? Is it useful? Or
      rather dangerous?

    • Both - it is useful and dangerous. But either way, reputational
      information, different reputational proxies and methods of
      quantifying and combining them are being used extensively on
      the Web and elsewhere.

    • An additional problem on the Web concerns the lack of visibility:
      for the users the metrics and algorithms behind different
      reputation tools are often unknown.

    • There is an epistemic duty of epistemologists and knowledge
      organization scholars to thoroughly analyze these different
      reputational practices from epistemological, ethical and political
      perspectives.
REPUTATION




                       Thank you for your attention!


 Contact
 Judith Simon
 Institut Jean Nicod
 Ecole Normale Supérieure
 29, rue d'Ulm
 F-75005 Paris
 email: judith.simon@ens.fr
 www: http://www.institutnicod.org
 tel: +33 (0) 1 443 22 6464
 fax: +33 (0) 1 443 22 699
REPUTATION: Problems for the Epistemic Use of Reputation


    • Two major problems of using reputation for
       epistemic purpose
    1) the use of reputation to assess content can be
         epistemically beneficial while being morally
         questionable
    2) limits of the epistemic usefulness of reputation itself
REPUTATION: as Evaluative Social Information


    • We want to explore the epistemic value of reputation,
      while being aware of the ethical and political
      problems that might come with using it for epistemic
      purpose.

    • Using the judgment on past records to classify an
      agent or an item can be epistemologically useful in
      the absence or - as is especially relevant today -
      overabundance of information. But it has to be and
      remain open to constant scrutiny and revision to be
      epistemically useful and ethically just.
REPUTATION: Reputational Tools on the Web


    • Early years of 2000: focus on personalized information (My-
      Features)
    • Now: trend towards systems of shared preferences, were
      people can rely on others‟ preferences and rankings to construct
      there own access to and categorization of information
    • Examples:
       – Flickr‟s Interestingess
       – Twitter-Logic of Followers and Leaders
       – LiquidJournal: people or groups create their own journals by selecting
         (existing) content and making it available via their selection
REPUTATION: Background


    • Epistemic Use and Value of Reputation as ongoing
      inquiry by two authors fuelled by different sources
REPUTATION: Background




                          Two authors -
                         Two perspectives
REPUTATION: Conclusions

    • What is the epistemic values of reputation? Is it useful? Or
      rather dangerous?

    • Both - it is useful and dangerous.
REPUTATION: Conclusions

    • What is the epistemic values of reputation? Is it useful? Or
      rather dangerous?

    • Both - it is useful and dangerous. But either way, reputational
      information, different methods or reputational cues of assessing
      it are being used extensively on the Web and elsewhere.

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:15
posted:11/29/2011
language:English
pages:44