Docstoc

Open Archive Initiative in Europe and Germany

Document Sample
Open Archive Initiative in Europe and Germany Powered By Docstoc
					      Open Archives Initiative
      in Europe and Germany


                    Uwe Müller
     Humboldt University Berlin, Germany
          Electronic Publishing Group
University Library / Computer and Media Service
          u.mueller@cms.hu-berlin.de
                Agenda


        1. Open Archives

        2. The Open Archives Initiative

        3. OAForum: European Activities

        4. DINI: German Activities




Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                Open Archives

         Archive
                   “repository” of digital information
                   text documents, images, revords, audio/video sequences, ...
         Open Archive

                          provides open machine interface for
                           making content externally available

                                  mostly: usage of open
                             standards as exchange protocols

                          not necessarily: open (= free) usage
                            of metadata (and digital objects)


Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                Value of Open Archives




                                          external service
                                                   Interface




                Interface                         Interface                          Interface


       “open” archive A                  “open” archive B                   “open” archive C


Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                Open Archives

              Different types of “archives”
                       scholarly publication server (pre-prints, e-prints)
                       libraries (OPAC, e-journals)
                       museum databases (object metadata)
                       archives (historical documents) and cultural heritage
                       education
              Origin
                       self archiving
                       unclosing existing databases
                       establishing new databases
              “open archives” approach gains popularity
                       cross archive access
                       low cost dissemination of previously “hidden” resources
                       building of new service provision

Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                Open Archives: Problems


         lacking interoperability
                       different metadata standards
                             formats (DC, MAB, Marc ...)
                             interpretations (Creator:
                             Author vs. Artist vs. Photographer)
                       different terminology
                       different languages
                       different access strategies
                       different interfaces / transfer protocols
                       different copyright regulations
         Difficulty to establish joint services based on open
          archives
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                General Search Methods


         Cross Search Approach (e.g. Z39.50)
          1 Request                                           2
                                      service                                archive
          4 Answer                                            3


         Harvest Approach (e.g. DIENST protocol)
          4 Request                                           1
                                      service                                archive
          6 Answer                                            2
                                  5             3


                                  database

Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                Agenda


        1. Open Archives

        2. The Open Archives Initiative

        3. OAForum: European Activities

        4. DINI: German Activities




Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                            The Open Archives Initiative (OAI)

                          Main ideas
                                   world-wide consolidation of scholarly archives
                                   free access on the archives (at least: metadata)
                                   consistent interfaces for archives and service provider
                                   effortless implementation
                                   based on existing standards (e.g. HTTP, XML, DC)
                          Basic functioning

                                                   Requests (based on HTTP)
                   Metadata                                                                             Metadata
                                                                                                        (Documents)
„Service”                                          Metadata (encoded in XML)
                                    Harvester                                           Repository

                  Service Provider                                                                     Data Provider



            Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                OAI: General Assumptions

            exchange metadata, not digital objects themselves
            based on harvest approach (asynchronous)
            two groups of participants
            Data Providers (Open Archives, Repositories)
                   free access of metadata
                   not necessarily: free access and usage of resources
                   easy to implement, low barriers
                             (useable for small institutions)
         Service Providers
                   use OAI interfaces of the Data Providers
                   harvest and store metadata
                   may select certain subsets from Data Providers
                               (set hierarchy, date stamp)
                   may enrich metadata
                   offer (value-added) service on the basis of the metadata
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                 OAI: Technical Model




                                                                                     Provider
                                                                                                e-prints
                                                                                                 e-print




                                                                                      Data
                             Requests:
                                                                        Repository
                              Identify
                              ListMetadataformats




                                                                                     Provider
                              ListSets                                                           Images
                                                                                                 e-print




                                                                                      Data
                              ListIdentifiers
                                                                        Repository
                              ListRecords
                              GetRecord
                 Provider
                 Service




                                                                                     Provider
                                                                                                 OPAC
                                                                                                 e-print




                                                                                      Data
                            Harvester                                   Repository
      Provider
       Data




                             Responses:




                                                                                     Provider
                                                                                                 Museum




                                                                                      Data
                              General information                                                e-print

                              Metadata formats                          Repository
                              Set structure
                              Record identifier




                                                                                     Provider
                                                                                                Archive
                                                                                                e-print




                                                                                      Data
                              Metadata

                                                                        Repository


Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                OAI-Protocol for Metadata Harvesting

              Basics of OAI-PMH
                       protocol based on HTTP
                       request arguments as GET or POST parameters
                       six request types
                       e.g. http://archive.org?verb=ListRecords&from=2003-08-01
                       responses are encoded in XML syntax
                       supports any metadata format (at least: Dublin Core)
              Details of OAI-PMH
                       logical set hierarchy (definition: data providers)
                       date stamps (last change of metadata set)
                       error messages
                       flow control


Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                OAI-Protocol for Metadata Harvesting


         Metadata sets (Records)
                  1. header
                         unique identifier (key for further archive requests),
                         e.g. oai:HUBerlin.de:30000231
                         datestamp, e.g. 2003-08-11
                         logical sets in which the record is contained
                  2. metadata
                         metadata prefix (identifier for metadata format)
                         metadata set (at least: Dublin Core, but arbitrary
                         other metadata formats can be transmitted)




Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                Example: http://edoc.hu-berlin.de/OAI-2.0?
                verb=ListIdentifiers&from=2002-01-03&until=2002-01-08&
                metadataPrefix=oai_dc&set=doctypes:dissertations

       <?xml version="1.0" encoding="UTF-8"?>
       <OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/"
                   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                   xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/
                                        http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd">
         <responseDate>2002-10-22T17:49:49+01:00</responseDate>
         <request verb="ListIdentifiers" from="2002-01-03" until="2002-01-08" metadataPrefix="oai_dc"
                    set="doctypes:dissertations">http://edoc.hu-berlin.de/OAI-2.0</request>
         <ListIdentifiers>
            <header>
               <identifier>oai:HUBerlin.de:3000819</identifier>
               <datestamp>2002-01-08</datestamp>
               <setSpec>doctypes</setSpec>
               <setSpec>doctypes:dissertations</setSpec>
               <setSpec>dnb</setSpec>
               <setSpec>dnb:dnb33</setSpec>
            </header>
            <header>
               <identifier>oai:HUBerlin.de:3000831</identifier>
               <datestamp>2002-01-07</datestamp>
               <setSpec>doctypes</setSpec>
               <setSpec>doctypes:dissertations</setSpec>
               <setSpec>dnb</setSpec>
               <setSpec>dnb:dnb27</setSpec>
            </header>
         </ListIdentifiers>
       </OAI-PMH>
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                OAI: Data Provider Architecture



       OAI request                                                                  Web server
     (HTTP request)                           Programming                    (e.g. Apache, IIS)
                                                extension
                                             (e.g. PHP, Perl)




      OAI response                        SQL                                               DB
     (XML instance)                      request                 SQL-                    response
                                                                Database



                                    OAI Data Provider


Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
              OAI: Service Provider Architecture
         User             Harvester             User



                                  OAI Service Provider
      Service                                                               Scheduler
      module
                                       Normaliser
                                                                            Update
                                                                           mechanism
     Database

                                       XML Parser
                                                                           Flow control
   Dublication
    checker


              Data Provider                 Data DCMI - "Open ArchivesData Provider and Germany"
Uwe Müller, 11.08.2003: Information Technology andProvider            Initiative in Europe
                Problems beyond the OAI-PMH


         agreement on metadata usage (except DC)
                       semantics
                       XML schema
         agreement on set definitions
                       selective harvesting
                       e.g. subject gateways
         definition of rights statements
                       agreement on different right states
                       machine readable information




Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                OAI: Examples / Participants


         Data Provider / Repositories
                   see http://www.openarchives.org/Register/BrowseSites.pl
         Service Provider
                   Repository Explorer:
                   http://oai.dlib.vt.edu/cgi-bin/Explorer/oai2.0/testoai/
                   Cross Archive Searching Service: http://arc.cs.odu.edu/
                   MyOAI: http://www.myoai.org/
                   DINI: http://edoc.hu-berlin.de/e_suche/oai.php
                   Physnet: http://physnet.uni-oldenburg.de/oai/query.php
                   …



Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                Agenda


        1. Open Archives

        2. The Open Archives Initiative

        3. OAForum: European Activities

        4. DINI: German Activities




Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                OAForum: Background

              Project Open Archives Forum
                       European Union Information Society Technologies (IST)
                       Programme
                       accompanying measure
                       project start: October 2001 (duration: 2 years)
                       project partners:
                           UKOLN, University of Bath, United Kingdom
                           I.E.I.-CNR, Pisa, Italy
                           Humboldt University Berlin, Germany
              http://www.oaforum.org/
              Motivation
                       increasing discussion about open archives approach
                       setting up a framework for the approach in general
                       promotion of the open archives approach
                       European view ...
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                OAForum: Project Partners


         UKOLN, Bath
                       various projects contexts of metadata and
                       interoperability, cross searching
                       Renardus project, Schemas, DESIRE
         I.E.I.-CNR, Pisa
                       CYCLADES project, DELOS
                       development of services on top of OAI specification
         Humboldt University, Berlin
                       Dissertation Online, NDLTD
                       DINI workshops on OAI



Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                OAForum: Objectives


         raise and sharpen awareness on the open
          archives approach
                       promote “low barrier” interoperability
                       opening cultural resources
                       detecting potentials for new services
         encourage collaborative development of solutions
                       discussion of problems
                       exchange of experiences and information
         support European liaison with OAI

                     … build community of interest
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                OAForum: Objectives (2)

              Establish an information repository
                       activities related to the open archives approach
                       experiences, developed services (e.g. document delivery,
                       searching, browsing, summarisation, linking)
                       share the database with other organisations (e.g. OAI)
              Validation of European experience concerning
                       implementing and using the OAI-PMH and other similar
                       approaches
                       requirements from implementers and users
              Organisational review and analysis
                       possible business models
                       Intellectual Property Rights
              Dissemination of the open archives approach
                       workshops

Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                OAForum: Ask and Answer Questions

         Not: yet another OAI implementation project …
         supporting activities
                       European initiatives with open archives based approach
         clustering activities
                       existing and new communities
                       IST projects
                       national initiatives
         dissemination activities
                       share experiences on Open Archives
                       investigation of usage: different paradigms
                       global availability
                       share developments

Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                OAForum: Benefits


         tool to reach communities
         bring together what is happening in Europe
         raise awareness on and discuss main issues
                       common terminology on digital repositories
                       metadata / full text harvesting models
                       needs of users and communities
                       advanced services
         make European projects ready for action
                       develop possible solutions
                       establish business models


Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                OAForum: Participants

         Carriers and Users of Open Archives
        Communities                                      Service Provider
                      institutions of cultural                         e-print archives
                      heritage                                         subject gateways with
                      museums                                          aggregating functions
                      European digitising                              value-added services
                      projects
                      scholarly institutions
                      public libraries                    Data Provider
                      special user groups                              existing metadata
                      publishers                                       repositories
                      commercial sector                                new metadata collections
                      education


Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                OAForum: Topics

        1.     Workshops, Distribution
                       distribute information and experiences on technologies, etc.
                       produce interest for topics connected with open archives
                       articles, talks, etc.
        2.     Organisational Evaluation
                       analyse business models
                       tackle issues of copyrights (IPR …)
        3.     Technical Evaluation
                       apply the technical framework of OAI
                       develop an information portal with
                             information on projects, repositories, service provider
                             metadata schemas
                             software and implementations
                       discuss problems of interoperability

Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                OAForum: Workshops, Distribution

        Contact: I.E.I.-CNR, Pisa – Donatella Castelli
         May 2002, Pisa, Italy
                       experiences from the European e-prints community
                       establish the forum
              December 2002, Lisbon, Portugal
                       archives and libraries
                       open access to hidden archives
              March 2003, Berlin, Germany
                       metadata schemas
                       networking multimedia resources
              September 2003, Bath, United Kingdom
                       “In Practice – Best Practice”
                       final workshop, EU review
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                OAForum: Organisational Evaluation

        Contact: UKOLN, Bath – Philip Hunter
         Business models
                       co-operations between data providers to establish service
                       networks
                       metadata exchange between archives and services
                       provision of value-added services (e.g. enrichment of
                       metadata, automatic classification, OpenURL)
              Copyright issues
                       IPR and copyright (influence on producers and distributors)
                       property rights on metadata (collective use of metadata,
                       metadata exchange, agreements with publishers etc.)
                       long-term availability of digital resources
              Discussion group
                       Dennis Nicholson (University Glasgow)
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                OAForum: Technical Evaluation

        Contact: Humboldt University, Berlin – Susanne Dobratz
         Interoperability
                       integration of new (open archive) approaches with existing
                       technologies
                       Is unqualified Dublin Core sufficient?
              Issues on database management
                       concurrency and update problems
                       scalability
                       de-dublication
              Software and tools
                       collect and share experiences and existing solutions
                       estimation of necessary expenditure
                             establish data provider / service …
                             skills, manpower, time
              Tutorials on OAI technologies
                       online tutorial – will be presented at Bath Workshop
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                     OAI Activities in Europe
 Overview of OAI activity (continents)
           60
                53
                                                                                 (numbers from November 2002)
60
50                                         Europe
40                                         America
30                                         Australia
20                                         Asia
                      4    2
10
 0                                                                                                            UK
                                    Overview of european countries engaged in OAI implementation
                                                                                                              Germany
                                                                                                              France
                               16     15
                                                                                                              Sweden
                               14                                                                             Italy
                                             12
                                                                                                              Netherlands
                               12
                                                                                                              Austria
                               10                                                                             Finland
                                                                                                              Belorussia
                               8                     7   7
                                                             6                                                Belgium
                               6                                                                              Denmark

                               4                                 3                                            Ireland
                                                                     2   2                                    Norway
                               2                                             1    1   1   1   1   1   1   1
                                                                                                              Portugal

                               0                                                                              Russia
                                                                                                              Switzerland

     Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
http://www.oaforum.org/resources/tecvalq2.php




   Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
http://www.oaforum.org/resources/glossary.php




   Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
http://www.oaforum.org/oaf_db/register/




Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
www.oaforum.org/oaf_db/list_db/list_services.php




     Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                Agenda


        1. Open Archives

        2. The Open Archives Initiative

        3. OAForum: European Activities

        4. DINI: German Activities




Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                DINI recommendations for usage of
                OAI-PMH

              created by DINI-OAI working group
              http://www.dini.de/
              target: agreement on syntax and semantics of OAI set
               definitions for German data and service providers
              enhance retrieval quality and support subject gateways (e.g.
               Physnet, Dissertation search engine, ...)
              definition of three classification types
                       subjects (according to DNB)
                       formal publication types (e.g. dissertation)
                       formal document types (e.g. text, audio)
              example service provider based on recommended sets:
               http://edoc.hu-berlin.de/e_suche/oai.php


Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                Classification according to subjects

        SetSpec                   SetName

        dnb:01                    Knowledge and Culture in General
        dnb:02                    Books and Libraries, Information and Documentation
        dnb:03                    Reference Books, Bibliographies
        dnb:04                    Directories and Phone Books
        dnb:05                    Calendars
        dnb:06                    Journalism
        dnb:07                    Children's and Youth Literature
        dnb:08                    Comics, Cartoons, Caricatures Miscellanea
        dnb:09                    Esoterica Manuscripts, Book Art
        dnb:10                    Philosophy
        dnb:11                    Psychology
        dnb:12                    Christianity
        dnb:13                    General and Comparative Theology, Non-Christian Religion
        dnb:14                    Sociology, Sociography
        ...                       ...
        dnb:65                    Economic History

Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                Classification according to formal
                publication types

        SetSpec                                SetName

        pub-type:monograph                     Books, Monographs
        pub-type:article                       Journal Articles
        pub-type:dissertation                  Dissertations and Professional Dissertations
        pub-type:masterthesis                  Diploma Theses
        pub-type:report                        Report
        pub-type:paper                         Paper
        pub-type:conf-proceeding               Conference Proceedings
        pub-type:lecture                       Lectures
        pub-type:music                         Music
        pub-type:program                       Programs
        Pub-type:play                          Play
        Pub-type:news                          News
        Pub-type:standards                     Standards
Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                Classification according to formal
                document types

        SetSpec                                SetName

        doc-type:text                          Text
        doc-type:notes                         Notes
        doc-type:image                         Image
        doc-type:audio                         Audio
        doc-type:video                         Video
        doc-type:multimedia                    Multimedia
        doc-type:data                          Data
        doc-type-binary                        Binary data, (executable) program



Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                Multiple Data and Service Providers

                                    Data providers




                                                                                    Harvesting
                                                                                    based on
                                                                                    OAI-PMH




                                    Service providers

Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                Aggregators – Example: HBZ Köln

                                    Data providers




                                                                                     Aggregator




                                    Service providers

Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                Hybrid Search – Example: Metalib at HU

                                    Data providers


                                                                                      Harvesting
                                                                                      based on
                                                                                      OAI-PMH



                                                                                   Searching
                                                                                   based on
                                                                                   Z39.50 or
                                                                                   SRW



                                    Service providers

Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                A German OAI Example: ProPrint

              project of SUB Göttingen and CMS of HU Berlin
              duration: 2000 – 2003
              target: integrate heterogenous document servers in order to
               provide PoD service
              components:
                       search engine for documents
                       PDF documents preview
                       generation of compound PDF file with front page and table of
                       contents
                       production and delivery by selected print service provider
              underlying technology: extension of Dublin Core and OAI-
               PMH (with an extension for document exchange)
              http://www.proprint-service.de/

Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                Thank you …


        Questions?




                                                                                               Uwe Müller
                                                                     Humboldt University Berlin, Germany
                                                                           u.mueller@cms.hu-berlin.de



Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"
                Additional Information
              Open Archives Initiative
                  http://www.openarchives.org/
                  http://www.openarchives.org/OAI/openarchivesprotocol.html (OAI-
                  PMH)
                  http://www.openarchives.org/service/listproviders.html (Service
                  Providers)
                  http://www.openarchives.org/Register/BrowseSites.pl (Data
                  Providers)
                  http://www.openarchives.org/tools/index.html (Tools, ...)
              Open Archives Forum
                  http://www.oaforum.org/
                  http://www.oaforum.org/workshops/bath_invitation.php
                  (workshop, Bath, September 2003)
                  http://www.oaforum.org/resources/tecvalq2.php (Technical
                  Validation Questionnaire)
              DINI
                  http://www.dini.de/
                  http://edoc.hu-berlin.de/e_suche/oai.php (OAI search engine)

Uwe Müller, 11.08.2003: Information Technology and DCMI - "Open Archives Initiative in Europe and Germany"

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:4
posted:7/2/2012
language:English
pages:46