Data Grids_ D-Libs_ DigitalWell

Document Sample
Data Grids_ D-Libs_ DigitalWell Powered By Docstoc
					Data Grids, D-Libs, DigitalWell
Chicago, December 5, 2006

Jim DeRoest
Director, Streaming Media Technologies
 University of Washington
 ResearchChannel
Director, e-Science Initiatives
 Pacific Northwest Gigapop

Myke Smith
Manager, Streaming Media Technologies
 University of Washington
 ResearchChannel




      THINK FORWARD. THINK RESEARCHCHANNEL.
e-Science
Building a Knowledge and Learning
Cyberinfrastructure



        Science and research are intrinsically a
         global activities.
        Creativity, research, discovery and
         analysis are parallel processes.
        Pervasive connectivity – eliminating
         boarders and boundaries.
e-Science
Building a Knowledge and Learning
Cyberinfrastructure



        The five big ideas …
           • Resource Sharing
           • Secure Access
           • Resource Use
           • The Death of Distance
           • Open Standards

       … CERN GridCafe: gridcafe.web.cern.ch/gridcafe/
e-Science
Building a Knowledge and Learning
Cyberinfrastructure

          Converging infrastructures …
            • Digital Libraries
                • Collection management, taxonomy management, search, and
                  preservation.
            • Science Grids
                • Interoperability, scheduling, latency management, networked
                  instrumentation, object semantics.
            • Telepresence
                • 2D video, 3D rendering, social network environments.

          Connect the dots …
            • Libraries, museums, data, instruments, and people.
e-Science
Building a Knowledge and Learning
Cyberinfrastructure



                        High-performance computing
                        for modeling, simulation, data
                             processing/mining

       Humans
                                                              Instruments for
                                                              observation and
                                                              characterization.

      Individual &                 Global
    Group Interfaces             Connectivity                  Physical World
     & Visualization
                                                         Facilities for activation,
     Collaboration                                         manipulation and
       Services                                                construction




                            Knowledge management
                       institutions for collection building
                       and curation of data, information,       Source: Dan Atkins
                            literature, digital objects


 Grid implies global (international) system for collaboration
    Projects, initiatives, consortia …
        Telepresence

    Access Grid
        http://www.accessgrid.org

    UltraGrid
        http://www.digitalpreservation.gov/index.html

    UIC EVL - Sage, CineGrid
        http://www.evl.uic.edu/index2.php

    ResearchChannel iHDTV N-Way
        http://www.researchchannel.org/tech/ihdtv.asp

    MSR ConferenceXP
        http://research.microsoft.com/conferencexp/
Projects, initiatives, consortia …
     Grids

       Enabling Grids for E-sciencE
            www.eu-egee.org

       Worldwide Universities Network
            wungrid.org

       Open Science Grid
            www.opensciencegrid.org/

       Open Grid Forum
            www.ogf.org

       Pacific Rim Applications and Grid Middleware
        Assembly
            www.pragma-grid.net/
Projects, initiatives, consortia …
    Digital Libraries

       Digital Library Foundation – Aquifer
            http://www.diglib.org/aquifer/

       Library of Congress – NDIIPP
            http://www.digitalpreservation.gov/index.html

       D-Space

       Fedora
            http://www.fedora.info/


       DigitalWell
            http://www.digitalwell.org
 Grid Enabled Digital Asset Management
     What we’ve done so far and why …

 Identify and/or develop constructs for building a
  moldable digital management and preservation
  architecture – Something to build upon …
  • Develop a modular, extensible, scalable digital asset
    management system – DigitalWell Project
  • Investigate Data Grid technologies to address and
    abstract federation, archival, namespace and
    infrastructure issues
  • Integration experiments with heterogeneous Data Grid,
    DAMS and Digital Library environments
 Grid Enabled Digital Asset Management
     Project Focus …

 Federated metadata mapping and synchronization.
   • Discipline specific mapping.
   • Standard base schema – Dublin Core, IEEE LOM, XMF

 20GBps asset capture and transfer rate.
   • Format agnostic, 24x7
   • Staging, analysis, preservation.

 Federated access controls.
 Object relationships, web semantics.
          … Prototype project – Neptune …
               http://www.neptune.washington.edu/
Internet HDTV: Neptune VISIONS’05
Neptune Sensor Network
Neptune Fiber and Power Grid
Visions ‘05 Jason and HD Camera
Neptune VISIONS’05
Data Grid and Metadata
Technologies …

      Grid data movers.
       • SDSC Storage Resource Broker
       • GFarm
       • IBM GPFS

      Metadata mapping and synchronization.
       • OAI
       • MetBroker/MetSOAP
  iHDTV N-Way + Data Grid + DigitalWell
  SC’05 SRB Prototype …


 Collaborative iHDTV Capture/Edit/Search
  • “Worlds Largest TiVo”

 iHDTV N-Way
  • Multipoint uncompressed video conference
  • Australia, UMichigan, UWisc, SURFnet, UWash, Tokyo

 Content delivered via SRB data grid
 Final edit – UWash Kane Studios
  • SC’05 interaction with UW editors via iHD N-Way
Global collaborative HD Editing – SC’05




                                   Uncompressed 1.5Gbps
                                   HD capture and
                                   edit via SRB data grid
                                   conduit.
 Storage Resource Broker Repository Abstraction



         Data Access Methods (Web Browser,WSAPI, OAI-PMH)

                                          Data Collection

      Storage Repository                       Data Grid
• Storage location                 • Logical resource name space
• User name                        • Logical user name space
• File name                        • Logical file name space
• File context (creation date,…)   • Logical context (metadata)
• Access constraints               • Control/consistency constraints
   SRB – Data Grid Federation


          Data Access Methods (Web Brower, WSAPI, OAI-PMH)

         Data Collection A                    Data Collection B

            Data Grid                             Data Grid
• Logical resource name space         • Logical resource name space
• Logical user name space             • Logical user name space
• Logical file name space             • Logical file name space
• Logical context (metadata)          • Logical context (metadata)
• Control/consistency constraints      • Control/consistency constraints
               Access controls and consistency constraints
               on cross registration of digital entities
  e-Science Environment


                Secure, seamless Access to Multiple Resrouces


  Synchronize, Share,                              Metadata, Collections,
  Mirror, Preserve …                               Applications, Rights …


Collaborators
                                     Grid
                                 Interconnect

                    Analysis &       D-Lib                Other Grids
                    Rendering        Sites
DigitalWell Development


  Open Source – Soliciting partners
  Data Grid
    • Federation
    • Investigate storage bandwidth issues for HD

  Authentication/Authorization
    • Shibboleth Integration

  Discovery & Metadata
    • Search, Transcripts, Mapping, Inheritance
www.pnw-gigapop.net
www.researchchannel.org

Contact: Jim DeRoest
deroest@washington.edu




     THINK FORWARD. THINK RESEARCHCHANNEL.