Data Grids_ D-Libs_ DigitalWell

Document Sample
Data Grids_ D-Libs_ DigitalWell Powered By Docstoc
					Data Grids, D-Libs, DigitalWell
Chicago, December 5, 2006

Jim DeRoest
Director, Streaming Media Technologies
 University of Washington
Director, e-Science Initiatives
 Pacific Northwest Gigapop

Myke Smith
Manager, Streaming Media Technologies
 University of Washington

Building a Knowledge and Learning

        Science and research are intrinsically a
         global activities.
        Creativity, research, discovery and
         analysis are parallel processes.
        Pervasive connectivity – eliminating
         boarders and boundaries.
Building a Knowledge and Learning

        The five big ideas …
           • Resource Sharing
           • Secure Access
           • Resource Use
           • The Death of Distance
           • Open Standards

       … CERN GridCafe:
Building a Knowledge and Learning

          Converging infrastructures …
            • Digital Libraries
                • Collection management, taxonomy management, search, and
            • Science Grids
                • Interoperability, scheduling, latency management, networked
                  instrumentation, object semantics.
            • Telepresence
                • 2D video, 3D rendering, social network environments.

          Connect the dots …
            • Libraries, museums, data, instruments, and people.
Building a Knowledge and Learning

                        High-performance computing
                        for modeling, simulation, data

                                                              Instruments for
                                                              observation and

      Individual &                 Global
    Group Interfaces             Connectivity                  Physical World
     & Visualization
                                                         Facilities for activation,
     Collaboration                                         manipulation and
       Services                                                construction

                            Knowledge management
                       institutions for collection building
                       and curation of data, information,       Source: Dan Atkins
                            literature, digital objects

 Grid implies global (international) system for collaboration
    Projects, initiatives, consortia …

    Access Grid

    UltraGrid

    UIC EVL - Sage, CineGrid

    ResearchChannel iHDTV N-Way

    MSR ConferenceXP
Projects, initiatives, consortia …

       Enabling Grids for E-sciencE

       Worldwide Universities Network

       Open Science Grid

       Open Grid Forum

       Pacific Rim Applications and Grid Middleware
Projects, initiatives, consortia …
    Digital Libraries

       Digital Library Foundation – Aquifer

       Library of Congress – NDIIPP

       D-Space

       Fedora

       DigitalWell
 Grid Enabled Digital Asset Management
     What we’ve done so far and why …

 Identify and/or develop constructs for building a
  moldable digital management and preservation
  architecture – Something to build upon …
  • Develop a modular, extensible, scalable digital asset
    management system – DigitalWell Project
  • Investigate Data Grid technologies to address and
    abstract federation, archival, namespace and
    infrastructure issues
  • Integration experiments with heterogeneous Data Grid,
    DAMS and Digital Library environments
 Grid Enabled Digital Asset Management
     Project Focus …

 Federated metadata mapping and synchronization.
   • Discipline specific mapping.
   • Standard base schema – Dublin Core, IEEE LOM, XMF

 20GBps asset capture and transfer rate.
   • Format agnostic, 24x7
   • Staging, analysis, preservation.

 Federated access controls.
 Object relationships, web semantics.
          … Prototype project – Neptune …
Internet HDTV: Neptune VISIONS’05
Neptune Sensor Network
Neptune Fiber and Power Grid
Visions ‘05 Jason and HD Camera
Neptune VISIONS’05
Data Grid and Metadata
Technologies …

      Grid data movers.
       • SDSC Storage Resource Broker
       • GFarm
       • IBM GPFS

      Metadata mapping and synchronization.
       • OAI
       • MetBroker/MetSOAP
  iHDTV N-Way + Data Grid + DigitalWell
  SC’05 SRB Prototype …

 Collaborative iHDTV Capture/Edit/Search
  • “Worlds Largest TiVo”

 iHDTV N-Way
  • Multipoint uncompressed video conference
  • Australia, UMichigan, UWisc, SURFnet, UWash, Tokyo

 Content delivered via SRB data grid
 Final edit – UWash Kane Studios
  • SC’05 interaction with UW editors via iHD N-Way
Global collaborative HD Editing – SC’05

                                   Uncompressed 1.5Gbps
                                   HD capture and
                                   edit via SRB data grid
 Storage Resource Broker Repository Abstraction

         Data Access Methods (Web Browser,WSAPI, OAI-PMH)

                                          Data Collection

      Storage Repository                       Data Grid
• Storage location                 • Logical resource name space
• User name                        • Logical user name space
• File name                        • Logical file name space
• File context (creation date,…)   • Logical context (metadata)
• Access constraints               • Control/consistency constraints
   SRB – Data Grid Federation

          Data Access Methods (Web Brower, WSAPI, OAI-PMH)

         Data Collection A                    Data Collection B

            Data Grid                             Data Grid
• Logical resource name space         • Logical resource name space
• Logical user name space             • Logical user name space
• Logical file name space             • Logical file name space
• Logical context (metadata)          • Logical context (metadata)
• Control/consistency constraints      • Control/consistency constraints
               Access controls and consistency constraints
               on cross registration of digital entities
  e-Science Environment

                Secure, seamless Access to Multiple Resrouces

  Synchronize, Share,                              Metadata, Collections,
  Mirror, Preserve …                               Applications, Rights …


                    Analysis &       D-Lib                Other Grids
                    Rendering        Sites
DigitalWell Development

  Open Source – Soliciting partners
  Data Grid
    • Federation
    • Investigate storage bandwidth issues for HD

  Authentication/Authorization
    • Shibboleth Integration

  Discovery & Metadata
    • Search, Transcripts, Mapping, Inheritance

Contact: Jim DeRoest