CADAL Digital Library

Document Sample
CADAL Digital Library Powered By Docstoc
					           The 2nd International Conference
             on Universal Digital Library
                      (ICUDL 2006)


  CADAL Digital Library

     Wu Jiang-Qin,Zhuang Yue-Ting 
              Pan Yun-he
 College of Computer Science, Zhejiang 
            University,China
          November 18,2006  
Outline
          1
          1        Introduction

              2
              2     Unified Paralleling Search

                  3 Multimedia Analysis and
                  3
                     Retrieval

                   4 Bilingual services
                   4

                  5
                  5 Chinese Calligraphy
                     Character Retrieval
              6
              6   Conclusion and Future Work
Outline
          1
          1        Introduction

              2
              2     Unified Paralleling Search

                  3 Multimedia Analysis and
                  3
                     Retrieval

                   4 Bilingual services
                   4

                  5
                  5 Chinese Calligraphy
                     Character Retrieval
              6
              6   Conclusion and Future Work
    CADAL
o   The China-Us Million Book Digital
    Library(CADAL) is an international
    cooperation program between China and
    the US.
o   The objective of CADAL project , is to
    create a free-to-read, searchable
    collection of one million book, available to
    everyone over the internet.

o    CADAL is the important part of Universal
    Digital Library(UDL), universal access to
    human knowledge.
    The challenges and services (1)
o   the amount of the digital resources including
    digital books and multimedia for research
    and education can reach 100 terabyte(The
    number of digital books is 1,023,425 by
    October of 2006,including previous Chinese
    ancient books, Chinese minguo books ,Chinese
    Modern books, Chinese degree
    dissertation,English books,image,video etc..
o   active services of unified paralleling search
    for the different types of digital resources
    The challenges and services (2)

o   image, video,3-D model and other types of
    media resources, various types of media
    resources are included in the CADAL
    resources.
o   the services of quickly retrieving and
    structurally browsing of multimedia
    documents including image, video
    The challenges and services (3)

o   there are two kinds of language digital
    books. Chinese and English, in the CADAL
    resources.
o    the services of bilingual translation
    The challenges and services (4)

o   traditional Chinese culture resources are
    important part of the CADAL resources.
o   the services related to Chinese traditional
    culture resources.
Outline
          1
          1        Introduction

              2
              2     Unified Paralleling Search

                  3 Multimedia Analysis and
                  3
                     Retrieval

                   4 Bilingual services
                   4

                  5
                  5 Chinese Calligraphy
                     Character Retrieval
              6
              6   Conclusion and Future Work
    Background
o   TB volume of various types of digital
    resources, such as dissertation, ancient minguo
    book, modern book, minguo journal, English
    book, drawing, video and illustration are
    available in the CADAL, which is one of the
    distinct characteristic of CADAL. So CADAL
    presents a challenge for the technique of
    searching resources based on metadata.
    Metadata
o   Dublin core metadata is used to
    describe the million digital books in
    the   CADAL    project.    Metadata
    corresponding to the other types of
    multimedia resources are used to
    describe them. Independent data
    map is designed for each kind of
    resource metadata.
    Unified parallel searching
o   In order to meet the requirements
    of different users and improve the
    user’s interactive experience, the
    service for the different types of
    digital resources is provided for
    users’ convenient searching.
Outline
          1
          1        Introduction

              2
              2     Unified Paralleling Search

                  3 Multimedia Analysis and
                  3
                     Retrieval

                   4 Bilingual services
                   4

                  5
                  5 Chinese Calligraphy
                     Character Retrieval
              6
              6   Conclusion and Future Work
    Background
o   As the digital library contains unstructured
    multimedia resources such as images,
    videos, audios etc besides digital books,
    effective and efficient analysis and retrieval
    of multimedia resources is a challenging
    problem in the CADAL digital library.
o   Here we examine the analysis and retrieval
    issues related to two primary kinds of
    multimedia, image and video.
    Contents
o   Content-based Image Retrieval
o   Image retrieval by peer indexing
o   Image annotation
o   Image search engine
o   Video analysis system
o   Video Browser(structure and
    summary)
o   Metadata-based Video Retrieval
        Content based image retrieval

o   Extracting visual features
    n   color feature:color histogram, color moment,
        color coherence vector, color correlgram
    n   texture:Tamura textural feature and co-
        occurrence textural feature


o   relevance feedback
    n   Make image retrieval coincide with
        user’s requirement
        Content based image retrieval



                   Query example



                      Negative example

                                         Relevance
   Image                                 feedback
   searching



Positive example
    Image retrieval by peer index
o   A new scheme for image indexing, Peer Index,
    is the method that describe images through
    semantically relevant peer images.
o   In particular, each image is associated with a
    two-level peer index, including
    n   global    peer    index:    describing  the    “data
        characteristics” of this image
    n   personal peer indexes: describing the “user
        characteristics” of an individual user with respect to
        this specific image
o   Both types of peer index are learned
    interactively and incrementally from user
    feedback information.
       Peerindex-based image retrieval

                           semantic relevance
                           feedback




Semantic query
    Image annotation
o   Automatic semantic annotation for images by machine
    learning and statistical modeling
    n  Classify the training images, and create a semantic
       skeleton for each class of the training image.

    n   Classify new image with Support Vector Machine
        automatically, and describe it using the semantic
        skeleton

    n   Select the key words for the image by statistical
        methods
Image annotation         images

            ............
 Images
                     images
          classify                     statistical
                                       learning

                                                                   Semantic
                                                          Image
                                    annotation                     skeleton
                                                           blobs

                                     tige
                                     r




                         annotate

              classify                           Visual similar
              segment
Text based image retrieval
              Query text:bonsai
    Image search engine
o   We implemented an image search engine,
    Octopus, which provides Peer Index and
    relevance feedback to avoid the gap
    between the semantics and low-level
    features, according to the intuitive and
    simple idea that the semantic concept is
    hidden in each image and the semantic
    concept appears apparently in the relation
    between the image and the other images.
       Integrating into CADAL DL
                                          Other
                                          images




                                          images

                WWW




       Books          scanner
                                          images                 Image
                                                                 Manager

                                                                      store
       Browse               CADAL portal
       Retrieve
       Relevance feedback                          CADAL image repository
                              Image
                              retrieval
user                          system                images     metadata       feature
The image retrieval interface
     Our target for video
o    Analyze multimodal information, 
     such as the visual, the audial, 
     motion and caption to  generate 
     structural information and video 
     summary
  
o    Support video browsing and video 
     retrieval based on metadata and
     structural information efficiently
    Main idea

o   Nonlinear browsing:Generate structural
    indexing such as key frame, shot and shot group
    from the original video stream


o   Content compression:Analyze time
    sequence in video stream, eliminate redundant
    data, and generate the summary and the highlight
    scene for the original video.
    System
o   Video Fusion Analysis System (VideoFAS)
o   VideoBrowser
                                                        CADAL
                                                        portal


    Video data
                              Video fusion
                              analysis
                              system

                                                        Video     user
                    CADAL video repository              browser



                   video         Metadata     Feature
                 repository      database    database
VideoFAS-system interface
                               Original Video



      Similar Video shots
      are Clustered together




                 Video shot
    VideoFAS-system functions

o   Basic operation
    n   Importing and Saving

    n   Appending

    n   Separating the video stream into video and
        audio data

    n   transcoding and compressing
        VideoFAS-system functions
o   Feature Extraction
    n    Visual feature
        o  color :color histogram, color moment, color
            coherence vector, color correlgram
        o   Texture :Tamura textural feature and co-
            occurrence textural feature
        o  shape:contour feature
    n    Audial feature
        o  temporal feature:zero-crossing rate
        o  Frequency feature:Mel coefficient、tone and
            sub-band statistical feature
    n   Target Feature
        o   Integrate OpenCV face detection module into the
            system Extract the face features
    VideoFAS-system functions
o   Video structuring
    n   shot detection
        o   Cut shot detection
        o   Transition shot detection


    n   key frame extraction

    n   Similar shot grouping
        o   group the shots based on Support Vector Machine
Original Video Shot Sequence




                               Video Shot Clustering




Video Shot Cluster A



Video Shot Cluster B



Video Shot Cluster C



Video Shot Cluster D



Video Shot Cluster E
        VideoFAS-system functions
o   Video summarization
    n     Summarize by Mining Non-Trivial Repeating Patterns
         o  Extract frequent and non-trivial shot sequence to
            generate video summary
        Original Video Shot Sequence




         A          B       C          D    A         C   D     E     A   B    C     D




                                A          B      C       D
    VideoFAS-system functions
o   Metadata annotation
    n   Annotate Video clip with metadata conform
        to Dublin Core Standard



    n   Save the metadata and the video structural
        information in database
    VideoBrowser-framework


Original video data

Video catalog           Video         WWW
                      repository
                                                     user
Video summary
                                   Content service
VideoBrowser-system interface
VideoBrowser-system interface



   metadata
                                media player




 Video structural information
        System architecture

  Web          Movies
                           Web server                   Internet
                                                                        Retrieval
   video data                                                           service


                                          firewall
                          switcher


Online storage
Disk array



  annotation     structuring   summarization   Archive server
                                                                   taper(offline
                                                                   storage)
Outline
          1
          1        Introduction

              2
              2     Unified Paralleling Search

                  3 Multimedia Analysis and
                  3
                     Retrieval

                   4 Bilingual services
                   4

                  5
                  5 Chinese Calligraphy
                     Character Retrieval
              6
              6   Conclusion and Future Work
    Background
o   As there are both English and Chinese books in
    CADAL, bilingual services are required for
    users to access resources in any language.
    Services
o   Some technologies and prototypes have been
    developed by north technical center on how to carry
    out the multi-layered bilingual machine translation in
    English and Chinese books, such as
    n   the metadata translation between English and Chinese
    n   the accurate translation of proper nouns such as names
        for unique individuals, events,or places
    n   the selective translation in a full-text context
    n   the translation of Old Chinese text
    n   the distributed translation service technique.
    Services
o   An online translation service is integrated into
    the CADAL digital library.
    n   Users can be directly conducted semantic-based
        multi-linguistics retrieval of required
        information in our CADAL digital library.
    n   The translation of contents of a page on line.
    n   The translation of metadata of a digital book.
Bilingual Search
The translation of contents of a page
Outline
          1
          1        Introduction

              2
              2     Unified Paralleling Search

                  3 Multimedia Analysis and
                  3
                     Retrieval

                   4 Bilingual services
                   4

                  5
                  5 Chinese Calligraphy
                     Character Retrieval
              6
              6   Conclusion and Future Work
    Background
o   Since most people are interested in the art of
    the beautiful styles of calligraphy character
    rather than the meaning of the character, the
    service of Chinese calligraphy character
    retrieval is provided in the CADAL digital
    library, treating them just as they are images
    without recognizing them like OCR does.
Calligraphy art still alive in:
    Key issues
o   Feature extraction: character complexity, stroke
    density and shape, the three kinds of features of
    the calligraphy character are proposed
o   similarity matching cost: retrieve relevant
    images according to it.
    Contents
o   Chinese Calligraphy Page Segmentation
o   Features Extraction
o   Character Image Retrieval
    Chinese Calligraphy Page Segmentation
o   The page image are binarized with characters in
    black and the background in white.
o   Cut the page into columns according to the
    vertical projecting histogram, and columns
    continued to be cut into individual characters.
o    All the characters are normalized in order to
    keep scale invariant Contour information,
Page segmentation
    Features Extraction
o   shape
o   character complexity
    shape representation
o   Calligraphic character’s shape is represented by
    its contour points.
o   The polar coordinates is used to describe
    directional relationship of points instead of the
    Cartesian coordinates.
o   For direction, we use 8 bins in equal degree
    size to divide the whole space into 8 directions.
o   For radius, we use 4 bins
o   For each point of a given point set composed of
    sampling points, its approximate shape context
    is described by its relationship with the
    remaining points in weighted bins.
shape representation




                       Contour point
    Calligraphy Character Complexity
o   We use Calligraphy Character Complexity as a
    filter at the beginning to discard the calligraphy
    character that has no possibility to be similar to
    the query.




    L be the number of sampled contour points from the
    query and Li be the number of sampled contour points
    from candidate image. a is the threshold obtained by
    experience.
    Character Image Retrieval
o   Compute the values of the character complexity of the
    calligraphy character.
o   Normalize the scale size of the query and sample its
    contour points.
o   Filter the candidate images by character complexity
o   Extract the shape feature and employ the shape
    matching method introduced in [6] to compute the
    matching cost for every remaining candidate image and
    the query.
o   Rank the results according to the matching cost, and
    return.
The calligraphy character retrieval
interface of browsing the original works
Outline
          1
          1        Introduction

              2
              2     Unified Paralleling Search

                  3 Multimedia Analysis and
                  3
                     Retrieval

                   4 Bilingual services
                   4

                  5
                  5 Chinese Calligraphy
                     Character Retrieval
              6
              6   Conclusion and Future Work
All the services have been accessed by the users
  from over 70 countries 280.000 times per day.
    Conclusion and Future Work

o   With the increase of the number of the
    users and the amount of the resources.
    future work with CADAL digital library will
    proceed in several directions:
    n   Improving the performance of the current
        services, to be more complete and be more
        stable;
    n   Continuing exploring the application of
        multimedia in Digital Library.
                             Thanks !
      Welcome Visiting 
    CADAL Digital Library
  (WWW.CADAL.ZJU.EDU.CN
                                         )


Email: wujq@cs.zju.edu.cn yzhuang@zju.edu.cn

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:0
posted:7/23/2013
language:English
pages:66