How to Publish Data in the VO

Workshop Garching, June 27 – July 1 2005 How to Publish Data in the VO Christophe Arviset ESAC Science Archives and VO Team ESA-VO, ESAC Christophe.Arviset@esa.int Publishing Data on the Internet WEB SERVER Publish Document Create document Robots makes Inventory Server Christophe Arviset Publish in the VO, Page 2 Publishing Data in the VO SSA SERVER S Create VO Service Produce Data Register VO Service Find List of VO Services Christophe Arviset Publish in the VO, Page 3 Existing Protocols to Publish Data  SIAP : Simple Image Access Protocol – Access to images  SSAP : Simple Spectra Access Protocol – Access to Spectra  ADQL / SkyNode – Data Query Language to access VO (meta)data  Web Services  In the future: – Simple Line Access Protocol, Theoretical Data Christophe Arviset Publish in the VO, Page 4 Issues to Publish Data in the VO  Availability of Data – FITS file, SED DM, ASCII, VOTable, etc…  Description of metadata – Data Model : same representation – UCD : same language  Quality of the data – Responsibility of the Data Provider – Description of quality – no common approach yet Christophe Arviset Publish in the VO, Page 5 Example from a Data Provider  European Space Agency in Spain – ISO Data Archive  Standard User Interface : http://www.iso.vilspa.esa.es/ida/  SIAP and SSA servers  Basic SkyNode – XMM-Newton Data Archive  Standard User Interface : http://xmm.vilspa.esa.es/xsa/  SIAP server  Basic and Full SkyNode (point source catalogue) – Others archives in progress and planned (Integral, Herschel, Planck, …) Christophe Arviset Publish in the VO, Page 6 Ways of Accessing ESA Data  Standard user friendly and powerful web based (Java) Graphical User Interface – Good for all users  Scriptable Interface for Project Experts  VO Services – Build extra layer on top of existing Scriptable Interface system to comply with Virtual Observatory standards – Flexibility for any future standard changes Christophe Arviset Publish in the VO, Page 7 Existing Access – GUI GUI Database/Product Request DB/Product Request GUI Database Result / Product DB Result / Product      Project Business Logic User friendly web access : Java Applet Powerful queries, organized by panel, hundreds of queriable and displayable parameters Public data accessible to all, proprietary data accessible only to privilege users Images preview (icons, full image) Quick download (1 click) at various processing levels, shopping basket retrieval Project prior to the VO – Existing data model – Existing data format – Existing access mechanisms Christophe Arviset Meta Data in Database Products Product Request  Translation Layer Data Products on hard disks Publish in the VO, Page 8 Existing Access : Scriptable I/F GUI Database/Product Request DB/Product Request GUI Database Result / Product DB Result / Product Project Product Request DB/Product Request Meta Data in Database foreach obs in (observation_list) AIOgetfile (obs, file_required) analyse_process file_required save results delete file_required end Project Product Project Business Logic Scriptable Interface DB Result / Product Products Product Request   Access to metadata (via XML scripting) Access to data through socket (direct data stream) or FTP (several files or individual file) Own Interoperability standards and needs as required by the project Not using VO protocols, but ready for it… Christophe Arviset   Translation Layer Data Products on hard disks Publish in the VO, Page 9 VO compliant Architecture GUI Database/Product Request Existing scriptable I/F already carried all the functionalities required by the new VO standards GUI Database Result / Product – Access to metadata (database) – Access to products (data repository) Project Product Request  By using “Translation Layer”, easy to convert the VO standards interfaces into our existing I/F Project Product (Scriptable Interface) required – Little effort foreach obs in (observation_list) – AIOgetfile (obs,Layer easy to code using XML Translation file_required) analyse_process file_required save results  As VO standards evolve, our scriptable I/F system delete file_required endremains, but we just have to adapt our translation layers  DB/Product Request DB Result / Product DB/Product Request Meta Data in Database DB Result / Product Project Business Logic Scriptable Interface Products Product Request SIAP/SSA Request Project Product Request DB/Product Request VOTable / Product Project Product DB Result / Product Translation Layer Christophe Arviset Translation Layer Data Products on hard disks Publish in the VO, Page 10 Conclusions  Publishing data in the VO can be done without changing your project back-end architecture – Use of multi-tier architecture – Use of translation layers  Publishing your data through VO compliant services will make your data more visible – accessible from any VO client / application Christophe Arviset Publish in the VO, Page 11 Acknowledgements ESAC Science Archive and VO Team : Isa Barbarisi, John Dowson, Nicolas Fajersztejn, Marco Freschi, Jose Hernández, Daniel Marin, Iñaki Ortiz, Pedro Osuna, Esther Parrilla, Jesus Salgado, Aurélien Stebe Christophe Arviset Publish in the VO, Page 12