AMIGA : SKA-Link: combining knowledge to pioneer Big-Data solutions for SKA Data Centres

PAST EVENTS SKA-Link kick-off meeting - 3rd & 4th April 2017 SKA-Link is a 2-year project led by the AMIGA team that belongs to the i-LINK progra

PAST EVENTS SKA-Link kick-off meeting - 3rd & 4th April 2017 SKA-Link is a 2-year project led by the AMIGA team that belongs to the i-LINK programme (Programa CSIC Conexión Internacional “i-LINK”). This call has the objective of promoting international scientific collaborations with foreign institutions. The AMIGA team aims to create a deep understanding of the best technical strategies for successfully exploiting the immense flow of science-ready data that SKA will generate. To achieve this goal, it has taken advantageof its experience in e-Science and SKA-SDP (SKA Science Data Processor) membership to establish a collaboration and produce a general framework of Best Practices to be considered in the design of the SKA Regional Centres. In order to elaborate this general framework of Best Practices, SKA-Link will promote the collaboration between 1) key members of the SDP consortium plus the ones involved in the design of SKA Regional Centres, 2) experts on cutting edge e-Science technologies for the scientific exploitation of Distributed Computing Infrastructures (DCIs ) and 3) Fujitsu Laboratories of Europe, contributing to make of SKA a reference, not only in science and technology, but in the scientific methodology too. In summary, the 3 main objectives of SKA-Link are: To make an inventory of technologies enabling scientists to exploit scientific data. For designing the SKA Regional Centres it will be necessary to have a deep knowledge about the state of the art tools of DCIs to perform efficient exploitations of the scientific data To assess combinations of those technologies supporting advances in the scientific methods. To describe the properties that SKA Regional Centres have to fulfil to be considered a reference in science, technology and scientific methodology. They will provide a framework that assists the astronomers to exploit the SKA data. This project aims at going further and study how this framework could assist the astronomers to increase the quality of their scientific methods and results, with an emphasis on tools that facilitate the reproducibility of the scientific methods and their verification. Participants SKA-Link coordinator: IAA-CSIC The SKA-Link project is coordinated by the AMIGA group, which belongs to the Instituto de Astrofísica de Andalucía (IAA-CSIC). This group is comprised by scientists and engineers, focused on the study of galaxies, with a technical component that has provided support via e-Science activities. AMIGA developed the first VO compliant data model for a single-dish radio astronomical archive, implemented in the IRAM-30m VO Archive (J.D. Santander-Vela et al., 2012) and has collaborated in the development of several VO standards. AMIGA has led one of the WPs in the Wf4Ever EU FP7 funded project “Advanced Workflow Preservation Technologies for Enhanced Science” in order to contribute to the development of standards and models for the preservation of scientific workflows. Besides, being aware of the data flux that will come from SKA, this group also collaborates with the Fundación de Supercomputación de Castilla y León and the Barcelona Supercomputing Centre to develop a federated layer, which facilitates astronomers launching their workflows in heterogeneous DCIs. AMIGA collaborates with the Asteroseismology group of IAA-CSIC, led by Prof. R. Garrido in the area of e-Science provision. His group is a pioneer in developing asteroseismic tools in the VO. In addition, this team has also ported some of their tools and applications to the Grid environment, using the infrastructure of EGI Members of the SDP consortium and responsible for designing the SKA Regional Centres University of Cambridge (UCAM). It is heavily involved in the SKA design, covering a range or areas. The SKA-Link PI from this university, Prof. Paul Alexander, is the leader of the SKA SDP consortium and was the leader of the SKA Design Studies. His group has expertise in exploiting radio astronomy infrastructures that include SKA pathfinders such as the Atacama Large Millimetre Array (ALMA) or the Low Frequency Array (LOFAR). This university and their members participating in SKA-Link are also members of the AENEAS H2020 project. University of Manchester (UMAN). It hosts the Headquarters of the SKA Organisation at the Jodrell Bank Centre for Astrophysics. The SKA-Link PI from this university, Prof. Anna Scaife, is leader of the working package of AENEAS in charge of defining the computing and processing requirements for an SRC. University of Cape Town (UCT). It is located in the oldest university in South Africa, one of the two SKA host countries. Through numerous legacy survey projects, this university is linked with SKA, especially with one of the SKA precursors, MeerKAT. The SKA-Link PI from this university, Prof. Rob Simmonds, is leader of the SKA SDP sub-working package in charge of delivering the data to the community. He also participates in the AENEAS H2020 project. International Centre for Radio Astronomy Research (ICRAR). This centre is located in Australia, one of the two SKA host countries, and was created specifically to support the development of the SKA project in this country. The SKA-link PI from this centre, Prof. Andreas Wicenec led the ICT program that includes four main projects, the Conceptual SKA Design, the Data Intensive Research Pathfinder, the High Performance Computing and the LSST collaboration on database research. He has contributed to the VO and was involved in the design and implementation of ESO and ALMA science archives. Netherlands Institute for Radio Astronomy (ASTRON). This institute designed, built and now operates the International LOFAR Telescope (a recognized pathfinder for the SKA). It leads both the SKA Low Frequency Aperture Array and the SKA Mid Frequency Aperture Array consortia while it plays a prominent role in the SDP consortium. The SKA-Link PI from this institute, Prof. Michael Wise, is also the coordinator of the AENEAS H2020 project and a member of the committee group created by the SKAO to define the SKA Regional Centres. Experts on cutting edge e-Science technologies University of Edinburgh (UEDIN). The group from this university participating in SKA-Link - Data-Intensive Research (DIR) group - has provided major contributions to the EU projects ADMIRE, ENVRI, EUDAT and VERCE, and it leads the design of the e-Infrastructure in the H2020 projects EUDAT2020 to greatly accelerate IO rates in exascale system. Its PI, Prof. Malcolm Atkinson, is Professor of e-Science in this university.He was director of the e-Science institute, which launched the DIR group. The Institute for Computer Science and Control (MTA SZTAKI). It is one of the largest IT research institutes in Central Europe playing a key role in Grid and Cloud computing related research and development. The group from this institute participating in SKA-Link is expert in developing generic science gateway frameworks based on workflows (gUSE/WS-PGRADE) for various executing infrastructures built on Grid, Cloud and cluster technologies. Its PI, Prof. Peter Kacsuk, is Head of the Laboratory of Parallel and Distributed Systems of SZTAKI and has been involved in many WU Grid projects (EDG, GridLab, EGEE, SEEGRID, CoreGrid, GridCoord, ICEAGE, CancerGrid, EDGes). The Eberhard­‐Karls Universität Tübingen (EKUT, University of Tübingen). The group from this university participating in SKA-link, the Applied Bioinformatics Group (ABI) is expert in providing science gateway solutions and workflow management technology, highlighting its contribution to the MoSGrid and SCI-Bus EU projects in order to build a Science Gateway for the Chemistry community. The SKA-Link PI of this group, Dr Jens Krüger, has been responsible for the workflow development and portal working packages of the MosGrid project. Royal Netherlands Meteorological Institute (KNMI). The Data-Technologies group of KNMI is participating in SKA-Link, providing its expertise on tools, web services and Science Gateways adopted across different disciplines in Earth Science. The SKA-Link PI, Dr Alessandro Spinuso, is involved in a number of EU initiatives and projects, focusing on the deployment of an e-science infrastructure for Solid Earth Science research in Europe (NERA, VERCE, EPOS). British Geological Survey (BGS). This centre belongs to the Environmental Science Centre, an international leader in Geology. This group is expert on the development of data-intensive applications. The SKA-Link PI, Dr Rosa Filgueira, is currently a senior Data Scientist in the BGS and one of the leaders of dispel4py, a Python framework for Data-Intensive scientific computing. Fujitsu Laboratories of Europe (FLE). FLE is a wholly owned subsidiary of Fujitsu Laboratories Ltd, Japan. FLE R&D activities are focused on advanced cloud computing technologies and Linked Open Data technology, next generation networks and wireless technologies, M2M service platforms and sensors. FLE has a strong collaboration mission and is carrying out pre-competitive R&D and EU collaborative projects. FLE has extensive knowledge of standardization in many fields, including wireless communication, Grid computing, Web services, and Linked Open Data. This project is funded under the call i-LINK+ 2016 (I-LINK1122), a grant from CSIC.
 

SKA-Link: combining knowledge to pioneer Big-Data solutions for SKA Data Centres

PAST EVENTS
SKA-Link kick-off meeting - 3rd & 4th April 2017

SKA-Link is a 2-year project led by the AMIGA team that belongs to the i-LINK programme (Programa CSIC Conexión Internacional “i-LINK”). This call has the objective of promoting international scientific collaborations with foreign institutions. 

The AMIGA team aims to create a deep understanding of the best technical strategies for successfully exploiting the immense flow of science-ready data that SKA will generate. To achieve this goal, it has taken advantage of its experience in e-Science and SKA-SDP (SKA Science Data Processor) membership to establish a collaboration and produce a general framework of Best Practices to be considered in the design of the SKA Regional Centres.

In order to elaborate this general framework of Best Practices, SKA-Link will promote the collaboration between 1) key members of the SDP consortium plus the ones involved in the design of SKA Regional Centres, 2) experts on cutting edge e-Science technologies for the scientific exploitation of Distributed Computing Infrastructures (DCIs ) and 3) Fujitsu Laboratories of Europe, contributing to make of SKA a reference, not only in science and technology, but in the scientific methodology too.

In summary, the 3 main objectives of SKA-Link are:
  1. To make an inventory of technologies enabling scientists to exploit scientific data. For designing the SKA Regional Centres it will be necessary to have a deep knowledge about the state of the art tools of DCIs to perform efficient exploitations of the scientific data
  2. To assess combinations of those technologies supporting advances in the scientific methods.
  3. To describe the properties that SKA Regional Centres have to fulfil to be considered a reference in science, technology and scientific methodology. They will provide a framework that assists the astronomers to exploit the SKA data.

This project aims at going further and study how this framework could assist the astronomers to increase the quality of their scientific methods and results, with an emphasis on tools that facilitate the reproducibility of the scientific methods and their verification.

 

Participants

SKA-Link coordinator: IAA-CSIC

The SKA-Link project is coordinated by the AMIGA group, which belongs to the Instituto de Astrofísica de Andalucía (IAA-CSIC). This group is comprised by scientists and engineers, focused on the study of galaxies, with a technical component that has provided support via e-Science activities. AMIGA developed the first VO compliant data model for a single-dish radio astronomical archive, implemented in the IRAM-30m VO Archive (J.D. Santander-Vela et al., 2012) and has collaborated in the development of several VO standards. AMIGA has led one of the WPs in the Wf4Ever EU FP7 funded project “Advanced Workflow Preservation Technologies for Enhanced Science” in order to contribute to the development of standards and models for the preservation of scientific workflows. Besides, being aware of the data flux that will come from SKA, this group also collaborates with the Fundación de Supercomputación de Castilla y León and the Barcelona Supercomputing Centre to develop a federated layer, which facilitates astronomers launching their workflows in heterogeneous DCIs. AMIGA collaborates with the Asteroseismology group of IAA-CSIC, led by Prof. R. Garrido in the area of e-Science provision. His group is a pioneer in developing asteroseismic tools in the VO. In addition, this team has also ported some of their tools and applications to the Grid environment, using the infrastructure of EGI

Members of the SDP consortium and responsible for designing the SKA Regional Centres

  • University of Cambridge (UCAM). It is heavily involved in the SKA design, covering a range or areas. The SKA-Link PI from this university, Prof. Paul Alexander, is the leader of the SKA SDP consortium and was the leader of the SKA Design Studies. His group has expertise in exploiting radio astronomy infrastructures that include SKA pathfinders such as the Atacama Large Millimetre Array (ALMA) or the Low Frequency Array (LOFAR). This university and their members participating in SKA-Link are also members of the AENEAS H2020 project.
  • University of Manchester (UMAN). It hosts the Headquarters of the SKA Organisation at the Jodrell Bank Centre for Astrophysics. The SKA-Link PI from this university, Prof. Anna Scaife, is leader of the working package of AENEAS in charge of defining the computing and processing requirements for an SRC.
  • University of Cape Town (UCT). It is located in the oldest university in South Africa, one of the two SKA host countries. Through numerous legacy survey projects, this university is linked with SKA, especially with one of the SKA precursors, MeerKAT. The SKA-Link PI from this university, Prof. Rob Simmonds, is leader of the SKA SDP sub-working package in charge of delivering the data to the community. He also participates in the AENEAS H2020 project.
  • International Centre for Radio Astronomy Research (ICRAR). This centre is located in Australia, one of the two SKA host countries, and was created specifically to support the development of the SKA project in this country. The SKA-link PI from this centre, Prof. Andreas Wicenec led the ICT program that includes four main projects, the Conceptual SKA Design, the Data Intensive Research Pathfinder, the High Performance Computing and the LSST collaboration on database research. He has contributed to the VO and was involved in the design and implementation of ESO and ALMA science archives.
  • Netherlands Institute for Radio Astronomy (ASTRON). This institute designed, built and now operates the International LOFAR Telescope (a recognized pathfinder for the SKA). It leads both the SKA Low Frequency Aperture Array and the SKA Mid Frequency Aperture Array consortia while it plays a prominent role in the SDP consortium. The SKA-Link PI from this institute, Prof. Michael Wise, is also the coordinator of the AENEAS H2020 project and a member of the committee group created by the SKAO to define the SKA Regional Centres.


Experts on cutting edge e-Science technologies

  • University of Edinburgh (UEDIN). The group from this university participating in SKA-Link - Data-Intensive Research (DIR) group - has provided major contributions to the EU projects ADMIRE, ENVRI, EUDAT and VERCE, and it leads the design of the e-Infrastructure in the H2020 projects EUDAT2020 to greatly accelerate IO rates in exascale system. Its PI, Prof. Malcolm Atkinson, is Professor of e-Science in this university.He was director of the e-Science institute, which launched the DIR group.
  • The Institute for Computer Science and Control (MTA SZTAKI). It is one of the largest IT research institutes in Central Europe playing a key role in Grid and Cloud computing related research and development. The group from this institute participating in SKA-Link is expert in developing generic science gateway frameworks based on workflows (gUSE/WS-PGRADE) for various executing infrastructures built on Grid, Cloud and cluster technologies. Its PI, Prof. Peter Kacsuk, is Head of the Laboratory of Parallel and Distributed Systems of SZTAKI and has been involved in many WU Grid projects (EDG, GridLab, EGEE, SEEGRID, CoreGrid, GridCoord, ICEAGE, CancerGrid, EDGes).
  • The Eberhard­‐Karls Universität Tübingen (EKUT, University of Tübingen). The group from this university participating in SKA-link, the Applied Bioinformatics Group (ABI) is expert in providing science gateway solutions and workflow management technology, highlighting its contribution to the MoSGrid and SCI-Bus EU projects in order to build a Science Gateway for the Chemistry community. The SKA-Link PI of this group, Dr Jens Krüger, has been responsible for the workflow development and portal working packages of the MosGrid project.
  • Royal Netherlands Meteorological Institute (KNMI). The Data-Technologies group of KNMI is participating in SKA-Link, providing its expertise on tools, web services and Science Gateways adopted across different disciplines in Earth Science. The SKA-Link PI, Dr Alessandro Spinuso, is involved in a number of EU initiatives and projects, focusing on the deployment of an e-science infrastructure for Solid Earth Science research in Europe (NERA, VERCE, EPOS).
  • British Geological Survey (BGS). This centre belongs to the Environmental Science Centre, an international leader in Geology. This group is expert on the development of data-intensive applications. The SKA-Link PI, Dr Rosa Filgueira, is currently a senior Data Scientist in the BGS and one of the leaders of dispel4py, a Python framework for Data-Intensive scientific computing.

Fujitsu Laboratories of Europe (FLE).

FLE is a wholly owned subsidiary of Fujitsu Laboratories Ltd, Japan. FLE R&D activities are focused on advanced cloud computing technologies and Linked Open Data technology, next generation networks and wireless technologies, M2M service platforms and sensors. FLE has a strong collaboration mission and is carrying out pre-competitive R&D and EU collaborative projects. FLE has extensive knowledge of standardization in many fields, including wireless communication, Grid computing, Web services, and Linked Open Data.


 

This project is funded under the call i-LINK+ 2016 (I-LINK1122), a grant from CSIC.