Astroinformatics and Heliophysics at the FHNW Institute for Data Science

Big Data Streaming on the Web

MSE | Research projects in the MRU Computer Science

Introduction

Scientific instruments are probably most ahead indata production, but paradoxically it is not clear how to deal with these data once they are archived. For instance, the camera AIA on board of the spacecraft SDO has produced since 2010 images of the Sun every second, accumulating over a Petabyte per year.

Solar Dynamic Observatory SDO, artistic representation, NASA/JPL

The trend continues and the data deluge isn’t stopping yet: Once in operation, the solar telescope DKIST will generate far more images than the AIA camera. Who knows what is in these images? People tend to look at always the same time intervals, but there are tools to browse the archives. One basic tool is offered by a project called JHelioviewer 3.0. This is a Java application not only able to display individual images out of the data but also to play movies; it is based on a special format called JPEG 2000. Another application, called helioviewer.org is Web-based, but it cannot display movies due to this weird JPEG 2000. However, the development of JHelioviewer 3.0 has stopped because we believe that the web version needs to be upgraded rather than the java version extended.

 

Your work

The goal of this project is to port the functionality of JHelioviewer into the Web application. The challenge is that the original images are 4k, and the user needs to be able to zoom in and out while playing the movies. The frame rates need to be selected in order to play long time sequences, such as, for example an entire year of observation within a 10 minute video. Web browser cannot rely on JPEG 2000 and as a result, the existing back-end needs to be adapted. We have already started to experiment by pre-producing video at different time and spatial resolutions (lots of videos!). The client then chooses the adequate videos and displays them as expected. We are eager to have a person who can build up on our actual experience and dedicates his or her time to work out how this should be done. There is a lot of research associated with it, we believe. Even if we think that we are able to manage the principles, there is still a lot to be done to find an optimal solution.

 

Individual tasks

The work will proceed in three steps, associated with the Project numbers P7, P8, and P9. Note that the exact definition of the projects will be discussed extensively and adapted to the personal interests and skills of the candidate.

  • P7: Make it run: First steps in the project: How do the data look, what has already been done? Learn to know JHelioviewer, and develop the first rough Web-based viewer
  • P8: Make it right: Optimization of the rendering algorithms on the front end, and of the data management on the back end. Inclusion in the actual helioviewer.org app
  • P9: Make it fast: Realization of the publicly avaialble application with consideration of scalability hundred of thousands of users, installation on a NASA server, testing (possibly on site at NASA GSFC in Washington DC).

Type of study
[x] Full time
[x] Part time 50% combined with a assistant position at i4Ds

Project organization
Much of the work will be in collaboration with our partners at NASA Goddard Space Flight Center in Washington, DC.

Advisor
Prof. Dr. André Csillagh