IRIS BIG DATA


Automatic Analysis of Solar Eruptions
in Data from the NASA space telescope IRIS
using machine learning methods

ROLE OF THE INSTITUTE FOR DATA SCIENCE


> Computer science: software architecture, machine learning, algorithms

> Solar Physics: Physics on the sun, characteristics of the data collected by NASA’s IRIS instrument

Project lead at i4Ds: Martin Melchior, Säm Krucker

Partners: UniGE

Funding: National Research Programme NRP 75 “Big Data” (extended)
NRP75 project description

Duration: 2017-2021


Keywords: data science, big data, machine learning, space weather, heliophysics, IRIS

    

SUMMARY

Current telescopes deliver huge amounts of data which cannot be handled by traditional methods anymore. This project uses machine learning to detect and analyse solar flares in data from the NASA space telescope IRIS. The new methods are expected to significantly contribute to the understanding of the physics behind solar flares. They will also improve capabilities to predict them, a core element in space weather prediction.

Interface Region Imaging Spectrograph (IRIS) is a NASA Small Explorer Mission designed to observe the transition region between the solar chromosphere and corona. It records around 12 GB of image data every day, amounting to a current total of >35 TB of available data.

PEOPLE @I4DS WHO WORK ON IRIS BIG DATA

Dr. Cédric Huwyler

Data Scientist, Astrophysicist

Prof. Dr. Säm Krucker

Astrophysicist

Prof. Dr. Martin Melchior

Physicist, Deputy Head I4DS

PEOPLE @I4DS WHO WORKED ON IRIS BIG DATA BEFORE

  • Dr. Lucia Kleint (until 2018, continues at UNIGE)
  • Dr. Brandon Panos (until 2021, continues at UNIGE)

STUDENT PARTICIPATION

OPEN RESOURCES AND RESULTS

Presentation Dublin 2018

Presentation Dublin 2018

Presentation Dublin 2018

Presentation Dublin 2018

Presentation Dublin 2018

Presentation Dublin 2018

 
 

Presentation Dublin 2018

Presentation Dublin 2018

Presentation Dublin 2018

Presentation Dublin 2018

Presentation Dublin 2018

Presentation Dublin 2018

Presentation Dublin 2018

Presentation Dublin 2018

HelioML: Machine Learning, Statistics, and Data Mining for Heliophysics

This is how reliable research should be done in the future: make method and code public so the community can reproduce the results.

The book by Monica Bobra and James Mason includes a collection of interactive Jupyter notebooks, written in Python, that explicitly shows the reader how to use machine learning, statistics, and data minining techniques on various kinds of heliophysics data sets to reproduce published results.

Chapter 6 ‘Spectra of Flaring Active Regions’ by I4DS/UniGE PhD student Brandon Panos.

Presentation Knowledge Exchange Platform @I4DS 2019

Clustering Flares, presentation about research for the NFP75 project IRIS Big Data by Brandon Panos, KEP@I4DS 2019

Clustering flares, presentation KEP@I4DS 2019

The team: Brandon Panos, Martin Melchior, Dennis Ullmann, Cedric Huwyler, Sviatoslav Voloshynovskiy,Lucia Kleint, Samuel Krucker

Clustering flares, presentation KEP@I4DS 2019

Our instrument IRIS

Clustering flares, presentation KEP@I4DS 2019

Data from the IRIS mission

Clustering flares, presentation KEP@I4DS 2019

Spectra: Where does it come from?

Clustering flares, presentation KEP@I4DS 2019

First major research question: Is the chromospheric physics of all flares the same?

Clustering flares, presentation KEP@I4DS 2019

Magnesium spectra: Mg II h&k lines

Clustering flares, presentation KEP@I4DS 2019

Mg II k-line: Quiet sun vs. Flare

Clustering flares, presentation KEP@I4DS 2019

Questions related to Mg II profiles in solar flares

Clustering flares, presentation KEP@I4DS 2019

K-means

Clustering flares, presentation KEP@I4DS 2019

Turning spectra into vectors

Clustering flares, presentation KEP@I4DS 2019

KMeans

Clustering flares, presentation KEP@I4DS 2019

Method: Millions of spectra organized into 53 groups by k-means

Clustering flares, presentation KEP@I4DS 2019

K-means groups projected onto a single SJI

Clustering flares, presentation KEP@I4DS 2019

Temporal evolution: K-means groups projected onto an M-class flare

Clustering flares, presentation KEP@I4DS 2019

Universal flaring profiles: Single peaked

Clustering flares, presentation KEP@I4DS 2019

Ribbon-front IRIS profiles: Unique profiles at the ribbon front

Clustering flares, presentation KEP@I4DS 2019

Temporal and spatial correlations with x-ray signatures: Correlated with GOES and within RHESSI contours

Clustering flares, presentation KEP@I4DS 2019

Link to publication

Clustering flares, presentation KEP@I4DS 2019

Research questions: Is the physics of the chromosphere the same in all flares?

Clustering flares, presentation KEP@I4DS 2019

Multi-line analysis: Tools: K-means, Isolation forest, Hierarchical clustering, Traveling salesman

Clustering flares, presentation KEP@I4DS 2019

TSP

Clustering flares, presentation KEP@I4DS 2019

Multi-line analysis: TSP

Clustering flares, presentation KEP@I4DS 2019

Multi-line analysis: TSP

Clustering flares, presentation KEP@I4DS 2019

Multi-line analysis: Centroids

Clustering flares, presentation KEP@I4DS 2019

Flare prediction

Clustering flares, presentation KEP@I4DS 2019

t-SNE: Quiet Sun  | Sunspot | Active region | Pre-Flare

Clustering flares, presentation KEP@I4DS 2019

LSTM results for flare prediction coming soon, please ask questions!

Clustering flares, presentation KEP@I4DS 2019

Thank you for your time

VISUALS AND AUDIO

IRIS is a NASA solar mission launched in 2013. It produces about 12 GB of data every day. Scientists use them to determine the thermodynamical properties on the Sun.

IRIS records the Sun in unprecedented details, both in the form of images and as spectra. The IRIS Big Data project aims at better understanding the physics behind solar flares and at developing an algorithm that automatically detects solar flares among the huge amount of data.

As a first step, an algorithm for classifying UV-spectra on the solar surface was developed (image right). The classification was layered over a close up of the sun as coloured dots (image left).

MEDIA COVERAGE

Automatic Analysis of Solar Eruptions

Interview with the principal investigators of this NRP75 project on the occasion of its completion.

Big Data Dialog, 16. November 2021

Wettervorhersage für die Sonne

FHNW E-Magazin (mit Video), 26. November 2019

Sonneneruptionen besser verstehen mit Machine Learning

Dr. Cédric Huwyler, Institut für Data Science I4DS, Fachhochschule Nordwestschweiz

Big Data Dialog, 3. Dezember 2018