Data analysis is a versatile field that deals with data in many forms, such as structured tabular data, images, time series, etc. While the first might look easier to handle, images are undoubtedly more complicated to process: they usually contain more information, often hidden by a superposition of different sources of signal. Data under this form are particularly interesting, specifically due to their rich scientific content. We shall specifically focus on hyperspectral imaging (HSI). Differently from standard RGB pictures, these images are created by systems that capture information across many different spectral bands (from tens to thousands, depending on the sensor), which might include wavelengths beyond what the human eye can see. This translates to larger volumes of data to digest in order to make sense of the huge amount of information contained in each pixel. By analysing this form of data, we can gain detailed insights into the chemical composition and physical properties of materials, hence the specific interest for our use cases.

For this internship we propose to draw inspiration from recent work in hyperspectral image classification for Mars exploration, though several datasets will indeed play different roles in the development. Differently from mainstream computer vision tasks, the analysis is complicated by the presence of strong correlations amongst absorption bands, by the spectral interference of different components, as well as by a non negligible impact of the sensor used for capturing the scene. The correct assessment of important elements in the spectra and their neighbouring correlations becomes then vital for a correct classification of species involved. The use of state-of-the-art (SOTA) AI algorithms such as multi-head self-attention, coupled to standard signal processing tools (e.g. convolutional neural networks), might then capture the essential information by facilitating the extraction of important features. However, simple classification (or semantic segmentation) techniques do not take into consideration the extremely large variability of classes that materials might present.

Following these considerations, during the internship, we would like to investigate the following points:

• dimensionality reduction: revise SOTA approaches in HSI and analyse their capacity to preserve the information contained in spectra;
• pixel classification: standard approaches in material science require the correct classification by spectrum (i.e. by pixel), though neighbouring relations might be used to promote this to a segmentation task in computer vision;
• zero-shot classification: while most approaches grant an in-distribution generalisation, the ideal algorithm should be able to apply to unseen classes (e.g. a technique à la CLIP for material science);
• uncertainty quantification: study SOTA methods for the quantification of uncertainties for scientifically sound results.

We look for a passionate student at the end of their studies (e.g. the French M2 level), with a good understanding of machine learning and coding techniques. Good knowledge of any deep learning framework (PyTorch, JAX, Tensorflow) in Python is mandatory. A basic understanding of physics (such as spectroscopy or basic material science) is appreciated and considered a plus, though not necessary.