A dashboard for exploring MA-XRF data

Given our precomputed .datastack files present in our Nextcloud storage we can now visualize our MA-XRF data in a Dashboard viewer after downloading them once.

Warning

The total size of these 16 files is pretty big (56.5 GB) and downloading on my computer took about half an hour!

from fairdatanow import DataViewer
import matplotlib.pyplot as plt 
import os
import numpy as np

configuration = {
    'url': "https://laboppad.nl/falnama-project", 
    'user':    os.getenv('NC_AUTH_USER'),
    'password': os.getenv('NC_AUTH_PASS')
}

filters = {'extensions': ['.datastack']} 

dv = DataViewer(configuration, **filters)
datastack_files = dv.download_filtered()

Ready with downloading 16 selected remote files to local cache: /home/frank/.cache/fairdatanow                                                                      

Ok, let’s start our exploration by plotting a thumbnail overview of all scanned areas.

import maxrf4u as mx
import matplotlib.pyplot as plt
import re
from falnama import plot_roi_dashboard, get_roi_ims
from matplotlib.patches import Rectangle

imvis_list = []
extent_list = []
for datastack_file in datastack_files: 
    ds = mx.DataStack(datastack_file)
    im = ds.read('imvis_reg')
    extent = ds.read('imvis_extent')
    imvis_list.append(im)
    extent_list.append(extent)

object_nums = [re.sub(r'.*(WM-71803-\d\d).*', r'\1', dsf) for dsf in datastack_files]

fig, axs = plt.subplots(ncols=8, nrows=2, figsize=[15, 6])
axs = axs.flatten()
for i, [im, ax, extent] in enumerate(zip(imvis_list, axs, extent_list)): 
    ax.imshow(im, extent=extent)
    ax.set_title(f'[{i}] {object_nums[i]}', fontsize=8)
    ax.axis('off')

ROI tabs Dashboard

So far my latest dashboard code made for the MuseumNext conference is not very interactive at all, but would seem to approach a layout that seems ok. We need to adapt a lot of things still…

A first step now is to manually select regions of interest for page 35 (index 15)…

n = 15
datastack_file = datastack_files[n]
ds = mx.DataStack(datastack_file)

object_num = object_nums[n]

#cube = ds.read('maxrf_cube', compute=False)
imvis_highres = ds.read('imvis_reg_highres')
#imvis = ds.read('imvis_reg')
extent = ds.read('imvis_extent')
#x_keVs = ds.read('maxrf_energies')
#y_maxspectrum = ds.read('maxrf_maxspectrum')

#nmf_elementmaps = ds.read('nmf_elementmaps') 

#ppa = mx.Peak_Pattern_Atlas(datastack_file=datastack_file, tube_keV=23)

#ds.tree()

xylims_list = [[int(ax.get_xlim()[0]), int(ax.get_xlim()[1]), int(ax.get_ylim()[0]), int(ax.get_ylim()[1])] for ax in axs]

[[29, 51, 106, 84],
 [39, 88, 136, 91],
 [154, 183, 242, 208],
 [275, 296, 249, 227],
 [40, 59, 289, 269],
 [72, 124, 441, 397],
 [146, 168, 462, 443],
 [139, 161, 393, 370],
 [245, 268, 454, 433],
 [197, 219, 300, 278]]

xylims_list = [[29, 51, 106, 84], [39, 88, 136, 91], [154, 183, 242, 208], [275, 296, 249, 227], [40, 59, 289, 269],
               [72, 124, 441, 397], [146, 168, 462, 443], [139, 161, 393, 370], [245, 268, 454, 433], [197, 219, 300, 278]]

roi_ims = get_roi_ims(datastack_file, xylims_list)

fig, axs = plt.subplots(ncols=len(roi_ims)+1, figsize=[20, 4], squeeze=True)

edgecolors = ['violet', 'cyan', 'blue', 'red', 'white', 'orange', 'brown', 'green', 'maroon', 'black']

axs[0].imshow(imvis_highres, extent=extent)

for i, xylims in enumerate(xylims_list):
    x0, x1, y0, y1 = xylims
    axs[0].add_patch(Rectangle([x0, y1], x1 -x0, y0 - y1, linewidth=1, edgecolor=edgecolors[i], facecolor='none'))

for i, [ax, im] in enumerate(zip(axs[1:], roi_ims)): 
    
    ax.imshow(im)
    ax.set_title(f'[{i}] {edgecolors[i]}')

Here is the latest still very rough code for an ROI tabs dashboard.

roi_elements_list = [['Pb', 'K', 'Ca', 'Fe'], #0 We see a clear 
                     ['Pb', 'K', 'Ca', 'As', 'S'], #1
                     ['Pb', 'Fe', 'Ca', 'Hg'], #2
                     ['Pb'], #3
                     ['Pb', 'Fe', 'Ca'], #4
                     ['Pb', 'As', 'S', 'Fe'], #5 Here we see that the NMF maps are more clear the the alpha slices
                     ['Au', 'Pb', 'As'], #6 Clearly gold on red lead. The Pb alpha slice map is confusing   
                     ['Pb', 'As', 'S', 'Au'], #7 The yellow robe is clearly orpiment. Here again the the overlapping Pb alpha with As is confusing 
                     ['As', 'S', 'Au', 'Pb'], #8 According to the NMF map there should be no lead present in the green robe 
                     ['Pb']] #9

plot_roi_dashboard(datastack_file, xylims_list, roi_elements_list)

Please wait while initializing peak pattern atlas...
                                                  

plt.close('all')

FUNCTIONS

---

source

plot_roi_dashboard

 plot_roi_dashboard (datastack_file, xylims_list, roi_elements_list)

Create tabbed plot for roi spectra.

	Details
datastack_file
xylims_list
roi_elements_list	, roi_cubes, roi_ims, , nmf_elementmaps, x_keVs, imvis_highres, extent, object_num, xylims_list):

---

source

plot_roi_peak_patterns

 plot_roi_peak_patterns (roi_cube, prominence=2, ax=None)

---

source

get_roi_ims

 get_roi_ims (datastack_file, xylims_list)

Get high resolution roi images for xylims_list.

---

source

get_slice_map

 get_slice_map (datastack_file, element, peak_idx=0)

*Compute simple single channel map from datastack_file cube for element alpha channel.

Choose option peak_idx=1 for beta peak.

Returns: element map*

---

source

pick_nmf_elementmap

 pick_nmf_elementmap (datastack_file, nmf_elementmaps, element)

Faster get nmf element map

---

source

get_nmf_map

 get_nmf_map (datastack_file, element)

Read computed NMF element map for element from datastack.

---

source

plot_roi_maps

 plot_roi_maps (roi_cubes, xylims_list, roi_n, elements,
                y_maxspectrum=None, exc_keV=23)

*Plot roi alpha maps for list of elements.

If y_maxspectrum is specified normalize with maximum at channel index.*

---

source

get_roi_maps

 get_roi_maps (roi_cube, elements, x_keVs, exc_keV=23)

*Slice roi cube at alpha index positions for each element in elements.

Returns: alpha_idxs, roi_maps*

---

source

plot_element_lines

 plot_element_lines (element, x_keVs, y=None, ax=None, exc_keV=23,
                     color=None)

Add emission lines to spectral plot.

---

source

get_element_lines

 get_element_lines (element, x_keVs, exc_keV=23)

Get sorted element peak idxs.

---

source

plot_hotmax_spectra

 plot_hotmax_spectra (spectra, x_keVs, hotmax_idxs, elems=None, xlim=[-1,
                      22], elem_colors={'Pb': 'k', 'Au': 'orange', 'Fe':
                      'brown', 'Ca': 'g', 'K': 'b', 'As': 'magenta', 'Na':
                      'blue', 'Ni': 'red', 'Cu': 'green', 'Ti': 'blue',
                      'Zn': 'red'}, y_maxspectrum=None)

*Plot roi hotmax spectra and add emission lines for list of elements elems.

Returns: axs*

---

source

get_hotmax_spectra

 get_hotmax_spectra (roi_cube)

*Get hotmax spectra for roi_cube.

Returns: hotspot_idxs,spectra`*

---

source

plot_roi_peak_patterns

 plot_roi_peak_patterns (roi_cube, prominence=2, ax=None)