from fairdatanow import DataViewer
import os
Exploring half a million molecules
Exploring our Nextcloud data
For our ASAP project research team we created a Nextcloud folder that we can use to easily share our data with fairdatanow Python package. Let’s take a look what Wim has uploaded so far.
= {
configuration 'url': "https://laboppad.nl/asap-data",
'user': os.getenv('NC_AUTH_USER'),
'password': os.getenv('NC_AUTH_PASS')
}
= DataViewer(configuration) dv
dv
For this example we want to download a demo data file Ref0443_casein_asap01.RAW
from the Nextcloud server to our local computer. This can be done by typing the filename and Enter key in the search bar. We can now select the file by clicking the row by clicking. Downloading the selection is done with the method .download_selected()
= dv.download_selected() files
Ready with downloading 1 selected remote files to local cache: /home/frank/.cache/fairdatanow
Due to security restrictions at this moment you can not yet download from our Nextcloud server unless you are a member of our team! We are working to open up our data for you soon!
It is also possible to export the complete filter settings with the .export_filters()
method.
dv.export_filters()
{'columns': ['path', 'size', 'modified'],
'extensions': ['.raw'],
'search': 'ref0443',
'show_directories': False,
'show_filters': True,
'use_regex': False}
Reading our first .raw file
First step is to read an .raw file containing (already centroided) ASAP-HRMS data. The data can be loaded into a positive and a negative mode dataframe as with the read_raw()
function which returns two dataframes for positive and negative mode. This function is based on the pyRawTools python package.
from kendrick import read_raw
= '../downloads/kendrick-data/Ref0443_casein_asap01.RAW' raw_file
= read_raw(raw_file) df_pos, df_neg
Let’s focus on the positive mode data for now. Here is what the first and last rows of the dataframe looks like.
df_pos
RT | mz | inty | |
---|---|---|---|
Scan | |||
1 | 0.003945 | 125.023285 | 10221.680664 |
1 | 0.003945 | 125.059830 | 81648.109375 |
1 | 0.003945 | 125.132690 | 35949.582031 |
1 | 0.003945 | 126.062935 | 6891.418945 |
1 | 0.003945 | 126.099777 | 18121.056641 |
... | ... | ... | ... |
390 | 3.005163 | 546.237732 | 35105.105469 |
390 | 3.005163 | 548.254578 | 52922.070312 |
390 | 3.005163 | 596.266724 | 100884.796875 |
390 | 3.005163 | 610.248840 | 50182.472656 |
390 | 3.005163 | 722.359741 | 28877.074219 |
271982 rows × 3 columns
Inspecting the df_pos
dataframe we find 271982 rows with three columns: 1) RT retention time, 2) mz mass per electrical charge, and 3) inty number of ions. From the first column one can see that this experiment lasted 3 minutes.
As we will see, m/z values for identical molecules are slightly jittered due to limited instrumental precision. In order to determine the abundance of different molecules present in the sample, we now need to create time averaged centroided m/z values. This can be achieved by 1) first binning the data in a histogram, 2) then Gaussian smoothing the histogram and locating the peaks. These steps are implemented in the functions histogram()
and get_time_averaged_centroids()
.
Next step is to explore the data in an interactive visualization. In order to plot half a million data points in a single plot we need to import a special function interactive_plot()
. This function makes heavily use of a powerful python package datashader that is designed for fast plotting huge numbers of data points.
Note that in order to activate interactive plotting in a Jupyter notebook you need to execute the following notebook magic command in a code cell: %matplotlib widget
from kendrick import histogram, get_time_averaged_centroids, interactive_plot
= histogram(df_pos)
mz_hist = get_time_averaged_centroids(mz_hist)
mz_centroids
interactive_plot(df_pos, mz_hist, mz_centroids)
interactive_plot
interactive_plot (df, mz_hist, mz_centroids)
Create interactive plot for dataframe df
.
get_time_averaged_centroids
get_time_averaged_centroids (mz_hist_w_xy)
Get peaks (centroids) from histogram.
histogram
histogram (df)
Create intensity weighed histogram.
read_mzml
read_mzml (mzml_file)
*Read mzml_file
.
Returns positive and negative mode dataframes df_pos
and df_min
.*
read_raw
read_raw (raw_file)
Read raw_file
into positive and negative mode data frames.