[DOC] Section on extending junifer (datagrabbers and markers) #124
No reviewers
Labels
No labels
CRITICAL
Stale
WIP
bug
concept
coordinate
dataset
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
maintenance
maps
marker
mask
on hold
parcellation
preprocess
question
ready
storage
template-space
triage
wontfix
No milestone
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
juaml/junifer!124
Loading…
Reference in a new issue
No description provided.
Delete branch "doc/extending"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Codecov Report
100.00% <ø> (ø)95.23% <100.00%> (+0.14%)Flags with carried forward coverage won't be shown. Click here to find out more.
95.00% <ø> (ø)97.72% <ø> (ø)97.75% <ø> (ø)100.00% <ø> (ø)100.00% <ø> (ø)100.00% <100.00%> (ø)100.00% <100.00%> (ø)91.11% <0.00%> (-1.75%)94.05% <0.00%> (-0.86%)97.26% <0.00%> (-0.42%)Test fail due to dataset id mismatch. Need to merge #123 first.
Also taclkes #117
@ -173,11 +173,11 @@ texts.################################################################################ The BIDS datagrabber requires three parameters: the types of data we want,# replaced in the patterns.@ -0,0 +1,424 @@.. include:: ../links.incItscoveredanthese... were scanned during 3 sessions ...... ``{session}`` ...Can we have a hyperlink reference for datalad?
This class will not only interpret patterns but also use datalad to `clone` and `get` the data.... .of... BOLD.canrepresent each of the items ...One
theshould be removed.@ -0,0 +45,4 @@Importantly, nothing prevents that one image is part of two different elements.For example, it is usually the case that the ``T1w`` image is not acquired foreach task, but once in the entire session. So in this case, the ``T1w`` imagefor the element (``sub001``, ``ses1``, ``rest``) will be the same as theMaybe:
("sub001", "ses1", "rest")?@ -0,0 +46,4 @@For example, it is usually the case that the ``T1w`` image is not acquired foreach task, but once in the entire session. So in this case, the ``T1w`` imagefor the element (``sub001``, ``ses1``, ``rest``) will be the same as the``T1w`` image for the element (``sub001``, ``ses1``, ``stroop``).And here maybe:
("sub001", "ses1", "stroop")?@ -0,0 +1,28 @@.. include:: ../links.incfunctionality to junifer at runtime.@ -0,0 +1,28 @@.. include:: ../links.incfunctionality@ -0,0 +1,276 @@.. include:: ../links.inc... with the data types that the marker ...In this example, the only parameter required for computation is the name of the parcellation to use.Rendering for this as well is weird.
usefulThe method ``store`` ...... simply ...Once all of the above steps are done, we ...parcellationparcellation_nameself.parcellation_name = parcellation_nameparcellation_nameself.parcellation_name = parcellation_name@ -0,0 +90,4 @@In this step, we will define the method that computes the marker. This method will be called by junifer when needed,using the data provided by the datagrabber, as configured by the user. The function ``compute`` has two arguments:* ``input``: a dictionary with the data to be used to compute the marker. This will be the corresponding element in theThe indentation for this is a bit weird when rendered.
@ -0,0 +124,4 @@)# Create a maskermasker = NiftiLabelsMasker(@ -0,0 +213,4 @@)# Create a maskermasker = NiftiLabelsMasker(@ -42,9 +42,21 @@ Data types* - ``BOLD``CONN toolbox?CONN toolbox?@ -0,0 +1,30 @@.. include:: ../links.incDataGrabberdatasetinstead ofdatabase?@ -0,0 +8,4 @@The junifer pipeline is the main execution path of junifer. It consists of five steps:1. :ref:`Data Grabber <datagrabber>`: Interpret the dataset and provide a list of files.2. :ref:`Data Reader <datareader>`: Read the files.DataReader@ -0,0 +9,4 @@1. :ref:`Data Grabber <datagrabber>`: Interpret the dataset and provide a list of files.2. :ref:`Data Reader <datareader>`: Read the files.3. :ref:`Pre-processing <preprocess>`: Prepare the images for marker computation.Preprocess@ -0,0 +10,4 @@1. :ref:`Data Grabber <datagrabber>`: Interpret the dataset and provide a list of files.2. :ref:`Data Reader <datareader>`: Read the files.3. :ref:`Pre-processing <preprocess>`: Prepare the images for marker computation.4. :ref:`Marker Computation <marker>`: Compute the marker.Marker@ -0,0 +1,204 @@.. include:: ../links.inc... datareader ...I believe the YAML syntax is
falseand when we load YAML, it should automatically convert it toFalse.Same as above but for
trueandTrue.compute@ -0,0 +52,4 @@it is possible to extend junifer with custom datareaders, and thus, it is also possible to configure this step.Data GrabberDataGrabber@ -0,0 +83,4 @@kind: OasisVBMTestingData ReaderDataReader@ -0,0 +168,4 @@storage:kind: SQLiteFeatureStorageuri: /data/junifer/example/oasis_vbm_testing.sqliteIn the storage types, we have
.dbextension for SQLite. Just to be consistent, maybe we use it here?@ -0,0 +1,89 @@.. include:: ../links.incIf you are in immediate need of any of these ...schedulerPythonPlease check my argument for having
true.@ -0,0 +1,60 @@.. include:: ../links.incconsoleinstead ofbash?consoleinstead ofbash?consoleinstead ofbash?consoleinstead ofbash?@ -0,0 +22,4 @@* ``--help``: Show a help message.* ``--verbose`` Set the verbosity level. Options are ``warning``, ``info``, ``debug``.* ``--element``: The *element* to run. If not specified, all elements will be run. This parameter can be specifiedThe rendering for this has some indentation issue.
@ -33,6 +33,30 @@ def register_datagrabber(klass: Type) -> Type:return klassdatareader@ -0,0 +45,4 @@Importantly, nothing prevents that one image is part of two different elements.For example, it is usually the case that the ``T1w`` image is not acquired foreach task, but once in the entire session. So in this case, the ``T1w`` imagefor the element (``sub001``, ``ses1``, ``rest``) will be the same as thenot as strings. I like it like that.
@ -0,0 +1,30 @@.. include:: ../links.incIt can be either. Given that we are coupled with datalad, I would keep it as dataset.
@ -0,0 +1,30 @@.. include:: ../links.incI stil prefer to use the two words to describe the concept and not the Class name.
@ -0,0 +8,4 @@The junifer pipeline is the main execution path of junifer. It consists of five steps:1. :ref:`Data Grabber <datagrabber>`: Interpret the dataset and provide a list of files.2. :ref:`Data Reader <datareader>`: Read the files.same as before
@ -0,0 +9,4 @@1. :ref:`Data Grabber <datagrabber>`: Interpret the dataset and provide a list of files.2. :ref:`Data Reader <datareader>`: Read the files.3. :ref:`Pre-processing <preprocess>`: Prepare the images for marker computation.same
@ -0,0 +10,4 @@1. :ref:`Data Grabber <datagrabber>`: Interpret the dataset and provide a list of files.2. :ref:`Data Reader <datareader>`: Read the files.3. :ref:`Pre-processing <preprocess>`: Prepare the images for marker computation.4. :ref:`Marker Computation <marker>`: Compute the marker.same
@ -0,0 +1,204 @@.. include:: ../links.incwas not sure about this.
@ -0,0 +52,4 @@it is possible to extend junifer with custom datareaders, and thus, it is also possible to configure this step.Data Grabberwill keep it as concepts and not class names
@ -0,0 +168,4 @@storage:kind: SQLiteFeatureStorageuri: /data/junifer/example/oasis_vbm_testing.sqliteit does not matter, the user sets the name and extension. It should be sqlite.
@ -0,0 +168,4 @@storage:kind: SQLiteFeatureStorageuri: /data/junifer/example/oasis_vbm_testing.sqliteIt indeed does not matter, my argument is just for the sake of consistency. I can imagine it be confusing for users who are not familiar with SQLite in that detail.
@ -0,0 +1,204 @@.. include:: ../links.incI think it does, can you please check it?
@ -0,0 +1,30 @@.. include:: ../links.incIn the Understanding section, we have
DataGrabberfor the concept as well. Again my reasoning is that if we keep it as one name throughout, users are not confused. And also, one can mentally link better to the name of the step being the class category's name.@ -0,0 +45,4 @@Importantly, nothing prevents that one image is part of two different elements.For example, it is usually the case that the ``T1w`` image is not acquired foreach task, but once in the entire session. So in this case, the ``T1w`` imagefor the element (``sub001``, ``ses1``, ``rest``) will be the same as theWith strings, you directly link it to the code which IMO is simpler.
@ -0,0 +45,4 @@Importantly, nothing prevents that one image is part of two different elements.For example, it is usually the case that the ``T1w`` image is not acquired foreach task, but once in the entire session. So in this case, the ``T1w`` imagefor the element (``sub001``, ``ses1``, ``rest``) will be the same as theAlso, the rendering is not very pretty.
@ -0,0 +45,4 @@Importantly, nothing prevents that one image is part of two different elements.For example, it is usually the case that the ``T1w`` image is not acquired foreach task, but once in the entire session. So in this case, the ``T1w`` imagefor the element (``sub001``, ``ses1``, ``rest``) will be the same as theWhat I actually meant was putting the whole thing as monospace.
@ -0,0 +192,4 @@datadir: /data/project/test/dataOptional: Using datalad..., but also ...@ -62,7 +62,7 @@ class MarkerCollection:----------Data Grabber🚀
Will merge after CI passes.