juaml/junifer
Watch
3
0
Fork
You've already forked junifer
 0
Code Issues 14 Pull requests 2 Packages Activity Actions

[BUG]: Cannot compute markers on AOMIC ID1000 using native space #487

Merged
synchon merged 2 commits from fix/mask-check-native into main 2026-03-03 15:07:05 +00:00
Conversation 2 Commits 2 Files changed 2 +11 -4
synchon commented 2026-03-02 15:10:47 +00:00 (Migrated from github.com)
Copy link

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

I want to compute ReHo/FC in native space. fMRIPrepConfoundRemover fails.

I get this error

2026-02-27 15:54:29 [info     ] Preprocessing data with fMRIPrepConfoundRemover [junifer]
2026-02-27 15:54:29 [info     ] Preprocessing BOLD             [junifer]
2026-02-27 15:54:30 [info     ] No `t_r` specified, using t_r from NIfTI header [junifer]
2026-02-27 15:54:30 [info     ] Read t_r from NIfTI header: 2.200000047683716 [junifer]
Traceback (most recent call last):
  File "/home/fraimondo/miniforge3/envs/junifer/bin/junifer", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/cli/cli.py", line 185, in run
    cli_func.run(
  File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/api/functions.py", line 223, in run
    mc.fit(datagrabber_object[t_element])
  File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/pipeline/marker_collection.py", line 103, in fit
    data = preprocessor.fit_transform(data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/pipeline/pipeline_step_mixin.py", line 232, in fit_transform
    return self._fit_transform(input=input, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/preprocess/base.py", line 181, in _fit_transform
    t_out = self.preprocess(input=t_input, extra_input=extra_input)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/preprocess/confounds/fmriprep_confound_remover.py", line 640, in preprocess
    mask_img = get_data(
               ^^^^^^^^^
  File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/data/_dispatch.py", line 160, in get_data
    return registry().get(
           ^^^^^^^^^^^^^^^
  File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/data/masks/_masks.py", line 487, in get
    set(masks) == {nilearn_callable_masks[0]}
    ^^^^^^^^^^
TypeError: unhashable type: 'CommentedMap'

Expected Behavior

Obtain markers in native space

Steps To Reproduce

  1. With junifer latest
  2. Run any element from the AOMIC ID1000 datasets
workdir: /tmp

datagrabber:
    kind: DataladAOMICID1000
    space: native

preprocess:
  - kind: fMRIPrepConfoundRemover
    detrend: true
    standardize: true
    strategy:
        motion: full
        wm_csf: full
        global_signal: full
    low_pass: 0.08
    high_pass: 0.01
    masks:
      - inherit
      - compute_epi_mask
      - threshold: 0
  - kind: SpaceWarper
    reference: T1w
    on: BOLD
    using: ants

markers:
  - name: ReHo-Julich-Brain_V3_1
    kind: ReHoParcels
    parcellation: Julich-Brain_V3_1
    using: afni
    masks: 
      - inherit

storage:
  kind: HDF5FeatureStorage
  uri: /data/group/riseml/fraimondo/2026_EBRAINS/features/ds003097_ReHo_native/ds003097_ReHo_native.hdf5

queue:
  jobname: aomic_reho_native
  kind: HTCondor
  env:
    kind: conda
    name: junifer
  mem: 8G
  disk: 5G
  verbose: info
  pre_run: |
    source /data/group/appliedml/tools/afni_24.3.06/afni.sh	
    source /data/group/appliedml/tools/ants_2.5.0/ants.sh
    source /data/group/appliedml/tools/fsl_6.0.4-patched2/fsl.sh

Environment

junifer:
  version: 0.0.7.dev320
python:
  version: 3.12.7
  implementation: CPython
dependencies:
  click: 8.1.7
  numpy: 1.26.4
  scipy: 1.14.1
  datalad: 1.1.3
  pandas: 2.2.3
  nibabel: 5.3.2
  nilearn: 0.10.4
  sqlalchemy: 2.0.36
  ruamel.yaml: 0.18.6
  templateflow: 24.2.2
  lazy_loader: '0.4'
  looseversion: None
  junifer_data: None
  structlog: 25.5.0
system:
  platform: Linux-6.12.9+bpo-amd64-x86_64-with-glibc2.36
environment:
  PATH: 
    /home/fraimondo/miniforge3/envs/junifer/bin:/home/fraimondo/.local/bin:/home/fraimondo/miniforge3/condabin:/usr/local/bin:/home/fraimondo/.vscode-server/data/User/globalStorage/github.copilot-chat/debugCommand:/home/fraimondo/.vscode-server/data/User/globalStorage/github.copilot-chat/copilotCli:/home/fraimondo/.vscode-server/cli/servers/Stable-072586267e68ece9a47aa43f8c108e0dcbf44622/server/bin/remote-cli:/home/fraimondo/.cargo/bin:/usr/local/bin:/usr/bin:/bin:/usr/games:/home/fraimondo/.vscode-server/extensions/ms-python.debugpy-2025.18.0-linux-x64/bundled/scripts/noConfigScripts

Relevant log output

2026-02-27 15:50:26 [warning  ] Requested extension 'next' is not available [datalad.support.entrypoints]
2026-02-27 15:50:31 [info     ] ===== Lib Versions =====       [junifer]
2026-02-27 15:50:31 [info     ] click: 8.1.7                   [junifer]
2026-02-27 15:50:31 [info     ] numpy: 1.26.4                  [junifer]
2026-02-27 15:50:31 [info     ] scipy: 1.14.1                  [junifer]
2026-02-27 15:50:31 [info     ] datalad: 1.1.3                 [junifer]
2026-02-27 15:50:31 [info     ] pandas: 2.2.3                  [junifer]
2026-02-27 15:50:31 [info     ] nibabel: 5.3.2                 [junifer]
2026-02-27 15:50:31 [info     ] nilearn: 0.10.4                [junifer]
2026-02-27 15:50:31 [info     ] sqlalchemy: 2.0.36             [junifer]
2026-02-27 15:50:31 [info     ] ruamel.yaml: 0.18.6            [junifer]
2026-02-27 15:50:31 [info     ] templateflow: 24.2.2           [junifer]
2026-02-27 15:50:31 [info     ] junifer_data: None             [junifer]
2026-02-27 15:50:31 [info     ] junifer: 0.0.7.dev320          [junifer]
2026-02-27 15:50:31 [info     ] ========================       [junifer]
2026-02-27 15:50:31 [info     ] Parsing yaml file: /home/fraimondo/dev/events/2026_EBRAINS/1_junifer/junifer_jobs/aomic_reho_native/config.yaml [junifer]
2026-02-27 15:50:31 [info     ] Registering PatternDataGrabber in datagrabber [junifer]
2026-02-27 15:50:31 [info     ] Registering PatternDataladDataGrabber in datagrabber [junifer]
2026-02-27 15:50:31 [info     ] Registering DataladAOMICID1000 in datagrabber [junifer]
2026-02-27 15:50:31 [info     ] `datadir` is None, creating a temporary directory [junifer]
2026-02-27 15:50:31 [info     ] `datadir` set to /tmp/tmpkal3yrx_/dataladu0wt22co/datadir [junifer]
2026-02-27 15:50:31 [info     ] Registering ParcelAggregation in marker [junifer]
2026-02-27 15:50:31 [info     ] Registering ReHoParcels in marker [junifer]
2026-02-27 15:50:31 [info     ] Registering HDF5FeatureStorage in storage [junifer]
2026-02-27 15:50:31 [info     ] Registering fMRIPrepConfoundRemover in preprocessing [junifer]
2026-02-27 15:50:31 [info     ] Registering SpaceWarper in preprocessing [junifer]
2026-02-27 15:50:31 [info     ] Validating Marker Collection   [junifer]
2026-02-27 15:50:31 [info     ] DataGrabber output type: ['BOLD', 'T1w', 'VBM_CSF', 'VBM_GM', 'VBM_WM', 'DWI', 'FreeSurfer', 'Warp'] [junifer]
2026-02-27 15:50:31 [info     ] Validating Data Reader:        [junifer]
2026-02-27 15:50:31 [info     ] Data Reader output type: ['BOLD', 'T1w', 'VBM_CSF', 'VBM_GM', 'VBM_WM', 'DWI', 'FreeSurfer', 'Warp'] [junifer]
2026-02-27 15:50:31 [info     ] Validating Preprocessor: fMRIPrepConfoundRemover [junifer]
2026-02-27 15:50:31 [info     ] Preprocessor input type: ['BOLD', 'T1w', 'VBM_CSF', 'VBM_GM', 'VBM_WM', 'DWI', 'FreeSurfer', 'Warp'] [junifer]
2026-02-27 15:50:31 [info     ] Preprocessor output type: ['VBM_GM', 'DWI', 'Warp', 'VBM_WM', 'T1w', 'VBM_CSF', 'FreeSurfer', 'BOLD'] [junifer]
2026-02-27 15:50:31 [info     ] Validating Preprocessor: SpaceWarper [junifer]
2026-02-27 15:50:31 [info     ] Preprocessor input type: ['VBM_GM', 'DWI', 'Warp', 'VBM_WM', 'T1w', 'VBM_CSF', 'FreeSurfer', 'BOLD'] [junifer]
2026-02-27 15:50:32 [info     ] Preprocessor output type: ['VBM_GM', 'DWI', 'Warp', 'T1w', 'VBM_WM', 'VBM_CSF', 'FreeSurfer', 'BOLD'] [junifer]
2026-02-27 15:50:32 [info     ] Validating Marker: ReHo-Julich-Brain_V3_1 [junifer]
2026-02-27 15:50:33 [info     ] Marker output type: ['vector'] [junifer]
2026-02-27 15:50:33 [info     ] Validating storage for ReHo-Julich-Brain_V3_1 [junifer]
2026-02-27 15:50:33 [info     ] Attempting a clone into /tmp/tmpkal3yrx_/dataladu0wt22co/datadir [datalad.core.distributed.clone] dlm_progress=cloneds dlm_progress_label=Cloning dlm_progress_total=2 dlm_progress_unit=' candidates'
2026-02-27 15:50:33 [info     ] Attempting to clone from https://github.com/OpenNeuroDatasets/ds003097.git to /tmp/tmpkal3yrx_/dataladu0wt22co/datadir [datalad.core.distributed.clone] dlm_progress=cloneds dlm_progress_increment=True dlm_progress_update=1
2026-02-27 15:51:39 [info     ] Completed clone attempts for Dataset(/tmp/tmpkal3yrx_/dataladu0wt22co/datadir) [datalad.core.distributed.clone] dlm_progress=cloneds
2026-02-27 15:51:46 [info     ] access to 1 dataset sibling s3-BACKUP not auto-enabled, enable with:
		datalad siblings -d "/tmp/tmpkal3yrx_/dataladu0wt22co/datadir" enable -s s3-BACKUP [datalad.core.distributed.clone]
2026-02-27 15:51:47 [info     ] Getting element sub-0001       [junifer]
2026-02-27 15:51:47 [info     ] Resolving path from pattern for BOLD [junifer]
2026-02-27 15:51:47 [info     ] Resolving path from pattern for BOLD.mask [junifer]
2026-02-27 15:51:47 [info     ] Resolving path from pattern for BOLD.confounds [junifer]
2026-02-27 15:51:47 [info     ] Resolving path from pattern for BOLD.reference [junifer]
2026-02-27 15:51:47 [info     ] Resolving path from pattern for T1w [junifer]
2026-02-27 15:51:47 [info     ] Resolving path from pattern for T1w.mask [junifer]
2026-02-27 15:51:47 [info     ] Resolving path from pattern for VBM_CSF [junifer]
2026-02-27 15:51:47 [info     ] Resolving path from pattern for VBM_GM [junifer]
2026-02-27 15:51:47 [info     ] Resolving path from pattern for VBM_WM [junifer]
2026-02-27 15:51:47 [info     ] Resolving path from pattern for DWI [junifer]
2026-02-27 15:51:47 [info     ] Resolving path from pattern for FreeSurfer [junifer]
2026-02-27 15:51:47 [info     ] Resolving path from pattern for FreeSurfer.aseg [junifer]
2026-02-27 15:51:47 [info     ] Resolving path from pattern for FreeSurfer.norm [junifer]
2026-02-27 15:51:47 [info     ] Resolving path from pattern for FreeSurfer.lh_white [junifer]
2026-02-27 15:51:47 [info     ] Resolving path from pattern for FreeSurfer.rh_white [junifer]
2026-02-27 15:51:47 [info     ] Resolving path from pattern for FreeSurfer.lh_pial [junifer]
2026-02-27 15:51:47 [info     ] Resolving path from pattern for FreeSurfer.rh_pial [junifer]
2026-02-27 15:51:47 [info     ] Resolving path from pattern for Warp.0 [junifer]
2026-02-27 15:51:47 [info     ] Resolving path from pattern for Warp.1 [junifer]
2026-02-27 15:54:29 [info     ] Fitting pipeline               [junifer]
2026-02-27 15:54:29 [info     ] Reading BOLD.mask from /tmp/tmpkal3yrx_/dataladu0wt22co/datadir/derivatives/fmriprep/sub-0001/func/sub-0001_task-moviewatching_space-T1w_desc-brain_mask.nii.gz [junifer]
2026-02-27 15:54:29 [info     ] BOLD.mask is of type NIFTI     [junifer]
2026-02-27 15:54:29 [info     ] Reading BOLD.confounds from /tmp/tmpkal3yrx_/dataladu0wt22co/datadir/derivatives/fmriprep/sub-0001/func/sub-0001_task-moviewatching_desc-confounds_regressors.tsv [junifer]
2026-02-27 15:54:29 [info     ] BOLD.confounds is of type TSV  [junifer]
2026-02-27 15:54:29 [info     ] Reading BOLD.reference from /tmp/tmpkal3yrx_/dataladu0wt22co/datadir/derivatives/fmriprep/sub-0001/func/sub-0001_task-moviewatching_space-T1w_boldref.nii.gz [junifer]
2026-02-27 15:54:29 [info     ] BOLD.reference is of type NIFTI [junifer]
2026-02-27 15:54:29 [info     ] Reading BOLD from /tmp/tmpkal3yrx_/dataladu0wt22co/datadir/derivatives/fmriprep/sub-0001/func/sub-0001_task-moviewatching_space-T1w_desc-preproc_bold.nii.gz [junifer]
2026-02-27 15:54:29 [info     ] BOLD is of type NIFTI          [junifer]
2026-02-27 15:54:29 [info     ] Reading T1w.mask from /tmp/tmpkal3yrx_/dataladu0wt22co/datadir/derivatives/fmriprep/sub-0001/anat/sub-0001_desc-brain_mask.nii.gz [junifer]
2026-02-27 15:54:29 [info     ] T1w.mask is of type NIFTI      [junifer]
2026-02-27 15:54:29 [info     ] Reading T1w from /tmp/tmpkal3yrx_/dataladu0wt22co/datadir/derivatives/fmriprep/sub-0001/anat/sub-0001_desc-preproc_T1w.nii.gz [junifer]
2026-02-27 15:54:29 [info     ] T1w is of type NIFTI           [junifer]
2026-02-27 15:54:29 [info     ] Reading VBM_CSF from /tmp/tmpkal3yrx_/dataladu0wt22co/datadir/derivatives/fmriprep/sub-0001/anat/sub-0001_label-CSF_probseg.nii.gz [junifer]
2026-02-27 15:54:29 [info     ] VBM_CSF is of type NIFTI       [junifer]
2026-02-27 15:54:29 [info     ] Reading VBM_GM from /tmp/tmpkal3yrx_/dataladu0wt22co/datadir/derivatives/fmriprep/sub-0001/anat/sub-0001_label-GM_probseg.nii.gz [junifer]
2026-02-27 15:54:29 [info     ] VBM_GM is of type NIFTI        [junifer]
2026-02-27 15:54:29 [info     ] Reading VBM_WM from /tmp/tmpkal3yrx_/dataladu0wt22co/datadir/derivatives/fmriprep/sub-0001/anat/sub-0001_label-WM_probseg.nii.gz [junifer]
2026-02-27 15:54:29 [info     ] VBM_WM is of type NIFTI        [junifer]
2026-02-27 15:54:29 [info     ] Reading DWI from /tmp/tmpkal3yrx_/dataladu0wt22co/datadir/derivatives/dwipreproc/sub-0001/dwi/sub-0001_desc-preproc_dwi.nii.gz [junifer]
2026-02-27 15:54:29 [info     ] DWI is of type NIFTI           [junifer]
2026-02-27 15:54:29 [info     ] Preprocessing data with fMRIPrepConfoundRemover [junifer]
2026-02-27 15:54:29 [info     ] Preprocessing BOLD             [junifer]
2026-02-27 15:54:30 [info     ] No `t_r` specified, using t_r from NIfTI header [junifer]
2026-02-27 15:54:30 [info     ] Read t_r from NIfTI header: 2.200000047683716 [junifer]
Traceback (most recent call last):
  File "/home/fraimondo/miniforge3/envs/junifer/bin/junifer", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/cli/cli.py", line 185, in run
    cli_func.run(
  File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/api/functions.py", line 223, in run
    mc.fit(datagrabber_object[t_element])
  File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/pipeline/marker_collection.py", line 103, in fit
    data = preprocessor.fit_transform(data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/pipeline/pipeline_step_mixin.py", line 232, in fit_transform
    return self._fit_transform(input=input, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/preprocess/base.py", line 181, in _fit_transform
    t_out = self.preprocess(input=t_input, extra_input=extra_input)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/preprocess/confounds/fmriprep_confound_remover.py", line 640, in preprocess
    mask_img = get_data(
               ^^^^^^^^^
  File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/data/_dispatch.py", line 160, in get_data
    return registry().get(
           ^^^^^^^^^^^^^^^
  File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/data/masks/_masks.py", line 487, in get
    set(masks) == {nilearn_callable_masks[0]}
    ^^^^^^^^^^
TypeError: unhashable type: 'CommentedMap'

Anything else?

No response

### Is there an existing issue for this? - [x] I have searched the existing issues ### Current Behavior I want to compute ReHo/FC in native space. fMRIPrepConfoundRemover fails. I get this error ``` 2026-02-27 15:54:29 [info ] Preprocessing data with fMRIPrepConfoundRemover [junifer] 2026-02-27 15:54:29 [info ] Preprocessing BOLD [junifer] 2026-02-27 15:54:30 [info ] No `t_r` specified, using t_r from NIfTI header [junifer] 2026-02-27 15:54:30 [info ] Read t_r from NIfTI header: 2.200000047683716 [junifer] Traceback (most recent call last): File "/home/fraimondo/miniforge3/envs/junifer/bin/junifer", line 8, in <module> sys.exit(cli()) ^^^^^ File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/click/core.py", line 1157, in __call__ return self.main(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/cli/cli.py", line 185, in run cli_func.run( File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/api/functions.py", line 223, in run mc.fit(datagrabber_object[t_element]) File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/pipeline/marker_collection.py", line 103, in fit data = preprocessor.fit_transform(data) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/pipeline/pipeline_step_mixin.py", line 232, in fit_transform return self._fit_transform(input=input, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/preprocess/base.py", line 181, in _fit_transform t_out = self.preprocess(input=t_input, extra_input=extra_input) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/preprocess/confounds/fmriprep_confound_remover.py", line 640, in preprocess mask_img = get_data( ^^^^^^^^^ File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/data/_dispatch.py", line 160, in get_data return registry().get( ^^^^^^^^^^^^^^^ File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/data/masks/_masks.py", line 487, in get set(masks) == {nilearn_callable_masks[0]} ^^^^^^^^^^ TypeError: unhashable type: 'CommentedMap' ``` ### Expected Behavior Obtain markers in native space ### Steps To Reproduce 1. With junifer latest 2. Run any element from the AOMIC ID1000 datasets ```yaml workdir: /tmp datagrabber: kind: DataladAOMICID1000 space: native preprocess: - kind: fMRIPrepConfoundRemover detrend: true standardize: true strategy: motion: full wm_csf: full global_signal: full low_pass: 0.08 high_pass: 0.01 masks: - inherit - compute_epi_mask - threshold: 0 - kind: SpaceWarper reference: T1w on: BOLD using: ants markers: - name: ReHo-Julich-Brain_V3_1 kind: ReHoParcels parcellation: Julich-Brain_V3_1 using: afni masks: - inherit storage: kind: HDF5FeatureStorage uri: /data/group/riseml/fraimondo/2026_EBRAINS/features/ds003097_ReHo_native/ds003097_ReHo_native.hdf5 queue: jobname: aomic_reho_native kind: HTCondor env: kind: conda name: junifer mem: 8G disk: 5G verbose: info pre_run: | source /data/group/appliedml/tools/afni_24.3.06/afni.sh source /data/group/appliedml/tools/ants_2.5.0/ants.sh source /data/group/appliedml/tools/fsl_6.0.4-patched2/fsl.sh ``` ### Environment ```markdown junifer: version: 0.0.7.dev320 python: version: 3.12.7 implementation: CPython dependencies: click: 8.1.7 numpy: 1.26.4 scipy: 1.14.1 datalad: 1.1.3 pandas: 2.2.3 nibabel: 5.3.2 nilearn: 0.10.4 sqlalchemy: 2.0.36 ruamel.yaml: 0.18.6 templateflow: 24.2.2 lazy_loader: '0.4' looseversion: None junifer_data: None structlog: 25.5.0 system: platform: Linux-6.12.9+bpo-amd64-x86_64-with-glibc2.36 environment: PATH: /home/fraimondo/miniforge3/envs/junifer/bin:/home/fraimondo/.local/bin:/home/fraimondo/miniforge3/condabin:/usr/local/bin:/home/fraimondo/.vscode-server/data/User/globalStorage/github.copilot-chat/debugCommand:/home/fraimondo/.vscode-server/data/User/globalStorage/github.copilot-chat/copilotCli:/home/fraimondo/.vscode-server/cli/servers/Stable-072586267e68ece9a47aa43f8c108e0dcbf44622/server/bin/remote-cli:/home/fraimondo/.cargo/bin:/usr/local/bin:/usr/bin:/bin:/usr/games:/home/fraimondo/.vscode-server/extensions/ms-python.debugpy-2025.18.0-linux-x64/bundled/scripts/noConfigScripts ``` ### Relevant log output ```shell 2026-02-27 15:50:26 [warning ] Requested extension 'next' is not available [datalad.support.entrypoints] 2026-02-27 15:50:31 [info ] ===== Lib Versions ===== [junifer] 2026-02-27 15:50:31 [info ] click: 8.1.7 [junifer] 2026-02-27 15:50:31 [info ] numpy: 1.26.4 [junifer] 2026-02-27 15:50:31 [info ] scipy: 1.14.1 [junifer] 2026-02-27 15:50:31 [info ] datalad: 1.1.3 [junifer] 2026-02-27 15:50:31 [info ] pandas: 2.2.3 [junifer] 2026-02-27 15:50:31 [info ] nibabel: 5.3.2 [junifer] 2026-02-27 15:50:31 [info ] nilearn: 0.10.4 [junifer] 2026-02-27 15:50:31 [info ] sqlalchemy: 2.0.36 [junifer] 2026-02-27 15:50:31 [info ] ruamel.yaml: 0.18.6 [junifer] 2026-02-27 15:50:31 [info ] templateflow: 24.2.2 [junifer] 2026-02-27 15:50:31 [info ] junifer_data: None [junifer] 2026-02-27 15:50:31 [info ] junifer: 0.0.7.dev320 [junifer] 2026-02-27 15:50:31 [info ] ======================== [junifer] 2026-02-27 15:50:31 [info ] Parsing yaml file: /home/fraimondo/dev/events/2026_EBRAINS/1_junifer/junifer_jobs/aomic_reho_native/config.yaml [junifer] 2026-02-27 15:50:31 [info ] Registering PatternDataGrabber in datagrabber [junifer] 2026-02-27 15:50:31 [info ] Registering PatternDataladDataGrabber in datagrabber [junifer] 2026-02-27 15:50:31 [info ] Registering DataladAOMICID1000 in datagrabber [junifer] 2026-02-27 15:50:31 [info ] `datadir` is None, creating a temporary directory [junifer] 2026-02-27 15:50:31 [info ] `datadir` set to /tmp/tmpkal3yrx_/dataladu0wt22co/datadir [junifer] 2026-02-27 15:50:31 [info ] Registering ParcelAggregation in marker [junifer] 2026-02-27 15:50:31 [info ] Registering ReHoParcels in marker [junifer] 2026-02-27 15:50:31 [info ] Registering HDF5FeatureStorage in storage [junifer] 2026-02-27 15:50:31 [info ] Registering fMRIPrepConfoundRemover in preprocessing [junifer] 2026-02-27 15:50:31 [info ] Registering SpaceWarper in preprocessing [junifer] 2026-02-27 15:50:31 [info ] Validating Marker Collection [junifer] 2026-02-27 15:50:31 [info ] DataGrabber output type: ['BOLD', 'T1w', 'VBM_CSF', 'VBM_GM', 'VBM_WM', 'DWI', 'FreeSurfer', 'Warp'] [junifer] 2026-02-27 15:50:31 [info ] Validating Data Reader: [junifer] 2026-02-27 15:50:31 [info ] Data Reader output type: ['BOLD', 'T1w', 'VBM_CSF', 'VBM_GM', 'VBM_WM', 'DWI', 'FreeSurfer', 'Warp'] [junifer] 2026-02-27 15:50:31 [info ] Validating Preprocessor: fMRIPrepConfoundRemover [junifer] 2026-02-27 15:50:31 [info ] Preprocessor input type: ['BOLD', 'T1w', 'VBM_CSF', 'VBM_GM', 'VBM_WM', 'DWI', 'FreeSurfer', 'Warp'] [junifer] 2026-02-27 15:50:31 [info ] Preprocessor output type: ['VBM_GM', 'DWI', 'Warp', 'VBM_WM', 'T1w', 'VBM_CSF', 'FreeSurfer', 'BOLD'] [junifer] 2026-02-27 15:50:31 [info ] Validating Preprocessor: SpaceWarper [junifer] 2026-02-27 15:50:31 [info ] Preprocessor input type: ['VBM_GM', 'DWI', 'Warp', 'VBM_WM', 'T1w', 'VBM_CSF', 'FreeSurfer', 'BOLD'] [junifer] 2026-02-27 15:50:32 [info ] Preprocessor output type: ['VBM_GM', 'DWI', 'Warp', 'T1w', 'VBM_WM', 'VBM_CSF', 'FreeSurfer', 'BOLD'] [junifer] 2026-02-27 15:50:32 [info ] Validating Marker: ReHo-Julich-Brain_V3_1 [junifer] 2026-02-27 15:50:33 [info ] Marker output type: ['vector'] [junifer] 2026-02-27 15:50:33 [info ] Validating storage for ReHo-Julich-Brain_V3_1 [junifer] 2026-02-27 15:50:33 [info ] Attempting a clone into /tmp/tmpkal3yrx_/dataladu0wt22co/datadir [datalad.core.distributed.clone] dlm_progress=cloneds dlm_progress_label=Cloning dlm_progress_total=2 dlm_progress_unit=' candidates' 2026-02-27 15:50:33 [info ] Attempting to clone from https://github.com/OpenNeuroDatasets/ds003097.git to /tmp/tmpkal3yrx_/dataladu0wt22co/datadir [datalad.core.distributed.clone] dlm_progress=cloneds dlm_progress_increment=True dlm_progress_update=1 2026-02-27 15:51:39 [info ] Completed clone attempts for Dataset(/tmp/tmpkal3yrx_/dataladu0wt22co/datadir) [datalad.core.distributed.clone] dlm_progress=cloneds 2026-02-27 15:51:46 [info ] access to 1 dataset sibling s3-BACKUP not auto-enabled, enable with: datalad siblings -d "/tmp/tmpkal3yrx_/dataladu0wt22co/datadir" enable -s s3-BACKUP [datalad.core.distributed.clone] 2026-02-27 15:51:47 [info ] Getting element sub-0001 [junifer] 2026-02-27 15:51:47 [info ] Resolving path from pattern for BOLD [junifer] 2026-02-27 15:51:47 [info ] Resolving path from pattern for BOLD.mask [junifer] 2026-02-27 15:51:47 [info ] Resolving path from pattern for BOLD.confounds [junifer] 2026-02-27 15:51:47 [info ] Resolving path from pattern for BOLD.reference [junifer] 2026-02-27 15:51:47 [info ] Resolving path from pattern for T1w [junifer] 2026-02-27 15:51:47 [info ] Resolving path from pattern for T1w.mask [junifer] 2026-02-27 15:51:47 [info ] Resolving path from pattern for VBM_CSF [junifer] 2026-02-27 15:51:47 [info ] Resolving path from pattern for VBM_GM [junifer] 2026-02-27 15:51:47 [info ] Resolving path from pattern for VBM_WM [junifer] 2026-02-27 15:51:47 [info ] Resolving path from pattern for DWI [junifer] 2026-02-27 15:51:47 [info ] Resolving path from pattern for FreeSurfer [junifer] 2026-02-27 15:51:47 [info ] Resolving path from pattern for FreeSurfer.aseg [junifer] 2026-02-27 15:51:47 [info ] Resolving path from pattern for FreeSurfer.norm [junifer] 2026-02-27 15:51:47 [info ] Resolving path from pattern for FreeSurfer.lh_white [junifer] 2026-02-27 15:51:47 [info ] Resolving path from pattern for FreeSurfer.rh_white [junifer] 2026-02-27 15:51:47 [info ] Resolving path from pattern for FreeSurfer.lh_pial [junifer] 2026-02-27 15:51:47 [info ] Resolving path from pattern for FreeSurfer.rh_pial [junifer] 2026-02-27 15:51:47 [info ] Resolving path from pattern for Warp.0 [junifer] 2026-02-27 15:51:47 [info ] Resolving path from pattern for Warp.1 [junifer] 2026-02-27 15:54:29 [info ] Fitting pipeline [junifer] 2026-02-27 15:54:29 [info ] Reading BOLD.mask from /tmp/tmpkal3yrx_/dataladu0wt22co/datadir/derivatives/fmriprep/sub-0001/func/sub-0001_task-moviewatching_space-T1w_desc-brain_mask.nii.gz [junifer] 2026-02-27 15:54:29 [info ] BOLD.mask is of type NIFTI [junifer] 2026-02-27 15:54:29 [info ] Reading BOLD.confounds from /tmp/tmpkal3yrx_/dataladu0wt22co/datadir/derivatives/fmriprep/sub-0001/func/sub-0001_task-moviewatching_desc-confounds_regressors.tsv [junifer] 2026-02-27 15:54:29 [info ] BOLD.confounds is of type TSV [junifer] 2026-02-27 15:54:29 [info ] Reading BOLD.reference from /tmp/tmpkal3yrx_/dataladu0wt22co/datadir/derivatives/fmriprep/sub-0001/func/sub-0001_task-moviewatching_space-T1w_boldref.nii.gz [junifer] 2026-02-27 15:54:29 [info ] BOLD.reference is of type NIFTI [junifer] 2026-02-27 15:54:29 [info ] Reading BOLD from /tmp/tmpkal3yrx_/dataladu0wt22co/datadir/derivatives/fmriprep/sub-0001/func/sub-0001_task-moviewatching_space-T1w_desc-preproc_bold.nii.gz [junifer] 2026-02-27 15:54:29 [info ] BOLD is of type NIFTI [junifer] 2026-02-27 15:54:29 [info ] Reading T1w.mask from /tmp/tmpkal3yrx_/dataladu0wt22co/datadir/derivatives/fmriprep/sub-0001/anat/sub-0001_desc-brain_mask.nii.gz [junifer] 2026-02-27 15:54:29 [info ] T1w.mask is of type NIFTI [junifer] 2026-02-27 15:54:29 [info ] Reading T1w from /tmp/tmpkal3yrx_/dataladu0wt22co/datadir/derivatives/fmriprep/sub-0001/anat/sub-0001_desc-preproc_T1w.nii.gz [junifer] 2026-02-27 15:54:29 [info ] T1w is of type NIFTI [junifer] 2026-02-27 15:54:29 [info ] Reading VBM_CSF from /tmp/tmpkal3yrx_/dataladu0wt22co/datadir/derivatives/fmriprep/sub-0001/anat/sub-0001_label-CSF_probseg.nii.gz [junifer] 2026-02-27 15:54:29 [info ] VBM_CSF is of type NIFTI [junifer] 2026-02-27 15:54:29 [info ] Reading VBM_GM from /tmp/tmpkal3yrx_/dataladu0wt22co/datadir/derivatives/fmriprep/sub-0001/anat/sub-0001_label-GM_probseg.nii.gz [junifer] 2026-02-27 15:54:29 [info ] VBM_GM is of type NIFTI [junifer] 2026-02-27 15:54:29 [info ] Reading VBM_WM from /tmp/tmpkal3yrx_/dataladu0wt22co/datadir/derivatives/fmriprep/sub-0001/anat/sub-0001_label-WM_probseg.nii.gz [junifer] 2026-02-27 15:54:29 [info ] VBM_WM is of type NIFTI [junifer] 2026-02-27 15:54:29 [info ] Reading DWI from /tmp/tmpkal3yrx_/dataladu0wt22co/datadir/derivatives/dwipreproc/sub-0001/dwi/sub-0001_desc-preproc_dwi.nii.gz [junifer] 2026-02-27 15:54:29 [info ] DWI is of type NIFTI [junifer] 2026-02-27 15:54:29 [info ] Preprocessing data with fMRIPrepConfoundRemover [junifer] 2026-02-27 15:54:29 [info ] Preprocessing BOLD [junifer] 2026-02-27 15:54:30 [info ] No `t_r` specified, using t_r from NIfTI header [junifer] 2026-02-27 15:54:30 [info ] Read t_r from NIfTI header: 2.200000047683716 [junifer] Traceback (most recent call last): File "/home/fraimondo/miniforge3/envs/junifer/bin/junifer", line 8, in <module> sys.exit(cli()) ^^^^^ File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/click/core.py", line 1157, in __call__ return self.main(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/cli/cli.py", line 185, in run cli_func.run( File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/api/functions.py", line 223, in run mc.fit(datagrabber_object[t_element]) File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/pipeline/marker_collection.py", line 103, in fit data = preprocessor.fit_transform(data) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/pipeline/pipeline_step_mixin.py", line 232, in fit_transform return self._fit_transform(input=input, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/preprocess/base.py", line 181, in _fit_transform t_out = self.preprocess(input=t_input, extra_input=extra_input) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/preprocess/confounds/fmriprep_confound_remover.py", line 640, in preprocess mask_img = get_data( ^^^^^^^^^ File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/data/_dispatch.py", line 160, in get_data return registry().get( ^^^^^^^^^^^^^^^ File "/home/fraimondo/miniforge3/envs/junifer/lib/python3.12/site-packages/junifer/data/masks/_masks.py", line 487, in get set(masks) == {nilearn_callable_masks[0]} ^^^^^^^^^^ TypeError: unhashable type: 'CommentedMap' ``` ### Anything else? _No response_
synchon commented 2026-03-02 15:17:59 +00:00 (Migrated from github.com)
Copy link

@fraimondo Can you check if this branch solves your issue? Also you are working with native space data from the beginning, the SpaceWarper is likely not needed.

@fraimondo Can you check if this branch solves your issue? Also you are working with native space data from the beginning, the SpaceWarper is likely not needed.
fraimondo (Migrated from github.com) approved these changes 2026-03-03 13:10:37 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
fraimondo
Labels
Clear labels   
CRITICAL
High Priority Issue

  
Stale

   
WIP
Work in progress

  
bug
Something isn't working

  
concept
Design/Implementation concepts

  
coordinate
Issues or pull requests related to coordinates (ROIs)

  
dataset
Issues or pull requests related to datagrabbers or datasets

  
dependencies
Pull requests that update a dependency file

  
documentation
Improvements or additions to documentation

  
duplicate
This issue or pull request already exists

  
enhancement
New feature or request

  
github_actions
Pull requests that update GitHub Actions code

  
good first issue
Good for newcomers

  
help wanted
Extra attention is needed

  
invalid
This doesn't seem right

  
maintenance
Issues or pull requests related to maintenance

  
maps
Issues or pull requests related to maps (probabilistic atlases)

  
marker
Issues or pull requests related to markers

  
mask
Issues or pull requests related to masks

  
on hold
Issue waiting on other issue(s)

  
parcellation
Issues or pull requests related to parcellations (deterministic atlases)

  
preprocess
Issues or pull requests related to preprocessors

  
question
Further information is requested

  
ready
Pull request is ready for review and merge

  
storage
Issues or pull requests related to marker storage or retrieval

  
template-space
Issues or pull requests related to template spaces

  
triage
New issues waiting to be reviewed

  
wontfix
This will not be worked on

No labels
CRITICAL
Stale
WIP
bug
concept
coordinate
dataset
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
maintenance
maps
marker
mask
on hold
parcellation
preprocess
question
ready
storage
template-space
triage
wontfix
Milestone
Clear milestone
No items
No milestone
0.0.7 (alpha 6)
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
juaml/junifer!487
No description provided.