[BUG]: External dependencies not found despite being there #385

Merged

synchon merged 2 commits from fix/ants-ext-deps-check into main

2024-12-02 12:19:34 +00:00

synchon commented

2024-12-02 11:22:57 +00:00

(Migrated from github.com)

Is there an existing issue for this?

I have searched the existing issues

Current Behavior

There's a warning that some external commands are not available (e.g. ANTs' ResampleImage) but in the end it works.

Expected Behavior

I would expect the warning to not be there.

Steps To Reproduce

In juseless, run any yaml

Environment

junifer:
  version: 0.0.6.dev176
python:
  version: 3.12.6
  implementation: CPython
dependencies:
  click: 8.1.7
  numpy: 1.26.4
  scipy: 1.14.1
  datalad: 1.1.3
  pandas: 2.2.2
  nibabel: 5.2.1
  ruamel.yaml: 0.17.40
  looseversion: None
system:
  platform: Linux-6.6.13+bpo-amd64-x86_64-with-glibc2.36
environment:
  PATH:
    /data/group/appliedml/tools/ants_2.5.0/binaries:/home/fraimondo/miniconda3/envs/junifer/bin:/home/fraimondo/miniconda3/condabin:/usr/local/bin:/usr/bin:/bin:/usr/games

Relevant log output

2024-10-24 10:35:45,780 [ WARNING] /home/fraimondo/miniconda3/envs/junifer/lib/python3.12/site-packages/junifer/pipeline/utils.py:248: RuntimeWarning: ANTs is installed but some of the required commands were not found. These are the results: {'ResampleImage': 'not found', 'antsApplyTransforms': 'not found'}
  warn_with_log(

2024-10-24 10:35:45,780 - JUNIFER - INFO - Preprocessing BOLD
2024-10-24 10:35:45,780 [    INFO] Preprocessing BOLD
2024-10-24 10:35:45,780 - JUNIFER - INFO - Warping to T1w space using SpaceWarper
2024-10-24 10:35:45,780 [    INFO] Warping to T1w space using SpaceWarper
2024-10-24 10:35:45,781 - JUNIFER - INFO - ResampleImage command to be executed:
ResampleImage 3 /tmp/tmpl7l_8aka/datadir/.git/annex/objects/91/63/MD5E-s31808414--24945231a2d73d0d08c4e402abd2f866.nii.gz/MD5E-s31808414--24945231a2d73d0d08c4e402abd2f866.nii.gz /tmp/tmp_t5pk2tp/ants_warperpjn8gvog/resampled_reference.nii.gz 3.0x3.0x3.0 0 3 3
2024-10-24 10:35:45,781 [    INFO] ResampleImage command to be executed:
ResampleImage 3 /tmp/tmpl7l_8aka/datadir/.git/annex/objects/91/63/MD5E-s31808414--24945231a2d73d0d08c4e402abd2f866.nii.gz/MD5E-s31808414--24945231a2d73d0d08c4e402abd2f866.nii.gz /tmp/tmp_t5pk2tp/ants_warperpjn8gvog/resampled_reference.nii.gz 3.0x3.0x3.0 0 3 3
2024-10-24 10:35:59,993 - JUNIFER - INFO - ResampleImage command succeeded with the following output:
/tmp/tmpl7l_8aka/datadir/.git/annex/objects/91/63/MD5E-s31808414--24945231a2d73d0d08c4e402abd2f866.nii.gz/MD5E-s31808414--24945231a2d73d0d08c4e402abd2f866.nii.gz is a file

Anything else?

Indeed the issue seems to be that sometimes the commands output a non-zero status just because the parameters were not set correctly:

(junifer)juseless ➜  junifer git:(main) ResampleImage
Singularity args:
Corrected args for ANTS: ResampleImage
Running command: /data/group/appliedml/tools/ants_2.5.0/singularity_cmd exec --cleanenv  /data/group/appliedml/tools/ants_2.5.0/ants_v2.5.0.sif ResampleImage
Usage: ResampleImage imageDimension inputImage outputImage MxNxO [size=1,spacing=0] [interpolate type] [pixeltype]
  Interpolation type:
    0. linear (default)
    1. nn
    2. gaussian [sigma=imageSpacing] [alpha=1.0]
    3. windowedSinc [type = 'c'osine, 'w'elch, 'b'lackman, 'l'anczos, 'h'amming]
    4. B-Spline [order=3]
 pixeltype  :  TYPE
  0  :  char
  1  :  unsigned char
  2  :  short
  3  :  unsigned short
  4  :  int
  5  :  unsigned int
  6  :  float (default)
  7  :  double
(junifer)juseless ➜  junifer git:(main) echo $?
1

And we are checking only for zero commands:

github.com/juaml/junifer@299b4549c8/junifer/pipeline/utils.py (L232-L243)

### Is there an existing issue for this? - [X] I have searched the existing issues ### Current Behavior There's a warning that some external commands are not available (e.g. ANTs' `ResampleImage`) but in the end it works. ### Expected Behavior I would expect the warning to not be there. ### Steps To Reproduce 1. In juseless, run any yaml ### Environment ```markdown junifer: version: 0.0.6.dev176 python: version: 3.12.6 implementation: CPython dependencies: click: 8.1.7 numpy: 1.26.4 scipy: 1.14.1 datalad: 1.1.3 pandas: 2.2.2 nibabel: 5.2.1 ruamel.yaml: 0.17.40 looseversion: None system: platform: Linux-6.6.13+bpo-amd64-x86_64-with-glibc2.36 environment: PATH: /data/group/appliedml/tools/ants_2.5.0/binaries:/home/fraimondo/miniconda3/envs/junifer/bin:/home/fraimondo/miniconda3/condabin:/usr/local/bin:/usr/bin:/bin:/usr/games ``` ### Relevant log output ```shell 2024-10-24 10:35:45,780 [ WARNING] /home/fraimondo/miniconda3/envs/junifer/lib/python3.12/site-packages/junifer/pipeline/utils.py:248: RuntimeWarning: ANTs is installed but some of the required commands were not found. These are the results: {'ResampleImage': 'not found', 'antsApplyTransforms': 'not found'} warn_with_log( 2024-10-24 10:35:45,780 - JUNIFER - INFO - Preprocessing BOLD 2024-10-24 10:35:45,780 [ INFO] Preprocessing BOLD 2024-10-24 10:35:45,780 - JUNIFER - INFO - Warping to T1w space using SpaceWarper 2024-10-24 10:35:45,780 [ INFO] Warping to T1w space using SpaceWarper 2024-10-24 10:35:45,781 - JUNIFER - INFO - ResampleImage command to be executed: ResampleImage 3 /tmp/tmpl7l_8aka/datadir/.git/annex/objects/91/63/MD5E-s31808414--24945231a2d73d0d08c4e402abd2f866.nii.gz/MD5E-s31808414--24945231a2d73d0d08c4e402abd2f866.nii.gz /tmp/tmp_t5pk2tp/ants_warperpjn8gvog/resampled_reference.nii.gz 3.0x3.0x3.0 0 3 3 2024-10-24 10:35:45,781 [ INFO] ResampleImage command to be executed: ResampleImage 3 /tmp/tmpl7l_8aka/datadir/.git/annex/objects/91/63/MD5E-s31808414--24945231a2d73d0d08c4e402abd2f866.nii.gz/MD5E-s31808414--24945231a2d73d0d08c4e402abd2f866.nii.gz /tmp/tmp_t5pk2tp/ants_warperpjn8gvog/resampled_reference.nii.gz 3.0x3.0x3.0 0 3 3 2024-10-24 10:35:59,993 - JUNIFER - INFO - ResampleImage command succeeded with the following output: /tmp/tmpl7l_8aka/datadir/.git/annex/objects/91/63/MD5E-s31808414--24945231a2d73d0d08c4e402abd2f866.nii.gz/MD5E-s31808414--24945231a2d73d0d08c4e402abd2f866.nii.gz is a file ``` ### Anything else? Indeed the issue seems to be that sometimes the commands output a non-zero status just because the parameters were not set correctly: ``` (junifer)juseless ➜ junifer git:(main) ResampleImage Singularity args: Corrected args for ANTS: ResampleImage Running command: /data/group/appliedml/tools/ants_2.5.0/singularity_cmd exec --cleanenv /data/group/appliedml/tools/ants_2.5.0/ants_v2.5.0.sif ResampleImage Usage: ResampleImage imageDimension inputImage outputImage MxNxO [size=1,spacing=0] [interpolate type] [pixeltype] Interpolation type: 0. linear (default) 1. nn 2. gaussian [sigma=imageSpacing] [alpha=1.0] 3. windowedSinc [type = 'c'osine, 'w'elch, 'b'lackman, 'l'anczos, 'h'amming] 4. B-Spline [order=3] pixeltype : TYPE 0 : char 1 : unsigned char 2 : short 3 : unsigned short 4 : int 5 : unsigned int 6 : float (default) 7 : double (junifer)juseless ➜ junifer git:(main) echo $? 1 ``` And we are checking only for zero commands: https://github.com/juaml/junifer/blob/299b4549c87b98c809c5f3cae18443e0ad364e13/junifer/pipeline/utils.py#L232-L243

codecov[bot] commented

2024-12-02 11:24:00 +00:00

(Migrated from github.com)

Attention: Patch coverage is 0% with 1 line in your changes missing coverage. Please review.

Project coverage is 0.01%. Comparing base (c2f3de8) to head (b58fb91).
Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
junifer/pipeline/utils.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@          Coverage Diff          @@
##            main    #385   +/-   ##
=====================================
  Coverage   0.01%   0.01%           
=====================================
  Files        133     133           
  Lines       5583    5583           
=====================================
  Hits           1       1           
  Misses      5582    5582

Flag	Coverage Δ
docs	`100.00% <ø> (ø)`
junifer	`0.00% <0.00%> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
junifer/pipeline/utils.py	`0.00% <0.00%> (ø)`

## [Codecov](https://app.codecov.io/gh/juaml/junifer/pull/385?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml) Report Attention: Patch coverage is `0%` with `1 line` in your changes missing coverage. Please review. > Project coverage is 0.01%. Comparing base [(`c2f3de8`)](https://app.codecov.io/gh/juaml/junifer/commit/c2f3de87de28919a051321c357309aa6cebeff55?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml) to head [(`b58fb91`)](https://app.codecov.io/gh/juaml/junifer/commit/b58fb9152bc8546c4556018c308512616a6d2997?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml). > Report is 3 commits behind head on main. | [Files with missing lines](https://app.codecov.io/gh/juaml/junifer/pull/385?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml) | Patch % | Lines | |---|---|---| | [junifer/pipeline/utils.py](https://app.codecov.io/gh/juaml/junifer/pull/385?src=pr&el=tree&filepath=junifer%2Fpipeline%2Futils.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml#diff-anVuaWZlci9waXBlbGluZS91dGlscy5weQ==) | 0.00% | [1 Missing :warning: ](https://app.codecov.io/gh/juaml/junifer/pull/385?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml) | <details><summary>Additional details and impacted files</summary> [![Impacted file tree graph](https://app.codecov.io/gh/juaml/junifer/pull/385/graphs/tree.svg?width=650&height=150&src=pr&token=5H21JuZXMw&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml)](https://app.codecov.io/gh/juaml/junifer/pull/385?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml) ```diff @@ Coverage Diff @@ ## main #385 +/- ## ===================================== Coverage 0.01% 0.01% ===================================== Files 133 133 Lines 5583 5583 ===================================== Hits 1 1 Misses 5582 5582 ``` | [Flag](https://app.codecov.io/gh/juaml/junifer/pull/385/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml) | Coverage Δ | | |---|---|---| | [docs](https://app.codecov.io/gh/juaml/junifer/pull/385/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml) | `100.00% <ø> (ø)` | | | [junifer](https://app.codecov.io/gh/juaml/junifer/pull/385/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml) | `0.00% <0.00%> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml#carryforward-flags-in-the-pull-request-comment) to find out more. | [Files with missing lines](https://app.codecov.io/gh/juaml/junifer/pull/385?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml) | Coverage Δ | | |---|---|---| | [junifer/pipeline/utils.py](https://app.codecov.io/gh/juaml/junifer/pull/385?src=pr&el=tree&filepath=junifer%2Fpipeline%2Futils.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml#diff-anVuaWZlci9waXBlbGluZS91dGlscy5weQ==) | `0.00% <0.00%> (ø)` | | </details>

github-actions[bot] commented

2024-12-02 11:39:37 +00:00

(Migrated from github.com)

PR Preview Action v1.4.8
Preview removed because the pull request was closed.
2024-12-02 12:31 UTC

[PR Preview Action](https://github.com/rossjrw/pr-preview-action) v1.4.8 :---: Preview removed because the pull request was closed. 2024-12-02 12:31 UTC

fraimondo (Migrated from github.com) approved these changes

2024-12-02 12:18:20 +00:00