[BUG]: HTCondor queue not working #143

New issue

Closed

opened 2022-11-24 12:49:48 +00:00 by fraimondo · 0 comments

fraimondo commented

2022-11-24 12:49:48 +00:00

(Migrated from github.com)

Is there an existing issue for this?

I have searched the existing issues

Current Behavior

When queueing:

11/24/22 13:38:37 ERROR: PREPROCESS_PIOP1.dag (line 2): Illegal character (") in or after macroname 0001_restingstate

Expected Behavior

Job running

Steps To Reproduce

Install junifer
Yaml

workdir: /tmp

datagrabber:
    kind: DataladAOMICPIOP1
    tasks: 
        - restingstate

preprocess:
    kind: fMRIPrepConfoundRemover
    strategy:
        motion: full
        wm_csf: full
        global_signal: full
    low_pass: 0.08
    high_pass: 0.008

markers:
  - name: Schaefer100x17_TianxS2x3TxMNInonlinear2009cAsym_FC
    kind: FunctionalConnectivityParcels
    parcellation: 
        - Schaefer100x17
        - TianxS2x3TxMNInonlinear2009cAsym
    cor_method: correlation

storage:
  kind: SQLiteFeatureStorage
  uri: /data/project/identification_prediction/data/junifer_aomic/storage/PIOP1/PIOP1

queue:
  jobname: PREPROCESS_PIOP1
  kind: HTCondor
  env:
    kind: conda
    name: junifer
  mem: 8G

Run junifer queue broken.yaml
Check the DAG for this lines:

```VARS run1 element="0002,restingstate log_element="0002_restingstate"``

Environment

(junifer) Mac-mini ➜  scratch git:(main) junifer wtf
junifer:
  version: 0.1.dev996
python:
  version: 3.11.0
  implementation: CPython
dependencies:
  click: 8.1.3
  numpy: 1.23.4
  datalad: 0.17.9
  pandas: 1.5.1
  nibabel: 4.0.2
  nilearn: 0.9.2
  sqlalchemy: 1.4.43
  yaml: '6.0'
system:
  platform: macOS-13.0.1-x86_64-i386-64bit
environment:
  LC_CTYPE: UTF-8
  PATH: /Users/fraimondo/Applications/workbench/bin_macosx64:/Users/fraimondo/.rbenv/shims:/Users/fraimondo/dev/tbox/freesurfer/bin:/Users/fraimondo/dev/tbox/freesurfer/fsfast/bin:/Users/fraimondo/dev/tbox/fsl/bin:/Users/fraimondo/dev/tbox/freesurfer/mni/bin:/usr/local/opt/ruby/bin:/Users/fraimondo/miniconda3/envs/junifer/bin:/Users/fraimondo/miniconda3/condabin:/Users/fraimondo/dev/tbox/fsl/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/opt/X11/bin:/Library/Apple/usr/bin:/Users/fraimondo/.platformio/penv/bin:/Users/fraimondo/gems/bin:/Users/fraimondo/.gem/ruby/3.0.0/bin

Relevant log output

No response

Anything else?

No response

### Is there an existing issue for this? - [X] I have searched the existing issues ### Current Behavior When queueing: 11/24/22 13:38:37 ERROR: PREPROCESS_PIOP1.dag (line 2): Illegal character (") in or after macroname 0001_restingstate ### Expected Behavior Job running ### Steps To Reproduce 1. Install junifer 2. Yaml ``` workdir: /tmp datagrabber: kind: DataladAOMICPIOP1 tasks: - restingstate preprocess: kind: fMRIPrepConfoundRemover strategy: motion: full wm_csf: full global_signal: full low_pass: 0.08 high_pass: 0.008 markers: - name: Schaefer100x17_TianxS2x3TxMNInonlinear2009cAsym_FC kind: FunctionalConnectivityParcels parcellation: - Schaefer100x17 - TianxS2x3TxMNInonlinear2009cAsym cor_method: correlation storage: kind: SQLiteFeatureStorage uri: /data/project/identification_prediction/data/junifer_aomic/storage/PIOP1/PIOP1 queue: jobname: PREPROCESS_PIOP1 kind: HTCondor env: kind: conda name: junifer mem: 8G ``` 3. Run `junifer queue broken.yaml` 4. Check the DAG for this lines: ```VARS run1 element="0002,restingstate log_element="0002_restingstate"`` ### Environment ```markdown (junifer) Mac-mini ➜ scratch git:(main) junifer wtf junifer: version: 0.1.dev996 python: version: 3.11.0 implementation: CPython dependencies: click: 8.1.3 numpy: 1.23.4 datalad: 0.17.9 pandas: 1.5.1 nibabel: 4.0.2 nilearn: 0.9.2 sqlalchemy: 1.4.43 yaml: '6.0' system: platform: macOS-13.0.1-x86_64-i386-64bit environment: LC_CTYPE: UTF-8 PATH: /Users/fraimondo/Applications/workbench/bin_macosx64:/Users/fraimondo/.rbenv/shims:/Users/fraimondo/dev/tbox/freesurfer/bin:/Users/fraimondo/dev/tbox/freesurfer/fsfast/bin:/Users/fraimondo/dev/tbox/fsl/bin:/Users/fraimondo/dev/tbox/freesurfer/mni/bin:/usr/local/opt/ruby/bin:/Users/fraimondo/miniconda3/envs/junifer/bin:/Users/fraimondo/miniconda3/condabin:/Users/fraimondo/dev/tbox/fsl/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/opt/X11/bin:/Library/Apple/usr/bin:/Users/fraimondo/.platformio/penv/bin:/Users/fraimondo/gems/bin:/Users/fraimondo/.gem/ruby/3.0.0/bin ``` ### Relevant log output _No response_ ### Anything else? _No response_