juaml/junifer
Watch
3
0
Fork
You've already forked junifer
 0
Code Issues 14 Pull requests 2 Packages Activity Actions

[CI]: Fix junifer-ci Docker image #468

Merged
synchon merged 13 commits from fix/ci-image into main 2025-09-22 15:00:15 +00:00
Conversation 2 Commits 13 Files changed 6 +99 -164
6 changed files with 99 additions and 164 deletions
Download patch file Download diff file Expand all files Collapse all files

118
.github/workflows/ci.yml vendored
View file

@ -13,6 +13,7 @@ on:
- '.github/workflows/**' - '.github/workflows/**'
- 'tox.ini' - 'tox.ini'
- 'pyproject.toml' - 'pyproject.toml'
workflow_dispatch: {}
jobs: jobs:
run-tests: run-tests:
@ -24,63 +25,62 @@ jobs:
fail-fast: false fail-fast: false
matrix: matrix:
python-version: ['3.9', '3.10', '3.11', '3.12'] python-version: ['3.9', '3.10', '3.11', '3.12']
steps: steps:
- run: df -h - run: df -h
- name: Clean runner - name: Clean runner
run: | run: |
rm -rf /usr/share/dotnet \ rm -rf /usr/share/dotnet \
/usr/local/lib/android \ /usr/local/lib/android \
/opt/ghc \ /opt/ghc \
/opt/hostedtoolcache/CodeQL /opt/hostedtoolcache
- run: df -h - run: df -h
- name: Check ANTs - name: Check ANTs
run: | run: |
echo "Using ANTs : $(antsRegistration --version)" echo "Using ANTs : $(antsRegistration --version)"
- name: Check AFNI - name: Check AFNI
run: | run: |
echo "Using AFNI : $(afni --version)" echo "Using AFNI : $(afni --version)"
- name: Check FSL - name: Check FSL
run: | run: |
echo "Using FSL : $(flirt -version)" echo "Using FSL : $(flirt -version)"
- name: Install system dependencies - name: Install system dependencies
# actions/checkout fails to identify Git as it's not in the CI image # actions/checkout fails to identify Git as it's not in the CI image
run: | run: |
apt-get -qq update apt-get -qq update
apt-get -qq install -y \ apt-get -qq install -y \
git \ git \
git-annex \ git-annex \
make \ make \
jq jq
- name: Copy junifer-data directory - name: Copy junifer-data directory
run: | run: |
mkdir -p $HOME/junifer_data/v4 mkdir -p $HOME/junifer_data/v4
cp -ar /root/junifer_data/v4/. $HOME/junifer_data/v4/ cp -ar /root/junifer_data/v4/. $HOME/junifer_data/v4/
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@v5 uses: actions/checkout@v5
with: with:
submodules: true submodules: true
- name: Set up Python ${{ matrix.python-version }} - name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v6 uses: actions/setup-python@v6
with: with:
python-version: ${{ matrix.python-version }} python-version: ${{ matrix.python-version }}
- name: Install Python dependencies - name: Install Python dependencies
run: | run: |
python -m pip install --upgrade pip setuptools wheel python -m pip install --upgrade pip setuptools wheel
python -m pip install tox tox-gh-actions python -m pip install tox tox-gh-actions
- name: Configure Git for DataLad - name: Configure Git for DataLad
run: | run: |
git config --global user.email "junifer-runner@github.com" git config --global user.email "junifer-runner@github.com"
git config --global user.name "GitHub Runner" git config --global user.name "GitHub Runner"
- name: Test with tox - name: Test with tox
# https://github.com/actions/checkout/issues/1169#issuecomment-2291682583 # https://github.com/actions/checkout/issues/1169#issuecomment-2291682583
run: | run: |
git config --system --add safe.directory $(pwd) git config --system --add safe.directory $(pwd)
tox tox
- name: Upload coverage to Codecov - name: Upload coverage to Codecov
uses: codecov/codecov-action@v5 uses: codecov/codecov-action@v5
with: with:
token: ${{ secrets.CODECOV_TOKEN }} token: ${{ secrets.CODECOV_TOKEN }}
fail_ci_if_error: true fail_ci_if_error: true
flags: junifer flags: junifer
if: success() && matrix.python-version == 3.12 if: success() && matrix.python-version == 3.12

64
.github/workflows/docker-ci-publish.yml vendored
View file

@ -1,83 +1,58 @@
name: Docker for CI name: Docker for CI
# This workflow uses actions that are not certified by GitHub.
# They are provided by a third-party and are governed by
# separate terms of service, privacy policy, and support
# documentation.
on: on:
push: push:
# Only run when the file is edited on main
branches: [ "main" ] branches: [ "main" ]
paths: [ "Dockerfile-ci" ] paths: [ "Dockerfile-ci" ]
pull_request: pull_request:
# Only run when the file is edited on main
branches: [ "main" ] branches: [ "main" ]
paths: [ "Dockerfile-ci" ] paths: [ "Dockerfile-ci" ]
workflow_dispatch: {} workflow_dispatch: {}
env: env:
# Use docker.io for Docker Hub if empty
REGISTRY: ghcr.io REGISTRY: ghcr.io
# github.repository as <account>/<repo>
IMAGE_NAME: ${{ github.repository }}-ci IMAGE_NAME: ${{ github.repository }}-ci
jobs: jobs:
build: build:
runs-on: ubuntu-24.04 runs-on: ubuntu-24.04
permissions: permissions:
contents: read contents: read
packages: write packages: write
# This is used to complete the identity challenge
# with sigstore/fulcio when running outside of PRs.
id-token: write id-token: write
steps: steps:
# Taken from https://github.com/orgs/community/discussions/25678#discussioncomment-5242449 - run: df -h
- name: Delete huge unnecessary tools folder - name: Clean runner
run: rm -rf /opt/hostedtoolcache run: |
sudo rm -rf /usr/share/dotnet \
/usr/local/lib/android \
/opt/ghc \
/opt/hostedtoolcache
sudo docker image prune --all --force
sudo docker builder prune -a
- run: df -h
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@v5 uses: actions/checkout@v5
# Install the cosign tool except on PR
# https://github.com/sigstore/cosign-installer
- name: Install cosign - name: Install cosign
if: github.event_name != 'pull_request' if: github.event_name != 'pull_request'
uses: sigstore/cosign-installer@d7543c93d881b35a8faa02e8e3605f69b7a1ce62 #v3.10.0 uses: sigstore/cosign-installer@v3.10.0
with:
cosign-release: 'v2.2.4'
# Set up BuildKit Docker container builder to be able to build
# multi-platform images and export cache
# https://github.com/docker/setup-buildx-action
- name: Set up Docker Buildx - name: Set up Docker Buildx
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1 uses: docker/setup-buildx-action@v3.11.1
# Login against a Docker registry except on PR
# https://github.com/docker/login-action
- name: Log into registry ${{ env.REGISTRY }} - name: Log into registry ${{ env.REGISTRY }}
if: github.event_name != 'pull_request' if: github.event_name != 'pull_request'
uses: docker/login-action@184bdaa0721073962dff0199f1fb9940f07167d1 # v3.5.0 uses: docker/login-action@v3.5.0
with: with:
registry: ${{ env.REGISTRY }} registry: ${{ env.REGISTRY }}
username: ${{ github.actor }} username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }} password: ${{ secrets.GITHUB_TOKEN }}
# Extract metadata (tags, labels) for Docker
# https://github.com/docker/metadata-action
- name: Extract Docker metadata - name: Extract Docker metadata
id: meta id: meta
uses: docker/metadata-action@c1e51972afc2121e065aed6d45c65596fe445f3f # v5.8.0 uses: docker/metadata-action@v5.8.0
with: with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
# Build and push Docker image with Buildx (don't push on PR)
# https://github.com/docker/build-push-action
- name: Build and push Docker image - name: Build and push Docker image
id: build-and-push id: build-and-push
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0 uses: docker/build-push-action@v6.18.0
with: with:
context: . context: .
file: Dockerfile-ci file: Dockerfile-ci
@ -86,18 +61,9 @@ jobs:
labels: ${{ steps.meta.outputs.labels }} labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha cache-from: type=gha
cache-to: type=gha,mode=max cache-to: type=gha,mode=max
# Sign the resulting Docker image digest except on PRs.
# This will only write to the public Rekor transparency log when the Docker
# repository is public to avoid leaking data. If you would like to publish
# transparency data even for private images, pass --force to cosign below.
# https://github.com/sigstore/cosign
- name: Sign the published Docker image - name: Sign the published Docker image
if: ${{ github.event_name != 'pull_request' }} if: ${{ github.event_name != 'pull_request' }}
env: env:
# https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions#using-an-intermediate-environment-variable
TAGS: ${{ steps.meta.outputs.tags }} TAGS: ${{ steps.meta.outputs.tags }}
DIGEST: ${{ steps.build-and-push.outputs.digest }} DIGEST: ${{ steps.build-and-push.outputs.digest }}
# This step uses the identity token to provision an ephemeral certificate
# against the sigstore community Fulcio instance.
run: echo "${TAGS}" | xargs -I {} cosign sign --yes {}@${DIGEST} run: echo "${TAGS}" | xargs -I {} cosign sign --yes {}@${DIGEST}

61
.github/workflows/docker-docs-publish.yml vendored
View file

@ -1,80 +1,58 @@
name: Docker for docs name: Docker for docs
# This workflow uses actions that are not certified by GitHub.
# They are provided by a third-party and are governed by
# separate terms of service, privacy policy, and support
# documentation.
on: on:
push: push:
# Only run when the file is edited on main
branches: [ "main" ] branches: [ "main" ]
paths: [ "Dockerfile-docs" ] paths: [ "Dockerfile-docs" ]
pull_request: pull_request:
# Only run when the file is edited on main
branches: [ "main" ] branches: [ "main" ]
paths: [ "Dockerfile-docs" ] paths: [ "Dockerfile-docs" ]
workflow_dispatch: {} workflow_dispatch: {}
env: env:
# Use docker.io for Docker Hub if empty
REGISTRY: ghcr.io REGISTRY: ghcr.io
# github.repository as <account>/<repo>
IMAGE_NAME: ${{ github.repository }}-docs IMAGE_NAME: ${{ github.repository }}-docs
jobs: jobs:
build: build:
runs-on: ubuntu-24.04 runs-on: ubuntu-24.04
permissions: permissions:
contents: read contents: read
packages: write packages: write
# This is used to complete the identity challenge
# with sigstore/fulcio when running outside of PRs.
id-token: write id-token: write
steps: steps:
- run: df -h
- name: Clean runner
run: |
rm -rf /usr/share/dotnet \
/usr/local/lib/android \
/opt/ghc \
/opt/hostedtoolcache
docker image prune --all --force
docker builder prune -a
- run: df -h
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@v5 uses: actions/checkout@v5
# Install the cosign tool except on PR
# https://github.com/sigstore/cosign-installer
- name: Install cosign - name: Install cosign
if: github.event_name != 'pull_request' if: github.event_name != 'pull_request'
uses: sigstore/cosign-installer@d7543c93d881b35a8faa02e8e3605f69b7a1ce62 #v3.10.0 uses: sigstore/cosign-installer@v3.10.0
with:
cosign-release: 'v2.2.4'
# Set up BuildKit Docker container builder to be able to build
# multi-platform images and export cache
# https://github.com/docker/setup-buildx-action
- name: Set up Docker Buildx - name: Set up Docker Buildx
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1 uses: docker/setup-buildx-action@v3.11.1
# Login against a Docker registry except on PR
# https://github.com/docker/login-action
- name: Log into registry ${{ env.REGISTRY }} - name: Log into registry ${{ env.REGISTRY }}
if: github.event_name != 'pull_request' if: github.event_name != 'pull_request'
uses: docker/login-action@184bdaa0721073962dff0199f1fb9940f07167d1 # v3.5.0 uses: docker/login-action@v3.5.0
with: with:
registry: ${{ env.REGISTRY }} registry: ${{ env.REGISTRY }}
username: ${{ github.actor }} username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }} password: ${{ secrets.GITHUB_TOKEN }}
# Extract metadata (tags, labels) for Docker
# https://github.com/docker/metadata-action
- name: Extract Docker metadata - name: Extract Docker metadata
id: meta id: meta
uses: docker/metadata-action@c1e51972afc2121e065aed6d45c65596fe445f3f # v5.8.0 uses: docker/metadata-action@v5.8.0
with: with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
# Build and push Docker image with Buildx (don't push on PR)
# https://github.com/docker/build-push-action
- name: Build and push Docker image - name: Build and push Docker image
id: build-and-push id: build-and-push
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0 uses: docker/build-push-action@v6.18.0
with: with:
context: . context: .
file: Dockerfile-docs file: Dockerfile-docs
@ -83,18 +61,9 @@ jobs:
labels: ${{ steps.meta.outputs.labels }} labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha cache-from: type=gha
cache-to: type=gha,mode=max cache-to: type=gha,mode=max
# Sign the resulting Docker image digest except on PRs.
# This will only write to the public Rekor transparency log when the Docker
# repository is public to avoid leaking data. If you would like to publish
# transparency data even for private images, pass --force to cosign below.
# https://github.com/sigstore/cosign
- name: Sign the published Docker image - name: Sign the published Docker image
if: ${{ github.event_name != 'pull_request' }} if: ${{ github.event_name != 'pull_request' }}
env: env:
# https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions#using-an-intermediate-environment-variable
TAGS: ${{ steps.meta.outputs.tags }} TAGS: ${{ steps.meta.outputs.tags }}
DIGEST: ${{ steps.build-and-push.outputs.digest }} DIGEST: ${{ steps.build-and-push.outputs.digest }}
# This step uses the identity token to provision an ephemeral certificate
# against the sigstore community Fulcio instance.
run: echo "${TAGS}" | xargs -I {} cosign sign --yes {}@${DIGEST} run: echo "${TAGS}" | xargs -I {} cosign sign --yes {}@${DIGEST}

17
Dockerfile-ci
View file

@ -1,6 +1,6 @@
# syntax=docker/dockerfile:1 # syntax=docker/dockerfile:1
FROM python:3.12-slim FROM python:3.12-slim-bookworm
LABEL org.opencontainers.image.source=https://github.com/juaml/junifer LABEL org.opencontainers.image.source=https://github.com/juaml/junifer
LABEL org.opencontainers.image.description="Junifer CI container image" LABEL org.opencontainers.image.description="Junifer CI container image"
@ -60,14 +60,13 @@ ENV PATH="/opt/afni/bin:$PATH"
# Patch gsl # Patch gsl
RUN ln -s /usr/lib/x86_64-linux-gnu/libgsl.so.27 /usr/lib/x86_64-linux-gnu/libgsl.so.23 RUN ln -s /usr/lib/x86_64-linux-gnu/libgsl.so.27 /usr/lib/x86_64-linux-gnu/libgsl.so.23
# disabled as standard github runner fails to install due to insufficient space on device # Add FSL
# # Add FSL RUN curl https://fsl.fmrib.ox.ac.uk/fsldownloads/fslconda/releases/fslinstaller.py | \
# RUN curl https://fsl.fmrib.ox.ac.uk/fsldownloads/fslconda/releases/fslinstaller.py | \ python - -d /opt/fsl/ --skip_registration
# python - -d /opt/fsl/ --skip_registration # Set env vars for FSL
# # Set env vars for FSL ENV FSLDIR=/opt/fsl \
# ENV FSLDIR=/opt/fsl \ FSLOUTPUTTYPE=NIFTI_GZ \
# FSLOUTPUTTYPE=NIFTI_GZ \ PATH="/opt/fsl/share/fsl/bin:$PATH"
# PATH="/opt/fsl/share/fsl/bin:$PATH"
# Clean apt cache # Clean apt cache
RUN apt-get autoremove --purge && apt-get clean RUN apt-get autoremove --purge && apt-get clean

2
docs/changes/newsfragments/467.misc
View file

@ -1 +1 @@
Disable FSL installation for `junifer-ci` Docker image build with `junifer-data` v4 by `Synchon Mandal`_ Disable FSL installation for ``junifer-ci`` Docker image build with ``junifer-data`` v4 by `Synchon Mandal`_

1
docs/changes/newsfragments/468.misc Normal file
View file

@ -0,0 +1 @@
Pin ``junifer-ci`` Docker image to ``python:3.12-slim-bookworm`` and re-enable FSL installation by `Synchon Mandal`_