From 88262005ecbbc59f1170cd676f05993a58c59772 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Szczepanik?= Date: Wed, 16 Sep 2026 19:17:41 +0200 Subject: [PATCH 1/7] Update Z03 storage graph with Cerebra This adds cerebra & cerebra-mask remotes to the schematic representation of motor assesment data storage locations. The JuGit + Sciebo pair is surrounded by a box labelled "deprecated". This is a half-step; the document needs to be rewritten to describe Cerebra access (it should become simpler, as the WebDAV caveats will go away). --- content/z03/access-data.md | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/content/z03/access-data.md b/content/z03/access-data.md index 7df6744..244c577 100644 --- a/content/z03/access-data.md +++ b/content/z03/access-data.md @@ -104,17 +104,26 @@ There are two components to the dataset: - The Git repository (which we can call a "lightweight dataset") contains information about content identity and availability, but does not contain the actual data. - The content managed by git-annex ("annexed data"), i.e. the actual data. -For technical reasons they are stored and shared separately. +For technical reasons, they can be stored and shared together (as is the case with Cerebra) or separately (JuGit + Sciebo). +The chart below illustrates the Git and git-annex repositories which can be configured for a local clone to access. In the chart below, a square box symbolizes a Git remote, and cylinders symbolize Git-annex special remotes. ```mermaid flowchart TD A(Your local dataset clone) - A ---o |clone, update| B["**JuGit** (public)"] - A ---o |initremote, get| C[("**Sciebo** - (restricted, encrypted)")] - A ---o |"🚫"| D[("**INM-7 storage** - (restricted, encrypted)")] + AA["**Cerebra** (restricted)"] + AB[("**Cerebra-mask** (restricted, encrypted)")] + subgraph Deprecated + B["**JuGit** (public)"] + C[("**Sciebo** (restricted, encrypted)")] + end + D[("**INM-7 storage** (restricted, encrypted)")] + AA --- AB + A ---o |clone, update| AA + A ---o |enableremote, get| AB + A ---o |clone, update| B + A ---o |initremote, get| C + A ---o |"🚫"| D ``` Retrieving data requires interacting with both the Git remote and the Git-annex special remote (key commands are shown over the lines). -- 2.52.0 From 6838f116e9bfe2916276b2da7ea44b92a67952a3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Szczepanik?= Date: Thu, 17 Sep 2026 18:21:47 +0200 Subject: [PATCH 2/7] Rewrite Z03 docs to focus on Cerebra This gets rid of all the Sciebo/WebDAV quirks, but still keeps some mentions of JuGit + Sciebo (valid until we fully phase it out). --- content/z03/access-data.md | 269 ++++++++++++------------------ content/z03/encrypted-workflow.md | 18 ++ 2 files changed, 126 insertions(+), 161 deletions(-) diff --git a/content/z03/access-data.md b/content/z03/access-data.md index 244c577..8e4934e 100644 --- a/content/z03/access-data.md +++ b/content/z03/access-data.md @@ -1,7 +1,7 @@ +++ date = '2025-02-12T18:01:59+01:00' draft = false -title = 'Accessing motor assessment data collection on Windows' +title = 'Accessing motor assessment data collection' weight = 4 +++ @@ -9,11 +9,86 @@ This note will explain in detail how to set-up your local copy of the aggregated Overview of the encrypted workflow and prerequisites are explained in the [encrypted workflow](z3-encrypted-workflow/) page. It is highly recommended that you read it first. -The note assumes that you are working on Windows and using cmd shell. It also assumes that you have access to the shared Sciebo folder, and the data is being encrypted to your GPG key. +The note assumes that you have access to a shared copy of the dataset, and the data is being encrypted to your GPG key. + +## Distributed storage + +There are two components to the dataset: + +- The Git repository (which we can call a "lightweight dataset") contains information about content identity and availability, but does not contain the actual data. +- The content managed by git-annex ("annexed data"), i.e. the actual data. + +Each can be stored in / retrieved from multiple locations. +Depending on the platform used, the Git repository and annexed data can be stored and shared together (as is the case with Cerebra) or separately (JuGit + Sciebo). +The chart below illustrates the Git and git-annex repositories which can be configured as "remotes" for a local clone. +In the chart below, a square box symbolizes a Git remote, and cylinders symbolize Git-annex special remotes. +Retrieving data requires interacting with both kinds of remotes, and the main operations are shown over the lines. + +```mermaid +flowchart TD + A(Your local dataset clone) + AA["**Cerebra** (restricted)"] + AB[("**Cerebra-mask** (restricted, encrypted)")] + subgraph Deprecated + B["**JuGit** (public)"] + C[("**Sciebo** (restricted, encrypted)")] + end + D[("**INM-7 storage** (restricted, encrypted)")] + AA --- AB + A ---o |clone, update| AA + A ---o |enableremote, get| AB + A ---o |clone, update| B + A ---o |initremote, get| C + A ---o |"🚫"| D +``` + +Note that Cerebra is a special case: it could store the Git repository and its annex in a single remote. However, using encryption requires us to set up a [mask special remote](https://git-annex.branchable.com/special_remotes/mask/) for the annex, which shows separately in the list of annex remotes, but uses the same transport (think: the same credentials) as the Git remote. + +Because no data are stored in JuGit, the repository can be made public, and fetching information from JuGit requires no credentials. +However, access to the data stored in Sciebo or Cerebra-mask requires not only credentials, but also possession of a GPG key for which the data are encrypted. +Note that the Git repository also indexes data stored in INM7-storage, but that special remote is not accessible outside INM-7. + +The JuGit + Sciebo setup continues to be maintained for a transition period, but will be phased out in favour of Cerebra. + ## Before you begin -### GPG +### Cerebra token + +Instead of using your password (the one you use to log in to Cerebra), for DataLad (Git, git-annex) operations you should be using a token. Tokens are like passwords, but they are generated by the website rather than created by the user. A token can be revoked when it is no longer used or there is a risk that it got exposed. + +To generate a token: + +- Visit Cerebra and log in. +- Click your avatar in the top-right corner, and navigate to Settings → Applications. +- Under *Access tokens* click *New access token* and fill in the form: + - give the token a recognizable name; + - choose either "All" or "Specific repositories" for *Repository and organization access* (if you choose the latter, don't forget to select the repositories); + - under *Select permissions*, enable "repository: Read and write"; + - click *Generate token*. +- The token appears as a long string. Copy the generated value, and store it safely. It disappears after refreshing. + +Treat your token like a password. Do not share it with anyone. Once generated, the token can be used until revoked. + +### Git credential helper + +To avoid having to enter the token multiple times, we recommend using a Git credential helper. + +The easiest one to configure is [git-credential-cache](https://git-scm.com/docs/git-credential-cache), which temporarily (default timeout: 15 minutes) stores passwords in memory. It can be made default by setting a global Git configuration): + +``` +git config --global credential.helper cache +``` + +You may also configure a credential helper with persistent storage, for example one which interacts with your preferred password manager. You can also use OAuth (redirects to browser login and creates a temporary bearer token). Credential management is a matter of personal taste, so we can not document all possibilities here. Instead, see Git documentation about [credential helpers](https://git-scm.com/doc/credential-helpers) for a list of available options [^3]. + +### GPG (Windows) + +{{% notice style="info" %}} +This section is Windows-specific, and assumes you are using cmd shell. + +On GNU/Linux (and likely MacOS), no additional configuration is needed. +{{% /notice %}} If you install Git (which is a requirement for using DataLad), it comes with its own `gpg` executable. If you install gpg4win (which is the solution recommended for digital signatures in our docs on [Input signing](digital-signatures) it naturally comes with a separate `gpg` executable. This means it is possible to have two GPG programs... which store their data (e.g. keys) in completely different locations on your computer. @@ -35,130 +110,15 @@ Then (if it is gpg4win) tell Git to use this one. Git-annex obeys the same setti git config gpg.program PATH/TO/GPG ``` - -### GPG-agent - -Operations which use your secret key (like downloading encrypted data with git-annex or listing your secret keys with `gpg --list-secret-keys`) require interactions with a program called `gpg-agent` [^2]. Normally, `gpg-agent` should be running in the background, and there is nothing to do. - -If you notice gpg-agent is not running (e.g. `get` fails with an error message which says that gpg-agent is not running, or you can not list your secret keys), you can start the gpg-agent with: - -``` -gpg-connect-agent reloadagent /bye -``` - -### Sciebo token (app password) - -Instead of using your password (the one you use to log in to Sciebo), it is recommended to generate a token for use with DataLad & git-annex. Tokens are like passwords, but they are generated by the website rather than created by the user. A token can be revoked when it is no longer used or there is a risk that it got exposed. - -To generate a token: - -- log in to Sciebo -- click your username on the upper right and go to settings -- select Security -- scroll down to App passwords / tokens -- enter a token name and click "Create new app passcode" -- copy (and store safely) the username and token - -The token name can be anything - it is only for your information, and will be displayed in the list of tokens, with the date of last use and an option to delete (revoke) the token. - - -## WebDAV URLs - -When using DataLad & git-annex, you will connect to Sciebo through a protocol called WebDAV. For this, you need to know your WebDAV URL, which is slightly different than the URL you use to visit Sciebo's web interface. - -### Base WebDAV URL - -To find your WebDAV address: - -- go to Sciebo (file view) -- click ⚙️ Settings in the lower left corner -- a "drawer" will slide up, showing your WebDAV URL - -The WebDAV URL will look like this: - -``` -https://fz-juelich.sciebo.de/remote.php/dav/files/j.doe%40fz-juelich.de/ -``` - -### WebDAV URLs to folders - -The URL above is pointing to your home directory. To access a specific folder, you will need to add some path components manually. - -Most likely, a folder named `Z03` will be shared with you. It will contain some folders; `motor_export_data` is the relevant one. - -The folder path relative to your home directory will depend on whether you move the shared folder or not. Assuming that `Z03` is in your home directory on Sciebo, the relative paths will be `Z03/motor_export_repo` and `Z03/motor_export_data`, respectively. - -For use with DataLad, you will need to put the base URL and the relative paths together. Using the example above, -and assuming that the shared folder is in the top-level directory, the data folder URL (for initremote) is: - - -``` -https://fz-juelich.sciebo.de/remote.php/dav/files/j.doe%40fz-juelich.de/Z03/motor_export_data -``` - - -## Distributed storage - -There are two components to the dataset: - -- The Git repository (which we can call a "lightweight dataset") contains information about content identity and availability, but does not contain the actual data. -- The content managed by git-annex ("annexed data"), i.e. the actual data. - -For technical reasons, they can be stored and shared together (as is the case with Cerebra) or separately (JuGit + Sciebo). -The chart below illustrates the Git and git-annex repositories which can be configured for a local clone to access. -In the chart below, a square box symbolizes a Git remote, and cylinders symbolize Git-annex special remotes. - -```mermaid -flowchart TD - A(Your local dataset clone) - AA["**Cerebra** (restricted)"] - AB[("**Cerebra-mask** (restricted, encrypted)")] - subgraph Deprecated - B["**JuGit** (public)"] - C[("**Sciebo** (restricted, encrypted)")] - end - D[("**INM-7 storage** (restricted, encrypted)")] - AA --- AB - A ---o |clone, update| AA - A ---o |enableremote, get| AB - A ---o |clone, update| B - A ---o |initremote, get| C - A ---o |"🚫"| D -``` - -Retrieving data requires interacting with both the Git remote and the Git-annex special remote (key commands are shown over the lines). -Because no data are stored in JuGit, the repository can be made public, and fetching information from JuGit requires no credentials. -However, access to the data stored in Sciebo requires not only credentials, but also possession of a GPG key for which the data are encrypted. -Note that the Git repository also indexes data stored in INM7-storage, but that special remote is not accessible outside INM-7. - ## Initial setup -#### Let git-annex know your token - -Git-annex needs WebDAV credentials to access the stored repository and data. -It expects them to be provided via environment variables. -Set the environment variables with the `set` command (substitute your Sciebo username and token): - -``` -set WEBDAV_USERNAME=j.doe@fz-juelich.de -set WEBDAV_PASSWORD=XXXXX-XXXXX-XXXXX-XXXXX -``` - -Important: do not put the password in quotes (`""`), because `set` command treats them literally. - -Tip: you can check the value of an environment variable with `echo`, e.g. `echo %WEBDAV_USERNAME%`. - -The environment variables persist until you close your cmd. -However, these credentials will later be cached in a owner-read-only file in the clone, -so re-entering the credentials should not be needed for subsequent updates. - ### Clone the dataset -Clone the (lightweight) dataset from JuGit (using the URL below) into a directory with a name of your choice +Clone the (lightweight) dataset from Cerebra (using the URL below) into a directory with a name of your choice (here: `motor_export`): ``` -datalad clone https://jugit.fz-juelich.de/m.szczepanik/motor-assesment.git motor_export +datalad clone https://cerebra.fz-juelich.de/sfb1451-z03-internal/motor-assessment motor_export ``` Then change your working directory to go into the dataset: @@ -166,64 +126,38 @@ Then change your working directory to go into the dataset: cd motor_export ``` -### Initialize sciebo remote +### Enable the mask special remote -Git-annex special remotes represent methods of accessing data, and special remote configuration contains things like access URL or encryption type to be used. +Git-annex special remotes represent methods of transporting data. In this particular case, a *mask special remote* was set up to store encrypted data alongside the repository in Cerebra. Enabling it (and transferring data) requires having a matching GPG key. For this reason, it is not enabled automatically. -One weaknes of the Sciebo setup is that WebDAV URLs contain user names and instance names, so will be different for each user (even though the shared folder is, in principle, the same). The dataset already has a sciebo remote, but it points to FZJ's sciebo and Michał Szczepanik's user account. There is no easy way to reconfigure it, other than each user initializing a new remote pointing to *their* sciebo URL. - -The trick is to initialize a special remote, and mark it *same as* the one above (meaning that we expect to reach exactly the same place, just through different means). - -This is by far the most involved part of the setup. - -#### (optional): check available remotes -Each special remote has its own ID and assigned name, which can be used interchangeably with git-annex commands. To list remotes known to the dataset, run: +Optionally, you can start by checking the list of available special remotes. +Each special remote has its own ID and assigned name, which can be used interchangeably inside git-annex commands. To list remotes known to the dataset, run: ``` git annex info ``` -Pending future changes, the remote we want to enable is going to be named sciebo-storage, -i.e. it will be this one: +Pending future changes, the remote we want to enable is going to be named cerebra-storage, +and it should appear in the list of *semitrusted repositories* like this: ``` -325dd4e6-e71b-4512-bce6-0eb574b6ed15 -- sciebo-storage +f095040a-18b7-4648-849e-f154c53ce976 -- cerebra-mask ``` -#### Initialize your own private special remote - -As explained above, you will need to initialize a special remote for yourself, and mark it "same as" the one above. - -Here's the command broken up into multiple lines with a caret (`^`) for readability. +To enable the cerebra-mask special remote, run: ``` -git annex initremote my-sciebo-storage^ - --private^ - --sameas sciebo-storage^ - type=webdav^ - url="" +git annex enableremote cerebra-mask ``` -And this is the same as a one-liner: +You should see the following output: -``` -git annex initremote my-sciebo-storage --private --sameas sciebo-storage type=webdav url="" +``` output +enableremote cerebra-mask (encryption update) (to gpg keys: [abbreviated]) ok +(recording state in git...) ``` -Substitute the `` with your data URL (the one which ends with `motor_export_data`)! - -{{% notice style="note" %}} -In the example above, we enclosed the URL with double quotes (`"...")`. -Keep in mind that cmd distinguishes single- and double quotes and treats them differently! -Find out more in https://ss64.com/nt/syntax-esc.html -{{% /notice %}} - -#### Troubleshooting - -- If the error message complains about credentials (bearer authentication), most likely the credentials (environment variables) are incorrect. -- If the error message complains about gpg-agent not running, it is either not running or git-annex is using a different gpg than intended. See sections on gpg and gpg-agent above. -- If there is seemingly no error but data can not be downloaded, then probably the URL given to initremote was incorrect. Make sure that the URL points to `motor_export_data` (not `repo`). - +If that didn't work, see the *Troubleshooting* section below. ## Data access @@ -257,6 +191,19 @@ git log ``` +## Troubleshooting + +Operations which use your secret key (like downloading encrypted data with git-annex or listing your secret keys with `gpg --list-secret-keys`) require interactions with a program called `gpg-agent` [^2]. Normally, `gpg-agent` should be running in the background, and there is nothing to do. + +If you notice gpg-agent is not running (e.g. `enableremote` or `get` fails with an error message which says that gpg-agent is not running, or you can not list your secret keys), you can start the gpg-agent with: + +``` +gpg-connect-agent reloadagent /bye +``` + +If an operation (e.g. `enableremote` or `get`) fails with an error message which says that a private key is not available (but you know from running `gpg --list-secret-keys` that your private key is configured), or the message says gpg-agent is not running (but you know it is) it is possible that you have more than one gpg program installed, and Git-annex uses a different one that you would expect. See the *GPG (Windows)* part of the *Before you begin* section above. + + [^1]: It would, at least in theory, be possible to modify the [PATH variable](https://en.wikipedia.org/wiki/PATH_(variable)) so that running `gpg` would use the Git-bundled gpg instead of the gpg4win gpg. [^2]: You call `datalad get`, DataLad calls `git annex get`; git-annex calls `gpg --decrypt`, and gpg asks gpg-agent for a matching secret key. -[^3]: The credential name and username are stored in global (i.e. user) Git configuration, the password is stored in Windows' secret storage. +[^3]: For a more technical introduction to what Git credential helpers do, see this blog post: [Git The Stupid Password Store](https://tylercipriani.com/blog/2024/07/31/git-as-a-password-prompt/) diff --git a/content/z03/encrypted-workflow.md b/content/z03/encrypted-workflow.md index 103c316..acc8d0b 100644 --- a/content/z03/encrypted-workflow.md +++ b/content/z03/encrypted-workflow.md @@ -31,8 +31,26 @@ an e-mail signed with a trusted GPG or S/MIME signature). INF their GPG key fingerprint (which, ideally, they would have confirmed in person), and their intended access rights. +## Cerebra - data engineering platform + +We are using [Cerebra](https://cerebra.fz-juelich.de/) to exchange +encrypted data. A Cerebra account is required for access. Repositories +are grouped in an organization, in which Z03 members have administrator +privileges and can manage user permissions. Both the organization and +repositories are private, and are only accessible to designated users. + +Cerebra is a data engineering platform operated by INM-7 and hosted on +JSC Cloud at FZJ. It is powered by +[Forgejo-aneksajo](https://codeberg.org/forgejo-aneksajo/forgejo-aneksajo/wiki), +a fork of [Forgejo](https://forgejo.org/) which adds git-annex +support. + ## Sciebo - cloud storage +{{% notice style="note" %}} +With the availability of Cerebra, the use of Sciebo will be phased out. +{{% /notice %}} + We are using Sciebo () to exchange encrypted data. Z03 members would likely already have accounts via UzK/UKK () or Jülich -- 2.52.0 From f7835f54f6d72b9cf0222a583fef9cdc416e36e1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Szczepanik?= Date: Fri, 13 Nov 2026 13:40:55 +0100 Subject: [PATCH 3/7] Update deploy workflow: new runner, git-pages --- .forgejo/workflows/deploy.yml | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/.forgejo/workflows/deploy.yml b/.forgejo/workflows/deploy.yml index 4a714b2..11cee0a 100644 --- a/.forgejo/workflows/deploy.yml +++ b/.forgejo/workflows/deploy.yml @@ -1,10 +1,5 @@ name: Deploy on webserver -defaults: - run: - # hugo container only has `sh` - shell: sh - on: push: branches: @@ -12,13 +7,8 @@ on: jobs: deploy: - if: github.repository_owner == 'www' name: Build site with hugo and deploy on success - runs-on: psyinf-hugo-site-build - container: - volumes: - # for depositing the built site directly - - /home/www/srv:/www:rw + runs-on: debian-13 steps: - name: Checkout uses: https://hub.datalad.org/forgejo/datalad-clone-action@v1 @@ -26,8 +16,20 @@ jobs: recursive: true get_file_content: false + - name: Setup Hugo + uses: https://github.com/peaceiris/actions-hugo@v3 + with: + hugo-version: '0.131.0' + extended: true + - name: Generate site run: hugo - name: Deploy - run: cp -r public /www/_rdm.sfb1451.de && rm -rf /www/rdm.sfb1451.de ; mv /www/_rdm.sfb1451.de /www/rdm.sfb1451.de + uses: actions/git-pages@v2 + with: + site: https://rdm.sfb1451.de/ + server: hub.psychoinformatics.de + password: ${{ secrets.DNS_CHALLENGE_PASSWORD }} + token: ${{ forgejo.token }} + source: public/ -- 2.52.0 From 760c18cd778fd8951da5780d090c865124923eaf Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Szczepanik?= Date: Fri, 13 Nov 2026 13:58:25 +0100 Subject: [PATCH 4/7] Fix deploy: password/token mutually exclusive --- .forgejo/workflows/deploy.yml | 1 - 1 file changed, 1 deletion(-) diff --git a/.forgejo/workflows/deploy.yml b/.forgejo/workflows/deploy.yml index 11cee0a..41667e6 100644 --- a/.forgejo/workflows/deploy.yml +++ b/.forgejo/workflows/deploy.yml @@ -31,5 +31,4 @@ jobs: site: https://rdm.sfb1451.de/ server: hub.psychoinformatics.de password: ${{ secrets.DNS_CHALLENGE_PASSWORD }} - token: ${{ forgejo.token }} source: public/ -- 2.52.0 From 4ffc2d3e0dfd09e4644dc86270b87e71a56330cd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Szczepanik?= Date: Fri, 13 Nov 2026 16:26:28 +0100 Subject: [PATCH 5/7] Update relearn theme to 9.5.1 --- themes/hugo-theme-relearn | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/themes/hugo-theme-relearn b/themes/hugo-theme-relearn index adb82b0..289a038 160000 --- a/themes/hugo-theme-relearn +++ b/themes/hugo-theme-relearn @@ -1 +1 @@ -Subproject commit adb82b040f7cda6de4d78892bb580a62f248d5ef +Subproject commit 289a03821aae0c5fc5dcd823f7bb000fd2b5d857 -- 2.52.0 From 5f2fd4547f0158d60fddda6abbdb2dc1b138502b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Szczepanik?= Date: Fri, 13 Nov 2026 16:36:58 +0100 Subject: [PATCH 6/7] Bump Hugo version; act on deprecations Relearn v9.5 requires hugo 0.166.0 or newer. We shall bump to latest available as of today, which is 0.169.0. Hugo deprecated `languageCode` (now called `locale`) and `_build` (now called `build`). The relearn theme deprecated the `notice` shortcode in favour of the new `callout` shortcode. The new version allows using Markdown callouts, which are IMO nicer. --- .forgejo/workflows/deploy.yml | 2 +- content/about/_index.md | 2 +- content/catalog/datalad-based-deposition.md | 3 +- content/datalad/extensions/_index.md | 19 ++-- content/walkthroughs/publication-catalog.md | 100 +++++++++----------- content/z03/access-data.md | 14 ++- content/z03/encrypted-workflow.md | 5 +- hugo.toml | 2 +- 8 files changed, 67 insertions(+), 80 deletions(-) diff --git a/.forgejo/workflows/deploy.yml b/.forgejo/workflows/deploy.yml index 41667e6..ea35087 100644 --- a/.forgejo/workflows/deploy.yml +++ b/.forgejo/workflows/deploy.yml @@ -19,7 +19,7 @@ jobs: - name: Setup Hugo uses: https://github.com/peaceiris/actions-hugo@v3 with: - hugo-version: '0.131.0' + hugo-version: '0.169.0' extended: true - name: Generate site diff --git a/content/about/_index.md b/content/about/_index.md index 50cdedb..7438e3d 100644 --- a/content/about/_index.md +++ b/content/about/_index.md @@ -1,7 +1,7 @@ +++ title = 'Extras' -[_build] +[build] list = 'never' publishResources = true render = 'always' diff --git a/content/catalog/datalad-based-deposition.md b/content/catalog/datalad-based-deposition.md index f272b6a..5afe4f2 100644 --- a/content/catalog/datalad-based-deposition.md +++ b/content/catalog/datalad-based-deposition.md @@ -77,9 +77,8 @@ the added metadata records. ### 4. Publish the update -{{% notice style="important" %}} +> [!important] This step assumes that the project has already been published (has a configured sibling). -{{% /notice %}} Then, push the new state of the dataset to an existing sibling: diff --git a/content/datalad/extensions/_index.md b/content/datalad/extensions/_index.md index 26fc116..6f35593 100644 --- a/content/datalad/extensions/_index.md +++ b/content/datalad/extensions/_index.md @@ -25,11 +25,10 @@ and in [DataLad-NEXT documentation](https://docs.datalad.org/projects/next/en/la ![DataLad graphical user interface](gooey_pushed.png) -{{% notice style="important" %}} -Currently, the DataLad-Gooey extension is *de facto* unmaintained. -However, it was an important proof of principle, and we expect it to -make a comeback in the future; likely in a completely new guise. -{{% /notice %}} +> [!important] +> Currently, the DataLad-Gooey extension is *de facto* unmaintained. +> However, it was an important proof of principle, and we expect it to +> make a comeback in the future; likely in a completely new guise. The DataLad-Gooey (pronounce \"GUI\") package provides a graphical user interface for DataLad. It is specifically aiming at making key data @@ -49,9 +48,8 @@ documentation](https://docs.datalad.org/projects/gooey/en/latest/index.html). ## DataLad-XNAT -{{% notice style="info" %}} -The DataLad-XNAT extension is currently unmaintained. -{{% /notice %}} +> [!info] +> The DataLad-XNAT extension is currently unmaintained. [XNAT](https://www.xnat.org/) is an open source imaging informatics platform developed by the Neuroinformatics Research Group at Washington @@ -80,9 +78,8 @@ documentation](http://docs.datalad.org/projects/datalad-xnat/en/latest/). ## DataLad-REDCap {#sec-datalad-redcap} -{{% notice style="info" %}} -The DataLad-REDCap extension is currently unmaintained. -{{% /notice %}} +> [!info] +> The DataLad-REDCap extension is currently unmaintained. DataLad-REDCap extension provides convenience commands for exporting data from [REDCap](https://project-redcap.org/) into DataLad datasets. diff --git a/content/walkthroughs/publication-catalog.md b/content/walkthroughs/publication-catalog.md index 47e76bd..dc3ec35 100644 --- a/content/walkthroughs/publication-catalog.md +++ b/content/walkthroughs/publication-catalog.md @@ -4,27 +4,23 @@ draft = false title = 'Publication Catalog' +++ -{{% notice style="important" %}} - -The workflow described here remains a valid description for dataset -publication on Sciebo. However, as a means for submitting datasets to -the SFB1451 catalog, it has been deprecated in favour of the procedure -described in -[Register data in the data catalog]({{% relref "/catalog/register-data" %}}). - -If publications are to be shown in the catalog, information about them -needs to be included directly in the dataset (metadata) files, -preferably using standard metadata formats (e.g as -[citation file format references field](https://github.com/citation-file-format/citation-file-format/blob/main/schema-guide.md#references)) -or as a [datalad-tabby publications table]({{% relref "/catalog/register-data/#table-publications" %}}). -All datasets with -project descriptions and publication metadata created according to the -workflow description below (either by INF or by respective projects) -have been republished to GitHub as the -[sfb1451/all-projects](https://github.com/sfb1451/all-projects) -superdataset. - -{{% /notice %}} +> [!important] +> The workflow described here remains a valid description for dataset +> publication on Sciebo. However, as a means for submitting datasets to +> the SFB1451 catalog, it has been deprecated in favour of the procedure +> described in [Register data in the data catalog]({{% relref "/catalog/register-data" %}}). +> +> If publications are to be shown in the catalog, information about them +> needs to be included directly in the dataset (metadata) files, +> preferably using standard metadata formats (e.g as +> [citation file format references field](https://github.com/citation-file-format/citation-file-format/blob/main/schema-guide.md#references)) +> or as a [datalad-tabby publications table]({{% relref "/catalog/register-data/#table-publications" %}}). +> All datasets with +> project descriptions and publication metadata created according to the +> workflow description below (either by INF or by respective projects) +> have been republished to GitHub as the +> [sfb1451/all-projects](https://github.com/sfb1451/all-projects) +> superdataset. ## Publication Catalog from Sciebo: a walkthrough @@ -79,11 +75,11 @@ it: cd my-project datalad run-procedure cfg_text2git -{{% notice style="info" %}} -The `run-procedure cfg_text2git` will configure DataLad so that text -files will not be annexed -- this is a useful setting for the files we -will be adding. -{{% /notice %}} +> [!info] +> The `run-procedure cfg_text2git` will configure DataLad so that text +> files will not be annexed -- this is a useful setting for the files we +> will be adding. + ### Create a citation file @@ -135,11 +131,11 @@ DataLad: datalad save -m "Add citation file" CITATION.cff -{{% notice style="info" %}} -Citation File Format is a standard for plain text files with human- and -machine-readable information for datasets (and software). You can read -more on the [CFF website](https://citation-file-format.github.io/). -{{% /notice %}} +> [!info] +> Citation File Format is a standard for plain text files with human- and +> machine-readable information for datasets (and software). You can read +> more on the [CFF website](https://citation-file-format.github.io/). + ### Add bibliographic information to the dataset @@ -178,29 +174,27 @@ If this is the first time you are using DataLad with Sciebo, you will be prompted for your credentials. -{{% notice style="info" %}} -Sciebo uses your entire e-mail address (name@example.com) as its username. -{{% /notice %}} +> [!info] +> Sciebo uses your entire e-mail address (name@example.com) as its username. -{{% notice style="info" %}} -The `--dataset` option lets us explicitly specify the dataset for which -we configure a sibling. The `--name` option sets the name which we will -use later when publishing. The `--mode filetree` option enables filetree -mode, meaning that the sibling will have the same, human-readable, file -tree layout as the folder on your drive. - -The WebDAV url can be obtained from Sciebo\'s web UI, by clicking at -\"Settings\" in the lower left. This URL points to the user\'s home -directory, and any subfolders must be appended \"by hand\". Nonexisting -subfolders will be created. The URLs will be different for different -instances and users. For example, with FZJ\'s instance, the full URL to -the dataset folder would look like this (`` and `` are -placeholders): - -``` -https://fz-juelich.sciebo.de/remote.php/dav/files///pub-dataset -``` -{{% /notice %}} +> [!info] +> The `--dataset` option lets us explicitly specify the dataset for which +> we configure a sibling. The `--name` option sets the name which we will +> use later when publishing. The `--mode filetree` option enables filetree +> mode, meaning that the sibling will have the same, human-readable, file +> tree layout as the folder on your drive. +> +> The WebDAV url can be obtained from Sciebo\'s web UI, by clicking at +> \"Settings\" in the lower left. This URL points to the user\'s home +> directory, and any subfolders must be appended \"by hand\". Nonexisting +> subfolders will be created. The URLs will be different for different +> instances and users. For example, with FZJ\'s instance, the full URL to +> the dataset folder would look like this (`` and `` are +> placeholders): +> +> ``` +> https://fz-juelich.sciebo.de/remote.php/dav/files///pub-dataset +> ``` ### Publish to Sciebo diff --git a/content/z03/access-data.md b/content/z03/access-data.md index 8e4934e..01cfc9b 100644 --- a/content/z03/access-data.md +++ b/content/z03/access-data.md @@ -84,11 +84,10 @@ You may also configure a credential helper with persistent storage, for example ### GPG (Windows) -{{% notice style="info" %}} -This section is Windows-specific, and assumes you are using cmd shell. - -On GNU/Linux (and likely MacOS), no additional configuration is needed. -{{% /notice %}} +> [!info] +> This section is Windows-specific, and assumes you are using cmd shell. +> +> On GNU/Linux (and likely MacOS), no additional configuration is needed. If you install Git (which is a requirement for using DataLad), it comes with its own `gpg` executable. If you install gpg4win (which is the solution recommended for digital signatures in our docs on [Input signing](digital-signatures) it naturally comes with a separate `gpg` executable. This means it is possible to have two GPG programs... which store their data (e.g. keys) in completely different locations on your computer. @@ -161,9 +160,8 @@ If that didn't work, see the *Troubleshooting* section below. ## Data access -{{% notice style="note" %}} -This assumes your working directory is in the dataset (that you `cd`'ed into the dataset). -{{% /notice %}} +> [!note] +> This assumes your working directory is in the dataset (that you `cd`'ed into the dataset). To download data, use: diff --git a/content/z03/encrypted-workflow.md b/content/z03/encrypted-workflow.md index acc8d0b..936695c 100644 --- a/content/z03/encrypted-workflow.md +++ b/content/z03/encrypted-workflow.md @@ -47,9 +47,8 @@ support. ## Sciebo - cloud storage -{{% notice style="note" %}} -With the availability of Cerebra, the use of Sciebo will be phased out. -{{% /notice %}} +> [!note] +> With the availability of Cerebra, the use of Sciebo will be phased out. We are using Sciebo () to exchange encrypted data. Z03 members would likely already have accounts via diff --git a/hugo.toml b/hugo.toml index 2fe0b4e..9d218e6 100644 --- a/hugo.toml +++ b/hugo.toml @@ -1,5 +1,5 @@ baseURL = 'https://rdm.sfb1451.de/' -languageCode = 'en-us' +locale = 'en-us' title = 'SFB1451 RDM docs' theme= 'hugo-theme-relearn' -- 2.52.0 From d0b2b81596957304a89fed1d571bd80214a5085d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Szczepanik?= Date: Fri, 13 Nov 2026 17:19:41 +0100 Subject: [PATCH 7/7] Speed up CI build with shallow theme clone The entire relearn theme repository provides for a massive checkout (about 1 min 30 s during recent builds), so we abandon the recursive datalad checkout action and initialize the submodule with depth 1. There is no annex in this repository, so we don't need more steps. --- .forgejo/workflows/deploy.yml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/.forgejo/workflows/deploy.yml b/.forgejo/workflows/deploy.yml index ea35087..1157023 100644 --- a/.forgejo/workflows/deploy.yml +++ b/.forgejo/workflows/deploy.yml @@ -11,10 +11,10 @@ jobs: runs-on: debian-13 steps: - name: Checkout - uses: https://hub.datalad.org/forgejo/datalad-clone-action@v1 - with: - recursive: true - get_file_content: false + uses: https://hub.psychoinformatics.de/actions/taiki-e-checkout-action@v1 + + - name: Fetch theme + run: git submodule update --init --depth 1 -- themes/hugo-theme-relearn - name: Setup Hugo uses: https://github.com/peaceiris/actions-hugo@v3 -- 2.52.0