diff --git a/docs/_static/css/custom.css b/docs/_static/css/custom.css index be6522b82..c929f98d0 100644 --- a/docs/_static/css/custom.css +++ b/docs/_static/css/custom.css @@ -20,4 +20,27 @@ #what-s-new li, #what-s-new a { font-size: medium; +} + +#first-steps-with-junifer div.mermaid { + overflow: scroll; +} + +#first-steps-with-junifer svg { + overflow: scroll; + width: 200%; +} + +body:not([data-theme="light"]) .mermaid svg .flowchart-link { + stroke: white !important; +} + +body:not([data-theme="light"]) .mermaid svg .edgeLabel { + color: white !important; + background-color: #202020 !important; +} + +body:not([data-theme="light"]) .mermaid svg .marker{ + stroke: white !important; + fill: white !important; } \ No newline at end of file diff --git a/docs/changes/newsfragments/210.doc b/docs/changes/newsfragments/210.doc new file mode 100644 index 000000000..007bc35f5 --- /dev/null +++ b/docs/changes/newsfragments/210.doc @@ -0,0 +1 @@ +Add sections *starting* and *help* in the documentation by `Fede Raimondo`_ \ No newline at end of file diff --git a/docs/conf.py b/docs/conf.py index 84ef66e28..c0cd0797d 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -51,6 +51,7 @@ extensions = [ "numpydoc", # support for NumPy style docstrings "gh_substitutions", # custom GitHub substitutions "sphinx_copybutton", # copy button for code blocks + "sphinxcontrib.mermaid", # mermaid support ] if use_multiversion: diff --git a/docs/help.rst b/docs/help.rst new file mode 100644 index 000000000..8362c8db7 --- /dev/null +++ b/docs/help.rst @@ -0,0 +1,108 @@ +.. include:: links.inc + +.. _help: + +Getting Help +============ + + | Help! I need somebody + | Help! not just anybody + | Help! you know I need someone + | Help! + + -- The Beatles, Help! + +This fragment of the song originally appeared in the 1965 film Help! and was written by Lennon and McCartney, just a +few years after the development of Mark 1, the first commercial computer. Maybe just a random coincidence, or maybe +Lennon was just trying to write an email to McCartney. + +While the song might have been written with another meaning in mind, it is a good way to describe the situation of many +researchers who are presented with a new toolbox. Indeed, the situation of many researchers is that the projects they +are working on are becoming more and more complex in terms of methods and data. Thus, we *open up the doors* to new +possibilites: + + | When I was younger, so much younger than today + | I never needed anybody's help in any way + | But now these days are gone, I'm not so self assured + | Now I find I've changed my mind and opened up the doors + + -- The Beatles, Help! + +The setback with modern research is that current methods are often more complex and require more computing, which +means that we need to learn concepts from computer science, mathematics, statistics, etc. This is a good thing, but it +also means that we need to learn new tools and new ways of thinking, which can be a bit overwhelming, to the point that +we start relying more and more on other researchers. In the end, we might feel like we lose our independence: + + | And now my life has changed in oh so many ways + | My independence seems to vanish in the haze + | But every now and then I feel so insecure + | I know that I just need you like I've never done before + + -- The Beatles, Help! + +We can continue with the song, but we think you get the point. The point is that we will need help, and we will need +to ask for it. We will need to ask for help from our colleagues, from our supervisors, from our friends. + +We are a small team of researchers and developers, and we are not experts in *everything at once*. Each one of us has +a specific expertise, and we are trying to use this expertise to create Junifer. When we conceived Junifer, we thought +of researchers' problems and tried to come up with the best way to help them, by building a tool that is easy to +understand, learn and use. Most importantly, we made it to help. We are here to help you and your research. + +If you have any questions, problems and / or suggestions, please do not hesitate to contact us. We will be happy to help you +and we will be happy to hear from you. + +Seems nice, no? But we have one condition: **help us help you**. + +Communication is the key for you to help us and in turn help you solve your problems. We cannot know what you are +trying to do, unless you tell us. **The more detailed explanation you give us, the faster we can help you**. +We have opened several communication channels so that you can contact us in the way that is most convenient for you. + +Some people prefer to **write, in detail, with code and figures**. If you are one of those, use the +`junifer Discussions`_. site on GitHub. This is a place where you can ask questions, and where you can discuss topics +such as potential new features, or potential new methods. + +Some people do **not have a clear idea of what they want**, but they know that they need help. This is not a +problem, but it is a bit more involved. Since it will require more frequent interactions to try to understand +what you are trying to do, we have the `junifer matrix channel`_ in which you can chat with us and other junifer users. + +Finally, some people **prefer to communicate verbally**. If you are one of those, you might want to join our +*office hours*. Given that our agenda might vary, office hours will be announced on the +`junifer matrix channel`_ chat. Feel free to join and just *listen in* if you are too shy to write. + +In short, these are the 3 communication channels to get help: + +#. Discussions on Github (`junifer Discussions`_): + + Pros: + + * You can write your question in detail. + * Your question will not get mixed with other conversations. + * Likely to be the fastest way to solve the issue in the long-run. + + Cons: + + * You need to write your question in detail. + +#. Chat (`junifer matrix channel`_): + + Pros: + + * You can ask questions in real-time. + * You can including code and figures in your question. + + Cons: + + * Real-time depends on the availability of the other users. + * It might be difficult to follow if several conversation happens at the same time. + +#. Video Calls (*office hours*) + + Pros: + + * You can get real-time feedback. + * You will get our undivided attention (no overlapping discussions). + + Cons: + + * We can't read or write code, or even propose solutions for you to test. + * You will need to wait for the next *office hour*. diff --git a/docs/images/pipeline/pipeline.001.png b/docs/images/pipeline/pipeline.001.png deleted file mode 100644 index 5f222fcd4..000000000 Binary files a/docs/images/pipeline/pipeline.001.png and /dev/null differ diff --git a/docs/images/pipeline/pipeline.002.png b/docs/images/pipeline/pipeline.002.png deleted file mode 100644 index dc1c8dfaf..000000000 Binary files a/docs/images/pipeline/pipeline.002.png and /dev/null differ diff --git a/docs/index.rst b/docs/index.rst index 74cb7f7ba..bd2b809b8 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -23,12 +23,14 @@ enabling others to extend it easily. :caption: Contents: installation + starting understanding/index.rst using/index.rst builtin extending/index.rst auto_examples/index.rst api/index.rst + help contribution maintaining faq diff --git a/docs/links.inc b/docs/links.inc index 791b99395..304214509 100644 --- a/docs/links.inc +++ b/docs/links.inc @@ -30,6 +30,7 @@ .. _`Github`: https://github.com/ .. _`junifer Github`: https://github.com/juaml/junifer .. _`junifer Discussions`: https://github.com/juaml/junifer/discussions +.. _`junifer matrix channel`: https://matrix.to/#/#junifer-gen:inm7.de .. _`Wikipedia`: https://wikipedia.org .. _`YAML`: https://yaml.org diff --git a/docs/starting.rst b/docs/starting.rst new file mode 100644 index 000000000..fc37c72b0 --- /dev/null +++ b/docs/starting.rst @@ -0,0 +1,158 @@ +.. include:: links.inc + +.. _starting: + +First steps with Junifer +======================== + + +.. note:: + To scroll the graph left and right, click on the graph and use the arrows on your keyboard. + +.. mermaid:: + + flowchart TD + start((( Start ))) + read_understanding(Read Understanding Junifer) + start --> read_understanding + question_features{Can I compute\nthe features I want\nusing junifer?} + read_understanding --> question_features + question_features -->|Yes| read_using_final + question_features -->|I'm not sure| read_using + question_features -->|No| question_feature_type + read_using --> question_features + + read_using(Read Using Junifer) + + question_feature_type{"What am I missing\nfrom junifer"} + question_feature_type --> missing_datagrabber + question_feature_type --> missing_preprocessing + question_feature_type --> missing_marker + question_feature_type --> missing_other + missing_datagrabber("A Dataset/DataGrabber") + missing_preprocessing("A Preprocessing") + missing_marker("A Marker") + missing_other("Something else") + + missing_datagrabber --> question_datagrabber_junifarm + question_datagrabber_junifarm{Is the\ndataset/datagrabber\nin juni-farm?} + question_datagrabber_junifarm -->|Yes| read_using_final + question_datagrabber_junifarm -->|No| read_extending_datagrabber_start + read_extending_datagrabber_start(Read Creating a Junifer extension) + read_extending_datagrabber_start --> read_extending_datagrabber + read_extending_datagrabber(Read Creating Data Grabbers) + read_extending_datagrabber --> question_datagrabber_kind + question_datagrabber_kind{Is your dataset\na datalad one?} + question_datagrabber_kind -->|Yes| extend_datagrabber_datalad + question_datagrabber_kind -->|No| extend_datagrabber_pattern + extend_datagrabber_pattern(Use\nPatternDataGrabber) + extend_datagrabber_datalad(Use\nPatternDataladDataGrabber) + extend_datagrabber_pattern --> question_datagrabber_solved + extend_datagrabber_datalad --> question_datagrabber_solved + question_datagrabber_solved{Can you use\nyour data now?} + question_datagrabber_solved -->|Yes| question_contribute_datagrabber + question_datagrabber_solved -->|No| contact_help + question_contribute_datagrabber{Do you think\nyour DataGrabber\nis useful for other users?} + question_contribute_datagrabber -->|Yes| contribute_datagrabber + question_contribute_datagrabber -->|No| final_run + contribute_datagrabber(Create a\nDATASET REQUEST\nissue on Github) + contribute_datagrabber --> final_run + + missing_marker --> question_marker_junifarm + question_marker_junifarm{Is the marker\nin juni-farm?} + question_marker_junifarm -->|Yes| read_using_final + question_marker_junifarm -->|No| read_extending_marker_start + read_extending_marker_start(Read Creating a Junifer extension) + read_extending_marker_start --> read_extending_marker + read_extending_marker(Read Creating Markers) + read_extending_marker --> question_marker_solved + question_marker_solved{Can you use\nyour marker now?} + question_marker_solved -->|Yes| question_contribute_marker + question_marker_solved -->|No| contact_help + question_contribute_marker{Do you think\nyour Marker\nis useful for other users?} + question_contribute_marker -->|Yes| contribute_marker + question_contribute_marker -->|No| final_run + contribute_marker(Create a\nMARKER REQUEST\nissue on Github) + contribute_marker --> final_run + + + missing_preprocessing --> contact_help + + missing_mask{"A Mask?"} + missing_parcellation{"A Parcellation?"} + missing_coordinates{"Coordinates?"} + missing_other_other{"Something else?"} + missing_other --> missing_mask + missing_other --> missing_parcellation + missing_other --> missing_coordinates + missing_other --> missing_other_other + missing_other_other --> contact_help + + contact_help(((Contact the\nJunifer team))) + + missing_mask --> read_adding_mask_start + read_adding_mask_start("Read Creating a Junifer extension") + read_adding_mask_start --> read_adding_mask + read_adding_mask("Read Adding Masks") + read_adding_mask --> missing_other_solved + + missing_parcellation --> read_adding_parcellation_start + read_adding_parcellation_start("Read Creating a Junifer extension") + read_adding_parcellation_start --> read_adding_parcellation + read_adding_parcellation("Read Adding Parcellations") + read_adding_parcellation --> missing_other_solved + + missing_coordinates --> read_adding_coordinates_start + read_adding_coordinates_start("Read Creating a Junifer extension") + read_adding_coordinates_start --> read_adding_coordinates + read_adding_coordinates("Read Adding Coordinates") + read_adding_coordinates --> missing_other_solved + + missing_other_solved{Did you solve your issue?} + missing_other_solved -->|Yes| read_using_final + missing_other_solved -->|No| missing_other_contact + missing_other_contact(Contact the\nJunifer team) + missing_other_contact --> missing_other_issue + missing_other_issue(((Submit a\nFEATURE REQUEST\nissue in Github))) + + read_using_final(Read Using Junifer) + read_using_final --> final_yaml + final_yaml(Create/edit the YAML file) + final_yaml --> final_run + final_run(Use junifer run to test your YAML configuration) + final_run --> question_final_run_worked + question_final_run_worked{"Did it work?"} + question_final_run_worked -->|No| question_error_run + question_error_run{"Is it an issue\nwith my YAML file?"} + question_error_run -->|Yes| final_yaml + question_error_run -->|No| error_contact + error_contact(Contact the\nJunifer team) + error_contact --> error_issue + error_issue(((Submit a\nBUG REPORT issue\nin Github))) + question_final_run_worked -->|Yes| final_queue + final_queue(Use junifer queue to compute your features) + final_queue --> final_magic + final_magic(((Let junifer do its magic!))) + + click read_understanding href "https://juaml.github.io/junifer/main/understanding/index.html" + click read_using href "https://juaml.github.io/junifer/main/using/index.html" + click read_using_final href "https://juaml.github.io/junifer/main/using/index.html" + click read_extending_datagrabber_start href "https://juaml.github.io/junifer/main/extending/extension.html" + click read_extending_datagrabber href "https://juaml.github.io/junifer/main/extending/datagrabber.html" + click read_extending_marker_start href "https://juaml.github.io/junifer/main/extending/extension.html" + click read_extending_marker href "https://juaml.github.io/junifer/main/extending/marker.html" + click read_adding_mask_start href "https://juaml.github.io/junifer/main/extending/extension.html" + click read_adding_mask href "https://juaml.github.io/junifer/main/extending/masks.html" + click read_adding_parcellation_start href "https://juaml.github.io/junifer/main/extending/extension.html" + click read_adding_parcellation href "https://juaml.github.io/junifer/main/extending/parcellations.html" + click read_adding_coordinates_start href "https://juaml.github.io/junifer/main/extending/extension.html" + click read_adding_coordinates href "https://juaml.github.io/junifer/main/extending/coordinates.html" + + click error_issue href "https://github.com/juaml/junifer/issues/new/choose" _blank + click missing_other_issue href "https://github.com/juaml/junifer/issues/new/choose" _blank + click contribute_marker href "https://github.com/juaml/junifer/issues/new/choose" _blank + click contribute_datagrabber href "https://github.com/juaml/junifer/issues/new/choose" _blank + + click error_contact href "https://juaml.github.io/junifer/main/help.html" + click contact_help href "https://juaml.github.io/junifer/main/help.html" + click missing_other_contact href "https://juaml.github.io/junifer/main/help.html" \ No newline at end of file diff --git a/docs/understanding/pipeline.rst b/docs/understanding/pipeline.rst index a19e94fe8..f70664167 100644 --- a/docs/understanding/pipeline.rst +++ b/docs/understanding/pipeline.rst @@ -17,14 +17,52 @@ The element that is passed accross the pipeline is called the :ref:`Data Object< The following is a graphical representation of the pipeline: -.. image:: ../images/pipeline/pipeline.001.png +.. mermaid:: + + flowchart LR + dg[Data Grabber] + dr[Data Reader] + pp[Pre-processing] + mc[Marker Computation] + st[Storage] + dg --> dr + dr --> pp + pp --> mc + mc --> st + However, it is usually the case that several markers are computed for the same data. Thus, the *markers* step of the pipeline is defined as a list of markers. The following is a graphical representation of the pipeline execution on multiple markers: -.. image:: ../images/pipeline/pipeline.002.png +.. mermaid:: + flowchart LR + dg[Data Grabber] + dr[Data Reader] + pp[Pre-processing] + mc1[Marker Computation] + mc2[Marker Computation] + mc3[Marker Computation] + mc4[Marker Computation] + mc5[Marker Computation] + st1[Storage] + st2[Storage] + st3[Storage] + st4[Storage] + st5[Storage] + dg --> dr + dr --> pp + pp --> mc1 + pp --> mc2 + pp --> mc3 + pp --> mc4 + pp --> mc5 + mc1 --> st1 + mc2 --> st2 + mc3 --> st3 + mc4 --> st4 + mc5 --> st5 .. note:: To avoid keeping in memory all of the computed marker, the storage step is called after each marker computation, releasing the memory used to compute each marker. \ No newline at end of file diff --git a/docs/using/codeless.rst b/docs/using/codeless.rst index 39117fa92..945e08397 100644 --- a/docs/using/codeless.rst +++ b/docs/using/codeless.rst @@ -11,7 +11,19 @@ achieved by using a configuration file that is written in YAML_. In this file, w As a reminder, this is how the pipeline looks like: -.. image:: ../images/pipeline/pipeline.001.png +.. mermaid:: + + flowchart LR + dg[Data Grabber] + dr[Data Reader] + pp[Pre-processing] + mc[Marker Computation] + st[Storage] + dg --> dr + dr --> pp + pp --> mc + mc --> st + Thus, the configuration file must configure each of the sections of the pipeline, as well as some general parameters. diff --git a/pyproject.toml b/pyproject.toml index 1f7863c65..db1ff6a46 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -71,6 +71,7 @@ docs = [ "julearn==0.2.5", "sphinx-copybutton==0.5.1", "towncrier==22.12.0", + "sphinxcontrib-mermaid==0.8.1", ] ################