juaml/junifer

Watch

Fork 0

[ENH] Implement `junifer selftest` to allow the users to test the library themselves #9

Merged

synchon merged 3 commits from feature/selftest-cmd into main

2022-10-25 06:57:12 +00:00

synchon commented

2022-10-23 08:10:35 +00:00

(Migrated from github.com)

It might be the case that some users have specific working environments that are not comprised within the tests from junifer (i.e. specific versions of the dependencies). For solving issues in a more efficient way, it might be good if we can have a junifer selftest command that run the test suite within the environment (so no tox, just pytest).

It might be the case that some users have specific working environments that are not comprised within the tests from junifer (i.e. specific versions of the dependencies). For solving issues in a more efficient way, it might be good if we can have a `junifer selftest` command that run the test suite within the environment (so no `tox`, just `pytest`).

synchon commented

2022-10-21 11:46:48 +00:00

(Migrated from github.com)

Do we want the user to see the pytest outputs in "real-time"?

Do we want the user to see the `pytest` outputs in "real-time"?

github-actions[bot] commented

2022-10-23 08:15:57 +00:00

(Migrated from github.com)

PR Preview Action v1.2.0
Preview removed because the pull request was closed.
2022-10-25 07:03 UTC

[PR Preview Action](https://github.com/rossjrw/pr-preview-action) v1.2.0 :---: Preview removed because the pull request was closed. 2022-10-25 07:03 UTC

synchon commented

2022-10-23 08:16:24 +00:00

(Migrated from github.com)

The pytest invocation is done via subprocess as running it from the same process will use the cache after the first run.
The current state redirects the subprocess stdout to sys.stdout and hence prints the pytest output in "real-time" but of course is dependent on system's buffer.
The junifer selftest command takes ~6-7 minutes to complete, so testing it by invoking in the test suite is a bit tricky now.

- The `pytest` invocation is done via `subprocess` as running it from the same process will use the cache after the first run. - The current state redirects the subprocess `stdout` to `sys.stdout` and hence prints the `pytest` output in "real-time" but of course is dependent on system's buffer. - The `junifer selftest` command takes ~6-7 minutes to complete, so testing it by invoking in the test suite is a bit tricky now.

codecov[bot] commented

2022-10-23 08:19:40 +00:00

(Migrated from github.com)

Codecov Report

Merging #9 (a8007eb) into main (9663700) will decrease coverage by 0.16%.
The diff coverage is 66.66%.

@@            Coverage Diff             @@
##             main       #9      +/-   ##
==========================================
- Coverage   88.11%   87.94%   -0.17%     
==========================================
  Files          50       50              
  Lines        1910     1925      +15     
  Branches      353      356       +3     
==========================================
+ Hits         1683     1693      +10     
- Misses        181      185       +4     
- Partials       46       47       +1

Impacted Files	Coverage Δ
junifer/api/cli.py	`78.94% <66.66%> (-3.02%)`	⬇️

# [Codecov](https://codecov.io/gh/juaml/junifer/pull/9?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml) Report > Merging [#9](https://codecov.io/gh/juaml/junifer/pull/9?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml) (a8007eb) into [main](https://codecov.io/gh/juaml/junifer/commit/96637005ae4f8be43d5e4e4d726c9bb9113c3ba0?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml) (9663700) will **decrease** coverage by `0.16%`. > The diff coverage is `66.66%`. [![Impacted file tree graph](https://codecov.io/gh/juaml/junifer/pull/9/graphs/tree.svg?width=650&height=150&src=pr&token=5H21JuZXMw&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml)](https://codecov.io/gh/juaml/junifer/pull/9?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml) ```diff @@ Coverage Diff @@ ## main #9 +/- ## ========================================== - Coverage 88.11% 87.94% -0.17% ========================================== Files 50 50 Lines 1910 1925 +15 Branches 353 356 +3 ========================================== + Hits 1683 1693 +10 - Misses 181 185 +4 - Partials 46 47 +1 ``` | [Impacted Files](https://codecov.io/gh/juaml/junifer/pull/9?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml) | Coverage Δ | | |---|---|---| | [junifer/api/cli.py](https://codecov.io/gh/juaml/junifer/pull/9/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml#diff-anVuaWZlci9hcGkvY2xpLnB5) | `78.94% <66.66%> (-3.02%)` | :arrow_down: |

fraimondo commented

2022-10-23 09:31:15 +00:00

(Migrated from github.com)

The pytest invocation is done via subprocess as running it from the same process will use the cache after the first run.

The current state redirects the subprocess stdout to sys.stdout and hence prints the pytest output in "real-time" but of course is dependent on system's buffer.

The junifer selftest command takes ~6-7 minutes to complete, so testing it by invoking in the test suite is a bit tricky now.

Was it better to use subprocess than just pytest python api? https://docs.pytest.org/en/7.1.x/how-to/usage.html#calling-pytest-from-python-code

> * The `pytest` invocation is done via `subprocess` as running it from the same process will use the cache after the first run. > * The current state redirects the subprocess `stdout` to `sys.stdout` and hence prints the `pytest` output in "real-time" but of course is dependent on system's buffer. > * The `junifer selftest` command takes ~6-7 minutes to complete, so testing it by invoking in the test suite is a bit tricky now. Was it better to use `subprocess` than just pytest python api? https://docs.pytest.org/en/7.1.x/how-to/usage.html#calling-pytest-from-python-code

synchon commented

2022-10-23 09:54:34 +00:00

(Migrated from github.com)

The pytest invocation is done via subprocess as running it from the same process will use the cache after the first run.

The current state redirects the subprocess stdout to sys.stdout and hence prints the pytest output in "real-time" but of course is dependent on system's buffer.

The junifer selftest command takes ~6-7 minutes to complete, so testing it by invoking in the test suite is a bit tricky now.

Was it better to use subprocess than just pytest python api? https://docs.pytest.org/en/7.1.x/how-to/usage.html#calling-pytest-from-python-code

Yeah it is related to the first point and since we intend to run it multiple times, we need separate process everytime. So, we create new process everytime, "why not just call the command?" was my thought.

> > * The `pytest` invocation is done via `subprocess` as running it from the same process will use the cache after the first run. > > > * The current state redirects the subprocess `stdout` to `sys.stdout` and hence prints the `pytest` output in "real-time" but of course is dependent on system's buffer. > > > * The `junifer selftest` command takes ~6-7 minutes to complete, so testing it by invoking in the test suite is a bit tricky now. > > > > Was it better to use `subprocess` than just pytest python api? https://docs.pytest.org/en/7.1.x/how-to/usage.html#calling-pytest-from-python-code Yeah it is related to the first point and since we intend to run it multiple times, we need separate process everytime. So, we create new process everytime, "why not just call the command?" was my thought.

fraimondo commented

2022-10-23 10:07:11 +00:00

(Migrated from github.com)

What about using --cache-clear?

What about using `--cache-clear`?

fraimondo commented

2022-10-23 10:07:15 +00:00

(Migrated from github.com)

https://docs.pytest.org/en/6.2.x/cache.html

synchon commented

2022-10-23 10:22:43 +00:00

(Migrated from github.com)

What about using --cache-clear?

Do you mean to use this along with the subprocess implementation?

> What about using `--cache-clear`? Do you mean to use this along with the subprocess implementation?

fraimondo commented

2022-10-23 17:25:50 +00:00

(Migrated from github.com)

No. I meant to use the python interface with cache clear.

synchon commented

2022-10-23 17:53:07 +00:00

(Migrated from github.com)

So I tested it and it works fine as well. Do we use this then?

fraimondo commented

2022-10-23 18:34:53 +00:00

(Migrated from github.com)

Yes, sure.

synchon commented

2022-10-23 18:41:46 +00:00

(Migrated from github.com)

I only see one problem with the pytest.main() approach which I find after using it multiple times is that it does not handle keyboard interrupt gracefully.

I only see one problem with the `pytest.main()` approach which I find after using it multiple times is that it does not handle keyboard interrupt gracefully.

fraimondo (Migrated from github.com) approved these changes

2022-10-25 06:56:01 +00:00