[ENH]: #316

New issue

Open

opened 2024-03-27 10:50:41 +00:00 by LeSasse · 0 comments

LeSasse commented

2024-03-27 10:50:41 +00:00

(Migrated from github.com)

Are you requiring a new dataset or marker?

I understand this is not a marker or dataset request

Which feature do you want to include?

I have prototyped a small command line utility to collect and aggregate logging information from junifer pipelines into convenient DataFrames. This DataFrame contains information about resources used (CPUs, time elapsed, memory, disk space used ...), warnings, job success etc. Each row corresponds to a junifer 'element'. For now, it is assumed that logs are from HTCondor as no other queueing backends are implemented in junifer.

As discussed with @fraimondo this could be integrated into junifer similarly to #315

How do you imagine this integrated in junifer?

as an additional command

or simply by linking to the tool in a list of curated external tools in the documentation

Do you have a sample code that implements this outside of junifer?

https://github.com/LeSasse/junilog

Anything else to say?

No response

### Are you requiring a new dataset or marker? - [X] I understand this is not a marker or dataset request ### Which feature do you want to include? I have prototyped a small command line utility to collect and aggregate logging information from junifer pipelines into convenient DataFrames. This DataFrame contains information about resources used (CPUs, time elapsed, memory, disk space used ...), warnings, job success etc. Each row corresponds to a junifer 'element'. For now, it is assumed that logs are from [HTCondor](https://htcondor.org/) as no other queueing backends are implemented in junifer. As discussed with @fraimondo this could be integrated into junifer similarly to #315 ### How do you imagine this integrated in junifer? as an additional command or simply by linking to the tool in a list of curated external tools in the documentation ### Do you have a sample code that implements this outside of junifer? ```shell https://github.com/LeSasse/junilog ``` ### Anything else to say? _No response_