kipe.provenance module

Provenance of a run: what ran, with which code, where; written next to the results.

recorded_run() writes, to the output directory of a subcommand (e.g., <output.path>/estimation/), next to the log and the results:

  • run.yaml: what the study does not hold: the status, running while the run lasts, then finished or failed, with start and end time; for estimation, the assimilation times (which may come from the data files).

  • study.yaml: the validated study options, with all defaults filled in: what actually ran. The comments of the study file are not kept; the CLI logs the study file’s path.

  • environment.yaml: host, platform, Python, MPI, thread settings, and the versions of kipe, of the forward solver’s package and of the main libraries. For kipe and the solver’s package, also their git state if they are run from a git checkout (e.g., an editable install).

  • <package>.patch: the uncommitted changes of such a checkout (git diff HEAD), so that a run from a modified checkout can be reproduced. Untracked files are not included.

  • packages.txt: all installed Python distributions, name==version.

  • spack.yaml and spack.lock, or conda_environment.yml: the environment, if the run happens in a Spack or conda environment.

Only rank 0 writes. The files are overwritten by the next run in the same directory.

kipe.provenance.recorded_run(output: Path, study: StudyOptions, extra: Mapping[str, Any] | None = None, comm: mpi4py.MPI.Comm = mpi4py.MPI.COMM_WORLD) → Iterator[None][source]

Write the provenance of the run in the with block, and its status at the end.

Parameters:

  • output – output directory of the subcommand

  • study – the study options

  • extra – further entries of run.yaml that the study file does not hold, e.g., the assimilation times

  • comm – communicator of the run; only its rank 0 writes

Yields:

nothing; the run happens in the with block

kipe.provenance.git_state(module: str, patch: Path | None = None) → dict[str, Any] | None[source]

Return the git state of the checkout a module is run from, if any.

Parameters:

  • module – import name of the module or package, e.g., "kipe"

  • patch – file to write the uncommitted changes to (git diff HEAD), if there are any

Returns:

repository, commit, describe (e.g., v0.1-3-gabc1234-dirty) and dirty; None if the module’s file is not tracked in a git checkout, e.g., for an installed package, or if git is not available