A saved model is only one part of an experiment. To understand or repeat the work, a later reader also needs to know which data, code, configuration and environment produced it, and what the reported result actually measures. A useful experiment record connects these pieces without relying on the original author's memory. This article proposes a compact manifest and review workflow. It is independently authored editorial guidance, not a description of how the historical research projects on this website were conducted.
The scikit-learn documentation on model persistence recommends retaining the training-data reference, source code, dependency versions and evaluation information alongside a saved model. It also warns that loading models across different scikit-learn versions is unsupported. Those points motivate a broader recordkeeping question: could another authorized reader identify the exact ingredients of the reported run before trying to execute anything?
Give the run an identity that survives a filename change
Assign a stable identifier to the experiment and record its date, purpose and responsible role. The identifier should connect the configuration, logs, outputs and evaluation note. A directory named final is not a sufficient identity when several later directories acquire the same name. Choose a convention that works with the project's existing tools and use it consistently. The important feature is traceability, not a particular naming pattern or a complicated tracking platform.
Write a short purpose statement describing the question the run was intended to address. Distinguish an exploratory attempt from a result selected for reporting. If a run failed or was abandoned, retain enough information to understand why it should not be mistaken for the published result. A tidy folder can conceal ambiguity if the status of its contents is unclear. The manifest should make the selected run and its relationship to earlier attempts explicit.
Reference the data without spreading private copies
Record the data source, version and relevant selection rules. Where possible, identify an immutable snapshot or another stable reference that an authorized collaborator can resolve. Do not place restricted or personal data in a public repository merely to make the experiment look reproducible. Document the access conditions and the steps needed to obtain the permitted input. A public manifest can describe a controlled data dependency without exposing the underlying records.
Include any exclusions, filtering or manual corrections that affected the run. A raw dataset name does not identify the actual training set when a separate preparation step selected only part of it. Record the preparation code and configuration as well as the input reference. If a manual decision cannot be reproduced exactly, state that limitation. The goal is to describe the real experiment faithfully, not to replace an inconvenient part of its history with a cleaner fictional pipeline.
Connect the result to an exact code revision
Save the repository revision and indicate whether the working directory contained uncommitted changes. If local edits were used, preserve a reviewable patch or another controlled record of those edits. A commit identifier alone can be misleading when the actual run included additional changes. Record the entry point and the command or configuration used to launch the experiment, with secrets removed and references to the approved secret-management process where necessary.
Keep configuration separate from the story you tell about the result. The configuration should contain the actual values used, while the narrative explains why those values were chosen. Avoid reconstructing parameters from memory after a successful run. If an automatic search selected a configuration, retain the selection procedure and the relevant outputs. This allows a later reader to distinguish a single fixed experiment from the final member of a broader search process.
Record the environment as an input
Capture the language runtime and dependency versions relevant to execution, including the method used to install them. Record important hardware and operating-system details when they affect the experiment or interpretation. An environment name on one machine may not identify a reproducible set of packages on another. Keep a lockfile or equivalent resolved specification where the project's tools support it, and explain any dependencies that require a separate installation step.
Do not promise bit-for-bit reproducibility merely because versions were recorded. Hardware behavior, parallel execution, external services and other factors may still matter. State what level of repeatability was actually checked. If a run was repeated, preserve the evidence and describe how the outputs were compared. If it was not repeated, label the record as preparation for reproduction rather than as proof that reproduction has already succeeded on another environment.
Preserve the evaluation definition
Record the evaluation data reference, metric definition and the procedure that produced the reported value. A score without this context is difficult to interpret. Identify any aggregation across groups, folds or repeated runs, and keep the underlying results needed to understand that aggregation. Do not mix an exploratory validation result with a final evaluation simply because both use the same metric name. Their roles in the development process may be different.
Write down which results influenced model or parameter selection. This makes the evaluation history easier to assess and connects the manifest with a leakage review. If a threshold was chosen after inspecting a particular set of outcomes, record that fact. The manifest does not need to defend every choice in a long essay, but it should provide enough information for a reviewer to find the decision and understand the evidence used to make it.
Treat a saved model as executable material
Record the serialization format, the tool that created it and the intended loading procedure. Do not open an unknown model artifact merely because it has a familiar extension. The scikit-learn documentation notes that pickle-based formats can execute arbitrary code during loading and should be used only with trusted, verified sources. Keep provenance and integrity information with the artifact, and use the project's approved handling process rather than improvising a quick inspection on a personal machine.
A checksum can help detect that a file changed, but it does not establish that the file is trustworthy or appropriate to execute. Keep those questions separate. The manifest should identify who produced the artifact, where it came from and what checks were performed. If the model depends on custom code, retain the corresponding source revision. A binary file without its required code and environment may be an output you can store but not an experiment you can meaningfully reconstruct.
Use a hypothetical handoff to test the record
Imagine handing a small classification experiment to a colleague who has access to the approved data but did not attend the project meetings. The colleague should be able to identify the selected run, locate the input snapshot, obtain the exact code and understand the evaluation before executing it. If they must guess which of three configuration files was used, the record is incomplete even if every file exists. Add the missing connection rather than expecting filename intuition to solve it.
Ask the colleague to describe what the manifest says, then compare that description with the intended experiment. This can reveal ambiguity without requiring a full retraining run. If execution is performed, record the environment and outcome of that attempt separately. A successful manifest review and a successful reproduction are different checks. Both can be valuable, but combining them under one verified label would hide what was actually tested.
Keep corrections visible and make the next step clear
When an error is found in the record, correct it and explain what changed. Do not silently replace a reported configuration if the change affects interpretation of the result. Preserve the relationship between the original report and the corrected evidence. A small correction log can be enough when it identifies the affected run and the reason for the update. This helps collaborators avoid comparing results generated from different assumptions while believing they are using the same experiment.
The research portfolio contains the original projects and their links; the blog offers separately authored methods notes. See the editorial policy for attribution. A useful experiment manifest is not a badge of perfect reproducibility. It is a clear map from a reported result to the data, code, environment and decisions that produced it, together with an honest account of what another reader has and has not been able to verify.