add cooldown period when resolving version numbers during bootstrapping

[dhellmann]

The resolver automatically pulls in the latest version of each package that matches the rule being resolved. This means that as soon as a new version of a package is released, the next bootstrap operation will pull that version into the build. We want to introduce a "cooldown" period so that newly uploaded versions are only included in builds after they have been public for some number of days. This will give us a window to identify potentially malicious content in packages that have had their release pipelines subverted.

Requirements

All bootstrap commands should accept an option to specify the minimum age of a release before it should be included in a build. The default for the minimum age should be 0, indicating that packages can be included immediately. Only whole numbers of days are supported. Negative numbers are equivalent to 0.

All resolvers should honor the minimum age value. For resolvers that look at tags in git repositories, the date should come from the timestamp of the tag, if it is an annotated tag, or from the commit timestamp of the commit that was tagged otherwise. Resolvers that look at packages uploaded to servers (PyPI, etc.) should use the timestamp from when the package was uploaded.

The age of a package should be calculated by subtracting the timestamp of the release from the start time of the bootstrap process (not the current time when the resolution is happening). This ensures that resolving the same package multiple times during a build will produce the same results.

The per-package settings should provide an override to change the default age requirement. This is useful for packages that are maintained internally and known to be safe, to avoid having a delay (by setting the value to 0) or for riskier packages (by setting the value to a default that is larger than the command line option). The configured value should take priority over the value provided by the command line. This allows for different jobs (one to test builds and another to run the production builds) to easily pass different ages on the command line, but for the overrides to still take effect in both cases.

The list-versions command should also take a flag to specify the minimum age. The default should be 0. The detailed output should show the timestamp and age in whole days of each release. The list-versions command should also take an option to ignore the configured overrides for minimum age for separate packages. The default should be to honor the configuration settings, so that the command produces the same information that a bootstrap command would use, but by having an option to ignore the configuration settings we can use list-versions to show versions that have been ignored during bootstrapping.

The list-overrides command should be extended to show any configured minimum age value for packages with overrides.

Background

• https://blog.yossarian.net/2025/11/21/We-should-all-be-using-dependency-cooldowns
• https://helixguard.ai/blog/malicious-sha1hulud-2025-11-24

[tiran]

Some comments:

A cooldown period has an additional benefit: The new version is tested by a broader audience before it gets consumed by Fromager. The community can detect and report issues in a release and upstream maintainers can yank a bad release.

PyPIProvider with pypi.org and JSON API and GitLabTagProvider get the release time stamp for free. The information is part of the JSON response. PyPIProvider with simple HTTP API, GenericProvider, and GitHubTagProvider do not have access to release timestamp. There might be a way to get the information from GitHub with GraphQL API, though. Otherwise we have to query each tag for its metadata.

How should the cooldown feature deal with explicit requirements? For example a requirements.txt has example==2.0.0 but 2.0.0 was releases less than 24 hours ago. Should Fromager ignore the cooldown period for >, >=, and ~=? If yes, should we treat default/global cooldown differently than per-package cooldown?

[dhellmann]

Some comments:

PyPIProvider with pypi.org and JSON API and GitLabTagProvider get the release time stamp for free. The information is part of the JSON response. PyPIProvider with simple HTTP API, GenericProvider, and GitHubTagProvider do not have access to release timestamp. There might be a way to get the information from GitHub with GraphQL API, though. Otherwise we have to query each tag for its metadata.

The reason the GitHub and gitlab providers are different is they use the respective APIs already. I know we have other metadata about the tag, like the SHA, when we use the GitHub API. The API response appears to include a date.

How should the cooldown feature deal with explicit requirements? For example a requirements.txt has example==2.0.0 but 2.0.0 was releases less than 24 hours ago. Should Fromager ignore the cooldown period for >, >=, and ~=? If yes, should we treat default/global cooldown differently than per-package cooldown?

I don't think we should treat the cooldown period differently based on the rule. Most of our rules are going to include >, >=, or ~= type settings, eventually, so the cooldown value would be ignored by more and more packages.

If we need a specific version that is so new, we can add a config override for it. To do that downstream today we have to make a new builder image, but we could come up with another way to handle that if we find ourselves needing to do it temporarily a lot. For example, we could merge a config snippet from the pipeline repository into the config present in the builder image. But I don't think this is likely to be a common case, do you?

[tiran]

The reason the GitHub and gitlab providers are different is they use the respective APIs already. I know we have other metadata about the tag, like the SHA, when we use the GitHub API. The API response appears to include a date.

That's the GitLab REST API endpoint to query a single, individual tag. The GitHub's list tags API endpoint is missing time stamps. GitLab API for listing tags returns both commit sha and dates.

It looks like the GitLab REST API cannot list tags with their dates. We either need to query each tag individually or look into GitHub's GraphQL API.

[dhellmann]

The reason the GitHub and gitlab providers are different is they use the respective APIs already. I know we have other metadata about the tag, like the SHA, when we use the GitHub API. The API response appears to include a date.

That's the GitLab REST API endpoint to query a single, individual tag. The GitHub's list tags API endpoint is missing time stamps. GitLab API for listing tags returns both commit sha and dates.

It looks like the GitLab REST API cannot list tags with their dates. We either need to query each tag individually or look into GitHub's GraphQL API.

OK. The data is there, so we just need to work out an efficient way to access it. The graphql approach may be most efficient, and I've used that API before with some success.