Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compiler-versions script: Compute supported compiler versions for all packages #639

Merged
merged 17 commits into from
Jul 27, 2023

Conversation

colinwahl
Copy link
Collaborator

@colinwahl colinwahl commented Jul 26, 2023

This PR builds on #632 and is a step towards #255. It extends to compiler-versions script with the option to compute the supported compiler versions for all packages in the manifest index.

The algorithm for finding the supported packages for a specific compiler version is the following:

  1. Topologically sort the current manifest index, initialize an empty DependencyIndex
  2. For each Manifest, try to solve via the previous DependencyIndex and then compile - if it succeeds, add the entry to the DependencyIndex, else record failure and continue

The end result is a DependencyIndex with as many entries as we could solve and then compile with that compiler version. Any Manifest not found there either didn't solve (meaning its dependencies are not supported by that compiler version) or didn't compiler (meaning its dependencies are supported by that compiler version, but that package is not). We record all entries and continue to the next compiler.

I am currently dumping results to a file and logging progress messages per entry we check. There are comprehensive debug logs. The failure results are sorted in topological order so that we can see how failures propagate throughout.

Usage:

For all packages, all compilers:
GITHUB_TOKEN=xxx spago run -p registry-scripts -m Registry.Scripts.CompilerVersions -- --all-packages --all-compilers

For all packages, with a single compiler:
GITHUB_TOKEN=xxx spago run -p registry-scripts -m Registry.Scripts.CompilerVersions -- --all-packages --compiler 0.13.0

For a single package, checking all compilers:
GITHUB_TOKEN=xxx spago run -p registry-scripts -m Registry.Scripts.CompilerVersions -- --package prelude@3.0.0 --all-compilers

For a single package, checking a single compiler:
GITHUB_TOKEN=xxx spago run -p registry-scripts -m Registry.Scripts.CompilerVersions -- --package prelude@3.0.0 --compiler 0.13.0

Future Work:

I've ran the script on 0.15.0 and 0.13.0 and the output looked fine at a glance. In the future we will need to add some verification checks (perhaps making sure the package sets for that compiler version are all included in this output), but I've left that as follow on work.

I've ran into spurious API failures while installing packages quite often, which should be alleviated by #638.

@thomashoneyman
Copy link
Member

I think the algorithm is sound and I've been able to run this with the 0.15.x series and 0.14.x series of the compiler (trying 0.13.x now). Once I've verified the various CLI combinations I'll take a deeper code review. At a glance there are some lingering Debug imports we should remove.

@thomashoneyman
Copy link
Member

I like the progress output, thanks for doing that!

[0.13.0 1343/10830] Checking aff-promise@2.0.0

@thomashoneyman
Copy link
Member

It's taking a very long time to verify the 0.13.0 compiler, but so far I'm about 7,000 package versions in and no random failures (with #638 included on top of this code).

@thomashoneyman thomashoneyman merged commit 92d014a into master Jul 27, 2023
15 checks passed
@thomashoneyman thomashoneyman deleted the colin/compiler-versions-in-metadata-2 branch July 27, 2023 17:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants