ci: slack notify integration

[PaulRBerg]

Closes this Notion task.

Using https://github.com/marketplace/actions/slack-notify to send a notification to the #ci-notifications channel on Slack whenever there's a failure in the scheduled workflow CI Deep.

@smol-ninja should this also be added to the fork tests workflow, as well as the staging branch?

[smol-ninja]

Nice and thanks for doing this.

• Yes, let's add it to the fork tests workflow so that we can track how many times they fail and if it's frequent, we can reduce runs or take some other action.

• Not required for the staging branch. The runs on the staging branch are triggered by the new PRs for which the owners of the PR always receive emails. Secondly, this also means it will spam the Slack channel since the failed runs on staging PRs are quite frequent .

[PaulRBerg]

Thanks. I will add it to the fork tests' workflow!

I now realize that we are no longer running CI Deep tests on the package-tethered version of the code base. Should we look into how to run the CI Deep workflow against staging instead of main?

[smol-ninja]

IIRC we never used to run CI Deep tests on the staging branch. Thats why they are not running against the package tethering version. Are you proposing the following?

• Run CI Fork tests only on main branch
• Run CI Deep tests only on staging branch

If so, it makes sense to me.

[smol-ninja]

I just realised CI Fork tests are not as deep as fork tests running in CI Deep. So we should keep running CI Deep against the main branch i.e.

• Run CI Fork tests only on main branch
• Run CI Deep tests on main and staging branch both

[PaulRBerg]

Running CI Fork tests only on main and CI Deep only on main and staging makes sense.

But unfortunately, it is not possible to specify the branch when scheduling workflows. GitHub will default to the main branch (our default branch). To run the workflows against a different branch, we would need to pass a with: parameter to the checkout action. This, in turn, would require modifying all the reusable workflows to accept a with: parameter. And worse still, doing so would make it impossible to run the CI workflows manually from a different branch using the GitHub UI.

Let's stick with the defaults for now. We can manually run CI Deep against staging from time to time.

[PaulRBerg]

So this should be good to merge now @smol-ninja.