Rerun GitHub Action workflows on the first failure

[desrosj]

It's fairly common for GitHub Action workflows to fail because of several different types of errors. Some examples include:

• Chromium timeout errors
• WordPress.org hiccups when making requests within PHP unit tests
• NPM install errors
• Docker container registry timout and connection issues.

In almost all cases (except when there are service level outages), rerunning the workflow will result in a successful outcome. Because this is currently manual, it's sometimes forgotten.

Our Actions workflows should attempt to fix these runs by automatically rerunning failed jobs.

This workflow will restart a failed or cancelled workflow when it has only been run once.

This is dependent on actions/github-script#283, which will add the needed .rest.actions.reRunWorkflowFailedJobs() function added in octokit/plugin-rest-endpoint-methods.js@v5.14.0.

Trac ticket: https://core.trac.wordpress.org/ticket/56407

There is some inconsistency, where github.rest.actions.createWorkflowDispatch is sometimes called with const result = await and sometimes just called on it's own. I think one should be picked and used throughout

Ah, good spot. This was a relic from when I was debugging why the workflow_dispatch call was not working in my fork.

One thing that is not clear to me reading through the action's documentation is whether github-script will cause a step or job to fail if a non 2xx response code is returned by an API call. If not, using await and checking the response for a problem manually would be preferred. But for now I have removed the one instance of that in the coding standards workflow.

[desrosj]

In 53947:

Build/Test Tools: Automatically rerun a workflow the first time it fails.

There are several common reoccurring issues that sometimes cause GitHub Action workflows to fail (connection timeouts to WordPress.org or the Docker container registry, npm install failures, Chromium issues, etc.). Except when there are service level outages, most of these issues can be resolved by simply rerunning the workflow.

This introduces a new step within each of Core’s GitHub Action workflows that attempts to rerun the failed jobs within the workflow that encountered a failure if they are running for the first time. Since a workflow is not allowed to restart itself, a new failed-workflow.yml callable workflow is being introduced.

Other related adjustments in this changeset:

• The actions/github-script 3rd-party action is also now updated to the latest version (v6.2.0).
• A new secret, GHA_WORKFLOW_DISPATCH, has been introduced. This will replace the current one in use (GHA_OLD_BRANCH_DISPATCH) with a less specific name.

Props jorbin, desrosj.
Fixes #56407.

Merged in https://core.trac.wordpress.org/changeset/53947.

[desrosj]

The workflow itself works when manually called and providing a cancelled or failed workflow run ID (see ​this test), but for some reason the step is being skipped in other workflows. It worked in the test repository I made on GitHub, so not sure what is missing.

Will dig in.

[desrosj]

Turns out to be an incorrect variable being used. github.run_number (the unique number for a run represents for a workflow over time) is not the same as github.run_attempt (the run attempt number for a given workflow run). The attempt is the correct one here.

[desrosj]

In 54039:

Build/Test Tools: Correct the context variable being used when auto-retrying failed and cancelled workflows.

The correct variable here is github.run_attempt, which represents the unique number for each attempt of a particular workflow run in a repository.

The github.run_number currently being used represents the unique number for each run of a particular workflow in a repository.

Follow up to [53947].

Fixes #56407.

[TobiasBg]

Hi @desrosj, thanks for your work on this! Cool idea!

To have it documented: Why is the GHA_WORKFLOW_DISPATCH secret needed over e.g. the default GITHUB_TOKEN, and what permission scope does it need? Thanks!

[desrosj]

Great question, @TobiasBg!

I definitely wanted to just use GITHUB_TOKEN instead. But in my testing, GITHUB_TOKEN I found that does not have the required permissions to modify workflow runs through the REST API.

The documentation for creating a workflow dispatch event mentions actions:write is ​required, but that specifically mentions GitHub Apps.

GitHub Actions does ​support specifying custom permissions in a workflow through permissions at the top or job level, but in my testing, even specifying permissions: write-all had no effect.

It can also be a bit hard to get to the bottom of because calling github.rest.actions.createWorkflowDispatch() with a token lacking the required permissions still returns a 204 status (the one ​documented as expected). But eventually, I found the ​right documentation on this here.

When you use the repository's GITHUB_TOKEN to perform tasks, events triggered by the GITHUB_TOKEN will not create a new workflow run. This prevents you from accidentally creating recursive workflow runs... If you do want to trigger a workflow from within a workflow run, you can use a personal access token instead of GITHUB_TOKEN to trigger events that require a token.

So seems that it's an intentional design decision to prevent user error.

[desrosj]

This came through today: ​https://github.blog/changelog/2022-09-08-github-actions-use-github_token-with-workflow_dispatch-and-repository_dispatch/. Looks like using the supplied token is now allowed! I’ll update the workflows through #55652.

[desrosj]

Reopening to add the enable_debug_logging parameter when restarting a workflow. See ​https://docs.github.com/en/rest/actions/workflow-runs#re-run-failed-jobs-from-a-workflow-run.

[desrosj]

In 54264:

Build/Test Tools: Enable debugging when rerunning a failed workflow.

Follow up to [53947], [54039].

Fixes #56407.

[desrosj]

In 55516:

Build/Test Tools: Backport updates to GitHub Actions.

This backports several changesets to GitHub Actions workflows. These changesets:

• address the deprecated notices related to save-output and set-output to ensure the workflows continue to run after these are removed.
• adds support for automatically retrying a failed workflow once.
• removes workflow files that are not applicable to the branch.

Merges [53736], [53737], [53940], [53947], [54039], [54108], [54293], [54313], [54342], [54343], [54373], [54511], [54649], [54650], [54651], [54674], [54750], [54852], [55152], [55487] to the 5.9 branch.
See #55652, #56407, #54695, #56820, #56816, #56793, #56820, #57572.

[desrosj]

In 55517:

Build/Test Tools: Backport updates to GitHub Actions.

This backports several changesets to GitHub Actions workflows. These changesets:

• address the deprecated notices related to save-output and set-output to ensure the workflows continue to run after these are removed.
• adds support for automatically retrying a failed workflow once.
• removes workflow files that are not applicable to the branch.
• backports some Docker environment related tooling updates for the sake of consistency across branches.

Merges [53736], [53737], [53940], [53947], [54039], [54096], [54108], [54293], [54313], [54342], [54343], [54373], [54511], [54649], [54650], [54651], [54674], [54750], [54852], [55152], [55487] to the 5.8 branch.
See #55652, #56407, #56528, #54695, #56820, #56816, #56793, #56820, #57572.

[desrosj]

In 55518:

Build/Test Tools: Backport updates to GitHub Actions.

This backports several changesets to GitHub Actions workflows. These changesets:

• address the deprecated notices related to save-output and set-output to ensure the workflows continue to run after these are removed.
• adds support for automatically retrying a failed workflow once.
• removes workflow files that are not applicable to the branch.
• backports some Docker environment related tooling updates for the sake of consistency across branches.

Merges [53736], [53737], [53940], [53947], [54039], [54096], [54108], [54293], [54313], [54342], [54343], [54373], [54511], [54649], [54650], [54651], [54674], [54750], [54852], [55152], [55487] to the 5.8 branch.
See #55652, #56407, #56528, #54695, #56820, #56816, #56793, #56820, #57572.

[desrosj]

In 55519:

Build/Test Tools: Backport updates to GitHub Actions.

This backports several changesets to GitHub Actions workflows. These changesets:

• address the deprecated notices related to save-output and set-output to ensure the workflows continue to run after these are removed.
• adds support for automatically retrying a failed workflow once.
• removes workflow files that are not applicable to the branch.
• backports some Docker environment related tooling updates for the sake of consistency across branches.

Merges [53736], [53737], [53940], [53947], [54039], [54096], [54108], [54293], [54313], [54342], [54343], [54373], [54511], [54649], [54650], [54651], [54674], [54750], [54852], [55152], [55487] to the 5.6 branch.
See #55652, #56407, #56528, #54695, #56820, #56816, #56793, #56820, #57572.

[desrosj]

In 55520:

Build/Test Tools: Backport updates to GitHub Actions.

This backports several changesets to GitHub Actions workflows. These changesets:

• address the deprecated notices related to save-output and set-output to ensure the workflows continue to run after these are removed.
• adds support for automatically retrying a failed workflow once.
• removes workflow files that are not applicable to the branch.
• backports some Docker environment related tooling updates for the sake of consistency across branches.

Merges [53736], [53737], [53940], [53947], [54039], [54096], [54108], [54293], [54313], [54342], [54343], [54373], [54511], [54649], [54650], [54651], [54674], [54750], [54852], [55152], [55487] to the 5.5 branch.
See #55652, #56407, #56528, #54695, #56820, #56816, #56793, #56820, #57572.

[desrosj]

In 55521:

Build/Test Tools: Backport updates to GitHub Actions.

This backports several changesets to GitHub Actions workflows. These changesets:

• address the deprecated notices related to save-output and set-output to ensure the workflows continue to run after these are removed.
• adds support for automatically retrying a failed workflow once.
• removes workflow files that are not applicable to the branch.
• backports some Docker environment related tooling updates for the sake of consistency across branches.

Merges [53736], [53737], [53940], [53947], [54039], [54096], [54108], [54293], [54313], [54342], [54343], [54373], [54511], [54649], [54650], [54651], [54674], [54750], [54852], [55152], [55487] to the 5.4 branch.
See #55652, #56407, #56528, #54695, #56820, #56816, #56793, #56820, #57572.

[desrosj]

In 55522:

Build/Test Tools: Backport updates to GitHub Actions.

This backports several changesets to GitHub Actions workflows. These changesets:

• address the deprecated notices related to save-output and set-output to ensure the workflows continue to run after these are removed.
• adds support for automatically retrying a failed workflow once.
• removes workflow files that are not applicable to the branch.
• backports some Docker environment related tooling updates for the sake of consistency across branches.

Merges [53736], [53737], [53940], [53947], [54039], [54096], [54108], [54293], [54313], [54342], [54343], [54373], [54511], [54649], [54650], [54651], [54674], [54750], [54852], [55152], [55487] to the 5.3 branch.
See #55652, #56407, #56528, #54695, #56820, #56816, #56793, #56820, #57572.

[desrosj]

In 55523:

Build/Test Tools: Backport updates to GitHub Actions.

This backports several changesets to GitHub Actions workflows. These changesets:

• address the deprecated notices related to save-output and set-output to ensure the workflows continue to run after these are removed.
• adds support for automatically retrying a failed workflow once.
• removes workflow files that are not applicable to the branch.
• backports some Docker environment related tooling updates for the sake of consistency across branches.

Merges [53736], [53737], [53940], [53947], [54039], [54096], [54108], [54293], [54313], [54342], [54343], [54373], [54511], [54649], [54650], [54651], [54674], [54750], [54852], [55152], [55487] to the 5.2 branch.
See #55652, #56407, #56528, #54695, #56820, #56816, #56793, #56820, #57572.

[desrosj]

In 55524:

Build/Test Tools: Backport updates to GitHub Actions.

This backports several changesets to GitHub Actions workflows. These changesets:

• address the deprecated notices related to save-output and set-output to ensure the workflows continue to run after these are removed.
• adds support for automatically retrying a failed workflow once.
• removes workflow files that are not applicable to the branch.
• backports some Docker environment related tooling updates for the sake of consistency across branches.

Merges [53736], [53737], [53940], [53947], [54039], [54096], [54108], [54293], [54313], [54342], [54343], [54373], [54511], [54650], [54651], [54674], [54750], [54852], [55152], [54651], [55487] to the 5.1 branch.
See #55652, #56407, #56528, #54695, #56820, #56816, #56793, #56820, #57572.

[desrosj]

In 55525:

Build/Test Tools: Backport updates to GitHub Actions.

This backports several changesets to GitHub Actions workflows. These changesets:

• address the deprecated notices related to save-output and set-output to ensure the workflows continue to run after these are removed.
• adds support for automatically retrying a failed workflow once.
• removes workflow files that are not applicable to the branch.
• backports some Docker environment related tooling updates for the sake of consistency across branches.

Merges [53736], [53737], [53940], [53947], [54039], [54096], [54108], [54293], [54313], [54342], [54343], [54373], [54511], [54650], [54651], [54674], [54750], [54852], [55152], [54651], [55487] to the 5.0 branch.
See #55652, #56407, #56528, #54695, #56820, #56816, #56793, #56820, #57572.

[desrosj]

In 55527:

Build/Test Tools: Backport updates to GitHub Actions.

This backports several changesets to GitHub Actions workflows. These changesets:

• address the deprecated notices related to save-output and set-output to ensure the workflows continue to run after these are removed.
• adds support for automatically retrying a failed workflow once.
• removes workflow files that are not applicable to the branch.
• backports some Docker environment related tooling updates for the sake of consistency across branches.

Merges [53736], [53737], [53940], [53947], [54039], [54096], [54108], [54293], [54313], [54342], [54343], [54373], [54511], [54650], [54651], [54674], [54750], [54852], [55152], [54651], [55487] to the 4.9 branch.
See #55652, #56407, #56528, #54695, #56820, #56816, #56793, #56820, #57572.

[desrosj]

In 55528:

Build/Test Tools: Backport updates to GitHub Actions.

This backports several changesets to GitHub Actions workflows. These changesets:

• address the deprecated notices related to save-output and set-output to ensure the workflows continue to run after these are removed.
• adds support for automatically retrying a failed workflow once.
• removes workflow files that are not applicable to the branch.
• backports some Docker environment related tooling updates for the sake of consistency across branches.

Merges [53736], [53737], [53940], [53947], [54039], [54096], [54108], [54293], [54313], [54342], [54343], [54373], [54511], [54650], [54651], [54674], [54750], [54852], [55152], [54651], [55487] to the 4.8 branch.
See #55652, #56407, #56528, #54695, #56820, #56816, #56793, #56820, #57572.

[desrosj]

In 55529:

Build/Test Tools: Backport updates to GitHub Actions.

This backports several changesets to GitHub Actions workflows. These changesets:

• address the deprecated notices related to save-output and set-output to ensure the workflows continue to run after these are removed.
• adds support for automatically retrying a failed workflow once.
• removes workflow files that are not applicable to the branch.
• backports some Docker environment related tooling updates for the sake of consistency across branches.

Merges [53736], [53737], [53940], [53947], [54039], [54096], [54108], [54293], [54313], [54342], [54343], [54373], [54511], [54650], [54651], [54674], [54750], [54852], [55152], [54651], [55487] to the 4.7 branch.
See #55652, #56407, #56528, #54695, #56820, #56816, #56793, #56820, #57572.

[desrosj]

In 55530:

Build/Test Tools: Backport updates to GitHub Actions.

This backports several changesets to GitHub Actions workflows. These changesets:

• address the deprecated notices related to save-output and set-output to ensure the workflows continue to run after these are removed.
• adds support for automatically retrying a failed workflow once.
• removes workflow files that are not applicable to the branch.
• backports some Docker environment related tooling updates for the sake of consistency across branches.

Merges [53736], [53737], [53940], [53947], [54039], [54096], [54108], [54293], [54313], [54342], [54343], [54373], [54511], [54650], [54651], [54674], [54750], [54852], [55152], [54651], [55487] to the 4.6 branch.
See #55652, #56407, #56528, #54695, #56820, #56816, #56793, #56820, #57572.

[desrosj]

In 55531:

Build/Test Tools: Backport updates to GitHub Actions.

This backports several changesets to GitHub Actions workflows. These changesets:

• address the deprecated notices related to save-output and set-output to ensure the workflows continue to run after these are removed.
• adds support for automatically retrying a failed workflow once.
• removes workflow files that are not applicable to the branch.
• backports some Docker environment related tooling updates for the sake of consistency across branches.

Merges [53736], [53737], [53940], [53947], [54039], [54096], [54108], [54293], [54313], [54342], [54343], [54373], [54511], [54650], [54651], [54674], [54750], [54852], [55152], [54651], [55487] to the 4.5 branch.
See #55652, #56407, #56528, #54695, #56820, #56816, #56793, #56820, #57572.

[desrosj]

In 55532:

Build/Test Tools: Backport updates to GitHub Actions.

This backports several changesets to GitHub Actions workflows. These changesets:

• address the deprecated notices related to save-output and set-output to ensure the workflows continue to run after these are removed.
• adds support for automatically retrying a failed workflow once.
• removes workflow files that are not applicable to the branch.
• backports some Docker environment related tooling updates for the sake of consistency across branches.

Merges [53736], [53737], [53940], [53947], [54039], [54096], [54108], [54293], [54313], [54342], [54343], [54373], [54511], [54650], [54651], [54674], [54750], [54852], [55152], [54651], [55487] to the 4.4 branch.
See #55652, #56407, #56528, #54695, #56820, #56816, #56793, #56820, #57572.

[desrosj]

In 55533:

Build/Test Tools: Backport updates to GitHub Actions.

This backports several changesets to GitHub Actions workflows. These changesets:

• address the deprecated notices related to save-output and set-output to ensure the workflows continue to run after these are removed.
• adds support for automatically retrying a failed workflow once.
• removes workflow files that are not applicable to the branch.
• backports some Docker environment related tooling updates for the sake of consistency across branches.

Merges [53736], [53737], [53940], [53947], [54039], [54096], [54108], [54293], [54313], [54342], [54343], [54373], [54511], [54650], [54651], [54674], [54750], [54852], [55152], [54651], [55487] to the 4.3 branch.
See #55652, #56407, #56528, #54695, #56820, #56816, #56793, #56820, #57572.

[desrosj]

In 55534:

Build/Test Tools: Backport updates to GitHub Actions.

This backports several changesets to GitHub Actions workflows. These changesets:

• address the deprecated notices related to save-output and set-output to ensure the workflows continue to run after these are removed.
• adds support for automatically retrying a failed workflow once.
• removes workflow files that are not applicable to the branch.
• backports some Docker environment related tooling updates for the sake of consistency across branches.

Merges [53736], [53737], [53940], [53947], [54039], [54096], [54108], [54293], [54313], [54342], [54343], [54373], [54511], [54650], [54651], [54674], [54750], [54852], [55152], [54651], [55487] to the 4.2 branch.
See #55652, #56407, #56528, #54695, #56820, #56816, #56793, #56820, #57572.

[desrosj]

In 55535:

Build/Test Tools: Backport updates to GitHub Actions.

This backports several changesets to GitHub Actions workflows. These changesets:

• address the deprecated notices related to save-output and set-output to ensure the workflows continue to run after these are removed.
• adds support for automatically retrying a failed workflow once.
• removes workflow files that are not applicable to the branch.
• backports some Docker environment related tooling updates for the sake of consistency across branches.

Merges [53736], [53737], [53940], [53947], [54039], [54096], [54108], [54293], [54313], [54342], [54343], [54373], [54511], [54650], [54651], [54674], [54750], [54852], [55152], [54651], [55487] to the 4.1 branch.
See #55652, #56407, #56528, #54695, #56820, #56816, #56793, #56820, #57572.