June 13, 2021

Preventing concurrent GitHub Actions

What happens when you trigger a GitHub Actions workflow which is already running? Workflows which depend on being run one-at-a-time might fail.

I recently encountered this with a workflow for publishing a static website. This workflow generates HTML files which are pushed to another git repo for publishing by GitHub Pages.

Screenshot of Github Actions log showing failed git push

When two workflows try to push to a checked-out repo at the same time, one will fail because it is missing the last commit from the other.

This is just one example where concurrent workflows are problematic. Workflows which automate deployments have the same problem.

A number of 3rd party solutions exist, but these introduce additional waiting costs or other issues. For one of my projects, I host a lock-service, just to force concurrent workflows to exit quietly, and then auto-trigger re-runs.

Finally, on April 19, 2021,
this appeared in the GitHub Blog.

Screenshot of GitHub Blog from April 19, 2021 announcing the new concurrency key in GitHub Actions

In the case of using actions to generate a GitHub Pages website, the feature works exactly as required.

The first workflow will run to completion.
Subsequent concurrent workflows will either be delayed or cancelled.
In the end only the first and last of the overlapping workflows will be run.

And all you need is 2 lines of yaml. This is from the workflow which generates jldec.uk.

The group can be any string - workflows in the same group are effectively serialized.

Thank you GitHub!

debug

user: anonymous

{
  "path": "/blog/preventing-concurrent-github-actions",
  "attrs": {
    "alias": "/serializing-github-actions-with-workers-durable-objects",
    "title": "Preventing concurrent GitHub Actions",
    "splash": {
      "image": "/images/rose.jpg"
    },
    "date": "2021-06-13",
    "layout": "BlogPostLayout",
    "excerpt": "Learn how the new concurrency group feature for GitHub Actions prevents concurrent workflows."
  },
  "md": "# Preventing concurrent GitHub Actions\n\nWhat happens when you trigger a [GitHub Actions](github-actions-101) workflow which is already running? Workflows which depend on being run one-at-a-time might fail.\n\nI recently encountered this with a [workflow](https://github.com/jldec/cloudflare-pages-test/blob/main/.github/workflows/generate.yaml) for publishing a static website. This workflow generates HTML files which are pushed to another git repo for publishing by [GitHub Pages](https://pages.github.com/).\n\n![Screenshot of Github Actions log showing failed git push](/images/fail-generate.png)\n\nWhen two workflows try to push to a checked-out repo at the same time, one will fail because it is missing the last commit from the other.\n\n This is just one example where concurrent workflows are problematic. Workflows which [automate](https://github.community/t/serializing-workflow-runs-in-the-context-of-continuous-deployment/17559) [deployments](https://github.community/t/how-to-limit-concurrent-workflow-runs/16844) have the same [problem](https://github.community/t/serializing-queueing-deployment-workflows-aws-re-invent/17152).\n\nA number of 3rd party [solutions](https://github.com/softprops/turnstyle) exist, but these introduce additional waiting costs or other issues. For one of my projects, I host a lock-service, just to force concurrent workflows to exit quietly, and then auto-trigger re-runs.\n\n> Finally, on April 19, 2021,  \n[this](https://github.blog/changelog/2021-04-19-github-actions-limit-workflow-run-or-job-concurrency/) appeared in the GitHub Blog.\n\n![Screenshot of GitHub Blog from April 19, 2021 announcing the new concurrency key in GitHub Actions](/images/github-actions-concurrency-announcement.png)\n\nIn the case of using actions to generate a GitHub Pages website, [the feature works](https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#concurrency) exactly as required.\n\n- The first workflow will run to completion.\n- Subsequent concurrent workflows will either be delayed or cancelled.\n- In the end only the first and last of the overlapping workflows will be run.\n\nAnd all you need is [2 lines of yaml](https://github.com/jldec/cloudflare-pages-test/blob/main/.github/workflows/generate.yaml#L5-L6).\nThis is from the workflow which generates [jldec.uk](first-steps-using-cloudflare-pages).\n\n[![Screenshot of yaml for GitHub Action with concurrency group](/images/github-actions-concurrency-yaml.png)](https://github.com/jldec/cloudflare-pages-test/blob/main/.github/workflows/generate.yaml#L5-L6)\n\nThe `group` can be any string - workflows in the same group are effectively serialized.\n\n> Thank you GitHub!\n\n\n",
  "html": "<h1>Preventing concurrent GitHub Actions</h1>\n<p>What happens when you trigger a <a href=\"github-actions-101\">GitHub Actions</a> workflow which is already running? Workflows which depend on being run one-at-a-time might fail.</p>\n<p>I recently encountered this with a <a href=\"https://github.com/jldec/cloudflare-pages-test/blob/main/.github/workflows/generate.yaml\">workflow</a> for publishing a static website. This workflow generates HTML files which are pushed to another git repo for publishing by <a href=\"https://pages.github.com/\">GitHub Pages</a>.</p>\n<p><img src=\"/images/fail-generate.png\" alt=\"Screenshot of Github Actions log showing failed git push\"></p>\n<p>When two workflows try to push to a checked-out repo at the same time, one will fail because it is missing the last commit from the other.</p>\n<p>This is just one example where concurrent workflows are problematic. Workflows which <a href=\"https://github.community/t/serializing-workflow-runs-in-the-context-of-continuous-deployment/17559\">automate</a> <a href=\"https://github.community/t/how-to-limit-concurrent-workflow-runs/16844\">deployments</a> have the same <a href=\"https://github.community/t/serializing-queueing-deployment-workflows-aws-re-invent/17152\">problem</a>.</p>\n<p>A number of 3rd party <a href=\"https://github.com/softprops/turnstyle\">solutions</a> exist, but these introduce additional waiting costs or other issues. For one of my projects, I host a lock-service, just to force concurrent workflows to exit quietly, and then auto-trigger re-runs.</p>\n<blockquote>\n<p>Finally, on April 19, 2021,<br>\n<a href=\"https://github.blog/changelog/2021-04-19-github-actions-limit-workflow-run-or-job-concurrency/\">this</a> appeared in the GitHub Blog.</p>\n</blockquote>\n<p><img src=\"/images/github-actions-concurrency-announcement.png\" alt=\"Screenshot of GitHub Blog from April 19, 2021 announcing the new concurrency key in GitHub Actions\"></p>\n<p>In the case of using actions to generate a GitHub Pages website, <a href=\"https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#concurrency\">the feature works</a> exactly as required.</p>\n<ul>\n<li>The first workflow will run to completion.</li>\n<li>Subsequent concurrent workflows will either be delayed or cancelled.</li>\n<li>In the end only the first and last of the overlapping workflows will be run.</li>\n</ul>\n<p>And all you need is <a href=\"https://github.com/jldec/cloudflare-pages-test/blob/main/.github/workflows/generate.yaml#L5-L6\">2 lines of yaml</a>.\nThis is from the workflow which generates <a href=\"first-steps-using-cloudflare-pages\">jldec.uk</a>.</p>\n<p><a href=\"https://github.com/jldec/cloudflare-pages-test/blob/main/.github/workflows/generate.yaml#L5-L6\"><img src=\"/images/github-actions-concurrency-yaml.png\" alt=\"Screenshot of yaml for GitHub Action with concurrency group\"></a></p>\n<p>The <code>group</code> can be any string - workflows in the same group are effectively serialized.</p>\n<blockquote>\n<p>Thank you GitHub!</p>\n</blockquote>\n"
}