til: github-actions_conditionally-run-a-second-job.md
This data as json
| path | topic | title | url | body | html | shot | created | created_utc | updated | updated_utc | shot_hash | slug |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| github-actions_conditionally-run-a-second-job.md | github-actions | Conditionally running a second job in a GitHub Actions workflow | https://github.com/simonw/til/blob/main/github-actions/conditionally-run-a-second-job.md | My [simonwillisonblog-backup workflow](https://github.com/simonw/simonwillisonblog-backup/blob/main/.github/workflows/backup.yml) periodically creates a JSON backup of my blog's PostgreSQL database, using [db-to-sqlite](https://datasette.io/tools/db-to-sqlite) and [sqlite-diffable](https://datasette.io/tools/sqlite-diffable). It then commits any changes back to the repo using this pattern: ```yaml - name: Commit any changes run: |- git config user.name "Automated" git config user.email "actions@users.noreply.github.com" git add simonwillisonblog timestamp=$(date -u) git commit -m "Latest data: ${timestamp}" || exit 0 git push ``` I decided to upgrade it to also build and deploy a SQLite database of the content to [datasette.simonwillison.net](https://datasette.simonwillison.net/) - but only if a change had been detected. I figured out the following pattern for doing that. First, I added a line to the above block that set a `change_detected` [output variable](https://docs.github.com/en/actions/using-jobs/defining-outputs-for-jobs) for that step if it made it past the `|| exit 0`. I also added an `id` to the step so I could reference it later on: ```yaml - name: Commit any changes id: commit_and_push run: |- git config user.name "Automated" git config user.email "actions@users.noreply.github.com" git add simonwillisonblog timestamp=$(date -u) git commit -m "Latest data: ${timestamp}" || exit 0 git push echo "::set-output name=change_detected::1" ``` This next piece took me a while to figure out: I also had to declare that output variable at the top of the initial job, copying the result of the named step: ```yaml jobs: backup: runs-on: ubuntu-latest outputs: change_detected: ${{ steps.commit_and_push.outputs.change_detected }} ``` Without this, the output is not visible to the second job. My second job started like this: ```yaml build_and_deploy: runs-on: ubuntu-latest needs: backup if: ${{ inputs.force_deploy || needs.backup.outputs.change_detected }} steps: ``` This second job is called `build_and_deploy` and specify that it `needs: backup` - so it should run directly after that backup job completes. That new job has an `if:` expression which looks at `needs.backup.outputs.change_detected` to read the variable that was set by my `echo "::set-output` line. I'm also checking `inputs.force_deploy` here. That's a separate mechanism, which allows me to trigger the workflow with manually and specify that a deploy should happen even if no changes were detected - useful for when I alter the code that configures the deployed Datasette instance. The `force_deploy` variable comes from this section at the start of the YAML: ```yaml on: push: branches: - main workflow_dispatch: inputs: force_deploy: description: 'Deploy even if no changes detected' required: false type: boolean ``` This configuration adds the following UI which I can use to manually trigger the workflow:  | <p>My <a href="https://github.com/simonw/simonwillisonblog-backup/blob/main/.github/workflows/backup.yml">simonwillisonblog-backup workflow</a> periodically creates a JSON backup of my blog's PostgreSQL database, using <a href="https://datasette.io/tools/db-to-sqlite" rel="nofollow">db-to-sqlite</a> and <a href="https://datasette.io/tools/sqlite-diffable" rel="nofollow">sqlite-diffable</a>. It then commits any changes back to the repo using this pattern:</p> <div class="highlight highlight-source-yaml"><pre> - <span class="pl-ent">name</span>: <span class="pl-s">Commit any changes</span> <span class="pl-ent">run</span>: <span class="pl-s">|-</span> <span class="pl-s"> git config user.name "Automated"</span> <span class="pl-s"> git config user.email "actions@users.noreply.github.com"</span> <span class="pl-s"> git add simonwillisonblog</span> <span class="pl-s"> timestamp=$(date -u)</span> <span class="pl-s"> git commit -m "Latest data: ${timestamp}" || exit 0</span> <span class="pl-s"> git push</span></pre></div> <p>I decided to upgrade it to also build and deploy a SQLite database of the content to <a href="https://datasette.simonwillison.net/" rel="nofollow">datasette.simonwillison.net</a> - but only if a change had been detected.</p> <p>I figured out the following pattern for doing that.</p> <p>First, I added a line to the above block that set a <code>change_detected</code> <a href="https://docs.github.com/en/actions/using-jobs/defining-outputs-for-jobs">output variable</a> for that step if it made it past the <code>|| exit 0</code>. I also added an <code>id</code> to the step so I could reference it later on:</p> <div class="highlight highlight-source-yaml"><pre> - <span class="pl-ent">name</span>: <span class="pl-s">Commit any changes</span> <span class="pl-ent">id</span>: <span class="pl-s">commit_and_push</span> <span class="pl-ent">run</span>: <span class="pl-s">|-</span> <span class="pl-s"> git config user.name "Automated"</span> <span class="pl-s"> git config user.email "actions@users.noreply.github.com"</span> <span class="pl-s"> git add simonwillisonblog</span> <span class="pl-s"> timestamp=$(date -u)</span> <span class="pl-s"> git commit -m "Latest data: ${timestamp}" || exit 0</span> <span class="pl-s"> git push</span> <span class="pl-s"> echo "::set-output name=change_detected::1"</span></pre></div> <p>This next piece took me a while to figure out: I also had to declare that output variable at the top of the initial job, copying the result of the named step:</p> <div class="highlight highlight-source-yaml"><pre><span class="pl-ent">jobs</span>: <span class="pl-ent">backup</span>: <span class="pl-ent">runs-on</span>: <span class="pl-s">ubuntu-latest</span> <span class="pl-ent">outputs</span>: <span class="pl-ent">change_detected</span>: <span class="pl-s">${{ steps.commit_and_push.outputs.change_detected }}</span></pre></div> <p>Without this, the output is not visible to the second job.</p> <p>My second job started like this:</p> <div class="highlight highlight-source-yaml"><pre> <span class="pl-ent">build_and_deploy</span>: <span class="pl-ent">runs-on</span>: <span class="pl-s">ubuntu-latest</span> <span class="pl-ent">needs</span>: <span class="pl-s">backup</span> <span class="pl-ent">if</span>: <span class="pl-s">${{ inputs.force_deploy || needs.backup.outputs.change_detected }}</span> <span class="pl-ent">steps</span>:</pre></div> <p>This second job is called <code>build_and_deploy</code> and specify that it <code>needs: backup</code> - so it should run directly after that backup job completes.</p> <p>That new job has an <code>if:</code> expression which looks at <code>needs.backup.outputs.change_detected</code> to read the variable that was set by my <code>echo "::set-output</code> line.</p> <p>I'm also checking <code>inputs.force_deploy</code> here. That's a separate mechanism, which allows me to trigger the workflow with manually and specify that a deploy should happen even if no changes were detected - useful for when I alter the code that configures the deployed Datasette instance.</p> <p>The <code>force_deploy</code> variable comes from this section at the start of the YAML:</p> <div class="highlight highlight-source-yaml"><pre><span class="pl-ent">on</span>: <span class="pl-ent">push</span>: <span class="pl-ent">branches</span>: - <span class="pl-s">main</span> <span class="pl-ent">workflow_dispatch</span>: <span class="pl-ent">inputs</span>: <span class="pl-ent">force_deploy</span>: <span class="pl-ent">description</span>: <span class="pl-s"><span class="pl-pds">'</span>Deploy even if no changes detected<span class="pl-pds">'</span></span> <span class="pl-ent">required</span>: <span class="pl-c1">false</span> <span class="pl-ent">type</span>: <span class="pl-s">boolean</span></pre></div> <p>This configuration adds the following UI which I can use to manually trigger the workflow:</p> <p><a href="https://user-images.githubusercontent.com/9599/178353732-77e58ddd-c78c-4032-aeea-cb388bac8ec6.jpg" target="_blank" rel="nofollow"><img src="https://user-images.githubusercontent.com/9599/178353732-77e58ddd-c78c-4032-aeea-cb388bac8ec6.jpg" alt="Screenshot of the UI showing the force_deploy checkbox" style="max-width:100%;"></a></p> | <Binary: 70,861 bytes> | 2022-07-11T13:39:01-07:00 | 2022-07-11T20:39:01+00:00 | 2022-07-11T17:05:01-07:00 | 2022-07-12T00:05:01+00:00 | 2c87c48078adb1b230e8e2e14af183e9 | conditionally-run-a-second-job |