til: cloudrun_using-build-args-with-cloud-run.md
This data as json
| path | topic | title | url | body | html | shot | created | created_utc | updated | updated_utc | shot_hash | slug |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| cloudrun_using-build-args-with-cloud-run.md | cloudrun | Using build-arg variables with Cloud Run deployments | https://github.com/simonw/til/blob/main/cloudrun/using-build-args-with-cloud-run.md | For [datasette/issues/1522](https://github.com/simonw/datasette/issues/1522) I wanted to use a Docker build argument in a `Dockerfile` that would then be deployed to Cloud Run. I needed this to be able to control the version of Datasette that was deployed. Here's my simplified `Dockerfile`: ```dockerfile FROM python:3-alpine ARG DATASETTE_REF # Copy to environment variable for use in CMD later ENV VERSION_NOTE=$DATASETTE_REF RUN pip install https://github.com/simonw/datasette/archive/${DATASETTE_REF}.zip # Need to use "shell form" here to get variable substition: CMD datasette -h 0.0.0.0 -p 8080 --version-note $VERSION_NOTE ``` I can build this on my laptop like so: docker build -t datasette-build-arg-demo . \ --build-arg DATASETTE_REF=c617e1769ea27e045b0f2907ef49a9a1244e577d Then run it like this: docker run -p 5000:8080 --rm datasette-build-arg-demo And visit `http://localhost:5000/-/versions` to see the version number to confirm it worked. I wanted to deploy this to Cloud Run, using [this recipe](https://til.simonwillison.net/cloudrun/ship-dockerfile-to-cloud-run). Unfortunately, the `gcloud builds submit` command doesn't have a mechanism for specifying `--build-arg`. Instead, you need to use a YAML file and pass it with the `gcloud builds submit --config cloudbuild.yml` option. The YAML should look like this: ```yaml steps: - name: 'gcr.io/cloud-builders/docker' args: ['build', '-t', 'gc.io/MY-PROJECT/MY-NAME', '.', '--build-arg', 'DATASETTE_REF=c617e1769ea27e045b0f2907ef49a9a1244e577d'] - name: 'gcr.io/cloud-builders/docker' args: ['push', $IMAGE] ``` Since I want to dynamically populate my YAML file, I ended up using the following pattern in a `./deploy.sh` script: ```bash #!/bin/bash # https://til.simonwillison.net/cloudrun/using-build-args-with-cloud-run if [[ -z "$DATASETTE_REF" ]]; then echo "Must provide DATASETTE_REF environment variable" 1>&2 exit 1 fi NAME="datasette-apache-proxy-demo" PROJECT=$(gcloud config get-value project) IMAGE="gcr.io/$PROJECT/$NAME" # Need YAML so we can set --build-arg echo " steps: - name: 'gcr.io/cloud-builders/docker' args: ['build', '-t', '$IMAGE', '.', '--build-arg', 'DATASETTE_REF=$DATASETTE_REF'] - name: 'gcr.io/cloud-builders/docker' args: ['push', '$IMAGE'] " > /tmp/cloudbuild.yml gcloud builds submit --config /tmp/cloudbuild.yml rm /tmp/cloudbuild.yml gcloud run deploy $NAME \ --allow-unauthenticated \ --platform=managed \ --image $IMAGE \ --port 80 ``` | <p>For <a href="https://github.com/simonw/datasette/issues/1522">datasette/issues/1522</a> I wanted to use a Docker build argument in a <code>Dockerfile</code> that would then be deployed to Cloud Run.</p> <p>I needed this to be able to control the version of Datasette that was deployed. Here's my simplified <code>Dockerfile</code>:</p> <div class="highlight highlight-source-dockerfile"><pre><span class="pl-k">FROM</span> python:3-alpine <span class="pl-k">ARG</span> DATASETTE_REF <span class="pl-c"><span class="pl-c">#</span> Copy to environment variable for use in CMD later</span> <span class="pl-k">ENV</span> VERSION_NOTE=$DATASETTE_REF <span class="pl-k">RUN</span> pip install https://github.com/simonw/datasette/archive/${DATASETTE_REF}.zip <span class="pl-c"><span class="pl-c">#</span> Need to use "shell form" here to get variable substition:</span> <span class="pl-k">CMD</span> datasette -h 0.0.0.0 -p 8080 --version-note $VERSION_NOTE</pre></div> <p>I can build this on my laptop like so:</p> <pre><code>docker build -t datasette-build-arg-demo . \ --build-arg DATASETTE_REF=c617e1769ea27e045b0f2907ef49a9a1244e577d </code></pre> <p>Then run it like this:</p> <pre><code>docker run -p 5000:8080 --rm datasette-build-arg-demo </code></pre> <p>And visit <code>http://localhost:5000/-/versions</code> to see the version number to confirm it worked.</p> <p>I wanted to deploy this to Cloud Run, using <a href="https://til.simonwillison.net/cloudrun/ship-dockerfile-to-cloud-run" rel="nofollow">this recipe</a>.</p> <p>Unfortunately, the <code>gcloud builds submit</code> command doesn't have a mechanism for specifying <code>--build-arg</code>.</p> <p>Instead, you need to use a YAML file and pass it with the <code>gcloud builds submit --config cloudbuild.yml</code> option. The YAML should look like this:</p> <div class="highlight highlight-source-yaml"><pre><span class="pl-ent">steps</span>: - <span class="pl-ent">name</span>: <span class="pl-s"><span class="pl-pds">'</span>gcr.io/cloud-builders/docker<span class="pl-pds">'</span></span> <span class="pl-ent">args</span>: <span class="pl-s">['build', '-t', 'gc.io/MY-PROJECT/MY-NAME', '.', '--build-arg', 'DATASETTE_REF=c617e1769ea27e045b0f2907ef49a9a1244e577d']</span> - <span class="pl-ent">name</span>: <span class="pl-s"><span class="pl-pds">'</span>gcr.io/cloud-builders/docker<span class="pl-pds">'</span></span> <span class="pl-ent">args</span>: <span class="pl-s">['push', $IMAGE]</span></pre></div> <p>Since I want to dynamically populate my YAML file, I ended up using the following pattern in a <code>./deploy.sh</code> script:</p> <div class="highlight highlight-source-shell"><pre><span class="pl-c"><span class="pl-c">#!</span>/bin/bash</span> <span class="pl-c"><span class="pl-c">#</span> https://til.simonwillison.net/cloudrun/using-build-args-with-cloud-run</span> <span class="pl-k">if</span> [[ <span class="pl-k">-z</span> <span class="pl-s"><span class="pl-pds">"</span><span class="pl-smi">$DATASETTE_REF</span><span class="pl-pds">"</span></span> ]]<span class="pl-k">;</span> <span class="pl-k">then</span> <span class="pl-c1">echo</span> <span class="pl-s"><span class="pl-pds">"</span>Must provide DATASETTE_REF environment variable<span class="pl-pds">"</span></span> <span class="pl-k">1>&2</span> <span class="pl-c1">exit</span> 1 <span class="pl-k">fi</span> NAME=<span class="pl-s"><span class="pl-pds">"</span>datasette-apache-proxy-demo<span class="pl-pds">"</span></span> PROJECT=<span class="pl-s"><span class="pl-pds">$(</span>gcloud config get-value project<span class="pl-pds">)</span></span> IMAGE=<span class="pl-s"><span class="pl-pds">"</span>gcr.io/<span class="pl-smi">$PROJECT</span>/<span class="pl-smi">$NAME</span><span class="pl-pds">"</span></span> <span class="pl-c"><span class="pl-c">#</span> Need YAML so we can set --build-arg</span> <span class="pl-c1">echo</span> <span class="pl-s"><span class="pl-pds">"</span></span> <span class="pl-s">steps:</span> <span class="pl-s">- name: 'gcr.io/cloud-builders/docker'</span> <span class="pl-s"> args: ['build', '-t', '<span class="pl-smi">$IMAGE</span>', '.', '--build-arg', 'DATASETTE_REF=<span class="pl-smi">$DATASETTE_REF</span>']</span> <span class="pl-s">- name: 'gcr.io/cloud-builders/docker'</span> <span class="pl-s"> args: ['push', '<span class="pl-smi">$IMAGE</span>']</span> <span class="pl-s"><span class="pl-pds">"</span></span> <span class="pl-k">></span> /tmp/cloudbuild.yml gcloud builds submit --config /tmp/cloudbuild.yml rm /tmp/cloudbuild.yml gcloud run deploy <span class="pl-smi">$NAME</span> \ --allow-unauthenticated \ --platform=managed \ --image <span class="pl-smi">$IMAGE</span> \ --port 80</pre></div> | <Binary: 61,078 bytes> | 2021-11-19T16:24:56-08:00 | 2021-11-20T00:24:56+00:00 | 2021-11-19T16:32:57-08:00 | 2021-11-20T00:32:57+00:00 | 328fd8809d3af0f5cdde5c937c7085bd | using-build-args-with-cloud-run |