til: bash_loop-over-csv.md
This data as json
| path | topic | title | url | body | html | shot | created | created_utc | updated | updated_utc | shot_hash | slug |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| bash_loop-over-csv.md | bash | Looping over comma-separated values in Bash | https://github.com/simonw/til/blob/main/bash/loop-over-csv.md | Given a file (or a process) that produces comma separated values, here's how to split those into separate variables and use them in a bash script. The trick is to set the Bash `IFS` to a delimiter, then use `my_array=($my_string)` to split on that delimiter. Create a text file called `data.txt` containing this: ``` first,1 second,2 ``` You can create that by doing: ```bash echo 'first,1 second,2' > /tmp/data.txt ``` To loop over that file and print each line: ```bash for line in $(cat /tmp/data.txt); do echo $line done ``` To split each line into two separate variables in the loop, do this: ```bash for line in $(cat /tmp/data.txt); do IFS=$','; split=($line); unset IFS; # $split is now a bash array echo "Column 1: ${split[0]}" echo "Column 2: ${split[1]}" done ``` Outputs: ``` Column 1: first Column 2: 1 Column 1: second Column 2: 2 ``` Here's a script I wrote using this technique for the TIL [Use labels on Cloud Run services for a billing breakdown](https://til.simonwillison.net/til/til/cloudrun_use-labels-for-billing-breakdown.md): ```bash #!/bin/bash for line in $( gcloud run services list --platform=managed \ --format="csv(SERVICE,REGION)" \ --filter "NOT metadata.labels.service:*" \ | tail -n +2) do IFS=$','; service_and_region=($line); unset IFS; service=${service_and_region[0]} region=${service_and_region[1]} echo "service: $service region: $region" gcloud run services update $service \ --region=$region --platform=managed \ --update-labels service=$service echo done ``` | <p>Given a file (or a process) that produces comma separated values, here's how to split those into separate variables and use them in a bash script.</p> <p>The trick is to set the Bash <code>IFS</code> to a delimiter, then use <code>my_array=($my_string)</code> to split on that delimiter.</p> <p>Create a text file called <code>data.txt</code> containing this:</p> <pre><code>first,1 second,2 </code></pre> <p>You can create that by doing:</p> <div class="highlight highlight-source-shell"><pre><span class="pl-c1">echo</span> <span class="pl-s"><span class="pl-pds">'</span>first,1</span> <span class="pl-s">second,2<span class="pl-pds">'</span></span> <span class="pl-k">></span> /tmp/data.txt</pre></div> <p>To loop over that file and print each line:</p> <div class="highlight highlight-source-shell"><pre><span class="pl-k">for</span> <span class="pl-smi">line</span> <span class="pl-k">in</span> <span class="pl-s"><span class="pl-pds">$(</span>cat /tmp/data.txt<span class="pl-pds">)</span></span><span class="pl-k">;</span> <span class="pl-k">do</span> <span class="pl-c1">echo</span> <span class="pl-smi">$line</span> <span class="pl-k">done</span></pre></div> <p>To split each line into two separate variables in the loop, do this:</p> <div class="highlight highlight-source-shell"><pre><span class="pl-k">for</span> <span class="pl-smi">line</span> <span class="pl-k">in</span> <span class="pl-s"><span class="pl-pds">$(</span>cat /tmp/data.txt<span class="pl-pds">)</span></span><span class="pl-k">;</span> <span class="pl-k">do</span> IFS=<span class="pl-s"><span class="pl-pds">$'</span>,<span class="pl-pds">'</span></span><span class="pl-k">;</span> split=(<span class="pl-smi">$line</span>)<span class="pl-k">;</span> <span class="pl-c1">unset</span> IFS<span class="pl-k">;</span> <span class="pl-c"><span class="pl-c">#</span> $split is now a bash array</span> <span class="pl-c1">echo</span> <span class="pl-s"><span class="pl-pds">"</span>Column 1: <span class="pl-smi">${split[0]}</span><span class="pl-pds">"</span></span> <span class="pl-c1">echo</span> <span class="pl-s"><span class="pl-pds">"</span>Column 2: <span class="pl-smi">${split[1]}</span><span class="pl-pds">"</span></span> <span class="pl-k">done</span></pre></div> <p>Outputs:</p> <pre><code>Column 1: first Column 2: 1 Column 1: second Column 2: 2 </code></pre> <p>Here's a script I wrote using this technique for the TIL <a href="https://til.simonwillison.net/til/til/cloudrun_use-labels-for-billing-breakdown.md" rel="nofollow">Use labels on Cloud Run services for a billing breakdown</a>:</p> <div class="highlight highlight-source-shell"><pre><span class="pl-c"><span class="pl-c">#!</span>/bin/bash</span> <span class="pl-k">for</span> <span class="pl-smi">line</span> <span class="pl-k">in</span> <span class="pl-s"><span class="pl-pds">$(</span></span> <span class="pl-s"> gcloud run services list --platform=managed \</span> <span class="pl-s"> --format=<span class="pl-s"><span class="pl-pds">"</span>csv(SERVICE,REGION)<span class="pl-pds">"</span></span> \</span> <span class="pl-s"> --filter <span class="pl-s"><span class="pl-pds">"</span>NOT metadata.labels.service:*<span class="pl-pds">"</span></span> \</span> <span class="pl-s"> <span class="pl-k">|</span> tail -n +2<span class="pl-pds">)</span></span> <span class="pl-k">do</span> IFS=<span class="pl-s"><span class="pl-pds">$'</span>,<span class="pl-pds">'</span></span><span class="pl-k">;</span> service_and_region=(<span class="pl-smi">$line</span>)<span class="pl-k">;</span> <span class="pl-c1">unset</span> IFS<span class="pl-k">;</span> service=<span class="pl-smi">${service_and_region[0]}</span> region=<span class="pl-smi">${service_and_region[1]}</span> <span class="pl-c1">echo</span> <span class="pl-s"><span class="pl-pds">"</span>service: <span class="pl-smi">$service</span> region: <span class="pl-smi">$region</span><span class="pl-pds">"</span></span> gcloud run services update <span class="pl-smi">$service</span> \ --region=<span class="pl-smi">$region</span> --platform=managed \ --update-labels service=<span class="pl-smi">$service</span> <span class="pl-c1">echo</span> <span class="pl-k">done</span></pre></div> | <Binary: 51,126 bytes> | 2020-09-01T18:48:28-07:00 | 2020-09-02T01:48:28+00:00 | 2020-09-01T18:48:28-07:00 | 2020-09-02T01:48:28+00:00 | d06963c31326ae773a8e7face614668c | loop-over-csv |