til: electron_testing-electron-playwright.md
This data as json
| path | topic | title | url | body | html | shot | created | created_utc | updated | updated_utc | shot_hash | slug |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| electron_testing-electron-playwright.md | electron | Testing Electron apps with Playwright and GitHub Actions | https://github.com/simonw/til/blob/main/electron/testing-electron-playwright.md | Yesterday [I figured out (issue 133)](https://github.com/simonw/datasette-app/issues/133) how to use Playwright to run tests against my Electron app, and then execute those tests in CI using GitHub Actions, for my [datasett-app](https://github.com/simonw/datasette-app) repo for my [Datasette Desktop](https://datasette.io/desktop) macOS application. ## Installing @playwright/test You need to install the `@playwright/test` package. You can do that like so: npm i -D @playwright/test This adds it to `devDependencies` in your `package.json`, something like this: ``` "devDependencies": { "@playwright/test": "^1.23.2", ``` ## Writing a test I dropped the following into a `test/spec.mjs` file: ```javascript import { test, expect } from '@playwright/test'; import { _electron } from 'playwright'; test('App launches and quits', async () => { const app = await _electron.launch({args: ['main.js']); const window = await app.firstWindow(); await expect(await window.title()).toContain('Loading'); await app.close(); }); ``` The `.mjs` extension is necessary in order to use `import`, since it lets Node.js know that this file is a JavaScript module. The test can be run using `playwright test`. I later added it to my `package.json` section like this: ```json "scripts": { "test": "playwright test" } ``` Now I can run the Playwright tests using `npm test`. ## Recording video of the tests Recording videos of the test runs turns out to be easy: change the `_electron.launch()` line to look like this: ```javascript const app = await _electron.launch({ args: ['main.js'], recordVideo: {dir: 'test-videos'} }); ``` This creates the videos as `.webm` files in the `test-videos` directory. These videos can be opened in Chrome, or can be converted to `mp4` using `ffmpeg` (available on macOS via `brew install ffmpeg`): ffmpeg -i bc74c2a51bd91fe6f6cb815e6b99b6c7.webm bc74c2a51bd91fe6f6cb815e6b99b6c7.mp4 Converting to `.mp4` means you can drag and drop them onto a GitHub Issues thread and get an embedded video player. [Here's an example](https://github.com/simonw/datasette-app/issues/133#issuecomment-1182530789) I recorded. ## Custom timeouts Playwright has a default 30s timeout on every action it takes. This turned out to be a bit too short for one of my tests, which installs a Python interpreter and a bunch of Python packages and can take 57s. Here's how I fixed that so the test could pass: ```javascript test('App launches and quits', async () => { // This disables the global 30s timeout test.setTimeout(0); const app = await _electron.launch({ args: ['main.js'], recordVideo: {dir: 'test-videos'} }); const window = await app.firstWindow(); // This sets a timeout of 90s for the page to load and the // element with id="run-sql-link" to appear in the DOM: await window.waitForSelector('#run-sql-link', { timeout: 90000 }); await app.close(); }); ``` ## Running it in GitHub Actions I'm using the `macos-latest` image in my GitHub Actions workflow. The relevant configuration in my `.github/workflows/test.yml` file looks like this: ```yaml name: Test on: push jobs: test: runs-on: macos-latest steps: - uses: actions/checkout@v3 - name: Configure Node caching uses: actions/cache@v3 env: cache-name: cache-node-modules with: path: ~/.npm key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/package-lock.json') }} restore-keys: | ${{ runner.os }}-build-${{ env.cache-name }}- ${{ runner.os }}-build- ${{ runner.os }}- - name: Install Node dependencies run: npm install - name: Run tests run: npm test timeout-minutes: 5 - name: Upload test videos uses: actions/upload-artifact@v3 with: name: test-videos path: test-videos/ ``` This workflow configures NPM caching to avoid downloading everything every time, installs the dependencies, runs the tests, and then uploads the videos at the end. Those videos end up attached to the workflow run as an artifact that can be downloaded and viewed locally. | <p>Yesterday <a href="https://github.com/simonw/datasette-app/issues/133">I figured out (issue 133)</a> how to use Playwright to run tests against my Electron app, and then execute those tests in CI using GitHub Actions, for my <a href="https://github.com/simonw/datasette-app">datasett-app</a> repo for my <a href="https://datasette.io/desktop" rel="nofollow">Datasette Desktop</a> macOS application.</p> <h2> <a id="user-content-installing-playwrighttest" class="anchor" href="#installing-playwrighttest" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Installing @playwright/test</h2> <p>You need to install the <code>@playwright/test</code> package. You can do that like so:</p> <pre><code>npm i -D @playwright/test </code></pre> <p>This adds it to <code>devDependencies</code> in your <code>package.json</code>, something like this:</p> <pre><code> "devDependencies": { "@playwright/test": "^1.23.2", </code></pre> <h2> <a id="user-content-writing-a-test" class="anchor" href="#writing-a-test" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Writing a test</h2> <p>I dropped the following into a <code>test/spec.mjs</code> file:</p> <div class="highlight highlight-source-js"><pre><span class="pl-k">import</span> <span class="pl-kos">{</span> <span class="pl-s1">test</span><span class="pl-kos">,</span> <span class="pl-s1">expect</span> <span class="pl-kos">}</span> <span class="pl-k">from</span> <span class="pl-s">'@playwright/test'</span><span class="pl-kos">;</span> <span class="pl-k">import</span> <span class="pl-kos">{</span> <span class="pl-s1">_electron</span> <span class="pl-kos">}</span> <span class="pl-k">from</span> <span class="pl-s">'playwright'</span><span class="pl-kos">;</span> <span class="pl-en">test</span><span class="pl-kos">(</span><span class="pl-s">'App launches and quits'</span><span class="pl-kos">,</span> <span class="pl-k">async</span> <span class="pl-kos">(</span><span class="pl-kos">)</span> <span class="pl-c1">=></span> <span class="pl-kos">{</span> <span class="pl-k">const</span> <span class="pl-s1">app</span> <span class="pl-c1">=</span> <span class="pl-k">await</span> <span class="pl-s1">_electron</span><span class="pl-kos">.</span><span class="pl-en">launch</span><span class="pl-kos">(</span><span class="pl-kos">{</span><span class="pl-c1">args</span>: <span class="pl-kos">[</span><span class="pl-s">'main.js'</span><span class="pl-kos">]</span><span class="pl-kos"></span><span class="pl-kos">)</span><span class="pl-kos">;</span> <span class="pl-k">const</span> <span class="pl-s1">window</span> <span class="pl-c1">=</span> <span class="pl-k">await</span> <span class="pl-s1">app</span><span class="pl-kos">.</span><span class="pl-en">firstWindow</span><span class="pl-kos">(</span><span class="pl-kos">)</span><span class="pl-kos">;</span> <span class="pl-k">await</span> <span class="pl-en">expect</span><span class="pl-kos">(</span><span class="pl-k">await</span> <span class="pl-s1">window</span><span class="pl-kos">.</span><span class="pl-en">title</span><span class="pl-kos">(</span><span class="pl-kos">)</span><span class="pl-kos">)</span><span class="pl-kos">.</span><span class="pl-en">toContain</span><span class="pl-kos">(</span><span class="pl-s">'Loading'</span><span class="pl-kos">)</span><span class="pl-kos">;</span> <span class="pl-k">await</span> <span class="pl-s1">app</span><span class="pl-kos">.</span><span class="pl-en">close</span><span class="pl-kos">(</span><span class="pl-kos">)</span><span class="pl-kos">;</span> <span class="pl-kos">}</span><span class="pl-kos">)</span><span class="pl-kos">;</span></pre></div> <p>The <code>.mjs</code> extension is necessary in order to use <code>import</code>, since it lets Node.js know that this file is a JavaScript module.</p> <p>The test can be run using <code>playwright test</code>.</p> <p>I later added it to my <code>package.json</code> section like this:</p> <div class="highlight highlight-source-json"><pre> <span class="pl-ent">"scripts"</span>: { <span class="pl-ent">"test"</span>: <span class="pl-s"><span class="pl-pds">"</span>playwright test<span class="pl-pds">"</span></span> }</pre></div> <p>Now I can run the Playwright tests using <code>npm test</code>.</p> <h2> <a id="user-content-recording-video-of-the-tests" class="anchor" href="#recording-video-of-the-tests" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Recording video of the tests</h2> <p>Recording videos of the test runs turns out to be easy: change the <code>_electron.launch()</code> line to look like this:</p> <div class="highlight highlight-source-js"><pre> <span class="pl-k">const</span> <span class="pl-s1">app</span> <span class="pl-c1">=</span> <span class="pl-k">await</span> <span class="pl-s1">_electron</span><span class="pl-kos">.</span><span class="pl-en">launch</span><span class="pl-kos">(</span><span class="pl-kos">{</span> <span class="pl-c1">args</span>: <span class="pl-kos">[</span><span class="pl-s">'main.js'</span><span class="pl-kos">]</span><span class="pl-kos">,</span> <span class="pl-c1">recordVideo</span>: <span class="pl-kos">{</span><span class="pl-c1">dir</span>: <span class="pl-s">'test-videos'</span><span class="pl-kos">}</span> <span class="pl-kos">}</span><span class="pl-kos">)</span><span class="pl-kos">;</span></pre></div> <p>This creates the videos as <code>.webm</code> files in the <code>test-videos</code> directory.</p> <p>These videos can be opened in Chrome, or can be converted to <code>mp4</code> using <code>ffmpeg</code> (available on macOS via <code>brew install ffmpeg</code>):</p> <pre><code>ffmpeg -i bc74c2a51bd91fe6f6cb815e6b99b6c7.webm bc74c2a51bd91fe6f6cb815e6b99b6c7.mp4 </code></pre> <p>Converting to <code>.mp4</code> means you can drag and drop them onto a GitHub Issues thread and get an embedded video player. <a href="https://github.com/simonw/datasette-app/issues/133#issuecomment-1182530789">Here's an example</a> I recorded.</p> <h2> <a id="user-content-custom-timeouts" class="anchor" href="#custom-timeouts" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Custom timeouts</h2> <p>Playwright has a default 30s timeout on every action it takes. This turned out to be a bit too short for one of my tests, which installs a Python interpreter and a bunch of Python packages and can take 57s. Here's how I fixed that so the test could pass:</p> <div class="highlight highlight-source-js"><pre><span class="pl-en">test</span><span class="pl-kos">(</span><span class="pl-s">'App launches and quits'</span><span class="pl-kos">,</span> <span class="pl-k">async</span> <span class="pl-kos">(</span><span class="pl-kos">)</span> <span class="pl-c1">=></span> <span class="pl-kos">{</span> <span class="pl-c">// This disables the global 30s timeout</span> <span class="pl-s1">test</span><span class="pl-kos">.</span><span class="pl-en">setTimeout</span><span class="pl-kos">(</span><span class="pl-c1">0</span><span class="pl-kos">)</span><span class="pl-kos">;</span> <span class="pl-k">const</span> <span class="pl-s1">app</span> <span class="pl-c1">=</span> <span class="pl-k">await</span> <span class="pl-s1">_electron</span><span class="pl-kos">.</span><span class="pl-en">launch</span><span class="pl-kos">(</span><span class="pl-kos">{</span> <span class="pl-c1">args</span>: <span class="pl-kos">[</span><span class="pl-s">'main.js'</span><span class="pl-kos">]</span><span class="pl-kos">,</span> <span class="pl-c1">recordVideo</span>: <span class="pl-kos">{</span><span class="pl-c1">dir</span>: <span class="pl-s">'test-videos'</span><span class="pl-kos">}</span> <span class="pl-kos">}</span><span class="pl-kos">)</span><span class="pl-kos">;</span> <span class="pl-k">const</span> <span class="pl-s1">window</span> <span class="pl-c1">=</span> <span class="pl-k">await</span> <span class="pl-s1">app</span><span class="pl-kos">.</span><span class="pl-en">firstWindow</span><span class="pl-kos">(</span><span class="pl-kos">)</span><span class="pl-kos">;</span> <span class="pl-c">// This sets a timeout of 90s for the page to load and the</span> <span class="pl-c">// element with id="run-sql-link" to appear in the DOM:</span> <span class="pl-k">await</span> <span class="pl-s1">window</span><span class="pl-kos">.</span><span class="pl-en">waitForSelector</span><span class="pl-kos">(</span><span class="pl-s">'#run-sql-link'</span><span class="pl-kos">,</span> <span class="pl-kos">{</span> <span class="pl-c1">timeout</span>: <span class="pl-c1">90000</span> <span class="pl-kos">}</span><span class="pl-kos">)</span><span class="pl-kos">;</span> <span class="pl-k">await</span> <span class="pl-s1">app</span><span class="pl-kos">.</span><span class="pl-en">close</span><span class="pl-kos">(</span><span class="pl-kos">)</span><span class="pl-kos">;</span> <span class="pl-kos">}</span><span class="pl-kos">)</span><span class="pl-kos">;</span></pre></div> <h2> <a id="user-content-running-it-in-github-actions" class="anchor" href="#running-it-in-github-actions" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Running it in GitHub Actions</h2> <p>I'm using the <code>macos-latest</code> image in my GitHub Actions workflow. The relevant configuration in my <code>.github/workflows/test.yml</code> file looks like this:</p> <div class="highlight highlight-source-yaml"><pre><span class="pl-ent">name</span>: <span class="pl-s">Test</span> <span class="pl-ent">on</span>: <span class="pl-s">push</span> <span class="pl-ent">jobs</span>: <span class="pl-ent">test</span>: <span class="pl-ent">runs-on</span>: <span class="pl-s">macos-latest</span> <span class="pl-ent">steps</span>: - <span class="pl-ent">uses</span>: <span class="pl-s">actions/checkout@v3</span> - <span class="pl-ent">name</span>: <span class="pl-s">Configure Node caching</span> <span class="pl-ent">uses</span>: <span class="pl-s">actions/cache@v3</span> <span class="pl-ent">env</span>: <span class="pl-ent">cache-name</span>: <span class="pl-s">cache-node-modules</span> <span class="pl-ent">with</span>: <span class="pl-ent">path</span>: <span class="pl-s">~/.npm</span> <span class="pl-ent">key</span>: <span class="pl-s">${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/package-lock.json') }}</span> <span class="pl-ent">restore-keys</span>: <span class="pl-s">|</span> <span class="pl-s"> ${{ runner.os }}-build-${{ env.cache-name }}-</span> <span class="pl-s"> ${{ runner.os }}-build-</span> <span class="pl-s"> ${{ runner.os }}-</span> <span class="pl-s"></span> - <span class="pl-ent">name</span>: <span class="pl-s">Install Node dependencies</span> <span class="pl-ent">run</span>: <span class="pl-s">npm install</span> - <span class="pl-ent">name</span>: <span class="pl-s">Run tests</span> <span class="pl-ent">run</span>: <span class="pl-s">npm test</span> <span class="pl-ent">timeout-minutes</span>: <span class="pl-c1">5</span> - <span class="pl-ent">name</span>: <span class="pl-s">Upload test videos</span> <span class="pl-ent">uses</span>: <span class="pl-s">actions/upload-artifact@v3</span> <span class="pl-ent">with</span>: <span class="pl-ent">name</span>: <span class="pl-s">test-videos</span> <span class="pl-ent">path</span>: <span class="pl-s">test-videos/</span></pre></div> <p>This workflow configures NPM caching to avoid downloading everything every time, installs the dependencies, runs the tests, and then uploads the videos at the end.</p> <p>Those videos end up attached to the workflow run as an artifact that can be downloaded and viewed locally.</p> | <Binary: 66,234 bytes> | 2022-07-13T15:29:19-07:00 | 2022-07-13T22:29:19+00:00 | 2022-07-13T15:29:19-07:00 | 2022-07-13T22:29:19+00:00 | b6eb2943ffaec25569035cc04383de7d | testing-electron-playwright |