til: electron_python-inside-electron.md
This data as json
| path | topic | title | url | body | html | shot | created | created_utc | updated | updated_utc | shot_hash | slug |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| electron_python-inside-electron.md | electron | Bundling Python inside an Electron app | https://github.com/simonw/til/blob/main/electron/python-inside-electron.md | For [Datasette Desktop](https://datasette.io/desktop) I chose to bundle a full version of Python 3.9 inside my `Datasette.app` application. I did this in order to support installation of plugins via `pip install` - you can read more about my reasoning in [Datasette Desktop—a macOS desktop application for Datasette](https://simonwillison.net/2021/Sep/8/datasette-desktop/). I used [python-build-standalone](https://github.com/indygreg/python-build-standalone) for this, which provides a version of Python that is designed for easy of bundling - it's also used by [PyOxidize](https://github.com/indygreg/PyOxidizer). Both projects are created and maintained by Gregory Szorc. ## In development mode In my Electron app's root folder I ran the following: ``` wget https://github.com/indygreg/python-build-standalone/releases/download/20210724/cpython-3.9.6-x86_64-apple-darwin-install_only-20210724T1424.tar.gz tar -xzvf cpython-3.9.6-x86_64-apple-darwin-install_only-20210724T1424.tar.gz ``` This gave me a `python/` subfolder containing a full standalone Python, ready to run on my Mac. Running `python/bin/python3.9 --version` confirms that this is working correctly. ## Calling Python from Electron I used the Node.js `child_process.execFile()` function to execute Python scripts from inside Electron, like this: ```javascript const cp = require("child_process"); const util = require("util"); const execFile = util.promisify(cp.execFile); await execFile(path_to_python, ["-m", "random"]); ``` `path_to_python` needs to be the path to that `python3.9` executable. I wrote a `findPython()` function to find that, like so: ```javascript const fs = require("fs"); function findPython() { const possibilities = [ // In packaged app path.join(process.resourcesPath, "python", "bin", "python3.9"), // In development path.join(__dirname, "python", "bin", "python3.9"), ]; for (const path of possibilities) { if (fs.existsSync(path)) { return path; } } console.log("Could not find python3, checked", possibilities); app.quit(); } ``` The `resourcesPath` bit here works for the packaged and deployed application. ## Packaging the app I needed that `python` folder to be bundled up as part of the `Datasette.app` application bundle. I achieved that by adding the following to my `"build"` section in `package.json`: ```json "extraResources": [ { "from": "python", "to": "python", "filter": [ "**/*" ] } ] ``` This causes `electron-builder` to copy the contents of that `python` folder into `Datasette.app/Contents/Resources/python` in the packaged application. My `findPython()` function earlier knows to look for it there by creating a path to it starting with `process.resourcesPath`. ## Signing and notarizing I wrote extensive notes on signing and notarizing in [Signing and notarizing an Electron app for distribution using GitHub Actions](https://til.simonwillison.net/electron/sign-notarize-electron-macos), which includes details on how I signed the Python binaries that ended up included in the package due to this pattern. | <p>For <a href="https://datasette.io/desktop" rel="nofollow">Datasette Desktop</a> I chose to bundle a full version of Python 3.9 inside my <code>Datasette.app</code> application. I did this in order to support installation of plugins via <code>pip install</code> - you can read more about my reasoning in <a href="https://simonwillison.net/2021/Sep/8/datasette-desktop/" rel="nofollow">Datasette Desktop—a macOS desktop application for Datasette</a>.</p> <p>I used <a href="https://github.com/indygreg/python-build-standalone">python-build-standalone</a> for this, which provides a version of Python that is designed for easy of bundling - it's also used by <a href="https://github.com/indygreg/PyOxidizer">PyOxidize</a>. Both projects are created and maintained by Gregory Szorc.</p> <h2> <a id="user-content-in-development-mode" class="anchor" href="#in-development-mode" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>In development mode</h2> <p>In my Electron app's root folder I ran the following:</p> <pre><code>wget https://github.com/indygreg/python-build-standalone/releases/download/20210724/cpython-3.9.6-x86_64-apple-darwin-install_only-20210724T1424.tar.gz tar -xzvf cpython-3.9.6-x86_64-apple-darwin-install_only-20210724T1424.tar.gz </code></pre> <p>This gave me a <code>python/</code> subfolder containing a full standalone Python, ready to run on my Mac.</p> <p>Running <code>python/bin/python3.9 --version</code> confirms that this is working correctly.</p> <h2> <a id="user-content-calling-python-from-electron" class="anchor" href="#calling-python-from-electron" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Calling Python from Electron</h2> <p>I used the Node.js <code>child_process.execFile()</code> function to execute Python scripts from inside Electron, like this:</p> <div class="highlight highlight-source-js"><pre><span class="pl-k">const</span> <span class="pl-s1">cp</span> <span class="pl-c1">=</span> <span class="pl-en">require</span><span class="pl-kos">(</span><span class="pl-s">"child_process"</span><span class="pl-kos">)</span><span class="pl-kos">;</span> <span class="pl-k">const</span> <span class="pl-s1">util</span> <span class="pl-c1">=</span> <span class="pl-en">require</span><span class="pl-kos">(</span><span class="pl-s">"util"</span><span class="pl-kos">)</span><span class="pl-kos">;</span> <span class="pl-k">const</span> <span class="pl-s1">execFile</span> <span class="pl-c1">=</span> <span class="pl-s1">util</span><span class="pl-kos">.</span><span class="pl-en">promisify</span><span class="pl-kos">(</span><span class="pl-s1">cp</span><span class="pl-kos">.</span><span class="pl-c1">execFile</span><span class="pl-kos">)</span><span class="pl-kos">;</span> <span class="pl-k">await</span> <span class="pl-s1">execFile</span><span class="pl-kos">(</span><span class="pl-s1">path_to_python</span><span class="pl-kos">,</span> <span class="pl-kos">[</span><span class="pl-s">"-m"</span><span class="pl-kos">,</span> <span class="pl-s">"random"</span><span class="pl-kos">]</span><span class="pl-kos">)</span><span class="pl-kos">;</span></pre></div> <p><code>path_to_python</code> needs to be the path to that <code>python3.9</code> executable. I wrote a <code>findPython()</code> function to find that, like so:</p> <div class="highlight highlight-source-js"><pre><span class="pl-k">const</span> <span class="pl-s1">fs</span> <span class="pl-c1">=</span> <span class="pl-en">require</span><span class="pl-kos">(</span><span class="pl-s">"fs"</span><span class="pl-kos">)</span><span class="pl-kos">;</span> <span class="pl-k">function</span> <span class="pl-en">findPython</span><span class="pl-kos">(</span><span class="pl-kos">)</span> <span class="pl-kos">{</span> <span class="pl-k">const</span> <span class="pl-s1">possibilities</span> <span class="pl-c1">=</span> <span class="pl-kos">[</span> <span class="pl-c">// In packaged app</span> <span class="pl-s1">path</span><span class="pl-kos">.</span><span class="pl-en">join</span><span class="pl-kos">(</span><span class="pl-s1">process</span><span class="pl-kos">.</span><span class="pl-c1">resourcesPath</span><span class="pl-kos">,</span> <span class="pl-s">"python"</span><span class="pl-kos">,</span> <span class="pl-s">"bin"</span><span class="pl-kos">,</span> <span class="pl-s">"python3.9"</span><span class="pl-kos">)</span><span class="pl-kos">,</span> <span class="pl-c">// In development</span> <span class="pl-s1">path</span><span class="pl-kos">.</span><span class="pl-en">join</span><span class="pl-kos">(</span><span class="pl-s1">__dirname</span><span class="pl-kos">,</span> <span class="pl-s">"python"</span><span class="pl-kos">,</span> <span class="pl-s">"bin"</span><span class="pl-kos">,</span> <span class="pl-s">"python3.9"</span><span class="pl-kos">)</span><span class="pl-kos">,</span> <span class="pl-kos">]</span><span class="pl-kos">;</span> <span class="pl-k">for</span> <span class="pl-kos">(</span><span class="pl-k">const</span> <span class="pl-s1">path</span> <span class="pl-k">of</span> <span class="pl-s1">possibilities</span><span class="pl-kos">)</span> <span class="pl-kos">{</span> <span class="pl-k">if</span> <span class="pl-kos">(</span><span class="pl-s1">fs</span><span class="pl-kos">.</span><span class="pl-en">existsSync</span><span class="pl-kos">(</span><span class="pl-s1">path</span><span class="pl-kos">)</span><span class="pl-kos">)</span> <span class="pl-kos">{</span> <span class="pl-k">return</span> <span class="pl-s1">path</span><span class="pl-kos">;</span> <span class="pl-kos">}</span> <span class="pl-kos">}</span> <span class="pl-smi">console</span><span class="pl-kos">.</span><span class="pl-en">log</span><span class="pl-kos">(</span><span class="pl-s">"Could not find python3, checked"</span><span class="pl-kos">,</span> <span class="pl-s1">possibilities</span><span class="pl-kos">)</span><span class="pl-kos">;</span> <span class="pl-s1">app</span><span class="pl-kos">.</span><span class="pl-en">quit</span><span class="pl-kos">(</span><span class="pl-kos">)</span><span class="pl-kos">;</span> <span class="pl-kos">}</span></pre></div> <p>The <code>resourcesPath</code> bit here works for the packaged and deployed application.</p> <h2> <a id="user-content-packaging-the-app" class="anchor" href="#packaging-the-app" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Packaging the app</h2> <p>I needed that <code>python</code> folder to be bundled up as part of the <code>Datasette.app</code> application bundle. I achieved that by adding the following to my <code>"build"</code> section in <code>package.json</code>:</p> <div class="highlight highlight-source-json"><pre> <span class="pl-s"><span class="pl-pds">"</span>extraResources<span class="pl-pds">"</span></span>: [ { <span class="pl-s"><span class="pl-pds">"</span>from<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>python<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>to<span class="pl-pds">"</span></span>: <span class="pl-s"><span class="pl-pds">"</span>python<span class="pl-pds">"</span></span>, <span class="pl-s"><span class="pl-pds">"</span>filter<span class="pl-pds">"</span></span>: [ <span class="pl-s"><span class="pl-pds">"</span>**/*<span class="pl-pds">"</span></span> ] } ]</pre></div> <p>This causes <code>electron-builder</code> to copy the contents of that <code>python</code> folder into <code>Datasette.app/Contents/Resources/python</code> in the packaged application.</p> <p>My <code>findPython()</code> function earlier knows to look for it there by creating a path to it starting with <code>process.resourcesPath</code>.</p> <h2> <a id="user-content-signing-and-notarizing" class="anchor" href="#signing-and-notarizing" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Signing and notarizing</h2> <p>I wrote extensive notes on signing and notarizing in <a href="https://til.simonwillison.net/electron/sign-notarize-electron-macos" rel="nofollow">Signing and notarizing an Electron app for distribution using GitHub Actions</a>, which includes details on how I signed the Python binaries that ended up included in the package due to this pattern.</p> | <Binary: 87,046 bytes> | 2021-09-08T16:38:57-07:00 | 2021-09-08T23:38:57+00:00 | 2021-09-08T16:38:57-07:00 | 2021-09-08T23:38:57+00:00 | 8490359447794f9b8a23fb242946a61c | python-inside-electron |