24th March 2023 #
I built a ChatGPT plugin to answer questions about data hosted in Datasette describes a new experimental Datasette plugin to enable people to query data hosted in a Datasette interface via ChatGPT, asking human language questions that are automatically converted to SQL and used to generate a readable response.
23rd February 2023 #
Using Datasette in GitHub Codespaces is a new tutorial showing how Datasette can be run in GitHub's free Codespaces browser-based development environments, using the new datasette-codespaces plugin.
28th January 2023 #
Examples of sites built using Datasette now includes screenshots of Datasette deployments that illustrate a variety of problems that can be addressed using Datasette and its plugins.
13th January 2023 #
Semantic search answers: Q&A against documentation with GPT3 + OpenAI embeddings shows how Datasette can be used to implement semantic search and build a system for answering questions against an existing corpus of text, using two new plugins: datasette-openai and datasette-faiss, and a new tool: openai-to-sqlite.
9th January 2023 #
Datasette 0.64 is out, and includes a strong warning against running SpatiaLite in production without disabling arbitrary SQL queries, plus a new --setting default_allow_sql off setting to make it easier to do that. See Datasette 0.64, with a warning about SpatiaLite for more about this release. A new tutorial, Building a location to time zone API with SpatiaLite, describes how to safely use SpatiaLite and Datasette to build and deploy an API for looking up time zones for a latitude/longitude location.
15th December 2022 #
Datasette 1.0a2: Upserts and finely grained permissions describes the new upsert API and much improved permissions capabilities introduced in the latest Datasette 1.0a2 alpha release.
2nd December 2022 #
Datasette’s new JSON write API: The first alpha of Datasette 1.0 introduces the new write API shipped in the first of the Datasette 1.0 alpha series of releases, including detailed descriptions of two demos that show how the API can be used.
27th October 2022 #
Datasette 0.63 is out. Here are the annotated release notes.
8th September 2022 #
Exploring the training data behind Stable Diffusion describes the process of building and deploying a 4GB searchable SQLite database using Datasette, starting with Parquet data that was used to train the Stable Diffusion image generation model. See also Exploring 12 Million of the 2.3 Billion Images Used to Train Stable Diffusion’s Image Generator.
21st August 2022 #
Analyzing ScotRail audio announcements with Datasette—from prototype to production provides a detailed walk-through of the process of constructing an initial rapid prototype using Datasette Lite, extending it with a custom plugin and then deploying it as a full Datasette instance using GitHub Actions and Vercel.
14th August 2022 #
Datasette 0.62 introduces compatibility with Pyodide for Datasette Lite, and incorporates a number of bug fixes, plugin hook upgrades and other improvements.
31st July 2022 #
New tutorial and accompanying ten minute video: Cleaning data with sqlite-utils and Datasette.
30th June 2022 #
s3-ocr is a new tool which can run OCR (via Amazon Textract) against every PDF file in an S3 bucket and write the results to a searchable SQLite database, ready to use with Datasette. Read more about it in s3-ocr: Extract text from PDF files stored in an S3 bucket.
5th May 2022 #
Datasette Lite is a new way to run Datasette: entirely in your browser, thanks to the Pyodide project which provides a full Python environment compiled to WebAssembly. You can use it to explore any SQLite database file hosted on a CORS-enabled static hosting provider, which includes GitHub and GitHub Pages. Read more about this project in Datasette Lite: a server-side Python web application running in a browser.
12th April 2022 #
Datasette for geospatial analysis describes how Datasette can be used in conjunction with SpatiaLite to work with geospatial data, including details of several geospatial plugins and tools from the Datasette ecosystem.
23rd March 2022 #
Datasette 0.61 introduces two potentially backwards-incompatible changes in preparation for the forthcoming 1.0 release: hashed URL mode has been moved to a new plugin, and the way URLs are generated to tables or databases containing special characters such as
/ has changed. Datasette 0.61.1 fixes a small bug in that release. See also the annotated release notes for these two versions.
27th February 2022 #
The first two of an ongoing series of official Datasette tutorials are now available: Exploring a database with Datasette introduces the Datasette web interface and shows how it can be used to explore a new database, and Learn SQL with Datasette provides an introduction to SQL using Datasette as a learning environment.
13th January 2022 #
Datasette 0.60 adds a new
filters_from_request plugin hook, new internal methods for writing to the database, better performance and various faceting improvements. See also the annotated release notes.
5th December 2021 #
14th October 2021 #
Datasette 0.59 adds column descriptions in metadata, a new
register_command plugin hook, enhanced
--cors support and a bunch of other fixes and documentation improvements. See also the annotated release notes.
8th September 2021 #
Datasette Desktop is a new macOS desktop application version of Datasette, which supports opening SQLite files on your computer, importing CSV files and installing plugins. I wrote more about how it works in Datasette Desktop—a macOS desktop application for Datasette.
28th July 2021 #
The Baked Data architectural pattern describes a pattern commonly used with Datasette where the content for a site is bundled inside a SQLite database file and included alongside templates and application code in a deployment to a serverless hosting provider.
15th July 2021 #
Datasette 0.58 has new plugin hooks, a huge performance improvement for faceting, support for Unix domain sockets and several other improvements. Read the annotated release notes for extra background and context on the release.
5th June 2021 #
Datasette 0.57 is out with an important security patch plus a number of new features and bug fixes. Datasette 0.56.1, also out today, provides the security patch for users who are not yet ready to upgrade to the latest version.
10th May 2021 #
Django SQL Dashboard is a new tool that brings a useful authenticated subset of Datasette to Django projects that are built on top of PostgreSQL.
28th March 2021 #
Datasette 0.56 has bug fixes and documentation improvements, plus some new documented internal APIs for plugin authors and SpatiaLite 5 bundled with the official Datasette Docker container.
18th February 2021 #
Datasette 0.55 adds support for cross-database SQL queries. You can now run
datasette --crossdb one.db two.db and then run queries that join data from tables in both of those database files - see cross-database queries in the documentation for more details.
sqlite-utils 3.6 adds similar features: a
db.attach(alias, filepath) Python API method and
--attach alias filepath.db command-line option, both for attaching additional databases in order to execute cross-database queries.
7th February 2021 #
This new Video introduction to Datasette and sqlite-utils provides a full introduction to both Datasette and sqlite-utils in 17 minutes, including a live demo of creating a database from a CSV file and publishing it to Google Cloud Run.
3rd February 2021 #
Serving map tiles from SQLite with MBTiles and datasette-tiles. datasette-tiles is a new plugin that adds a tile server to Datasette, serving map tiles from databases that conform to the MBTiles specification. download-tiles is a tool for building these databases, and datasette-basemap is a plugin that bundles a 22MB SQLite database with OpenStreetMap tiles covering zoom levels 0-6 for the entire world.
25th January 2021 #
Datasette 0.54 is out today. Highlights include the new
24th January 2021 #
Drawing shapes on a map to query a SpatiaLite database introduces the new datasette-leaflet-freedraw plugin, which adds support for drawing shapes on a map to specify a GeoJSON MultiPolygon that can be used to query SpatiaLite databases.
7th January 2021 #
19th December 2020 #
New on this site: a Datasette Tools directory and a search engine that covers documentation, tools, plugins, releases and more. The search engine uses Dogsheep Beta - I wrote about how that works in Building a search engine for datasette.io.
13th December 2020 #
I wrote more about how this site works in Weeknotes: datasette.io, an official project website for Datasette.
10th December 2020 #
Datasette 0.53 -
datasette serve --create option,
?column__arraynotcontains= table filter,
?_header=off CSV option and updated links to reflect the new project website.
New Datasette project website - three years after the first release Datasette finally has its own website! The site includes project news, a plugin directory, use-cases and examples. The site is implemented using Datasette custom templates, and the source code is available on GitHub.
28th November 2020 #
Datasette 0.52 -
--config is now
database_actions plugin hook,
datasette publish cloudrun --apt-get-install option and several bug fixes.
31st October 2020 #
Datasette 0.51 - A new visual design, plugin hooks for adding navigation options, better handling of binary data, URL building utility methods and better support for running Datasette behind a proxy. Annotated release notes.
9th October 2020 #
Datasette 0.50 - New column actions menu.
datasette.client object for plugins to make internal API requests. Improved documentation on deploying Datasette. Annotated release notes.
14th September 2020 #
Datasette 0.49 - JSON API for writable canned queries, path parameters for custom pages. See also Datasette 0.49: The annotated release notes.
16th August 2020 #
Datasette 0.48 - Documentation now lives at docs.datasette.io, improvements to the
extra_body_script plugin hooks.
11th August 2020 #
Datasette 0.47 - Datasette can now be installed using Homebrew!
brew install simonw/datasette/datasette. Also new:
datasette install name-of-plugin and
datasette uninstall name-of-plugin commands, and
datasette --get '/-/versions.json' to output the result of Datasette HTTP calls on the command-line.
9th August 2020 #
Datasette 0.46 - security fix relating to CSRF protection for writable canned queries, a new logo, new debugging tools, improved file downloads and more.
6th August 2020 #
24th July 2020 #
Two new plugins: datasette-copyable and datasette-insert-api.
datasette-copyable adds copy-and-paste export options, and
datasette-insert-api lets you create tables and insert or update data by POSTing JSON directly to Datasette.
1st July 2020 #
Datasette 0.45 - Magic parameters for canned queries, a log out feature, improved plugin documentation and four new plugin hooks. See also Datasette 0.45: The annotated release notes.
20th June 2020 #
11th June 2020 #
Datasette 0.44 - Authentication and permissions, writable canned queries, flash messages, new plugin hooks and much, much more.
28th May 2020 #
Datasette 0.43 - Redesigned register_output_renderer plugin hook and various small improvements and fixes.
8th May 2020 #
Datasette 0.42 - Documented internal methods for plugins to execute read queries against a database.
6th May 2020 #
Datasette 0.41 - New mechanism for creating custom pages, new configuration directory mode, new
?column__notlike= table filter and various other smaller improvements.
21st April 2020 #
Datasette 0.40 - Metadata can now be provided as YAML instead of JSON. Publishing to Zeit Now v1 is no longer supported, but Now v2 support is provided by the new datasette-publish-now plugin. Various bug fixes.
24th March 2020 #
Datasette 0.39 - New
base_url configuration option for running Datasette under a different URL prefix,
"sort_desc" metadata options for setting a default sort order for a table.
8th March 2020 #
Datasette 0.38 - New
--memory option for
datasete publish cloudrun, Docker image upgraded to SQLite 3.31.1.
25th February 2020 #
Datasette 0.37 - new internal APIs enabling plugins to safely write to databases. Read more here: Datasette Writes.
21st February 2020 #
Datasette 0.36 - new internals documentation for plugins,
prepare_connection() now accepts optional
4th February 2020 #
Datasette 0.35 - new
.render_template() method for plugins.
29th January 2020 #
Datasette 0.34 - improvements to search,
datasette publish cloudrun and
21st January 2020 #
Deploying a data API using GitHub Actions and Cloud Run - how to use GitHub Actions and Google Cloud Run to automatically scrape data and deploy the result as an API with Datasette.
22nd December 2019 #
Datasette 0.33 - various small improvements.
19th December 2019 #
Building tools to bring data-driven reporting to more newsrooms - some notes on my JSK fellowship so far.
2nd December 2019 #
Niche Museums is a new site entirely powered by Datasette, using custom templates and plugins. niche-museums.com, powered by Datasette describes how the site works, and datasette-atom: Define an Atom feed using a custom SQL query describes how the new datasette-atom plugin was used to add an Atom syndication feed to the site.
14th November 2019 #
Datasette 0.32 now uses asynchronous rendering in Jinja templates, which means template functions can perform asynchronous operations such as executing SQL queries. datasette-template-sql is a new plugin uses this capability to add a new custom
sql(sql_query) template function.
11th November 2019 #
Datasette 0.31 - the first version of Datasette to support Python 3.8, which means dropping support for Python 3.5.
18th October 2019 #
13th July 2019 #
Single sign-on against GitHub using ASGI middleware talks about the implementation of datasette-auth-github in more detail.
7th July 2019 #
Datasette 0.29 - ASGI, new plugin hooks, facet by date and much, much more...
- datasette-auth-github - a new plugin for Datasette 0.29 that lets you require users to authenticate against GitHub before accessing your Datasette instance. You can whitelist specific users, or you can restrict access to members of specific GitHub organizations or teams.
23rd June 2019 #
21st May 2019 #
The anonymized raw data from the Stack Overflow Developer Survey 2019 has been published in partnership with Glitch, powered by Datasette.
19th May 2019 #
Datasette 0.28 - a salmagundi of new features!
- No longer immutable! Datasette now supports databases that change.
- Faceting improvements including facet-by-JSON-array and the ability to define custom faceting using plugins.
- datasette publish cloudrun lets you publish databases to Google's new Cloud Run hosting service.
- New register_output_renderer
plugin hook for adding custom output extensions to Datasette in addition to the default
- Dozens of other smaller features and tweaks - see the release notes for full details.
- Read more about this release here: Datasette 0.28 - and why master should always be releasable
24th February 2019 #
sqlite-utils: a Python library and CLI tool for building SQLite databases - a partner tool for easily creating SQLite databases for use with Datasette.
31st January 2019 #
Datasette 0.27 -
datasette plugins command, newline-delimited JSON export option, new documentation on The Datasette Ecosystem.
10th January 2019 #
Datasette 0.26.1 - SQLite upgrade in Docker image,
/-/versions now shows SQLite compile options.
2nd January 2019 #
Datasette 0.26 - minor bug fixes,
datasette publish now --alias argument.
18th December 2018 #
3rd October 2018 #
The interesting ideas in Datasette - a write-up of some of the less obvious interesting ideas embedded in the Datasette project.
19th September 2018 #
Datasette 0.25 - New plugin hooks, improved database view support and an easier way to use more recent versions of SQLite.
23rd July 2018 #
Datasette 0.24 - a number of small new features
29th June 2018 #
datasette-vega, a new plugin for visualizing data as bar, line or scatter charts
21st June 2018 #
Datasette 0.23.1 - minor bug fixes
18th June 2018 #
Datasette 0.23: CSV, SpatiaLite and more - CSV export, foreign key expansion in JSON and CSV, new config options, improved support for SpatiaLite and a bunch of other improvements
23rd May 2018 #
Datasette 0.22.1 bugfix plus we now use versioneer
20th May 2018 #
5th May 2018 #
25th April 2018 #
Exploring the UK Register of Members Interests with SQL and Datasette - a tutorial describing how register-of-members-interests.datasettes.com was built (source code here)
20th April 2018 #
Datasette plugins, and building a clustered map visualization - introducing Datasette's new plugin system and datasette-cluster-map, a plugin for visualizing data on a map
16th April 2018 #
14th April 2018 #
9th April 2018 #
28th March 2018 #
Baltimore Sun Public Salary Records - a data journalism project from the Baltimore Sun powered by Datasette - source code is available here
27th March 2018 #
Cloud-first: Rapid webapp deployment using containers - a tutorial covering deploying Datasette using Microsoft Azure by the Research Software Engineering team at Imperial College London
28th January 2018 #
Analyzing my Twitter followers with Datasette - a tutorial on using Datasette to analyze follower data pulled from the Twitter API