Datasette News

Datasette 0.55 adds support for cross-database SQL queries. You can now run datasette --crossdb one.db two.db and then run queries that join data from tables in both of those database files - see cross-database queries in the documentation for more details.

sqlite-utils 3.6 adds similar features: a db.attach(alias, filepath) Python API method and --attach alias filepath.db command-line option, both for attaching additional databases in order to execute cross-database queries.

This new Video introduction to Datasette and sqlite-utils provides a full introduction to both Datasette and sqlite-utils in 17 minutes, including a live demo of creating a database from a CSV file and publishing it to Google Cloud Run.

Serving map tiles from SQLite with MBTiles and datasette-tiles. datasette-tiles is a new plugin that adds a tile server to Datasette, serving map tiles from databases that conform to the MBTiles specification. download-tiles is a tool for building these databases, and datasette-basemap is a plugin that bundles a 22MB SQLite database with OpenStreetMap tiles covering zoom levels 0-6 for the entire world.

Datasette 0.54 is out today. Highlights include the new _internal in-memory database exposing details of connected tables, plus support for JavaScript modules in plugins and add-on scripts. More commentary on this release is available in the annotated release notes.

Drawing shapes on a map to query a SpatiaLite database introduces the new datasette-leaflet-freedraw plugin, which adds support for drawing shapes on a map to specify a GeoJSON MultiPolygon that can be used to query SpatiaLite databases.

APIs from CSS without JavaScript: the datasette-css-properties plugin introduces datasette-css-properties, a highly experimental plugin that can output table rows and SQL query results as CSS stylesheets defining custom properties that can then be used to customize a static HTML page.

New on this site: a Datasette Tools directory and a search engine that covers documentation, tools, plugins, releases and more. The search engine uses Dogsheep Beta - I wrote about how that works in Building a search engine for datasette.io.

I wrote more about how this site works in Weeknotes: datasette.io, an official project website for Datasette.

Datasette 0.53 - datasette serve --create option, ?column__arraynotcontains= table filter, ?_header=off CSV option and updated links to reflect the new project website.

New Datasette project website - three years after the first release Datasette finally has its own website! The site includes project news, a plugin directory, use-cases and examples. The site is implemented using Datasette custom templates, and the source code is available on GitHub.

Datasette 0.52 - --config is now --setting, new database_actions plugin hook, datasette publish cloudrun --apt-get-install option and several bug fixes.

Datasette 0.51 - A new visual design, plugin hooks for adding navigation options, better handling of binary data, URL building utility methods and better support for running Datasette behind a proxy. Annotated release notes.

Datasette 0.50 - New column actions menu. datasette.client object for plugins to make internal API requests. Improved documentation on deploying Datasette. Annotated release notes.

Datasette 0.49 - JSON API for writable canned queries, path parameters for custom pages. See also Datasette 0.49: The annotated release notes.

Datasette 0.48 - Documentation now lives at docs.datasette.io, improvements to the extra_template_vars, extra_css_urls, extra_js_urls and extra_body_script plugin hooks.

Datasette 0.47 - Datasette can now be installed using Homebrew! brew install simonw/datasette/datasette. Also new: datasette install name-of-plugin and datasette uninstall name-of-plugin commands, and datasette --get '/-/versions.json' to output the result of Datasette HTTP calls on the command-line.

Datasette 0.46 - security fix relating to CSRF protection for writable canned queries, a new logo, new debugging tools, improved file downloads and more.

GraphQL in Datasette with the new datasette-graphql plugin

Two new plugins: datasette-copyable and datasette-insert-api. datasette-copyable adds copy-and-paste export options, and datasette-insert-api lets you create tables and insert or update data by POSTing JSON directly to Datasette.

Datasette 0.45 - Magic parameters for canned queries, a log out feature, improved plugin documentation and four new plugin hooks. See also Datasette 0.45: The annotated release notes.

A cookiecutter template for writing Datasette plugins

Datasette 0.44 - Authentication and permissions, writable canned queries, flash messages, new plugin hooks and much, much more.

Datasette 0.43 - Redesigned register_output_renderer plugin hook and various small improvements and fixes.

Datasette 0.42 - Documented internal methods for plugins to execute read queries against a database.

Datasette 0.41 - New mechanism for creating custom pages, new configuration directory mode, new ?column__notlike= table filter and various other smaller improvements.

Datasette 0.40 - Metadata can now be provided as YAML instead of JSON. Publishing to Zeit Now v1 is no longer supported, but Now v2 support is provided by the new datasette-publish-now plugin. Various bug fixes.

Datasette 0.39 - New base_url configuration option for running Datasette under a different URL prefix, "sort" and "sort_desc" metadata options for setting a default sort order for a table.

Datasette 0.38 - New --memory option for datasete publish cloudrun, Docker image upgraded to SQLite 3.31.1.

Datasette 0.37 - new internal APIs enabling plugins to safely write to databases. Read more here: Datasette Writes.

Datasette 0.36 - new internals documentation for plugins, prepare_connection() now accepts optional database and datasette arguments.

Datasette 0.35 - new .render_template() method for plugins.

Datasette 0.34 - improvements to search, datasette publish cloudrun and datasette package.

Deploying a data API using GitHub Actions and Cloud Run - how to use GitHub Actions and Google Cloud Run to automatically scrape data and deploy the result as an API with Datasette.

Datasette 0.33 - various small improvements.

Building tools to bring data-driven reporting to more newsrooms - some notes on my JSK fellowship so far.

Niche Museums is a new site entirely powered by Datasette, using custom templates and plugins. niche-museums.com, powered by Datasette describes how the site works, and datasette-atom: Define an Atom feed using a custom SQL query describes how the new datasette-atom plugin was used to add an Atom syndication feed to the site.

Datasette 0.32 now uses asynchronous rendering in Jinja templates, which means template functions can perform asynchronous operations such as executing SQL queries. datasette-template-sql is a new plugin uses this capability to add a new custom sql(sql_query) template function.

Datasette 0.31 - the first version of Datasette to support Python 3.8, which means dropping support for Python 3.5.

Datasette 0.30

Single sign-on against GitHub using ASGI middleware talks about the implementation of datasette-auth-github in more detail.

Datasette 0.29 - ASGI, new plugin hooks, facet by date and much, much more...

• datasette-auth-github - a new plugin for Datasette 0.29 that lets you require users to authenticate against GitHub before accessing your Datasette instance. You can whitelist specific users, or you can restrict access to members of specific GitHub organizations or teams.
• datasette-cors - a plugin that lets you configure CORS access from a list of domains (or a set of domain wildcards) so you can make JavaScript calls to a Datasette instance from a specific set of other hosts.

Porting Datasette to ASGI, and Turtles all the way down

The anonymized raw data from the Stack Overflow Developer Survey 2019 has been published in partnership with Glitch, powered by Datasette.

Datasette 0.28 - a salmagundi of new features!

• No longer immutable! Datasette now supports databases that change.
• Faceting improvements including facet-by-JSON-array and the ability to define custom faceting using plugins.
• datasette publish cloudrun lets you publish databases to Google's new Cloud Run hosting service.
• New register_output_renderer plugin hook for adding custom output extensions to Datasette in addition to the default .json and .csv.
• Dozens of other smaller features and tweaks - see the release notes for full details.
• Read more about this release here: Datasette 0.28 - and why master should always be releasable

sqlite-utils: a Python library and CLI tool for building SQLite databases - a partner tool for easily creating SQLite databases for use with Datasette.

Datasette 0.27 - datasette plugins command, newline-delimited JSON export option, new documentation on The Datasette Ecosystem.

Datasette 0.26.1 - SQLite upgrade in Docker image, /-/versions now shows SQLite compile options.

Datasette 0.26 - minor bug fixes, datasette publish now --alias argument.

Fast Autocomplete Search for Your Website - a new tutorial on using Datasette to build a JavaScript autocomplete search engine.

The interesting ideas in Datasette - a write-up of some of the less obvious interesting ideas embedded in the Datasette project.

Datasette 0.25 - New plugin hooks, improved database view support and an easier way to use more recent versions of SQLite.

Datasette 0.24 - a number of small new features

datasette-vega, a new plugin for visualizing data as bar, line or scatter charts

Datasette 0.23.1 - minor bug fixes

Datasette 0.23: CSV, SpatiaLite and more - CSV export, foreign key expansion in JSON and CSV, new config options, improved support for SpatiaLite and a bunch of other improvements

Datasette 0.22.1 bugfix plus we now use versioneer

Datasette 0.22: Datasette Facets

Datasette 0.21: New _shape=, new _size=, search within columns

Exploring the UK Register of Members Interests with SQL and Datasette - a tutorial describing how register-of-members-interests.datasettes.com was built (source code here)

Datasette plugins, and building a clustered map visualization - introducing Datasette's new plugin system and datasette-cluster-map, a plugin for visualizing data on a map

Datasette 0.20: static assets and templates for plugins

Datasette 0.19: plugins preview

Datasette 0.18: units

Datasette 0.15: sort by column

Baltimore Sun Public Salary Records - a data journalism project from the Baltimore Sun powered by Datasette - source code is available here

Cloud-first: Rapid webapp deployment using containers - a tutorial covering deploying Datasette using Microsoft Azure by the Research Software Engineering team at Imperial College London

Analyzing my Twitter followers with Datasette - a tutorial on using Datasette to analyze follower data pulled from the Twitter API

Datasette Publish: a web app for publishing CSV files as an online database

Building a location to time zone API with SpatiaLite, OpenStreetMap and Datasette

Datasette 0.14: customization edition

New in Datasette: filters, foreign keys and search

Datasette: instantly create and publish an API for your SQLite databases