Find stories in data

PyPI Changelog Python 3.x License discord mastodon: @datasette

Annotated version of this introductory video

Datasette is a tool for exploring and publishing data. It helps people take data of any shape, analyze and explore it, and publish it as an interactive website and accompanying API.

Datasette is aimed at data journalists, museum curators, archivists, local governments, scientists, researchers and anyone else who has data that they wish to share with the world. It is part of a wider ecosystem of 46 tools and 145 plugins dedicated to making working with structured data as productive as possible.

Try a demo and explore 33,000 power plants around the world, then follow the tutorial or take a look at some other examples of Datasette in action.

Then read how to get started with Datasette, subscribe to the monthly-ish newsletter and consider signing up for office hours for an in-person conversation about the project.

New: Datasette Desktop - a macOS desktop application for easily running Datasette on your own computer!

Exploratory data analysis

Import data from CSVs, JSON, database connections and more. Datasette will automatically show you patterns in your data and help you share your findings with your colleagues.

Instant data publishing

datasette publish lets you instantly publish your data to hosting providers like Google Cloud Run, Heroku or Vercel.

Rapid prototyping

Spin up a JSON API for any data in minutes. Use it to prototype and prove your ideas without building a custom backend.

Latest news

18th February 2024 #

Datasette 1.0a10 is a focused alpha that changes some internal details about how Datasette handles transactions. The datasette.execute_write_fn() internal method now wraps the function in a database transaction unless you pass transaction=False.

16th February 2024 #

Datasette 1.0a9 adds basic alter table support to the JSON API, tweaks how permissions works and introduces some new plugin debugging utilities.

7th February 2024 #

Datasette 1.0a8 introduces several new plugin hooks, a JavaScript plugin system and moves plugin configuration from metadata.yaml to datasette.yaml. Read more about the release in the annotated release notes for 1.0a8.

1st December 2023 #

Datasette Enrichments is a new feature for Datasette that supports enriching data by running custom code against every selected row in a table. Read Datasette Enrichments: a new plugin framework for augmenting your data for more details, plus a video demo of enrichments for geocoding addresses and processing text and images using GPT-4.

30th November 2023 #

datasette-comments is a new plugin by Alex Garcia which adds collaborative commenting to Datasette. Alex built the plugin for Datasette Cloud, but it's also available as an open source package for people who are hosting their own Datasette instances. See Annotate and explore your data with datasette-comments on the Datasette Cloud blog for more details.

22nd August 2023 #

Datasette 1.0a4 has a fix for a security vulnerability in the Datasette 1.0 alpha series: the API explorer interface exposed the names of private databases and tables in public instances that were protected by a plugin such as datasette-auth-passwords, though not the actual content of those tables. See the security advisory for more details and workarounds for if you can't upgrade immediately. The latest edition of the Datasette Newsletter also talks about this issue.

15th August 2023 #

datasette-write-ui: a Datasette plugin for editing, inserting, and deleting rows introduces a new plugin adding add/edit/delete functionality to Datasette, developed by Alex Garcia. Alex built this for Datasette Cloud, and this post is the first announcement made on the new Datasette Cloud blog - see also Welcome to Datasette Cloud.

9th August 2023 #

Datasette 1.0a3 is an alpha release of Datasette that previews the new default JSON API design that’s coming in version 1.0 - the single most significant change planned for that 1.0 release.

1st July 2023 #

New tutorial: Data analysis with SQLite and Python. This tutorial, originally presented at PyCon 2023, includes a 2h45m video and an extensive handout that should be useful with or without the video. Topics covered include Python's sqlite3 module, sqlite-utils, Datasette, Datasette Lite, advanced SQL patterns and more.

24th March 2023 #

I built a ChatGPT plugin to answer questions about data hosted in Datasette describes a new experimental Datasette plugin to enable people to query data hosted in a Datasette interface via ChatGPT, asking human language questions that are automatically converted to SQL and used to generate a readable response.

23rd February 2023 #

Using Datasette in GitHub Codespaces is a new tutorial showing how Datasette can be run in GitHub's free Codespaces browser-based development environments, using the new datasette-codespaces plugin.

28th January 2023 #

Examples of sites built using Datasette now includes screenshots of Datasette deployments that illustrate a variety of problems that can be addressed using Datasette and its plugins.

13th January 2023 #

Semantic search answers: Q&A against documentation with GPT3 + OpenAI embeddings shows how Datasette can be used to implement semantic search and build a system for answering questions against an existing corpus of text, using two new plugins: datasette-openai and datasette-faiss, and a new tool: openai-to-sqlite.

9th January 2023 #

Datasette 0.64 is out, and includes a strong warning against running SpatiaLite in production without disabling arbitrary SQL queries, plus a new --setting default_allow_sql off setting to make it easier to do that. See Datasette 0.64, with a warning about SpatiaLite for more about this release. A new tutorial, Building a location to time zone API with SpatiaLite, describes how to safely use SpatiaLite and Datasette to build and deploy an API for looking up time zones for a latitude/longitude location.

15th December 2022 #

Datasette 1.0a2: Upserts and finely grained permissions describes the new upsert API and much improved permissions capabilities introduced in the latest Datasette 1.0a2 alpha release.

All news

Latest releases

25th February 2024

dclient 0.3 - A client CLI utility for Datasette instances

20th February 2024

datasette-events-forward 0.1a1 - Forward Datasette events to another instance

  • Silently ignore events if api_url has not been configured, rather than accumulating them in the datasette_events_to_forward table. #2
  • Events are now deleted from datasette_events_to_forward as soon as they have been delivired. #3

datasette-events-forward 0.1a0

  • Initial implementation. Forwards tracked events to another Datasette instance via the JSON Write API. #1

datasette-auth-tokens 0.4a9 - Datasette plugin for authenticating access using API tokens

  • Fixed a bug where the heading "Specific tables in specific databases" showed even for databases with no tables. #32

19th February 2024

datasette 1.0a11 - An open source multi-tool for exploring and publishing data

  • The "replace": true argument to the /db/table/-/insert API now requires the actor to have the update-row permission. (#2279)
  • Fixed some UI bugs in the interactive permissions debugging tool. (#2278)
  • The column action menu now aligns better with the cog icon, and positions itself taking into account the width of the browser window. (#2263)

18th February 2024

datasette-edit-schema 0.8a0 - Datasette plugin for modifying table schemas

This release introduces support for finely grained per-database and per-table permissions, plus integration with the new Datasette Events mechanism added in Datasette 1.0a8.

  • This release requires Datasette 1.0a9 or higher.
  • Finely grained permissions: the edit-schema permission is still supported, but you can also control access to this plugin using the create-table, drop-table and alter-table permissions from Datasette core. These can be used to control access on a per-table and per-database level. #22, #55
  • Now tracks create-table, alter-table and drop-table events using the new Datasette Events system introduced in Datasette 1.0a8. #50
  • Primary keys can now also be set as foreign keys, useful for many-to-many relationship tables. #52
  • Now runs VACUUM using db.execute_isolated_fn() to avoid being caught out by any connection permission handlers. #47
  • Tightened up display on mobile screens. #45
  • Fixed a bug where the SQL schema for a table was not shown correctly on the edit schema page. #49
  • Fixed a 500 error if a table only consists of primary keys. #51

datasette-unsafe-actor-debug 0.2 - Debug plugin that lets you imitate any actor

  • Plugin is now disabled unless you explicitly configure it. #2

Use this in your settings: yaml plugins: datasette-unsafe-actor-debug: enabled: true Or with Datasette 1.0 alpha start datasette with the following: bash datasette -s plugins.datasette-unsafe-actor-debug.enabled 1 - If enabled, the plugin now adds a link to the debug tool in the Datasette navigation menu. #3

datasette-unsafe-actor-debug 0.1

  • Initial plugin. Adds a /-/unsafe-actor debug page for imitating any actor. #1

datasette 1.0a10 - An open source multi-tool for exploring and publishing data

The only changes in this alpha correspond to the way Datasette handles database transactions. (#2277)

  • The database.execute_write_fn() method has a new transaction=True parameter. This defaults to True which means all functions executed using this method are now automatically wrapped in a transaction - previously the functions needed to roll transaction handling on their own, and many did not.
  • Pass transaction=False to execute_write_fn() if you want to manually handle transactions in your function.
  • Several internal Datasette features, including parts of the JSON write API, had been failing to wrap their operations in a transaction. This has been fixed by the new transaction=True default.

16th February 2024

datasette 1.0a9

This alpha release adds basic alter table support to the Datasette Write API and fixes a permissions bug relating to the /upsert API endpoint.

Alter table support for create, insert, upsert and update

The JSON write API can now be used to apply simple alter table schema changes, provided the acting actor has the new alter-table permission. (#2101)

The only alter operation supported so far is adding new columns to an existing table.

  • The /db/-/create API now adds new columns during large operations to create a table based on incoming example "rows", in the case where one of the later rows includes columns that were not present in the earlier batches. This requires the create-table but not the alter-table permission.
  • When /db/-/create is called with rows in a situation where the table may have been already created, an "alter": true key can be included to indicate that any missing columns from the new rows should be added to the table. This requires the alter-table permission.
  • /db/table/-/insert and /db/table/-/upsert and /db/table/row-pks/-/update all now also accept "alter": true, depending on the alter-table permission.

Operations that alter a table now fire the new alter-table event.

Permissions fix for the upsert API

The /database/table/-/upsert API had a minor permissions bug, only affecting Datasette instances that had configured the insert-row and update-row permissions to apply to a specific table rather than the database or instance as a whole. Full details in issue #2262.

To avoid similar mistakes in the future the datasette.permission_allowed() method now specifies default= as a keyword-only argument.

Permission checks now consider opinions from every plugin

The datasette.permission_allowed() method previously consulted every plugin that implemented the permission_allowed() plugin hook and obeyed the opinion of the last plugin to return a value. (#2275)

Datasette now consults every plugin and checks to see if any of them returned False (the veto rule), and if none of them did, it then checks to see if any of them returned True.

This is explained at length in the new documentation covering How permissions are resolved.

Other changes
  • The new DATASETTE_TRACE_PLUGINS=1 environment variable turns on detailed trace output for every executed plugin hook, useful for debugging and understanding how the plugin system works at a low level. (#2274)
  • Datasette on Python 3.9 or above marks its non-cryptographic uses of the MD5 hash function as usedforsecurity=False, for compatibility with FIPS systems. (#2270)
  • SQL relating to Datasette's internal database now executes inside a transaction, avoiding a potential database locked error. (#2273)
  • The /-/threads debug page now identifies the database in the name associated with each dedicated write thread. (#2265)
  • The /db/-/create API now fires a insert-rows event if rows were inserted after the table was created. (#2260)

datasette-enrichments-opencage 0.1.1 - Geocoding enrichment using OpenCage

  • Run insert SQL queries inside a transaction. #5

14th February 2024

datasette-auth-tokens 0.4a8 - Datasette plugin for authenticating access using API tokens

  • Fix for intermittent database is locked bug in managed tokens mode. #30

9th February 2024

datasette-enrichments-quickjs 0.1a0 - Enrich data with a custom JavaScript function

  • Initial release. Enrich data using a custom JavaScript function. #1

8th February 2024

datasette-events-db 0.1a0 - Log Datasette events to a database table

  • Initial plugin. Logs all events to a datasette_events table. #1

7th February 2024

datasette 1.0a8 - An open source multi-tool for exploring and publishing data

This alpha release continues the migration of Datasette's configuration from metadata.yaml to the new datasette.yaml configuration file, introduces a new system for JavaScript plugins and adds several new plugin hooks.

See Datasette 1.0a8: JavaScript plugins, new plugin hooks and plugin configuration in datasette.yaml for an annotated version of these release notes.

Configuration
  • Plugin configuration now lives in the datasette.yaml configuration file, passed to Datasette using the -c/--config option. Thanks, Alex Garcia. (#2093) bash datasette -c datasette.yaml Where datasette.yaml contains configuration that looks like this: yaml plugins: datasette-cluster-map: latitude_column: xlat longitude_column: xlon
  • Previously plugins were configured in metadata.yaml, which was confusing as plugin settings were unrelated to database and table metadata.

  • The -s/--setting option can now be used to set plugin configuration as well. See Configuration via the command-line for details. (#2252)

    The above YAML configuration example using -s/--setting looks like this: bash datasette mydatabase.db\ -s plugins.datasette-cluster-map.latitude_column xlat\ -s plugins.datasette-cluster-map.longitude_column xlon - The new /-/config page shows the current instance configuration, after redacting keys that could contain sensitive data such as API keys or passwords. (#2254)

  • Existing Datasette installations may already have configuration set in metadata.yaml that should be migrated to datasette.yaml. To avoid breaking these installations, Datasette will silently treat table configuration, plugin configuration and allow blocks in metadata as if they had been specified in configuration instead. (#2247) (#2248) (#2249)

Note that the datasette publish command has not yet been updated to accept a datasette.yaml configuration file. This will be addressed in #2195 but for the moment you can include those settings in metadata.yaml instead.

JavaScript plugins

Datasette now includes a JavaScript plugins mechanism, allowing JavaScript to customize Datasette in a way that can collaborate with other plugins.

This provides two initial hooks, with more to come in the future:

Thanks Cameron Yick for contributing this feature. (#2052)

Plugin hooks
  • New jinja2_environment_from_request(datasette, request, env) plugin hook, which can be used to customize the current Jinja environment based on the incoming request. This can be used to modify the template lookup path based on the incoming request hostname, among other things. (#2225)

  • New family of template slot plugin hooks: top_homepage, top_database, top_table, top_row, top_query, top_canned_query. Plugins can use these to provide additional HTML to be injected at the top of the corresponding pages. (#1191)

  • New track_event() mechanism for plugins to emit and receive events when certain events occur within Datasette. (#2240)

  • New internal function for plugin authors: await db.execute_isolated_fn(fn), for creating a new SQLite connection, executing code and then closing that connection, all while preventing other code from writing to that particular database. This connection will not have the prepare_connection() plugin hook executed against it, allowing plugins to perform actions that might otherwise be blocked by existing connection configuration. (#2218)

Documentation
Minor fixes
  • Datasette no longer attempts to run SQL queries in parallel when rendering a table page, as this was leading to some rare crashing bugs. (#2189)

  • Fixed warning: DeprecationWarning: pkg_resources is deprecated as an API (#2057)

  • Fixed bug where ?_extra=columns parameter returned an incorrectly shaped response. (#2230)

All releases