Annotated version of this introductory video
Datasette is a tool for exploring and publishing data. It helps people take data of any shape, analyze and explore it, and publish it as an interactive website and accompanying API.
Datasette is aimed at data journalists, museum curators, archivists, local governments, scientists, researchers and anyone else who has data that they wish to share with the world. It is part of a wider ecosystem of 46 tools and 145 plugins dedicated to making working with structured data as productive as possible.
New: Datasette Desktop - a macOS desktop application for easily running Datasette on your own computer!
Import data from CSVs, JSON, database connections and more. Datasette will automatically show you patterns in your data and help you share your findings with your colleagues.
datasette publish lets you instantly publish your data to hosting providers like Google Cloud Run, Heroku or Vercel.
Spin up a JSON API for any data in minutes. Use it to prototype and prove your ideas without building a custom backend.
18th February 2024 #
Datasette 1.0a10 is a focused alpha that changes some internal details about how Datasette handles transactions. The
datasette.execute_write_fn() internal method now wraps the function in a database transaction unless you pass
16th February 2024 #
Datasette 1.0a9 adds basic alter table support to the JSON API, tweaks how permissions works and introduces some new plugin debugging utilities.
7th February 2024 #
datasette.yaml. Read more about the release in the annotated release notes for 1.0a8.
1st December 2023 #
Datasette Enrichments is a new feature for Datasette that supports enriching data by running custom code against every selected row in a table. Read Datasette Enrichments: a new plugin framework for augmenting your data for more details, plus a video demo of enrichments for geocoding addresses and processing text and images using GPT-4.
30th November 2023 #
datasette-comments is a new plugin by Alex Garcia which adds collaborative commenting to Datasette. Alex built the plugin for Datasette Cloud, but it's also available as an open source package for people who are hosting their own Datasette instances. See Annotate and explore your data with datasette-comments on the Datasette Cloud blog for more details.
22nd August 2023 #
Datasette 1.0a4 has a fix for a security vulnerability in the Datasette 1.0 alpha series: the API explorer interface exposed the names of private databases and tables in public instances that were protected by a plugin such as datasette-auth-passwords, though not the actual content of those tables. See the security advisory for more details and workarounds for if you can't upgrade immediately. The latest edition of the Datasette Newsletter also talks about this issue.
15th August 2023 #
datasette-write-ui: a Datasette plugin for editing, inserting, and deleting rows introduces a new plugin adding add/edit/delete functionality to Datasette, developed by Alex Garcia. Alex built this for Datasette Cloud, and this post is the first announcement made on the new Datasette Cloud blog - see also Welcome to Datasette Cloud.
9th August 2023 #
Datasette 1.0a3 is an alpha release of Datasette that previews the new default JSON API design that’s coming in version 1.0 - the single most significant change planned for that 1.0 release.
1st July 2023 #
New tutorial: Data analysis with SQLite and Python. This tutorial, originally presented at PyCon 2023, includes a 2h45m video and an extensive handout that should be useful with or without the video. Topics covered include Python's
sqlite-utils, Datasette, Datasette Lite, advanced SQL patterns and more.
24th March 2023 #
I built a ChatGPT plugin to answer questions about data hosted in Datasette describes a new experimental Datasette plugin to enable people to query data hosted in a Datasette interface via ChatGPT, asking human language questions that are automatically converted to SQL and used to generate a readable response.
23rd February 2023 #
28th January 2023 #
Examples of sites built using Datasette now includes screenshots of Datasette deployments that illustrate a variety of problems that can be addressed using Datasette and its plugins.
13th January 2023 #
Semantic search answers: Q&A against documentation with GPT3 + OpenAI embeddings shows how Datasette can be used to implement semantic search and build a system for answering questions against an existing corpus of text, using two new plugins: datasette-openai and datasette-faiss, and a new tool: openai-to-sqlite.
9th January 2023 #
Datasette 0.64 is out, and includes a strong warning against running SpatiaLite in production without disabling arbitrary SQL queries, plus a new --setting default_allow_sql off setting to make it easier to do that. See Datasette 0.64, with a warning about SpatiaLite for more about this release. A new tutorial, Building a location to time zone API with SpatiaLite, describes how to safely use SpatiaLite and Datasette to build and deploy an API for looking up time zones for a latitude/longitude location.
15th December 2022 #
Datasette 1.0a2: Upserts and finely grained permissions describes the new upsert API and much improved permissions capabilities introduced in the latest Datasette 1.0a2 alpha release.
25th February 2024
dclient 0.3 - A client CLI utility for Datasette instances
20th February 2024
datasette-events-forward 0.1a1 - Forward Datasette events to another instance
- Initial implementation. Forwards tracked events to another Datasette instance via the JSON Write API. #1
datasette-auth-tokens 0.4a9 - Datasette plugin for authenticating access using API tokens
- Fixed a bug where the heading "Specific tables in specific databases" showed even for databases with no tables. #32
19th February 2024
datasette 1.0a11 - An open source multi-tool for exploring and publishing data
"replace": trueargument to the
/db/table/-/insertAPI now requires the actor to have the
- Fixed some UI bugs in the interactive permissions debugging tool. (#2278)
- The column action menu now aligns better with the cog icon, and positions itself taking into account the width of the browser window. (#2263)
18th February 2024
datasette-edit-schema 0.8a0 - Datasette plugin for modifying table schemas
This release introduces support for finely grained per-database and per-table permissions, plus integration with the new Datasette Events mechanism added in Datasette 1.0a8.
- This release requires Datasette 1.0a9 or higher.
- Finely grained permissions: the
edit-schemapermission is still supported, but you can also control access to this plugin using the
alter-tablepermissions from Datasette core. These can be used to control access on a per-table and per-database level. #22, #55
- Now tracks
drop-tableevents using the new Datasette Events system introduced in Datasette 1.0a8. #50
- Primary keys can now also be set as foreign keys, useful for many-to-many relationship tables. #52
- Now runs
db.execute_isolated_fn()to avoid being caught out by any connection permission handlers. #47
- Tightened up display on mobile screens. #45
- Fixed a bug where the SQL schema for a table was not shown correctly on the edit schema page. #49
- Fixed a 500 error if a table only consists of primary keys. #51
datasette-unsafe-actor-debug 0.2 - Debug plugin that lets you imitate any actor
- Plugin is now disabled unless you explicitly configure it. #2
Use this in your settings:
Or with Datasette 1.0 alpha start
datasette with the following:
datasette -s plugins.datasette-unsafe-actor-debug.enabled 1
- If enabled, the plugin now adds a link to the debug tool in the Datasette navigation menu. #3
- Initial plugin. Adds a
/-/unsafe-actordebug page for imitating any actor. #1
datasette 1.0a10 - An open source multi-tool for exploring and publishing data
The only changes in this alpha correspond to the way Datasette handles database transactions. (#2277)
- The database.execute_write_fn() method has a new
transaction=Trueparameter. This defaults to
Truewhich means all functions executed using this method are now automatically wrapped in a transaction - previously the functions needed to roll transaction handling on their own, and many did not.
execute_write_fn()if you want to manually handle transactions in your function.
- Several internal Datasette features, including parts of the JSON write API, had been failing to wrap their operations in a transaction. This has been fixed by the new
16th February 2024
This alpha release adds basic alter table support to the Datasette Write API and fixes a permissions bug relating to the
/upsert API endpoint.
Alter table support for create, insert, upsert and update
The only alter operation supported so far is adding new columns to an existing table.
- The /db/-/create API now adds new columns during large operations to create a table based on incoming example
"rows", in the case where one of the later rows includes columns that were not present in the earlier batches. This requires the
create-tablebut not the
/db/-/createis called with rows in a situation where the table may have been already created, an
"alter": truekey can be included to indicate that any missing columns from the new rows should be added to the table. This requires the
- /db/table/-/insert and /db/table/-/upsert and /db/table/row-pks/-/update all now also accept
"alter": true, depending on the
Operations that alter a table now fire the new alter-table event.
Permissions fix for the upsert API
The /database/table/-/upsert API had a minor permissions bug, only affecting Datasette instances that had configured the
update-row permissions to apply to a specific table rather than the database or instance as a whole. Full details in issue #2262.
To avoid similar mistakes in the future the datasette.permission_allowed() method now specifies
default= as a keyword-only argument.
Permission checks now consider opinions from every plugin
Datasette now consults every plugin and checks to see if any of them returned
False (the veto rule), and if none of them did, it then checks to see if any of them returned
This is explained at length in the new documentation covering How permissions are resolved.
- The new DATASETTE_TRACE_PLUGINS=1 environment variable turns on detailed trace output for every executed plugin hook, useful for debugging and understanding how the plugin system works at a low level. (#2274)
- Datasette on Python 3.9 or above marks its non-cryptographic uses of the MD5 hash function as
usedforsecurity=False, for compatibility with FIPS systems. (#2270)
- SQL relating to Datasette's internal database now executes inside a transaction, avoiding a potential database locked error. (#2273)
/-/threadsdebug page now identifies the database in the name associated with each dedicated write thread. (#2265)
/db/-/createAPI now fires a
insert-rowsevent if rows were inserted after the table was created. (#2260)
datasette-enrichments-opencage 0.1.1 - Geocoding enrichment using OpenCage
- Run insert SQL queries inside a transaction. #5
14th February 2024
datasette-auth-tokens 0.4a8 - Datasette plugin for authenticating access using API tokens
- Fix for intermittent
database is lockedbug in managed tokens mode. #30
9th February 2024
8th February 2024
datasette-events-db 0.1a0 - Log Datasette events to a database table
- Initial plugin. Logs all events to a
7th February 2024
datasette 1.0a8 - An open source multi-tool for exploring and publishing data
This alpha release continues the migration of Datasette's configuration from
metadata.yaml to the new
- Plugin configuration now lives in the datasette.yaml configuration file, passed to Datasette using the
-c/--configoption. Thanks, Alex Garcia. (#2093)
bash datasette -c datasette.yamlWhere
datasette.yamlcontains configuration that looks like this:
yaml plugins: datasette-cluster-map: latitude_column: xlat longitude_column: xlon
Previously plugins were configured in
metadata.yaml, which was confusing as plugin settings were unrelated to database and table metadata.
The above YAML configuration example using
-s/--settinglooks like this:
bash datasette mydatabase.db\ -s plugins.datasette-cluster-map.latitude_column xlat\ -s plugins.datasette-cluster-map.longitude_column xlon- The new
/-/configpage shows the current instance configuration, after redacting keys that could contain sensitive data such as API keys or passwords. (#2254)
Existing Datasette installations may already have configuration set in
metadata.yamlthat should be migrated to
datasette.yaml. To avoid breaking these installations, Datasette will silently treat table configuration, plugin configuration and allow blocks in metadata as if they had been specified in configuration instead. (#2247) (#2248) (#2249)
Note that the
datasette publish command has not yet been updated to accept a
datasette.yaml configuration file. This will be addressed in #2195 but for the moment you can include those settings in
This provides two initial hooks, with more to come in the future:
makeAboveTablePanelConfigs() can add additional panels to the top of the table page.
makeColumnActions() can add additional actions to the column menu.
New jinja2_environment_from_request(datasette, request, env) plugin hook, which can be used to customize the current Jinja environment based on the incoming request. This can be used to modify the template lookup path based on the incoming request hostname, among other things. (#2225)
New family of template slot plugin hooks:
top_canned_query. Plugins can use these to provide additional HTML to be injected at the top of the corresponding pages. (#1191)
Plugins can register additional event classes using register_events(datasette).
They can then trigger those events with the datasette.track_event(event) internal method.
Plugins can subscribe to notifications of events using the track_event(datasette, event) plugin hook.
Datasette core now emits
delete-rowevents, documented here.
New internal function for plugin authors: await db.execute_isolated_fn(fn), for creating a new SQLite connection, executing code and then closing that connection, all while preventing other code from writing to that particular database. This connection will not have the prepare_connection() plugin hook executed against it, allowing plugins to perform actions that might otherwise be blocked by existing connection configuration. (#2218)
The configuration documentation now shows examples of both YAML and JSON for each setting.
Datasette no longer attempts to run SQL queries in parallel when rendering a table page, as this was leading to some rare crashing bugs. (#2189)
DeprecationWarning: pkg_resources is deprecated as an API(#2057)
Fixed bug where
?_extra=columnsparameter returned an incorrectly shaped response. (#2230)