An open source multi-tool for exploring and publishing data

PyPI Changelog Python 3.x License

Annotated version of this introductory video

Datasette is a tool for exploring and publishing data. It helps people take data of any shape or size, analyze and explore it, and publish it as an interactive website and accompanying API.

Datasette is aimed at data journalists, museum curators, archivists, local governments, scientists, researchers and anyone else who has data that they wish to share with the world. It is part of a wider ecosystem of tools and plugins dedicated to making working with structured data as productive as possible.

Try a demo and explore 33,000 power plants around the world, then take a look at some other examples of Datasette in action.

Then read how to get started with Datasette, subscribe to the monthly-ish newsletter and consider signing up for office hours for an in-person conversation about the project.

New: Datasette Desktop - a macOS desktop application for easily running Datasette on your own computer!

Exploratory data analysis

Import data from CSVs, JSON, database connections and more. Datasette will automatically show you patterns in your data and help you share your findings with your colleagues.

Instant data publishing

datasette publish lets you instantly publish your data to hosting providers like Google Cloud Run, Heroku or Vercel.

Rapid prototyping

Spin up a JSON API for any data in minutes. Use it to prototype and prove your ideas without building a custom backend.

Latest news

13th January 2022 #

Datasette 0.60 adds a new filters_from_request plugin hook, new internal methods for writing to the database, better performance and various faceting improvements. See also the annotated release notes.

5th December 2021 #

Observable notebooks recently added a SQL cell type, allowing SQL queries to be executed as part of an interactive notebook workflow. Alex Garcia built a Datasette Client for these which allows you to excute queries against any Datasette instance and explore and visualize the results using JavaScript code running in a notebook.

14th October 2021 #

Datasette 0.59 adds column descriptions in metadata, a new register_command plugin hook, enhanced --cors support and a bunch of other fixes and documentation improvements. See also the annotated release notes.

8th September 2021 #

Datasette Desktop is a new macOS desktop application version of Datasette, which supports opening SQLite files on your computer, importing CSV files and installing plugins. I wrote more about how it works in Datasette Desktop—a macOS desktop application for Datasette.

28th July 2021 #

The Baked Data architectural pattern describes a pattern commonly used with Datasette where the content for a site is bundled inside a SQLite database file and included alongside templates and application code in a deployment to a serverless hosting provider.

15th July 2021 #

Datasette 0.58 has new plugin hooks, a huge performance improvement for faceting, support for Unix domain sockets and several other improvements. Read the annotated release notes for extra background and context on the release.

5th June 2021 #

Datasette 0.57 is out with an important security patch plus a number of new features and bug fixes. Datasette 0.56.1, also out today, provides the security patch for users who are not yet ready to upgrade to the latest version.

10th May 2021 #

Django SQL Dashboard is a new tool that brings a useful authenticated subset of Datasette to Django projects that are built on top of PostgreSQL.

28th March 2021 #

Datasette 0.56 has bug fixes and documentation improvements, plus some new documented internal APIs for plugin authors and SpatiaLite 5 bundled with the official Datasette Docker container.

18th February 2021 #

Datasette 0.55 adds support for cross-database SQL queries. You can now run datasette --crossdb one.db two.db and then run queries that join data from tables in both of those database files - see cross-database queries in the documentation for more details.

sqlite-utils 3.6 adds similar features: a db.attach(alias, filepath) Python API method and --attach alias filepath.db command-line option, both for attaching additional databases in order to execute cross-database queries.

7th February 2021 #

This new Video introduction to Datasette and sqlite-utils provides a full introduction to both Datasette and sqlite-utils in 17 minutes, including a live demo of creating a database from a CSV file and publishing it to Google Cloud Run.

3rd February 2021 #

Serving map tiles from SQLite with MBTiles and datasette-tiles. datasette-tiles is a new plugin that adds a tile server to Datasette, serving map tiles from databases that conform to the MBTiles specification. download-tiles is a tool for building these databases, and datasette-basemap is a plugin that bundles a 22MB SQLite database with OpenStreetMap tiles covering zoom levels 0-6 for the entire world.

25th January 2021 #

Datasette 0.54 is out today. Highlights include the new _internal in-memory database exposing details of connected tables, plus support for JavaScript modules in plugins and add-on scripts. More commentary on this release is available in the annotated release notes.

24th January 2021 #

Drawing shapes on a map to query a SpatiaLite database introduces the new datasette-leaflet-freedraw plugin, which adds support for drawing shapes on a map to specify a GeoJSON MultiPolygon that can be used to query SpatiaLite databases.

7th January 2021 #

APIs from CSS without JavaScript: the datasette-css-properties plugin introduces datasette-css-properties, a highly experimental plugin that can output table rows and SQL query results as CSS stylesheets defining custom properties that can then be used to customize a static HTML page.

All news

Latest releases

21st January 2022

datasette 0.60.1

  • Fixed a bug where installation on Python 3.6 stopped working due to a change to an underlying dependency. This release can now be installed on Python 3.6, but is the last release of Datasette that will support anything less than Python 3.7. (#1609)

18th January 2022

s3-credentials 0.9

  • New --prefix myprefix/ option to s3-credentials create, which configures the credentials to only allow access to keys within the S3 bucket that start with the provided prefix. #12
  • s3-credentials policy --prefix myprefix/ command for generating and outputting a JSON policy that is restricted to the specified prefix. You can see examples in the README.
  • New list-bucket command for listing the contents of a specified bucket. #28
  • The list-users, list-buckets and list-bucket command all default to outputting an indented JSON array - previously the outputted indented JSON objects separated by newlines. The --nl option can be used to return newline-delimited single line JSON objects. The new --csv and --tsv options can be used to return CSV or TSV output. #48

14th January 2022

datasette-pretty-traces 0.4

datasette-leaflet-freedraw 0.3

  • Map filtering interface now displays on table pages for tables with a SpatiaLite geometry column. #7
  • Coordinates in JSON are now truncated to 5 decimal places. Thanks, Matthew Somerville. #6

datasette 0.60

Plugins and internals

Faceting

  • The number of unique values in a facet is now always displayed. Previously it was only displayed if the user specified ?_facet_size=max. (#1556)
  • Facets of type date or array can now be configured in metadata.json, see Facets in metadata.json. Thanks, David Larlet. (#1552)
  • New ?_nosuggest=1 parameter for table views, which disables facet suggestion. (#1557)
  • Fixed bug where ?_facet_array=tags&_facet=tags would only display one of the two selected facets. (#625)

Other small fixes

  • Made several performance improvements to the database schema introspection code that runs when Datasette first starts up. (#1555)
  • Label columns detected for foreign keys are now case-insensitive, so Name or TITLE will be detected in the same way as name or title. (#1544)
  • Upgraded Pluggy dependency to 1.0. (#1575)
  • Now using Plausible analytics for the Datasette documentation.
  • explain query plan is now allowed with varying amounts of whitespace in the query. (#1588)
  • New CLI reference page showing the output of --help for each of the datasette sub-commands. This lead to several small improvements to the help copy. (#1594)
  • Fixed bug where writable canned queries could not be used with custom templates. (#1547)
  • Improved fix for a bug where columns with a underscore prefix could result in unnecessary hidden form fields. (#1527)

12th January 2022

datasette-graphql 2.0.1

  • Fixes broken demo links in the README. #86

11th January 2022

sqlite-utils 3.22

  • New CLI reference documentation page, listing the output of --help for every one of the CLI commands. (#383)
  • sqlite-utils rows now has --limit and --offset options for paginating through data. (#381)
  • sqlite-utils rows now has --where and -p options for filtering the table using a WHERE query, see Returning all rows in a table. (#382)

sqlite-utils 3.21

CLI and Python library improvements to help run ANALYZE after creating indexes or inserting rows, to gain better performance from the SQLite query planner when it runs against indexes.

Three new CLI commands: create-database, analyze and bulk.

More details and examples can be found in the annotated release notes.

  • New sqlite-utils create-database command for creating new empty database files. (#348)
  • New Python methods for running ANALYZE against a database, table or index: db.analyze() and table.analyze(), see Optimizing index usage with ANALYZE. (#366)
  • New sqlite-utils analyze command for running ANALYZE using the CLI. (#379)
  • The create-index, insert and upsert commands now have a new --analyze option for running ANALYZE after the command has completed. (#379)
  • New sqlite-utils bulk command which can import records in the same way as sqlite-utils insert (from JSON, CSV or TSV) and use them to bulk execute a parametrized SQL query. (#375)
  • The CLI tool can now also be run using python -m sqlite_utils. (#368)
  • Using --fmt now implies --table, so you don't need to pass both options. (#374)
  • The --convert function applied to rows can now modify the row in place. (#371)
  • The insert-files command supports two new columns: stem and suffix. (#372)
  • The --nl import option now ignores blank lines in the input. (#376)
  • Fixed bug where streaming input to the insert command with --batch-size 1 would appear to only commit after several rows had been ingested, due to unnecessary input buffering. (#364)

6th January 2022

sqlite-utils 3.20

  • sqlite-utils insert ... --lines to insert the lines from a file into a table with a single line column, see Inserting unstructured data with --lines and --text.
  • sqlite-utils insert ... --text to insert the contents of the file into a table with a single text column and a single row.
  • sqlite-utils insert ... --convert allows a Python function to be provided that will be used to convert each row that is being inserted into the database. See Applying conversions while inserting data, including details on special behavior when combined with --lines and --text. (#356)
  • sqlite-utils convert now accepts a code value of - to read code from standard input. (#353)
  • sqlite-utils convert also now accepts code that defines a named convert(value) function, see Converting data in columns.
  • db.supports_strict property showing if the database connection supports SQLite strict tables.
  • table.strict property (see .strict) indicating if the table uses strict mode. (#344)
  • Fixed bug where sqlite-utils upsert ... --detect-types ignored the --detect-types option. (#362)

23rd December 2021

datasette-tiddlywiki 0.1

  • Initial release.

All releases