recent_releases (view)

• Deployments that are configured to check organization membership now use read:org scope instead of the more permissive user scope. #73

• Fix ASGI lifespan bug. #1

• First working release

• Tool now also works against inputs that are JSON files consisting of a list of objects with identical keys, using --format=json. #12

• New sqlite-transform jsonsplit for converting foo,bar into ["foo", "bar"]. #10
• Added --dry-run option to sqlite-tranfsorm lambda for previewing the conversion that is about to be applied. #8

• New --sqlite database.db export option. #10

This release adds the ability to execute queries joining data from more than one database file - similar to the cross database querying feature introduced in Datasette 0.55.

• The db.attach(alias, filepath) Python method can be used to attach extra databases to the same connection, see db.attach() in the Python API documentation. (#113)
• The --attach option attaches extra aliased databases to run SQL queries against directly on the command-line, see attaching additional databases in the CLI documentation. (#236)

Support for cross-database SQL queries and built-in support for serving via HTTPS.

• The new --crossdb command-line option causes Datasette to attach up to ten database files to the same /_memory database connection. This enables cross-database SQL queries, including the ability to use joins and unions to combine data from tables that exist in different database files. See Cross-database queries for details. (#283)
• --ssl-keyfile and --ssl-certfile options can be used to specify a TLS certificate, allowing Datasette to serve traffic over https:// without needing to run it behind a separate proxy. (#1221)
• The /:memory: page has been renamed (and redirected) to /_memory for consistency with the new /_internal database introduced in Datasette 0.54. (#1205)
• Added plugin testing documentation on Using pdb for errors thrown inside Datasette. (#1207)
• The official Datasette Docker image now uses Python 3.7.10, applying the latest security fix for that Python version. (#1235)

• Fix broken tests against Datasette 0.54. #71
• Bundle JavaScript and CSS assets for GraphiQL, instead of loading them from a CDN. #73
• Pin to graphene dependency < 3.0. #72

• Fixed bug in released version caused by unnecessary attr import. #6

• sqlite-utils insert --sniff option for detecting the delimiter and quote character used by a CSV file, see Alternative delimiters and quote characters. (#230)
• The table.rows_where(), table.search() and table.search_sql() methods all now take optional offset= and limit= arguments. (#231)
• New --no-headers option for sqlite-utils insert --csv to handle CSV files that are missing the header row, see CSV files without a header row. (#228)
• Fixed bug where inserting data with extra columns in subsequent chunks would throw an error. Thanks @nieuwenhoven for the fix. (#234)
• Fixed bug importing CSV files with columns containing more than 128KB of data. (#229)
• Test suite now runs in CI against Ubuntu, macOS and Windows. Thanks @nieuwenhoven for the Windows test fixes. (#232)

• Explicitly open file using utf-8 encoding - thanks, RiverZhou. #10
• Fix import error if note has no "updated" element - thanks, Matej Korosec. #8

• Correctly handle entities in XHTML notes. #9
• Fixed a bug importing ENEX files on Windows. Thanks, dskrad. #11

• Fixed bug introduced in 0.6 which caused the non-JavaScript tile explorer to display incorrectly. #22

This release breaks backwards compatibility if you are using this as a tile server. Map tiles served by /-/tiles/... now use the OpenStreetMap coordinate system, where y=0 references the upper-most tile on the map. If you wish to continue using the TMS coordinate system you should switch to /-/tiles-tms/... instead. See Tile coordinate systems for more details.

• Support both OSM and TMS coordinate systems, use OSM by default. #15
• Hide non-JavaScript map as soon as the JavaScript loads in the tile explorer. #21

• Updated README examples to reflect function name changes in version 1.0. #3

• Fixed a code import bug that slipped in to 3.4. (#226)

Renamed two functions for compatibility with the latest Jellyfish - see their documentation for details.

• jaro_distance(a, b) is now jaro_similarity(a, b)
• jaro_winkler(a, b) is now jaro_winkler_similarity(a, b)

• sqlite-utils insert --csv now accepts optional --delimiter and --quotechar options. See Alternative delimiters and quote characters. (#223)

• Tile explorer interface now works without JavaScript! Thanks, Matthew Somerville. #14

• New /-/tiles-stack mechanism for serving tiles from multiple databases. #8
• "tiles-stack-order" plugin setting, for controlling the order in which tiles are served from the stack.
• Tile explorer page now also displays metadata for that tile database. #7
• Tile explorer uses name from metadata if it is available, and provides links to the database and table pages.
• Tile 404s are now served as blank PNGs. #9
• Renamed templates from mbtiles_*.html to tiles_*.html. #10

• --name= option for setting the name to be written into the metadata table. If you use the --city or --country options without passing the --name the city or country name will be used instead. #5
• --attribution=osm shortcut for setting the attribution to © OpenStreetMap contributors without having to type that out in full. #4

• New --attribution option for writing an attribution record to the metadata table. #3

• Attribution is now shown on the tile explorer page if it is included in the metadata table. #6

• Database now includes attribution in the metadata table. #3

• Fixed a bug where ?_search= and ?_sort= parameters were incorrectly duplicated when the filter form on the table page was re-submitted. (#1214)

• Table and database action menu items linking to the tile explorer. #4

• --city and --country options as an alternative to --bbox. #1

Initial working version.

• First non-alpha release.

• Improved README and live demo. #2

• Initial working version, with tile serving and tile explorer. #1

• Fixed bug with database bundling. #1

Initial release, bundling zoom levels 0-6. #1

• Upgraded datasette-leaflet dependency to get fix for marker icon bug. datasette-leaflet #6

• Fixed bug where Leaflet marker images were not bundled with the plugin. #6

• Updated README

• Pandas read_csv() streaming example now includes dtype= parameter with correct column types. #8

• Switched to using the register_routes() Datasette plugin hook. #2

• Updated for compatibility with latest Datasette. #3

• Now depends on datasette-leaflet. #28

• Now depends on datasette-leaflet. #28

• Now depends on datasette-leaflet for loading Leaflet JavaScript and CSS. #17

• Include source map in the bundle. #4

• Now depends on datasette-leaflet for loading Leaflet JavaScript and CSS. #3

• Expose URLs to bundled assets as JavaScript global variables datasette.leaflet.JAVASCRIPT_URL and datasette.leaflet.CSS_URL. #4

The two big new features in this release are the _internal SQLite in-memory database storing details of all connected databases and tables, and support for JavaScript modules in plugins and additional scripts.

For additional commentary on this release, see Datasette 0.54, the annotated release notes.

The _internal database

As part of ongoing work to help Datasette handle much larger numbers of connected databases and tables (see Datasette Library) Datasette now maintains an in-memory SQLite database with details of all of the attached databases, tables, columns, indexes and foreign keys. (#1150)

This will support future improvements such as a searchable, paginated homepage of all available tables.

You can explore an example of this database by signing in as root to the latest.datasette.io demo instance and then navigating to latest.datasette.io/_internal.

Plugins can use these tables to introspect attached data in an efficient way. Plugin authors should note that this is not yet considered a stable interface, so any plugins that use this may need to make changes prior to Datasette 1.0 if the _internal table schemas change.

Named in-memory database support

As part of the work building the _internal database, Datasette now supports named in-memory databases that can be shared across multiple connections. This allows plugins to create in-memory databases which will persist data for the lifetime of the Datasette server process. (#1151)

The new memory_name= parameter to the Database class can be used to create named, shared in-memory databases.

JavaScript modules

JavaScript modules were introduced in ECMAScript 2015 and provide native browser support for the import and export keywords.

To use modules, JavaScript needs to be included in <script> tags with a type="module" attribute.

Datasette now has the ability to output <script type="module"> in places where you may wish to take advantage of modules. The extra_js_urls option described in Custom CSS and JavaScript can now be used with modules, and module support is also available for the extra_body_script() plugin hook. (#1186, #1187)

datasette-leaflet-freedraw is the first example of a Datasette plugin that takes advantage of the new support for JavaScript modules. See Drawing shapes on a map to query a SpatiaLite database for more on this plugin.

Code formatting with Black and Prettier

Datasette adopted Black for opinionated Python code formatting in June 2019. Datasette now also embraces Prettier for JavaScript formatting, which like Black is enforced by tests in continuous integration. Instructions for using these two tools can be found in the new section on Code formatting in the contributors documentation. (#1167)

Other changes

• Datasette can now open multiple database files with the same name, e.g. if you run datasette path/to/one.db path/to/other/one.db. (#509)
• datasette publish cloudrun now sets force_https_urls for every deployment, fixing some incorrect http:// links. (#1178)
• Fixed a bug in the example nginx configuration in Running Datasette behind a proxy. (#1091)
• The Datasette Ecosystem documentation page has been reduced in size in favour of the datasette.io tools and plugins directories. (#1182)
• The request object now provides a request.full_path property, which returns the path including any query string. (#1184)
• Better error message for disallowed PRAGMA clauses in SQL queries. (#1185)
• datasette publish heroku now deploys using python-3.8.7.
• New plugin testing documentation on Testing outbound HTTP calls with pytest-httpx. (#1198)
• All ?_* query string parameters passed to the table page are now persisted in hidden form fields, so parameters such as ?_size=10 will be correctly passed to the next page when query filters are changed. (#1194)
• Fixed a bug loading a database file called test-database (1).sqlite. (#1181)

• Initial release, bundling Leaflet 1.7.1 as static assets that can be loaded as JavaScript modules.

• README now includes a demo. #2
• Note that this currently requires the Datasette 0.54a0 alpha.

This plugin now builds on the Authentication and permissions mechanism introduced in Datasette 0.44. This means it can be used in combination with other authentication plugins such as datasette-auth-tokens. #62

When a user signs in with GitHub, a signed ds_actor cookie is set with details of their GitHub profile. Fields from this dictionary can be used with Datasette allow blocks to control access to individual databases, tables and queries.

The new load_orgs and load_teams options can be used to check the user's membership of different organization and teams when they first sign in. These memberships will be encoded in the cookie and can be used to control access.

Other changes:

• The ASGI middleware version of this plugin has been split off into a separate asgi-auth-github package. #63
• Error messages are now rendered using a datasette_auth_github_error.html template, which can be customized by including a template of the same name in a custom templates directory. #36
• The /-/auth-callback path is now /-/github-auth-callback - you will need to update this in your GitHub OAuth application settings. #66

• Handle cases where columns may have null values. Thanks, Jose Rodriguez-Salinas. #27
• Now depends on Datasette >= 0.50

• Renamed /-/auth-callback to /-/github-auth-callback. #66

• Removed custom base template - Datasette provides a "logout" link in the default interface now.
• Added "Sign in with GitHub" menu action. #65
• Use datasette_datasette_auth_github_error.html template for error messages. #36

Alpha release. This is a big change - it switches the plugin over to using the authentication plugin mechanism introduced in Datasette 0.44.

• Plugin now only shows maps for the first ten rows, with a "Click to view map" prompt for rows past that point. This can be customized using the new default_maps_to_load plugin setting. #16

• Fixed bug where polygons drawn at a high zoom level were not correctly re-displayed after submitting the query. #1

• Bundle the CSS as well

• Added some static files that I missed from the packaging.

• Initial working release.

Alpha release. Release notes in progress.

• Improved support for named in-memory databases. (#1151)
• New _internal in-memory database tracking attached databases, tables and columns. (#1150)
• Support for JavaScript modules. (#1186, #1187)

• Fixed bug where foreign key labels were displayed as [object Object]. #26

• Better --help for sqlite-transform lambda. #7

• Fixed bug where tool could crash with an error about missing columns. #11

• The table.m2m() method now accepts an optional alter=True argument to specify that any missing columns should be added to the referenced table. See Working with many-to-many relationships. (#222)

• Upgraded tests for latest dependencies
• Switched CI to GitHub Actions

• Fixed a bug where .add_missing_columns() failed to take case insensitive column names into account. (#221)

• Observable export instructions are only shown if Datasette is run with the --cors option. #5
• Export notebook page links back to view the rows. #6
• CSV streaming export instructions are provided, if available. #7

Improved README with links to a live demo.

• .css pages now send the x-content-type-options: nosniff header, to protect against browsers incorrectly rendering the CSS as HTML which could be an XSS security hole. #1

• Improved README

First working version. #1

Initial prototype.

This release introduces a new mechanism for speeding up count(*) queries using cached table counts, stored in a _counts table and updated by triggers. This mechanism is described in Cached table counts using triggers, and can be enabled using Python API methods or the new enable-counts CLI command. (#212)

• table.enable_counts() method for enabling these triggers on a specific table.
• db.enable_counts() method for enabling triggers on every table in the database. (#213)
• New sqlite-utils enable-counts my.db command for enabling counts on all or specific tables, see Enabling cached counts. (#214)
• New sqlite-utils triggers command for listing the triggers defined for a database or specific tables, see Listing triggers. (#218)
• New db.use_counts_table property which, if True, causes table.count to read from the _counts table. (#215)
• table.has_counts_triggers property revealing if a table has been configured with the new _counts database triggers.
• db.reset_counts() method and sqlite-utils reset-counts command for resetting the values in the _counts table. (#219)
• The previously undocumented db.escape() method has been renamed to db.quote() and is now covered by the documentation: Quoting strings for use in SQL. (#217)
• New table.triggers_dict and db.triggers_dict introspection properties. (#211, #216)
• sqlite-utils insert now shows a more useful error message for invalid JSON. (#206)

• Fixed failing test caused by optimize sometimes creating larger database files. (#209)
• Documentation now lives on https://sqlite-utils.datasette.io/
• README now includes brew install sqlite-utils installation method.

• Fixed a bug with the datasette publish now alias for datasette publish vercel. #31

• Fix for error if search query contains hyphens such as github-to-sqlite. #31

• Now depends on sqlite-utils >= 3.0. #30
• The user's original search term is now passed to the display_sql SQL query as the :q parameter. #29

• --readme now stores null if the README file does not exist, rather than throwing an error. #57
• README HTML is rewritten to fix broken internal links, e.g. for a table of contents. #58

• New --setting option for specifying Datasette settings, as a working alternative to --extra-options. #30

• New command: sqlite-utils analyze-tables my.db outputs useful information about the table columns in the database, such as the number of distinct values and how many rows are null. See Analyzing tables for documentation. (#207)
• New table.analyze_column(column) Python method used by the analyze-tables command - see Analyzing a column.
• The table.update() method now correctly handles values that should be stored as JSON. Thanks, Andreas Madsack. (#204)

• Fixed various errors that occurred when using latest version of flyctl. #7

• Fixed bug where users with a "someusername_password_hash" value defined in metadata.json but no corresponding "actor" record were incorrectly signed in as an actor with "id": "username", when they should have been signed in as an actor with "id": "someusername". #14

Datasette has an official project website now, at https://datasette.io/. This release mainly updates the documentation to reflect the new site.

• New ?column__arraynotcontains= table filter. (#1132)
• datasette serve has a new --create option, which will create blank database files if they do not already exist rather than exiting with an error. (#1135)
• New ?_header=off option for CSV export which omits the CSV header row, documented here. (#1133)
• "Powered by Datasette" link in the footer now links to https://datasette.io/. (#1138)
• Project news no longer lives in the README - it can now be found at https://datasette.io/news. (#1137)

• Correctly parse column links. #3
• Show a message encouraging mobile users to rotate to landscape. #2

• Switched CI to GitHub Actions

• Fix for error caused by combining the _searchmode=raw and ?_search_COLUMN parameters. (#1134)

• Updated for latest sqlite-utils library
• Fixed bug where import failed if user did not provide a --pk option

• Chart rendering now takes place in a subprocess with a 5 second default time limit. Time limit can be customized using a new render_time_limit plugin configuration option. #4

• Show pysqlite3 version on /-/versions, if installed. (#1125)
• Errors output by Datasette (e.g. for invalid SQL queries) now go to stderr, not stdout. (#1131)
• Fix for a startup error on windows caused by unnecessary from os import EX_CANTCREAT - thanks, Abdussamet Koçak. (#1094)

• Form inputs are now wrapped in a paragraph, making form less cramped when displayed next to metadata. #12

• Fixed bug where static assets would 404 for Datasette installed on ARM Amazon Linux. (#1124)

• Generated columns from SQLite 3.31.0 or higher are now correctly displayed. (#1116)
• Error message if you attempt to open a SpatiaLite database now suggests using --load-extension=spatialite if it detects that the extension is available in a common location. (#1115)
• OPTIONS requests against the /database page no longer raise a 500 error. (#1100)
• Databases larger than 32MB that are published to Cloud Run can now be downloaded. (#749)
• Fix for misaligned cog icon on table and database pages. Thanks, Abdussamet Koçak. (#1121)

• Depend on datasette without version pinning.

• Fixed bug where in: and notin: filters raised an error when run against lists of integers rather than lists of strings. #68

• Suggested queries for tables (available from the table actions cog menu) now include foreign key expansions. #67

• README now links to example database tables for each command. #56
• Added documentation for github-to-sqlite releases.

• New command: github-to-sqlite workflows, for fetching the YAML configuration used by GitHub Actions workflows and using it to populate new workflows, jobs and steps tables. #54
• New command: github-to-sqlite pull-requests - thanks, @adamjonas! #48
• github-to-sqlite issues --issue option can now be used multiple times.

• Documentation on Testing plugins now recommends using datasette.client. (#1102)
• Fix bug where compound foreign keys produced broken links. (#1098)
• datasette --load-module=spatialite now also checks for /usr/local/lib/mod_spatialite.so. Thanks, Dan Peterson. (#1114)

This release includes a number of changes relating to an internal rebranding effort: Datasette's configuration mechanism (things like datasette --config default_page_size:10) has been renamed to settings.

• New --setting default_page_size 10 option as a replacement for --config default_page_size:10 (note the lack of a colon). The --config option is deprecated but will continue working until Datasette 1.0. (#992)
• The /-/config introspection page is now /-/settings, and the previous page redirects to the new one. (#1103)
• The config.json file in Configuration directory mode is now called settings.json. (#1104)
• The undocumented datasette.config() internal method has been replaced by a documented .setting(key) method. (#1107)

Also in this release:

• New plugin hook: database_actions(datasette, actor, database), which adds menu items to a new cog menu shown at the top of the database page. (#1077)
• datasette publish cloudrun has a new --apt-get-install option that can be used to install additional Ubuntu packages as part of the deployment. This is useful for deploying the new datasette-ripgrep plugin. (#1110)
• Swept the documentation to remove words that minimize involved difficulty. (#1089)

And some bug fixes:

• Foreign keys linking to rows with blank label columns now display as a hyphen, allowing those links to be clicked. (#1086)
• Fixed bug where row pages could sometimes 500 if the underlying queries exceeded a time limit. (#1088)
• Fixed a bug where the table action menu could appear partially obscured by the edge of the page. (#1084)

• Improved README with better tagline and better examples.