recent_releases (view)

• Fixed bug where foreign key labels were displayed as [object Object]. #26

• Better --help for sqlite-transform lambda. #7

• Fixed bug where tool could crash with an error about missing columns. #11

• The table.m2m() method now accepts an optional alter=True argument to specify that any missing columns should be added to the referenced table. See Working with many-to-many relationships. (#222)

• Upgraded tests for latest dependencies
• Switched CI to GitHub Actions

• Fixed a bug where .add_missing_columns() failed to take case insensitive column names into account. (#221)

• Observable export instructions are only shown if Datasette is run with the --cors option. #5
• Export notebook page links back to view the rows. #6
• CSV streaming export instructions are provided, if available. #7

Improved README with links to a live demo.

• .css pages now send the x-content-type-options: nosniff header, to protect against browsers incorrectly rendering the CSS as HTML which could be an XSS security hole. #1

• Improved README

First working version. #1

Initial prototype.

This release introduces a new mechanism for speeding up count(*) queries using cached table counts, stored in a _counts table and updated by triggers. This mechanism is described in Cached table counts using triggers, and can be enabled using Python API methods or the new enable-counts CLI command. (#212)

• table.enable_counts() method for enabling these triggers on a specific table.
• db.enable_counts() method for enabling triggers on every table in the database. (#213)
• New sqlite-utils enable-counts my.db command for enabling counts on all or specific tables, see Enabling cached counts. (#214)
• New sqlite-utils triggers command for listing the triggers defined for a database or specific tables, see Listing triggers. (#218)
• New db.use_counts_table property which, if True, causes table.count to read from the _counts table. (#215)
• table.has_counts_triggers property revealing if a table has been configured with the new _counts database triggers.
• db.reset_counts() method and sqlite-utils reset-counts command for resetting the values in the _counts table. (#219)
• The previously undocumented db.escape() method has been renamed to db.quote() and is now covered by the documentation: Quoting strings for use in SQL. (#217)
• New table.triggers_dict and db.triggers_dict introspection properties. (#211, #216)
• sqlite-utils insert now shows a more useful error message for invalid JSON. (#206)

• Fixed failing test caused by optimize sometimes creating larger database files. (#209)
• Documentation now lives on https://sqlite-utils.datasette.io/
• README now includes brew install sqlite-utils installation method.

• Fixed a bug with the datasette publish now alias for datasette publish vercel. #31

• Fix for error if search query contains hyphens such as github-to-sqlite. #31

• Now depends on sqlite-utils >= 3.0. #30
• The user's original search term is now passed to the display_sql SQL query as the :q parameter. #29

• --readme now stores null if the README file does not exist, rather than throwing an error. #57
• README HTML is rewritten to fix broken internal links, e.g. for a table of contents. #58

• New --setting option for specifying Datasette settings, as a working alternative to --extra-options. #30

• New command: sqlite-utils analyze-tables my.db outputs useful information about the table columns in the database, such as the number of distinct values and how many rows are null. See Analyzing tables for documentation. (#207)
• New table.analyze_column(column) Python method used by the analyze-tables command - see Analyzing a column.
• The table.update() method now correctly handles values that should be stored as JSON. Thanks, Andreas Madsack. (#204)

• Fixed various errors that occurred when using latest version of flyctl. #7

• Fixed bug where users with a "someusername_password_hash" value defined in metadata.json but no corresponding "actor" record were incorrectly signed in as an actor with "id": "username", when they should have been signed in as an actor with "id": "someusername". #14

Datasette has an official project website now, at https://datasette.io/. This release mainly updates the documentation to reflect the new site.

• New ?column__arraynotcontains= table filter. (#1132)
• datasette serve has a new --create option, which will create blank database files if they do not already exist rather than exiting with an error. (#1135)
• New ?_header=off option for CSV export which omits the CSV header row, documented here. (#1133)
• "Powered by Datasette" link in the footer now links to https://datasette.io/. (#1138)
• Project news no longer lives in the README - it can now be found at https://datasette.io/news. (#1137)

• Correctly parse column links. #3
• Show a message encouraging mobile users to rotate to landscape. #2

• Switched CI to GitHub Actions

• Fix for error caused by combining the _searchmode=raw and ?_search_COLUMN parameters. (#1134)

• Updated for latest sqlite-utils library
• Fixed bug where import failed if user did not provide a --pk option

• Chart rendering now takes place in a subprocess with a 5 second default time limit. Time limit can be customized using a new render_time_limit plugin configuration option. #4

• Show pysqlite3 version on /-/versions, if installed. (#1125)
• Errors output by Datasette (e.g. for invalid SQL queries) now go to stderr, not stdout. (#1131)
• Fix for a startup error on windows caused by unnecessary from os import EX_CANTCREAT - thanks, Abdussamet Koçak. (#1094)

• Form inputs are now wrapped in a paragraph, making form less cramped when displayed next to metadata. #12

• Fixed bug where static assets would 404 for Datasette installed on ARM Amazon Linux. (#1124)

• Generated columns from SQLite 3.31.0 or higher are now correctly displayed. (#1116)
• Error message if you attempt to open a SpatiaLite database now suggests using --load-extension=spatialite if it detects that the extension is available in a common location. (#1115)
• OPTIONS requests against the /database page no longer raise a 500 error. (#1100)
• Databases larger than 32MB that are published to Cloud Run can now be downloaded. (#749)
• Fix for misaligned cog icon on table and database pages. Thanks, Abdussamet Koçak. (#1121)

• Depend on datasette without version pinning.

• Fixed bug where in: and notin: filters raised an error when run against lists of integers rather than lists of strings. #68

• Suggested queries for tables (available from the table actions cog menu) now include foreign key expansions. #67

• README now links to example database tables for each command. #56
• Added documentation for github-to-sqlite releases.

• New command: github-to-sqlite workflows, for fetching the YAML configuration used by GitHub Actions workflows and using it to populate new workflows, jobs and steps tables. #54
• New command: github-to-sqlite pull-requests - thanks, @adamjonas! #48
• github-to-sqlite issues --issue option can now be used multiple times.

• Documentation on Testing plugins now recommends using datasette.client. (#1102)
• Fix bug where compound foreign keys produced broken links. (#1098)
• datasette --load-module=spatialite now also checks for /usr/local/lib/mod_spatialite.so. Thanks, Dan Peterson. (#1114)

This release includes a number of changes relating to an internal rebranding effort: Datasette's configuration mechanism (things like datasette --config default_page_size:10) has been renamed to settings.

• New --setting default_page_size 10 option as a replacement for --config default_page_size:10 (note the lack of a colon). The --config option is deprecated but will continue working until Datasette 1.0. (#992)
• The /-/config introspection page is now /-/settings, and the previous page redirects to the new one. (#1103)
• The config.json file in Configuration directory mode is now called settings.json. (#1104)
• The undocumented datasette.config() internal method has been replaced by a documented .setting(key) method. (#1107)

Also in this release:

• New plugin hook: database_actions(datasette, actor, database), which adds menu items to a new cog menu shown at the top of the database page. (#1077)
• datasette publish cloudrun has a new --apt-get-install option that can be used to install additional Ubuntu packages as part of the deployment. This is useful for deploying the new datasette-ripgrep plugin. (#1110)
• Swept the documentation to remove words that minimize involved difficulty. (#1089)

And some bug fixes:

• Foreign keys linking to rows with blank label columns now display as a hyphen, allowing those links to be clicked. (#1086)
• Fixed bug where row pages could sometimes 500 if the underlying queries exceeded a time limit. (#1088)
• Fixed a bug where the table action menu could appear partially obscured by the edge of the page. (#1084)

• Improved README with better tagline and better examples.

• Now adds a link to Datasette's global navigation menu. #16
• Code snippets in search scroll horizontally.

• Support for ?glob= option for adding -g file filter patterns. #9
• Improved configuration instructions. #15

• Show context around each match. #13
• Use :target CSS instead of JavaScript to highlight linked line. Thanks, Louis Lévêque. #14

• Option to ignore case. #12
• Option to treat input as a literal string, not a regular expression. #8

• Fixed a bug where patterns like -v could be interpreted as command-line arguments. #7
• Links to view files now correctly escape characters such as {.

• Search results now link to the full source code, with line numbers and highlights. #4
• Respect view-instance permission, so you can protect this using Datasette's authentication and permissions system. #6

• First non-alpha release
• Now implements both a time limit and a lines-returned limit. #3
• New time_limit and max_lines configuration options.
• Live demo at https://ripgrep.datasette.io/-/ripgrep

• Removed rogue print statement. #14

• Fixed bug where template was not packaged in the release. #1
• Fixed bug caused by call to proc.terminate()

• Initial alpha prototype

• Fix for forthcoming change to Datasette undocumented internal method. #66
• Now tested against Python 3.9

• Plugin now respects the view-instance and view-database permissions. Previously the plugin could expose schema details of databases that should not be visible, though not their actual row content. See security advisory: datasette-graphql leaks details of the schema of private database files . #65
• Adds links to the GraphQL API in the database and table action menus, including example table queries. #63
• Running the plugin against Datasette with no attached databases no longer returns a 500 error. #64

• Fixed 500 error with .copyable?_labels=on. #4

• Added loading indicators. #7
• URLs in search results are turned into links. #9
• Now uses datasette.urls to construct paths that work with the base_url config setting. #10
• Adds a link to Datasette navigation menu. #11

This release introduces a new sqlite-utils search command for searching tables, see Executing searches. (#192)

The table.search() method has been redesigned, see Searching with table.search(). (#197)

The release includes minor backwards-incompatible changes, hence the version bump to 3.0. Those changes, which should not affect most users, are:

• The -c shortcut option for outputting CSV is no longer available. The full --csv option is required instead.
• The -f shortcut for --fmt has also been removed - use --fmt.
• The table.search() method now defaults to sorting by relevance, not sorting by rowid. (#198)
• The table.search() method now returns a generator over a list of Python dictionaries. It previously returned a list of tuples.

Also in this release:

• The query, tables, rows and search CLI commands now accept a new --tsv option which outputs the results in TSV. (#193)
• A new table.virtual_table_using property reveals if a table is a virtual table, and returns the upper case type of virtual table (e.g. FTS4 or FTS5) if it is. It returns None if the table is not a virtual table. (#196)
• The new table.search_sql() method returns the SQL for searching a table, see Building SQL queries with table.search_sql().
• sqlite-utils rows now accepts multiple optional -c parameters specifying the columns to return. (#200)

Changes since the 3.0a0 alpha release:

• The sqlite-utils search command now defaults to returning every result, unless you add a --limit 20 option.
• The sqlite-utils search -c and table.search(columns=[]) options are now fully respected. (#201)

This release introduces a new sqlite-utils search command for searching tables, see Executing searches. (#192)

The table.search() method has been redesigned, see Searching with table.search(). (#197)

The release includes minor backwards-incompatible changes, hence the version bump to 3.0. Those changes, which should not affect most users, are:

• The -c shortcut option for outputting CSV is no longer available. The full --csv option is required instead.
• The -f shortcut for --fmt has also been removed - use --fmt.
• The table.search() method now defaults to sorting by relevance, not sorting by rowid. (#198)
• The table.search() method now returns a generator over a list of Python dictionaries. It previously returned a list of tuples.

Also in this release:

• The query, tables, rows and search CLI commands now accept a new --tsv option which outputs the results in TSV. (#193)
• A new table.virtual_table_using property reveals if a table is a virtual table, and returns the upper case type of virtual table (e.g. FTS4 or FTS5) if it is. It returns None if the table is not a virtual table. (#196)
• The new table.search_sql() method returns the SQL for searching a table, see Building SQL queries with table.search_sql().
• sqlite-utils rows now accepts multiple optional -c parameters specifying the columns to return. (#200)

• Upgraded dependencies, including Pandas. #73

• Improvements to the new Binary data documentation page.

• Link to interface from table actions menu. #12

• Adds "GraphQL API" link to Datasette's new navigation menu, added in Datasette 0.51. #62

• Registers a table action using the new table_actions() plugin hook in Datasette 0.51. #26

• Adds link to Datasette's new navigation menu, added in Datasette 0.51. #15

A new visual design, plugin hooks for adding navigation options, better handling of binary data, URL building utility methods and better support for running Datasette behind a proxy.

New visual design

Datasette is no longer white and grey with blue and purple links! Natalie Downe has been working on a visual refresh, the first iteration of which is included in this release. (#1056)

<img alt="datasette-0 51" src="https://user-images.githubusercontent.com/9599/97791267-d9011e80-1b8d-11eb-9b1b-42908266f36f.png">

Plugins can now add links within Datasette

A number of existing Datasette plugins add new pages to the Datasette interface, providig tools for things like uploading CSVs, editing table schemas or configuring full-text search.

Plugins like this can now link to themselves from other parts of Datasette interface. The menu_links(datasette, actor) hook (#1064) lets plugins add links to Datasette's new top-right application menu, and the table_actions(datasette, actor, database, table) hook (#1066) adds links to a new "table actions" menu on the table page.

The demo at latest.datasette.io now includes some example plugins. To see the new table actions menu first sign into that demo as root and then visit the facetable table to see the new cog icon menu at the top of the page.

Binary data

SQLite tables can contain binary data in BLOB columns. Datasette now provides links for users to download this data directly from Datasette, and uses those links to make binary data available from CSV exports. See Binary data for more details. (#1036 and #1034).

URL building

The new datasette.urls family of methods can be used to generate URLs to key pages within the Datasette interface, both within custom templates and Datasette plugins. See Building URLs within plugins for more details. (#904)

Running Datasette behind a proxy

The base_url configuration option is designed to help run Datasette on a specific path behind a proxy - for example if you want to run an instance of Datasette at /my-datasette/ within your existing site's URL hierarchy, proxied behind nginx or Apache.

Support for this configuration option has been greatly improved (#1023), and guidelines for using it are now available in a new documentation section on Running Datasette behind a proxy. (#1027)

Smaller changes

• Wide tables shown within Datasette now scroll horizontally (#998). This is achieved using a new <div class="table-wrapper"> element which may impact the implementation of some plugins (for example this change to datasette-cluster-map).
• New debug-menu permission. (#1068)
• Removed --debug option, which didn't do anything. (#814)
• Link: HTTP header pagination. (#1014)
• x button for clearing filters. (#1016)
• Edit SQL button on canned queries, (#1019)
• --load-extension=spatialite shortcut. (#1028)
• scale-in animation for column action menu. (#1039)
• Option to pass a list of templates to .render_template() is now documented. (#1045)
• New datasette.urls.static_plugins() method. (#1033)
• datasette -o option now opens the most relevant page. (#976)
• datasette --cors option now enables access to /database.db downloads. (#1057)
• Database file downloads now implement cascading permissions, so you can download a database if you have view-database-download permission even if you do not have permission to access the Datasette instance. (#1058)
• New documentation on Designing URLs for your plugin. (#1053)

• New load_template(template, request, datasette) plugin hook. (#1042)
• New debug-menu permission. (#1068)

• Registers a table action using the new table_actions() plugin hook. #26

• New colour scheme and improved visual design, courtesy of Natalie Downe. (#1056)
• scale-in animation for column action menu. (#1039)
• Wide tables now scroll horizontally. (#998)
• Option to pass a list of templates to .render_template() is now documented. (#1045)
• New datasette.urls.static_plugins() method. (#1033)
• BLOB column values can now be downloaded directly from the Datasette UI. (#1036)
• .csv exports now link to direct BLOB downloads. (#1034)
• datasette -o option now opens the most relevant page. (#976)
• datasette --cors option now enables access to /database.db downloads. (#1057)
• Database file downloads now implement cascading permissions, so you can download a database if you have view-database-download permission even if you do not have permission to access the Datasette instance. (#1058)
• New documentation on Designing URLs for your plugin. (#1053)
• New navigation menu plus a menu_links(datasette, actor) plugin hook to customize it. (#1064)
• table_actions(datasette, actor, database, table) plugin hook for the new table actions menu. (#1066)

• Now uses white-space: normal to fix a bug caused by Datasette's use of white-space: pre-wrap. #9

• table.m2m(other_table, records) method now takes any iterable, not just a list or tuple. Thanks, Adam Wolf. (#189)
• sqlite-utils insert now displays a progress bar for CSV or TSV imports. (#173)
• New @db.register_function(deterministic=True) option for registering deterministic SQLite functions in Python 3.8 or higher. (#191)

• Fixed a bug where tokens did not have spaces between them. #2

• Forwards compatibility with a change to Datasette's HTML scheduled for the next Datasette release. (Datasette #998)

• Tweaked dependencies for compatibility with new PyPI resolver.

• Tweaked dependencies for compatibility with new PyPI resolver.

• Fixed a bunch of issues relating to the base_url setting. (#1023)
• New datasette.urls URL builder for plugins, see Building URLs within plugins. (#904)
• Removed --debug option, which didn't do anything. (#814)
• Link: HTTP header pagination. (#1014)
• x button for clearing filters. (#1016)
• Edit SQL button on canned queries, (#1019)
• --load-extension=spatialite shortcut. (#1028)

• Default popup window row representation is now a definition list, not JSON. #20

• github-to-sqlite repos command now takes options --readme and --readme-html, which write the README or rendered HTML README into the readme or readme_html columns, respectively. #52
• New --accept option for github-to-sqlite get. #50

Initial alpha release.

• Facet by date. #16
• Now depends on Datasette 0.50.2
• Uses datasette.client and passes through authentication cookies. #28

• Fixed a bug where export files generated in languages other than English failed to import. #11
• CI now uses GitHub Actions

• New --encoding option for processing CSV and TSV files that use a non-utf-8 encoding, for both the insert and update commands. (#182)
• The --load-extension option is now available to many more commands. (#137)
• --load-extension=spatialite can be used to load SpatiaLite from common installation locations, if it is available. (#136)
• Tests now also run against Python 3.9. (#184)
• Passing pk=["id"] now has the same effect as passing pk="id". (#181)

• Fix for "table photos has no column named hasSticker" bug, thanks @mattiaborsoi. #10
• Switched CI to GitHub Actions.

• Notes now use an ID derived from a hash of their contents. #3
• Date columns are now in ISO format, and indexd. #4
• Notes and resources table now have full-text search configured. #4
• Improved how OCR is stored in the database, only storing the most confident matches. #6

• Initial release, with support for importing ENEX files.

• Fixed another bug introduced in 0.50 where column header links on the table page were broken. (#1011)

• Fixed a bug introduced in 0.50 where the export as JSON/CSV links on the table, row and query pages were broken. (#1010)

The key new feature in this release is the column actions menu on the table page (#891). This can be used to sort a column in ascending or descending order, facet data by that column or filter the table to just rows that have a value for that column.

Plugin authors can use the new datasette.client object to make internal HTTP requests from their plugins, allowing them to make use of Datasette's JSON API. (#943)

New Deploying Datasette documentation with guides for deploying Datasette on a Linux server using systemd or to hosting providers that support buildpacks. (#514, #997)

Other improvements in this release:

• Publishing to Google Cloud Run documentation now covers Google Cloud SDK options. Thanks, Geoffrey Hing. (#995)
• New datasette -o option which opens your browser as soon as Datasette starts up. (#970)
• Datasette now sets sqlite3.enable_callback_tracebacks(True) so that errors in custom SQL functions will display tracebacks. (#891)
• Fixed two rendering bugs with column headers in portrait mobile view. (#978, #980)
• New db.table_column_details(table) introspection method for retrieving full details of the columns in a specific table, see Database introspection.
• Fixed a routing bug with custom page wildcard templates. (#996)
• datasette publish heroku now deploys using Python 3.8.6.
• New datasette publish heroku --tar= option. (#969)
• OPTIONS requests against HTML pages no longer return a 500 error. (#1001)
• Datasette now supports Python 3.9.

• Column action menu now shows the column type. (#993)
• Column action sort links now correctly link to the first page of sorted results. (#989)
• Publishing to Google Cloud Run documentation now covers Google Cloud SDK options. Thanks, Geoffrey Hing. (#995)

• New column action menu - table columns now show a cog icon which provides a contextual menu for that column. (#981)
• New datasette -o option which opens your browser as soon as Datasette starts up. (#970
• sqlite3.enable_callback_tracebacks(True) so errors in custom SQL functions will now display tracebacks. (#891)
• Fixed two rendering bugs with column headers in portrait mobile view. (#978, #980)

• Fixed a bug where this could cause errors if other plugins provide custom templates. #19

• First non-alpha release.
• Ability to rename a table. #19
• Improved ability to add a column. #4

• Renamed to datasette-edit-schema. #20
• Now controlled by edit-schema permission. #9
• Improved /-/edit-schema/databasename page. #15
• Now ships with a copy of Draggable.js. #16
• Interface links back to table page. #18

• Fix for a bug where the map fails to display if the table is sorted by latitude or longitude. This fix depends on the upcoming release of Datasette (0.50). #18

• Fixed bug with case sensitivity, LATITUDE and LONGITUDE columns will now work. #17

• Accidental release, this does not change anything.

• Fix bug with editing column types. #17

• First prototype of transform table interface. #11

• Documentation showing how to use these functions in conjunction with json_each() to join against other tables. #2

• New dateutil_rrule() and dateutil_rrule_date() SQL functions. #1
• New dateutil_dates_between() SQL function. #1

• Tables that are currently importing now display a progress bar. #2

• First non-alpha release.
• Now checks for import-table permission, which by default is only granted to the root actor. #1
• Uses ?_size=max to import the maximum number of rows per batch, speeding up imports. #2

• Second attempt at fixing the template packaging issue.