Custom SQL query returning 101 rows (hide)

140912432 2021-06-19

This release introduces the sqlite-utils memory command, which can be used to load CSV or JSON data into a temporary in-memory database and run SQL queries (including joins across multiple files) directly against that data.

Also new: sqlite-utils insert --detect-types, sqlite-utils dump, table.use_rowid plus some smaller fixes.

sqlite-utils memory

This example of sqlite-utils memory retrieves information about the all of the repositories in the Dogsheep organization on GitHub using this JSON API, sorts them by their number of stars and outputs a table of the top five (using -t): ``` $ curl -s ''\ | sqlite-utils memory - ' select full_name, forks_count, stargazers_count from stdin order by stargazers_count desc limit 5 ' -t full_name forks_count stargazers_count


dogsheep/twitter-to-sqlite 12 225 dogsheep/github-to-sqlite 14 139 dogsheep/dogsheep-photos 5 116 dogsheep/ 7 90 dogsheep/healthkit-to-sqlite 4 85 The tool works against files on disk as well. This example joins data from two CSV files: $ cat creatures.csv species_id,name 1,Cleo 2,Bants 2,Dori 2,Azi $ cat species.csv id,species_name 1,Dog 2,Chicken $ sqlite-utils memory species.csv creatures.csv ' select * from creatures join species on creatures.species_id = ' [{"species_id": 1, "name": "Cleo", "id": 1, "species_name": "Dog"}, {"species_id": 2, "name": "Bants", "id": 2, "species_name": "Chicken"}, {"species_id": 2, "name": "Dori", "id": 2, "species_name": "Chicken"}, {"species_id": 2, "name": "Azi", "id": 2, "species_name": "Chicken"}] `` Here thespecies.csvfile becomes thespeciestable, thecreatures.csvfile becomes thecreatures` table and the output is JSON, the default output format.

You can also use the --attach option to attach existing SQLite database files to the in-memory database, in order to join data from CSV or JSON directly against your existing tables.

Full documentation of this new feature is available in Querying data directly using an in-memory database. (#272)

sqlite-utils insert --detect-types

The sqlite-utils insert command can be used to insert data from JSON, CSV or TSV files into a SQLite database file. The new --detect-types option (shortcut -d), when used in conjunction with a CSV or TSV import, will automatically detect if columns in the file are integers or floating point numbers as opposed to treating everything as a text column and create the new table with the corresponding schema. See Inserting CSV or TSV data for details. (#282)

Other changes

  • Bug fix: table.transform(), when run against a table without explicit primary keys, would incorrectly create a new version of the table with an explicit primary key column called rowid. (#284)
  • New table.use_rowid introspection property, see .use_rowid. (#285)
  • The new sqlite-utils dump file.db command outputs a SQL dump that can be used to recreate a database. (#274)
  • -h now works as a shortcut for --help, thanks Loren McIntyre. (#276)
  • Now using pytest-cov and Codecov to track test coverage - currently at 96%. (#275)
  • SQL errors that occur when using sqlite-utils query are now displayed as CLI errors.
2021-06-19T16:13:11Z ["python", "cli", "sqlite", "sqlite-database", "click", "datasette", "datasette-io", "datasette-tool"]
197431109 2021-06-13
  • Fixed bug where searches consisting only of whitespace caused a server error. #33
2021-06-13T14:38:09Z ["search", "datasette", "datasette-plugin", "dogsheep", "datasette-io", "datasette-tool"]
175550127 2021-06-13
  • New feature: yaml-to-sqlite dogs.db dogs.yaml --single-column=name can be used to import YAML files consisting of a list of items into a single table with a single primary key column. #5
  • Importing YAML files with new columns will now alter the table to add those columns. #4
2021-06-13T04:45:52Z ["yaml", "sqlite", "datasette-io", "datasette-tool"]
167759846 2021-06-13
  • BREAKING CHANGE: The option order for this command is now markdown-to-sqlite database.db tablename - prior to version 1.0 the order was markdown-to-sqlite database.db tablename. This change was made for improved consistency with other x-to-sqlite tools.
2021-06-13T04:17:15Z ["markdown", "yaml", "sqlite", "datasette-io", "datasette-tool"]
140912432 2021-06-13
  • Fixed bug when using table.upsert_all() to create a table with only a single column that is treated as the primary key. (#271)
2021-06-13T03:00:45Z ["python", "cli", "sqlite", "sqlite-database", "click", "datasette", "datasette-io", "datasette-tool"]
374846311 2021-06-13
  • Spatialite's geometry blob is now automatically parsed into GeoJSON, so the same query will output GeoJSON with and without Spatialite.
2021-06-13T00:04:19Z []
140912432 2021-06-12
2021-06-12T02:08:03Z ["python", "cli", "sqlite", "sqlite-database", "click", "datasette", "datasette-io", "datasette-tool"]
166159072 2021-06-11
  • New --postgres-schema option for specifying a different PostgreSQL schema, thanks Jeremy Dormitzer. #29
  • Correctly handles table names that need escaping, such as user in PostgreSQL. #25
  • Empty tables are now created in the SQLite database with a matching schema, rather than being skipped. #28
  • Connection strings starting with postgres:// are treated as if they started with postgresql://. #34
2021-06-11T22:55:56Z ["sqlalchemy", "sqlite", "datasette", "datasette-io", "datasette-tool"]
242260583 2021-06-10
  • No longer pins a specific version of Datasette.
2021-06-10T19:51:02Z ["datasette", "datasette-plugin", "datasette-io"]
107914493 2021-06-10
2021-06-10T04:52:45Z ["python", "docker", "json", "csv", "sql", "sqlite", "datasets", "asgi", "automatic-api", "datasette", "datasette-io"]
375546675 2021-06-10
  • First working version
2021-06-10T02:32:42Z ["datasette", "datasette-plugin", "datasette-io", "placekey"]
374846311 2021-06-09
2021-06-09T18:56:21Z []
374846311 2021-06-09
2021-06-09T18:09:41Z []
107914493 2021-06-08
  • Fixed visual display glitch with global navigation menu. (#1367)
  • No longer truncates the list of table columns displayed on the /database page. (#1364)
2021-06-08T16:28:31Z ["python", "docker", "json", "csv", "sql", "sqlite", "datasets", "asgi", "automatic-api", "datasette", "datasette-io"]
234825790 2021-06-07
  • Fixed bug using this plugin with Datasette 0.54 or higher. #19
2021-06-07T16:41:25Z ["csvs", "datasette", "datasette-plugin", "datasette-io"]
347263722 2021-06-06
  • This release includes a small potentially backwards-incompatible change: the description field for a saved dashboard is now treated as Markdown and rendered as such when the saved dashboard is displayed. It is very unlikely that this will affect any of your existing dashboards but you should still check before applying the upgrade. Thanks, Atul Varma. #115
2021-06-06T19:12:45Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
279357123 2021-06-05
2021-06-05T23:39:15Z ["datasette", "datasette-plugin", "datasette-io"]
107914493 2021-06-05

Warning: This release fixes a reflected cross-site scripting security hole with the ?_trace=1 feature. You should upgrade to this version, or to Datasette 0.56.1, as soon as possible. (#1360)

In addition to the security fix, this release includes ?_col= and ?_nocol= options for controlling which columns are displayed for a table, ?_facet_size= for increasing the number of facet results returned, re-display of your SQL query should an error occur and numerous bug fixes.

New features

  • If an error occurs while executing a user-provided SQL query, that query is now re-displayed in an editable form along with the error message. (#619)
  • New ?_col= and ?_nocol= parameters to show and hide columns in a table, plus an interface for hiding and showing columns in the column cog menu. (#615)
  • A new ?_facet_size= parameter for customizing the number of facet results returned on a table or view page. (#1332)
  • ?_facet_size=max sets that to the maximum, which defaults to 1,000 and is controlled by the the max_returned_rows setting. If facet results are truncated the ... at the bottom of the facet list now links to this parameter. (#1337)
  • ?_nofacet=1 option to disable all facet calculations on a page, used as a performance optimization for CSV exports and ?_shape=array/object. (#1349, #263)
  • ?_nocount=1 option to disable full query result counts. (#1353)
  • ?_trace=1 debugging option is now controlled by the new trace_debug setting, which is turned off by default. (#1359)

Bug fixes and other improvements

  • Custom pages now work correctly when combined with the base_url setting. (#1238)
  • Fixed intermittent error displaying the index page when the user did not have permission to access one of the tables. Thanks, Guy Freeman. (#1305)
  • Columns with the name "Link" are no longer incorrectly displayed in bold. (#1308)
  • Fixed error caused by tables with a single quote in their names. (#1257)
  • Updated dependencies: pytest-asyncio, Black, jinja2, aiofiles, click, and itsdangerous.
  • The official Datasette Docker image now supports apt-get install. (#1320)
  • The Heroku runtime used by datasette publish heroku is now python-3.8.10.
2021-06-05T22:11:18Z ["python", "docker", "json", "csv", "sql", "sqlite", "datasets", "asgi", "automatic-api", "datasette", "datasette-io"]
107914493 2021-06-05
2021-06-05T22:00:01Z ["python", "docker", "json", "csv", "sql", "sqlite", "datasets", "asgi", "automatic-api", "datasette", "datasette-io"]
347263722 2021-06-03
  • Fixed 500 error caused when markdown widget attempted to render a null value. #124
2021-06-03T16:27:35Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
140912432 2021-06-03
  • New sqlite-utils indexes command to list indexes in a database, see Listing indexes. (#263)
  • table.xindexes introspection property returning more details about that table's indexes, see .xindexes. (#261)
2021-06-03T05:17:33Z ["python", "cli", "sqlite", "sqlite-database", "click", "datasette", "datasette-io", "datasette-tool"]
140912432 2021-05-29
  • New table.pks_and_rows_where() method returning (primary_key, row_dictionary) tuples - see Listing rows with their primary keys. (#240)
  • Fixed bug with table.add_foreign_key() against columns containing spaces. (#238)
  • table_or_view.drop(ignore=True) option for avoiding errors if the table or view does not exist. (#237)
  • sqlite-utils drop-view --ignore and sqlite-utils drop-table --ignore options. (#237)
  • Fixed a bug with inserts of nested JSON containing non-ascii strings - thanks, Dylan Wu. (#257)
  • Suggest --alter if an error occurs caused by a missing column. (#259)
  • Support creating indexes with columns in descending order, see API documentation and CLI documentation. (#260)
  • Correctly handle CSV files that start with a UTF-8 BOM. (#250)
2021-05-29T05:49:59Z ["python", "cli", "sqlite", "sqlite-database", "click", "datasette", "datasette-io", "datasette-tool"]
107914493 2021-05-27
  • Docker image now allows apt-get install. #1320
  • ?_col= and ?_nocol= options to show and hide columns in a table, plus UI for doing this in the column cog menu. #615
  • New ?_facet_size=max option, linked to from the … on truncated facets. #1337
  • Heroku runtime used by datasette publish heroku is now python-3.8.10.
  • Multi-arch support for built Docker images using Buildx - thanks, Blair Drummand. #1319
2021-05-27T16:58:41Z ["python", "docker", "json", "csv", "sql", "sqlite", "datasets", "asgi", "automatic-api", "datasette", "datasette-io"]
347263722 2021-05-25
2021-05-25T00:34:05Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
107914493 2021-05-23

Mainly dependency bumps, plus a new ?_facet_size= argument.

  • Updated dependencies: pytest-asyncio, Black, jinja2, aiofiles, itsdangerous
  • Fixed bug where columns called "Link" were incorrectly displayed in bold. (#1308)
  • New ?_facet_size= argument for customizing the number of facet results returned on a page. (#1332)
2021-05-23T00:46:44Z ["python", "docker", "json", "csv", "sql", "sqlite", "datasets", "asgi", "automatic-api", "datasette", "datasette-io"]
237321267 2021-05-20
  • Fixed bug where the id column did not come first in the table schema. #20
2021-05-20T19:15:44Z ["geojson", "sqlite", "gis", "datasette-io", "datasette-tool"]
237321267 2021-05-17
  • New --nl option for consuming newline-delimited GeoJSON - thanks, Chris Amico! #13
  • New --spatial-index option which creates the table using SpatiaLite and adds a spatial index for the geometry column. #17
2021-05-17T00:14:27Z ["geojson", "sqlite", "gis", "datasette-io", "datasette-tool"]
347263722 2021-05-16
  • Fixed a security and permissions flaw, where users without the execute_sql permission could still run custom queries by editing saved dashboards using the Django admin interface. #94
  • Bar charts now preserve the order in which the data is returned by the query. #106
  • Example select statements now include explicit columns. #105
  • Columns on the dashboard page now respond to media queries, collapsing to a single column on narrow or mobile browser windows. #106
  • Fixed hard-coded /dashboard/ URL, thanks Flávio Juvenal da Silva Junior. #99
  • Fixed bug where ?_save- parameters could be accidentally reflected in the query string. #104
  • Explicitly require at least Django 3.0. #101
  • Fixed a warning about AutoField migrations with Django 3.2. #103
  • Fixed a bug where users lacking permissions could end up in an infinite redirect. #30
  • Configuration and security documentation now recommends using a read-only database replica as the most robust option. #95
  • Added screenshots and demo links for all of the included widgets. #96
2021-05-16T20:24:32Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
347263722 2021-05-10
  • New word cloud widget displayed when queries return wordcount_word and wordcount_count columns. #91
  • All pages are now served with cache-control: private header if the user is logged in. #92
  • Much improved README, including a detailed list of features. #40
2021-05-10T15:47:08Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
347263722 2021-05-09
  • Fixed bug where saved dashboards relating to groups could be displayed multiple times. #90
  • Removed duplicate "Run queries" button. #89
  • HTML page titles now include named parameter values, if available. #88
2021-05-09T06:40:54Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
347263722 2021-05-09
2021-05-09T05:47:24Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
347263722 2021-05-09
  • Improved and updated documentation for saved dashboards. #84
  • Added "Remove all queries" link for resetting interactive dashboard. #85
2021-05-09T04:41:38Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
347263722 2021-05-09

First non-alpha release! Django SQL Dashboard is now ready for people to use against their production Django applications.

  • Saved dashboards can now be created from the interactive dashboard page. #44
  • New progress bar widget for queries that return numeric columns total_count and completed_count. #77
  • The list of available tables now better reflects your current permissions, and shows the columns for each listed table. #79, #80
  • The dashboard index page now lists saved dashboards that the user is able to view or edit. #81
  • "Edit" link for dashboards links to the Django Admin, which now respects the edit policy set for a dashboard. #44
  • New documentation section covering security. #6
  • Show row count on non-truncated results. #76
  • More robust extraction of named parameteters from queries, fixing some errer-cases. #75
  • Custom widgets can now extend a django_sql_dashboard/widgets/_base_widget.html base template. #78
  • Fixed a bug caused by errors on saved dashboard pages being displayed as editable text. #74
  • Unlisted public dashboards now include a robots tag to avoid being indexed by search engines. #42
2021-05-09T04:30:09Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
347263722 2021-04-26
  • "Copy to clipboard" button on pretty-printed JSON results. #72
  • Saved dashboards no longer display broken export buttons. #71
2021-04-26T18:50:46Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
347263722 2021-04-25
  • Fixed bug where large SQL queries submitted by POST did not correctly evaluate any named parameters. #58
2021-04-25T23:24:58Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
347263722 2021-04-25
  • Handle SQL queries too long for a GET. #58
  • Export all results as CSV/TSV feature. #50
2021-04-25T23:08:38Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
347263722 2021-04-25
  • Use separate styles for pre.sql and pre.json
2021-04-25T17:31:00Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
347263722 2021-04-25
  • Do not show "[count]" links for unambiguous columns. #69
  • Better HTML titles for pages, for better browser history. #70
  • Pretty-print cells containing JSON. #62
2021-04-25T17:05:06Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
347263722 2021-04-14
  • Correctly sign ?sql= on redirects created using DASHBOARD_UPGRADE_OLD_BASE64_LINKS. #68
2021-04-14T06:22:53Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
347263722 2021-04-14
  • Fixed bug with DASHBOARD_UPGRADE_OLD_BASE64_LINKS setting. #67
2021-04-14T06:09:03Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
347263722 2021-04-14
  • Make it easy to provide a custom base template. #7
  • Content-Security-Policy: frame-ancestors header. #64
  • Signing no longer uses base64/json. #45
2021-04-14T05:55:20Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
347263722 2021-04-12
  • SQL textarea now always has at least 5 rows
  • Always strip trailing ; from SQL, fixing bug with "count" links. #56
  • URLize links in results. #61
2021-04-12T23:17:12Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
291339086 2021-04-09
  • Fixed bug with new --http-read-timeout feature
2021-04-09T23:13:21Z ["yaml", "airtable", "airtable-api", "datasette-io", "datasette-tool"]
347263722 2021-04-09
  • Fix for bug with columns with identical names. #52
  • DASHBOARD_ROW_LIMIT setting. #53
  • Size SQL textarea to fit SQL. #51
2021-04-09T04:44:11Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
291339086 2021-04-08
  • New --http-read-timeout option, thanks Eli Block! #14
2021-04-08T22:18:29Z ["yaml", "airtable", "airtable-api", "datasette-io", "datasette-tool"]
291339086 2021-04-02
  • New --user-agent= option for passing a custom user-agent to Airtable. #12
2021-04-02T05:42:52Z ["yaml", "airtable", "airtable-api", "datasette-io", "datasette-tool"]
107914493 2021-03-29

Documentation improvements, bug fixes and support for SpatiaLite 5.

  • The SQL editor can now be resized by dragging a handle. (#1236)
  • Fixed a bug with JSON faceting and the __arraycontains filter caused by tables with spaces in their names. (#1239)
  • Upgraded httpx dependency. (#1005)
  • JSON faceting is now suggested even if a column contains blank strings. (#1246)
  • New datasette.add_memory_database() method. (#1247)
  • The Response.asgi_send() method is now documented. (#1266)
  • The official Datasette Docker image now bundles SpatiaLite version 5. (#1278)
  • Fixed a no such table: pragma_database_list bug when running Datasette against SQLite versions prior to SQLite 3.16.0. (#1276)
  • HTML lists displayed in table cells are now styled correctly. Thanks, Bob Whitelock. (#1141, #1252)
  • Configuration directory mode now correctly serves immutable databases that are listed in inspect-data.json. Thanks Campbell Allen and Frankie Robertson. (#1031, #1229)
2021-03-29T00:43:36Z ["python", "docker", "json", "csv", "sql", "sqlite", "datasets", "asgi", "automatic-api", "datasette", "datasette-io"]
253632948 2021-03-26
  • Better error message for --project that contains uppercase characters. #37
  • Documentation on usage with GitHub Actions. #36
2021-03-26T19:12:33Z ["datasette", "zeit-now", "datasette-plugin", "datasette-io", "vercel"]
219372133 2021-03-24
  • --dayfirst and --yearfirst options for parsedate and parsedatetime commands. #12
2021-03-24T22:59:15Z ["sqlite", "datasette-io", "datasette-tool"]
347263722 2021-03-24
  • Link to dashboard from admin edit page. #48
  • Better error message for queries with % that needs to be escaped. #41
2021-03-24T14:42:32Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
346597557 2021-03-22
  • Pin underlying version of TableauScraper library to avoid problems caused by breaking changes. #4
2021-03-22T03:56:03Z ["datasette-io", "datasette-tool"]
347263722 2021-03-21
  • Removed unnecessary migration dependency, so this should now install on versions of Django prior to 3.1. Thanks, @jefftriplett. #46
2021-03-21T21:27:00Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
347263722 2021-03-21
  • Serve cache-control: private HTTP header on private dashboards. #43
2021-03-21T00:40:22Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
253632948 2021-03-20
  • Upgraded to latest @vercel/python builder, fixing a number of bugs involving incorrectly handled querystring characters. #28
2021-03-20T21:47:32Z ["datasette", "zeit-now", "datasette-plugin", "datasette-io", "vercel"]
279357123 2021-03-19
  • Optional support for HTTP Basic authentication. #15
2021-03-19T21:02:12Z ["datasette", "datasette-plugin", "datasette-io"]
347263722 2021-03-19
  • Documentation now lives at (#36)
  • Ability to copy and paste TSV from the default table display. (#29)
  • Fixed two bugs with the way count links in column headers work. (#31, #32)
  • New permissions system: a saved dashboard can now be made public, private, unlisted, group-only, staff-only or superuser-only. (#27)
2021-03-19T06:50:27Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
347263722 2021-03-16
  • Changed default permission policy: saved dashboards are now inaccessible to the public by default (#37). This will change when permissions are implemented fully in #27.
2021-03-16T00:14:51Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
347263722 2021-03-15
  • Added basic bar chart widget. #10
  • Added markdown widget. #12
  • Added html widget. #12
2021-03-15T04:47:46Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
347263722 2021-03-15
  • Bundle widget templates in release.
2021-03-15T03:13:20Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
347263722 2021-03-15
  • Fix for bug where templatetags were not bundled in the release package. #22
2021-03-15T03:09:46Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
347263722 2021-03-15
  • Run tests against read-only 'dashboard' connection, refs #18
  • Named SQL parameters are now extracted into form fields. #15
  • Configure URLs using include() pattern. #14
  • ?sql= params are now signed, must first be created using a POST request. #13
  • New custom widgets framework #20
  • Big number widget for displaying a big number. #20
2021-03-15T01:20:22Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
248903544 2021-03-13

Progress display now shows the total as well as the number currently in the queue. % hacker-news-to-sqlite trees hn.db 26440397 done: 1, in queue: 0, total: 1 done: 2, in queue: 28, total: 30 done: 3, in queue: 27, total: 30

2021-03-13T19:15:06Z ["hacker-news", "datasette", "dogsheep", "datasette-io", "datasette-tool"]
347263722 2021-03-13
  • Include templates and migrations in the release. #4
2021-03-13T05:50:05Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
347263722 2021-03-13
  • Workaround for psycopg2 dependency bug on macOS. #3
2021-03-13T05:38:20Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
347263722 2021-03-13
  • Initial working alpha.
2021-03-13T05:31:23Z ["django", "sql", "dashboards", "datasette-io", "datasette-tool"]
315796015 2021-03-11
  • Use white-space: pre-wrap when displaying lines, so lines now wrap. #21
  • Long lines in results are now truncated, with a [...] link to expand them to full length. #22
2021-03-11T20:03:38Z ["codesearch", "ripgrep", "datasette", "datasette-plugin", "datasette-io"]
346597557 2021-03-11
  • Support importing data from a full URL to a Tableau dashboard hosted somewhere other than #2
2021-03-11T06:52:22Z ["datasette-io", "datasette-tool"]
346597557 2021-03-11
  • No longer produces tables with single quotes in the name, which can cause bugs when viewed in Datasette. #1
2021-03-11T06:37:14Z ["datasette-io", "datasette-tool"]
346597557 2021-03-11

First working version.

2021-03-11T06:13:20Z ["datasette-io", "datasette-tool"]
195087137 2021-02-25
  • Deployments that are configured to check organization membership now use read:org scope instead of the more permissive user scope. #73
2021-02-25T06:40:17Z ["asgi", "datasette", "datasette-plugin", "datasette-io"]
342126610 2021-02-25
  • Fix ASGI lifespan bug. #1
2021-02-25T05:03:45Z ["datasette", "datasette-plugin", "datasette-io"]
342126610 2021-02-25
  • First working release
2021-02-25T04:52:07Z ["datasette", "datasette-plugin", "datasette-io"]
175321497 2021-02-23
  • Tool now also works against inputs that are JSON files consisting of a list of objects with identical keys, using --format=json. #12
2021-02-23T01:14:25Z ["diff", "csv", "click", "datasette-io", "git-scraping", "datasette-tool"]
219372133 2021-02-22
  • New sqlite-transform jsonsplit for converting foo,bar into ["foo", "bar"]. #10
  • Added --dry-run option to sqlite-tranfsorm lambda for previewing the conversion that is about to be applied. #8
2021-02-22T17:04:53Z ["sqlite", "datasette-io", "datasette-tool"]
291339086 2021-02-22
  • New --sqlite database.db export option. #10
2021-02-22T06:02:10Z ["yaml", "airtable", "airtable-api", "datasette-io", "datasette-tool"]
140912432 2021-02-19

This release adds the ability to execute queries joining data from more than one database file - similar to the cross database querying feature introduced in Datasette 0.55.

2021-02-19T05:19:00Z ["python", "cli", "sqlite", "sqlite-database", "click", "datasette", "datasette-io", "datasette-tool"]
107914493 2021-02-19

Support for cross-database SQL queries and built-in support for serving via HTTPS.

  • The new --crossdb command-line option causes Datasette to attach up to ten database files to the same /_memory database connection. This enables cross-database SQL queries, including the ability to use joins and unions to combine data from tables that exist in different database files. See Cross-database queries for details. (#283)
  • --ssl-keyfile and --ssl-certfile options can be used to specify a TLS certificate, allowing Datasette to serve traffic over https:// without needing to run it behind a separate proxy. (#1221)
  • The /:memory: page has been renamed (and redirected) to /_memory for consistency with the new /_internal database introduced in Datasette 0.54. (#1205)
  • Added plugin testing documentation on Using pdb for errors thrown inside Datasette. (#1207)
  • The official Datasette Docker image now uses Python 3.7.10, applying the latest security fix for that Python version. (#1235)
2021-02-19T02:02:29Z ["python", "docker", "json", "csv", "sql", "sqlite", "datasets", "asgi", "automatic-api", "datasette", "datasette-io"]
284383265 2021-02-18
  • Fix broken tests against Datasette 0.54. #71
  • Bundle JavaScript and CSS assets for GraphiQL, instead of loading them from a CDN. #73
  • Pin to graphene dependency < 3.0. #72
2021-02-18T23:37:44Z ["graphql", "sqlite", "datasette", "datasette-plugin", "datasette-io"]
335372050 2021-02-16
  • Fixed bug in released version caused by unnecessary attr import. #6
2021-02-16T04:19:59Z ["openstreetmap", "mbtiles", "datasette-io", "datasette-tool"]
140912432 2021-02-14
  • sqlite-utils insert --sniff option for detecting the delimiter and quote character used by a CSV file, see Alternative delimiters and quote characters. (#230)
  • The table.rows_where(), and table.search_sql() methods all now take optional offset= and limit= arguments. (#231)
  • New --no-headers option for sqlite-utils insert --csv to handle CSV files that are missing the header row, see CSV files without a header row. (#228)
  • Fixed bug where inserting data with extra columns in subsequent chunks would throw an error. Thanks @nieuwenhoven for the fix. (#234)
  • Fixed bug importing CSV files with columns containing more than 128KB of data. (#229)
  • Test suite now runs in CI against Ubuntu, macOS and Windows. Thanks @nieuwenhoven for the Windows test fixes. (#232)
2021-02-14T22:44:21Z ["python", "cli", "sqlite", "sqlite-database", "click", "datasette", "datasette-io", "datasette-tool"]
303218369 2021-02-11
  • Explicitly open file using utf-8 encoding - thanks, RiverZhou. #10
  • Fix import error if note has no "updated" element - thanks, Matej Korosec. #8
2021-02-11T22:40:40Z ["sqlite", "evernote", "dogsheep", "datasette-io", "datasette-tool"]
303218369 2021-02-11
  • Correctly handle entities in XHTML notes. #9
  • Fixed a bug importing ENEX files on Windows. Thanks, dskrad. #11
2021-02-11T22:16:20Z ["sqlite", "evernote", "dogsheep", "datasette-io", "datasette-tool"]
335175637 2021-02-11
  • Fixed bug introduced in 0.6 which caused the non-JavaScript tile explorer to display incorrectly. #22
2021-02-11T00:40:11Z ["mbtiles", "datasette", "datasette-plugin", "datasette-io"]
335175637 2021-02-11

This release breaks backwards compatibility if you are using this as a tile server. Map tiles served by /-/tiles/... now use the OpenStreetMap coordinate system, where y=0 references the upper-most tile on the map. If you wish to continue using the TMS coordinate system you should switch to /-/tiles-tms/... instead. See Tile coordinate systems for more details.

  • Support both OSM and TMS coordinate systems, use OSM by default. #15
  • Hide non-JavaScript map as soon as the JavaScript loads in the tile explorer. #21
2021-02-11T00:25:56Z ["mbtiles", "datasette", "datasette-plugin", "datasette-io"]
174715153 2021-02-06
  • Updated README examples to reflect function name changes in version 1.0. #3
2021-02-06T02:34:18Z ["datasette", "datasette-plugin", "datasette-io"]
140912432 2021-02-06
  • Fixed a code import bug that slipped in to 3.4. (#226)
2021-02-06T02:11:15Z ["python", "cli", "sqlite", "sqlite-database", "click", "datasette", "datasette-io", "datasette-tool"]
174715153 2021-02-06

Renamed two functions for compatibility with the latest Jellyfish - see their documentation for details.

  • jaro_distance(a, b) is now jaro_similarity(a, b)
  • jaro_winkler(a, b) is now jaro_winkler_similarity(a, b)
2021-02-06T02:00:51Z ["datasette", "datasette-plugin", "datasette-io"]
140912432 2021-02-06
2021-02-06T01:38:26Z ["python", "cli", "sqlite", "sqlite-database", "click", "datasette", "datasette-io", "datasette-tool"]
335175637 2021-02-04
2021-02-04T18:16:52Z ["mbtiles", "datasette", "datasette-plugin", "datasette-io"]
335175637 2021-02-03
  • New /-/tiles-stack mechanism for serving tiles from multiple databases. #8
  • "tiles-stack-order" plugin setting, for controlling the order in which tiles are served from the stack.
  • Tile explorer page now also displays metadata for that tile database. #7
  • Tile explorer uses name from metadata if it is available, and provides links to the database and table pages.
  • Tile 404s are now served as blank PNGs. #9
  • Renamed templates from mbtiles_*.html to tiles_*.html. #10
2021-02-03T22:24:18Z ["mbtiles", "datasette", "datasette-plugin", "datasette-io"]
335372050 2021-02-03
  • --name= option for setting the name to be written into the metadata table. If you use the --city or --country options without passing the --name the city or country name will be used instead. #5
  • --attribution=osm shortcut for setting the attribution to © OpenStreetMap contributors without having to type that out in full. #4
2021-02-03T03:32:11Z ["openstreetmap", "mbtiles", "datasette-io", "datasette-tool"]
335372050 2021-02-03
  • New --attribution option for writing an attribution record to the metadata table. #3
2021-02-03T00:35:59Z ["openstreetmap", "mbtiles", "datasette-io", "datasette-tool"]
335175637 2021-02-02
  • Attribution is now shown on the tile explorer page if it is included in the metadata table. #6
2021-02-02T23:41:32Z ["mbtiles", "datasette", "datasette-plugin", "datasette-io"]
335137108 2021-02-02
  • Database now includes attribution in the metadata table. #3
2021-02-02T23:33:57Z ["openstreetmap", "mbtiles", "datasette", "datasette-plugin", "datasette-io"]
107914493 2021-02-02
  • Fixed a bug where ?_search= and ?_sort= parameters were incorrectly duplicated when the filter form on the table page was re-submitted. (#1214)
2021-02-02T21:30:12Z ["python", "docker", "json", "csv", "sql", "sqlite", "datasets", "asgi", "automatic-api", "datasette", "datasette-io"]
335175637 2021-02-02
  • Table and database action menu items linking to the tile explorer. #4
2021-02-02T19:54:11Z ["mbtiles", "datasette", "datasette-plugin", "datasette-io"]
335372050 2021-02-02
  • --city and --country options as an alternative to --bbox. #1
2021-02-02T18:24:29Z ["openstreetmap", "mbtiles", "datasette-io", "datasette-tool"]
335372050 2021-02-02

Initial working version.

2021-02-02T17:39:35Z ["openstreetmap", "mbtiles", "datasette-io", "datasette-tool"]
335137108 2021-02-02
  • First non-alpha release.
2021-02-02T07:07:43Z ["openstreetmap", "mbtiles", "datasette", "datasette-plugin", "datasette-io"]
335175637 2021-02-02
  • Improved README and live demo. #2
2021-02-02T07:05:04Z ["mbtiles", "datasette", "datasette-plugin", "datasette-io"]
335175637 2021-02-02
  • Initial working version, with tile serving and tile explorer. #1
2021-02-02T06:47:44Z ["mbtiles", "datasette", "datasette-plugin", "datasette-io"]