Datasette 0.65.5 and 1.0a40 are out, with a security fix for a table permisisons bypass. Datasette 1.0a40 also adds a mechanism for plugins to run background tasks.

Here's a new TIL on using SQL functions in Google Sheets to fetch data from Datasette.

Datasette 1.0a27 changes how CSRF protection works in a way that simplifies form and API integration, and introduces a new RenameTableEvent for when a table is renamed by a SQL query.

Datasette 1.0a26 adds a column_types system allowing columns to carry additional type information such as email or url, or extra types defined by plugins. This alpha also adds a UI for selecting and sorting visible columns on a table page.

Datasette 1.0a25 adds write_wrapper() and register_token_handler() plugin hooks.

Datasette 1.0a24 introduces a new await request.form(files=True) method for accepting file uploads, plus various other changes.

Datasette 0.65.2 and Datasette 1.0a21 have been released with a security fix for an open redirect error, see this advisory. The latest Datasette alphas also include a breaking change to Datasette's permission system, described in detail in A new SQL-powered permissions system in Datasette 1.0a20.

Datasette 1.0a17 is the latest Datasette 1.0 alpha release, with bug fixes and small feature improvements from the last few months.

• Now compatible with Datasette 1.0a42.
• Now depends on datasette-leaflet 0.3, which upgrades Leaflet to 1.9.4 and fixes the "Access blocked" tiles returned by tile.openstreetmap.org. Tile requests now send the origin of your Datasette instance as their Referer header, but never the URL of the page showing the map. See Tile layers and the Referer header for how to change or disable this.
• The map is now only shown once a row with a valid latitude and longitude has loaded. Tables with latitude and longitude columns that contained no values previously displayed an empty grey map.
• The default tile layer is now https://tile.openstreetmap.org/{z}/{x}/{y}.png, without the {s} subdomain prefix, as recommended by OpenStreetMap. #48
• The "Showing X of Y" progress message now handles tables where Datasette did not return an exact count.
• Now requires Python 3.11 or higher.

• Upgraded the bundled Leaflet from 1.7.1 to 1.9.4. #8
• The JavaScript is still an ES module, loaded with import(datasette.leaflet.JAVASCRIPT_URL). Importing it continues to set a global window.L, so Leaflet plugins that expect that global, such as Leaflet.markercluster, keep working. #12
• Tile layers now send the origin of the Datasette site as their Referer header by default. Datasette serves table pages with Referrer-Policy: no-referrer, which caused tile.openstreetmap.org to respond with "Access blocked" tiles. The bundled Leaflet changes the default of the referrerPolicy tile layer option to strict-origin-when-cross-origin, so tile servers see https://example.com/ but never the path or query string of the page showing the map. This applies to every plugin that uses this copy of Leaflet.
• New tile_referrer_policy plugin setting for changing that policy, or turning it off with false. The value is available to JavaScript as datasette.leaflet.TILE_REFERRER_POLICY. The referrerPolicy option on an individual tile layer takes precedence. See Tile layers and the Referer header.
• New update_leaflet.py script for downloading and bundling a new version of Leaflet.
• Now requires Python 3.11 or higher.
• Added an Apache 2.0 LICENSE file. #11

• Security: Models in extra-openai-models.yaml with an api_base and an api_key_name no longer send the OPENAI_API_KEY environment variable to that endpoint when the named key has not been stored. They now fail with a No key found error instead. GHSA-v9h7-jj4g-f862
• Security: Models with responses: true and an api_base no longer send your stored openai key or the OPENAI_API_KEY environment variable to that endpoint when run with -o chat_completions 1. This also applied to llm openai endpoint URL --responses -o chat_completions 1 run without --key. GHSA-r639-rr7r-x72w
• -a/--attachment and --at/--attachment-type now work with pipes such as Bash process substitution, so llm "Describe this image" -a <(cat image.png) works. Previously LLM could not detect the type of a piped attachment, and --at could lose its content before it was sent. Thanks, Steven Huang. #971, #1857
• An empty attachment, such as llm -a - with nothing on standard input or -a with an empty file, now shows the usual Could not determine mimetype error instead of a traceback. Thanks, Ansansan. #1853, #1854
• llm openai endpoint add no longer breaks extra-openai-models.yaml when the existing list is indented or ends with a ... document end marker. Previously the file stopped parsing, so LLM could not load any models until it was fixed by hand. Thanks, ledd. #1864, #1865
• llm openai endpoint list --json no longer crashes with a TypeError when extra-openai-models.yaml contains an unquoted date, such as api_version: 2024-02-01. Thanks, Steven Huang. #1850
• In the Python API, reopening an embedding collection with a model= other than the one it was created with now uses the collection's stored model. Previously new embeddings were calculated with the supplied model, mixing vectors from two models in one collection. Thanks, Chen Yufeiyang. #1859

A redesigned homepage, redesigned table, row and query pages, a new "Export data" dialog with TSV support, a column statistics feature, and Python 3.11 is now the minimum supported version.

Python 3.11 or higher

• Datasette now requires Python 3.11 or higher. This allowed the removal of several compatibility shims. (#2998)
• Datasette is now tested on Python 3.11, 3.12, 3.13, 3.14, and 3.15.

Export data dialog and TSV

• Table, query and row pages replace the "This data as..." links and the "Advanced export" box with an Export data button. This opens a dialog listing every available format - TSV, CSV, JSON, JSON array, newline-delimited JSON, JSON object and any plugin formats - each with Copy, View, Download and Copy URL actions. Old #export links open the dialog. See CSV and TSV export. (#3037)
• New TSV export, for pasting into Excel and Google Sheets. TSV uses the same streaming code, options and URL parameters as CSV, with Excel-compatible quoting.
• register_output_renderer dictionaries can now include optional label and description keys, shown in the Export data dialog, plus is_binary and is_webpage flags that control which actions the dialog offers for that format.

Column statistics

• The column action menu has a new Column statistics item, available to actors with the execute-sql permission. It opens a dialog showing counts of rows, nulls and distinct values, a breakdown of storage types, numeric statistics with a histogram, box plot and percentiles, text length statistics, a chart of dates grouped by month, year or decade, and the most common values. Every query it runs is listed, with a link to open it in the SQL editor. (#3038)
• Column statistics are also available for the results of arbitrary SQL queries and stored queries.

Table page redesign

• A refreshed design for tables on the table, row and query pages, with right-aligned numbers and shadows indicating horizontally scrollable content. (#3032)
• Values taller than three lines are clamped to a preview with an "expand" button. Column headers offer an "expand all" toggle.
• A pin in the primary key column header freezes the header row and primary key columns while scrolling. On query pages it freezes just the header row. The choice is remembered.
• Tables that span more than one page now show a pagination bar, for example "Rows 101-150 of 1,761", with "Back to first page" and "Next page" buttons. This uses a new ?_count=before option for the count endpoint, which counts the rows before a ?_next= cursor. (#3043)
• The row count is now shown in the table page heading, and the pagination bar sits directly above the table. (#3048)
• The CREATE TABLE or CREATE VIEW statement is now collapsed into a "Table schema" or "View definition" disclosure at the bottom of the page.
• Facet values now show a faint bar indicating their share of the rows. Suggested facets and facet results have been restyled.
• New Clear filters button next to "Apply filters", shown when column filters are applied.

Row page redesign

• The row page now shows the row as a list with one field per line, each with a button to copy its value, in place of a single-row table. Long values are clamped with a toggle to show them in full, and the tables that reference the row are listed below it. (#3052)
• Backwards incompatible: the row page now includes a new _row_fields.html template instead of _table.html. A custom _table.html template that applies to every table no longer changes the row page - provide a _row_fields.html template instead. Templates for a specific table, _table-mydatabase-mytable.html and _table-row-mydatabase-mytable.html, still apply to the row page. See Custom templates.
• The row page no longer contains a table.rows-and-columns element, so plugin JavaScript that looks for that table on row pages will not find it. The .above-table-panel element plugins can attach to is still present.

Homepage redesign

• The homepage now shows one searchable catalog of tables, views and stored queries across every database, instead of grouping them by database. (#3049)
• A search box, focused when the page loads, matches table and view names, column names, table descriptions and the names, titles, descriptions and SQL of stored queries. Results update as you type and can be navigated with the arrow keys and Enter.
• Tables, views, queries and databases you visit are recorded in the browser's localStorage, separately for each signed-in actor, and shown in a Recently visited section. The "Recent" list in the Jump to menu now uses the same history.
• Database filter buttons (?database=) are shown when more than one database is attached. The table list is built from the internal catalog_* tables so it stays fast with thousands of tables, and is paginated 30 at a time.
• New makeHomepageSections() JavaScript plugin hook for adding sections to the homepage. Items added to Jump to using the jump_items_sql(datasette, actor, request) plugin hook are included in search results.
• The stored query browser at /-/queries has been redesigned to match the new homepage.
• Backwards incompatible: the index.html template context has changed and is now documented in Template context. Custom index.html templates that used keys such as tables_and_views_truncated or table_rows_sum will need to be updated. The /.json output is unchanged.

Other changes

• App, database, table, row and column action menus have been redesigned to share positioning, dismissal and keyboard navigation behavior. See the new /-/patterns/menus page. (#2984)
• The /-/jump endpoint now supports ?type= filtering, for example ?type=table&type=database. Items now include the URL arguments used to build their URL as fields, hidden tables are excluded, and stored queries use their title as their display name. (#3024, #3025)
• The internal catalog_tables table has a new hidden boolean column, populated for tables that Datasette treats as hidden from default table lists.
• Error responses now use the JSON error format when the request sends an Accept: application/json or Content-Type: application/json header, not just for URLs ending in .json. Thanks, kokokoXUY. (#1949, #2961)
• HTML table pages no longer send a rel="next" HTTP Link header. (#3046)
• edit-tools.js, export-dialog.js and column-stats.js are now loaded on demand, the first time they are needed, rather than on every page. The table page now only includes the editing tools loader if the actor can insert, alter, update or delete, indicated by a new row_mutation_ui template context variable. (#3042)

Bug fixes

• execute-write operations now work against tables with FTS index triggers, such as those created by sqlite-utils enable_fts(create_triggers=True). The SQL analysis table now shows why an operation was rejected.
• The about and about_url metadata are now displayed even if no source or license is set. (#512, #2985)
• The query page heading now says "returning more than N rows" when results are truncated.
• A stored query that references a missing :_actor_X key now raises a KeyError naming that key.
• Facet counts on the second and later pages of a table now count every matching row, and facet links no longer keep the ?_next= pagination cursor. Thanks, b6tang. (#3021, #3026)
• The ?_json= option now decodes JSON columns on table pages, no longer replaces row values when combined with ?_extra=columns, and works with ?_shape=arrays and ?_shape=arrayfirst. Thanks, IlyaKhalafi. (#3003, #3005)
• Views with a BLOB column no longer return a 500 error, and CSV exports of those views now include a link to download each value. (#2601)
• Fixed a bug where Datasette running with base_url produced links and redirects that repeated the prefix, such as /prefix/prefix/, when the prefix was included in the request path. This affected filter and sort redirects, column sort links, next page links, facet links and the query page. (#2347, #1633)
• Sorting a table that has no primary key by a column hidden using ?_nocol= or ?_col= no longer returns a 500 error when the results span more than one page. Thanks, feiiiiii5. (#2974, #2975)
• Full-text search now works for FTS tables that declare their content table unquoted or with single quotes or backticks, for example content=documents. Previously these tables showed no search box and ?_search= was ignored. Thanks, b84330808. (#1991, #3011)
• datasette.plugin_config(plugin_name, database=...) now falls back to the top-level plugin configuration for a database that has no configuration of its own for that plugin. Thanks, pujitha24. (#2959, #3041)

• llm openai endpoint add now works when extra-openai-models.yaml holds a list written in square brackets, such as the [] left behind by removing the last saved endpoint. Previously it wrote invalid YAML that stopped LLM from loading any models until the file was fixed by hand. Thanks, Steven Huang. #1845
• Passing both system= and messages= to prompt() or chain() in the Python API no longer drops the system prompt for models that build their requests from prompt.messages, including the default OpenAI Chat models. #1849
• Non-streaming prompts, such as those run with --no-stream, no longer crash with an AttributeError when an OpenAI-compatible server leaves usage out of its response. Thanks, Tao An. #1697, #1698
• llm embed --format blob now outputs exactly the bytes of the embedding. It previously added a newline at the end, which made the output one byte too long to decode. Thanks, Arjan Mossel. #1848
• llm logs no longer crashes with JSON cannot hold BLOB values when a tool result logged by an LLM version before 0.32 has an attachment stored in the database. Thanks, XQ. #1775
• llm logs -q no longer misses words at the boundary between two text parts, for example in a response that interleaves text with reasoning or tool calls.
• When a prompt passes several user messages using messages=, llm logs -q now searches all of them rather than only the last one.

New features:

• New OpenAI models: gpt-6.1-sol for GPT-6.1 Sol, gpt-6.1-terra for GPT-6.1 Terra and chat-latest for Chat Latest.
• The default model for users who have not set their own default is now GPT-6 Luna, OpenAI's cheapest frontier model. It was previously GPT-5.6 Luna. You can switch back using llm models default gpt-5.6-luna. #1746
• New llm openai endpoint add, list, remove and path commands for managing OpenAI-compatible models saved in extra-openai-models.yaml. See Configure an OpenAI-compatible model. #1838
• Embedding models can now accept typed attachments, which carry a MIME type and the path or URL they came from. Plugins declare the MIME types they support using attachment_types and receive matching inputs as llm.Attachment objects. llm embed, llm embed-multi and llm similar gain -a/--attachment and --at/--attachment-type options. Collections store the MIME type in a new content_type column, exposed as Entry.content_type. See llm embed and Writing plugins to add new embedding models. #1741
• llm chat now accepts -u/--usage to display token usage on standard error after each chat turn. Thanks, Bian Yuhan. #980, #1754
• New chain.on_response(callback) method in the Python API, which runs a callback for each completed response in a tool chain before the chain limit is checked or that response's tool calls are executed. See Running code when a response has completed. #1768
• Added support for Python 3.15 and dropped support for Python 3.10.
• Removed OpenAI models that have been shut down: gpt-3.5-turbo, gpt-3.5-turbo-16k, gpt-3.5-turbo-instruct, gpt-4, gpt-4-turbo, gpt-4-turbo-2024-04-09, gpt-4.1-nano, gpt-5.2-chat-latest, o1, o1-2024-12-17, o3-mini and o4-mini, along with their aliases such as 3.5, chatgpt, 4 and 4t. #1696

Bug fixes:

• llm chat now respects llm logs off. Previously it logged every response even when logging had been turned off. #1203
• Response.reply() now passes before_call and after_call to the tool calls it executes automatically, and keeps them for later replies. Previously a tool rejected by before_call would still run. Thanks, 0d00ciallo0721. #1835, #1836
• llm embed-multi --files no longer corrupts non-ASCII UTF-8 files by decoding them as latin-1, which stored and embedded "Café" as "Café". Run the same command again to repair an existing collection: only the affected files will be embedded again. Thanks, Alex. #283, #1750
• llm prompt and llm chat now log each response in a tool chain as soon as it completes, so an error or hitting the chain limit no longer discards the earlier responses in that chain from the logs. #1768
• Fixed SQLITE_BUSY errors when multiple processes logged responses to the same database at the same time. #1742
• Tool chains that finish within their chain limit no longer raise a Chain limit of N exceeded error. The limit now only applies when the final allowed response requests another round of tool calls. Thanks, Bishop Kammeraad. #1757, #1766
• Fixed AsyncChainResponse.log_to_db() raising RuntimeError inside a running event loop, for example straight after await chain.text(). Thanks, Christopher Krah. #1727, #1734
• Fixed llm prompt --usage crashing with an AttributeError when used with --async and tools. Thanks, Ethan Stoner. #1709
• Response.execute_tool_calls() now returns an error result for a tool with no implementation instead of raising ValueError, matching the behavior of AsyncResponse.execute_tool_calls(). Thanks, Chen Yufeiyang. #1724
• Toolboxes exposing more than one method now run prepare() and prepare_async() once per instance, instead of once per method. Thanks, Jiachen Xie. #1761
• Tool specs passed to -T/--tool with keyword arguments whose names start with true, false or null, such as MyTool(nullable=true), are no longer mistaken for a single positional JSON value. Thanks, Olamide Faniyan. #1704
• llm logs -s/--short now truncates tool call arguments. #1772
• A schema passed to a tool chain now applies to every request in that chain, including the final response. It was previously dropped after the first round of tool calls. Thanks, Chen Yufeiyang. #1222, #1825
• Collection.embed() no longer silently skips a new ID when another item in the collection has the same content, matching the behavior of Collection.embed_multi(). Thanks, nosmile99. #1789, #1812
• Collection.embed_multi() now raises ValueError if an embedding model returns the wrong number of embeddings for a batch, instead of silently losing rows or storing vectors against the wrong IDs. Thanks, Sharat Chandra Sai Boddu. #1722, #1726
• llm schemas list now includes schemas used by recently logged responses. Thanks, Jiachen Xie. #1808
• llm schemas list no longer crashes on schemas with boolean property definitions such as {"properties": {"x": true}}. Thanks, Jiachen Xie. #1748
• AsyncConversation.prompt() now uses the tools set on the conversation, matching Conversation.prompt(). Thanks, Bishop Kammeraad. #1790, #1791
• Calling response.log_to_db() on a synchronous response that has not yet been read now runs the response first. Previously the logged turn was missing its output and token usage. Thanks, Allen Flux. #1817

• ttok now defaults to the o200k_base tokenizer used by the GPT-5 and GPT-6 families. #25
• The ttok --list-models command now also shows how prefixes like gpt-5* are handled by the underlying tiktoken library.

• Fixed the UserWarning: The parameter --tokens is used more than once warning shown by recent versions of Click. #20, #24
• --allow-special can now be used when counting or truncating tokens, not just with --encode or --tokens. This means files-to-prompt . | ttok --allow-special now works. #17
• New ttok --list-models option for listing the models supported by your installed version of tiktoken. #18
• Documentation now lists newer models such as gpt-4o, gpt-4.1, gpt-5, o1, o3 and o4-mini, all of which use the o200k_base encoding. #15
• ttok now requires Python 3.10 or higher.
• Packaging moved from setup.py to pyproject.toml.

• Compatible with Datasette>1.0a41

OpenTelemetry support, a new JavaScript API for creating modal dialogs, and several smaller bug fixes.

OpenTelemetry

Datasette now supports OpenTelemetry traces and metrics for monitoring application performance. Thanks, Alex Garcia. (#1730, #2867)

• Traces cover HTTP requests, database queries and startup, including time spent waiting for SQL threads and queued writes.
• Metrics report query latency, time-limit interruptions, SQL thread usage, write queues and open connections.
• New tools for plugin authors help plugins add their own traces and metrics, with shared registry classes and pytest helpers.

To collect telemetry, configure an OpenTelemetry SDK and exporter, then run Datasette using opentelemetry-instrument. See OpenTelemetry for setup instructions.

Other features

• New DatasetteModal JavaScript API for plugins to create dialogs with Datasette's shared styles, keyboard behavior and focus handling. Datasette's built-in dialogs use the same API. (#2790, #2948)

Bug fixes

• Table pages now show measured query timings instead of always displaying 1.2ms. (#2446)
• Facet loading now ignores unrelated query parameters such as ?_facets=x, instead of returning a 500 error. Thanks, Peng Boyu. (#2949)
• Foreign key values no longer link to tables that do not exist. Thanks, Dipak Chaudhari. (#1515, #2952)
• Fixed missing punctuation between table and view counts on the homepage, such as 0 tables1view. Thanks, Dipak Chaudhari. (#2012, #2951)
• The sort menu now excludes primary keys that are not included in sortable_columns. Thanks, Sanjay Santhanam. (#1980, #2858)

New features:

• New OpenAI models: gpt-6-sol for GPT-6 Sol and gpt-6-luna for GPT-6 Luna. #1702
• Model plugins can now declare supports_conversation = False for models that only accept single-turn prompts. LLM raises llm.ConversationNotSupported when these models receive assistant or tool history, and llm chat rejects them before starting a session. See Models that do not support conversations. The first plugin to use this is llm-typesafe. #1692
• Reasoning traces in the Markdown output of llm logs are now wrapped in <details><summary> tags. #1701

Bug fixes:

• Fixed llm logs -t crashing with KeyError: 'options_json' when displaying truncated logs with options or schemas. Thanks, rdslw. #1656
• Fixed AsyncResponse.log_to_db() raising sqlite3.ProgrammingError after a response had been awaited. Structured messages and provider metadata are also preserved when logging restored async responses. Thanks, Roli Bosch. #1668
• Tool parameters and nested model fields named title are now preserved in generated input schemas. Thanks, Christopher Pruijsen. #1679, #1680
• Constructing llm.Prompt directly without options now initializes the model's default Options object, fixing errors in providers that access option attributes. Thanks, Jane Li. #1028, #1663
• Closing an asynchronous response.astream_events() stream before completion now closes the underlying provider generator, allowing it to release streaming connections and other resources. Thanks, Shunsuke. #1659
• SQLite connections opened by CLI commands are now closed when the command finishes, including on errors, avoiding unclosed-connection warnings.

Documentation:

• Corrected the structured output example in the advanced model plugins documentation to use supports_schema = True. Thanks, Ethan Stoner. #1686