Shillelagh 1.1.4

A small release allowing the S3 Select adapter to query JSON documents.

Shillelagh 1.1.3

A small improvement to the generic JSON adapter so it can handle nested fields:

sql> SELECT NS ->> '$[0]' AS first_ns FROM "https://api.domainsdb.info/v1/domains/search?domain=facebook&zone=com#$.domains[*]" LIMIT 1;
first_ns
----------------------
aron.ns.cloudflare.com
sql>

Shillelagh 1.1.2

A small fix ensuring only requested adapters are loaded when in non-safe mode, and a new adapter that handles generic JSON endpoints.

Shillelagh 1.1.1

A new minor version with small fixes.

Shillelagh 1.1.0 is out!

Heya!

Today we're releasing Shillelagh 1.1.0. There are no breaking changes, but some interesting new additions:

• Shillelagh now supports DROP TABLE. If you run DROP TABLE on a Google sheet URL, for example, the sheet will be deleted. This is only supported in the adapters where it makes sense. In the future we might also want to support CREATE TABLE.
• The CLI now supports multi-line statements, which means that statements must be terminated with a semicolon. Before, pressing enter would immediately run the query.
• The configuration file now uses appdirs for the directory location. if you're on a Mac you need to move your configuration files from ~/.config/shillelagh/ to ~/Library/Application\ Support/shillelagh/.
• New adapters: there is now an adapter for querying CSV/JSON/Parquet files in S3, and an adapter to query HTML tables from any page.
• Adapters can now request to handle LIMIT and OFFSET, instead of leaving the work to SQLite. This should greatly improve the performance of adapters, since it can greatly reduce the amount of data than needs to be fetched. Most of the builtin adapters have support for them in 1.1.0.
• Adapters can now be registered at runtime, similar to how SQLAlchemy does with dialects. Before, adapters had to be registered as entry points.
• Additional documentation on the architecture, custom dialects and custom fields was added.

1.0.16

This release has no external changes. The only change is how parameters are passed from SQLite to the adapter. The new serialize function uses pickle to allow passing more than just strings to adapters.

1.0.15

Includes a new adapter for S3 files, using S3 Select:

SELECT * FROM "s3://bucket-name/sample_data.csv";

Supports CSV, JSON, and Parquet.

Also changes the way integers work, now we manipulate them as strings inside SQLite to prevent overflows.

1.0.14

Includes a small fix by @AAfghahi allowing column names to have double quotes in them.

1.0.13

Release 1.0.13 drops support for Python 3.7. We're also starting to test shillelagh with Python 3.10, hopefully we'll add support soon.

Shillelagh 1.0

After almost 1 year of development and 4 of planning, I finally released Shillelagh 1.0. The initial release focused on a stable and flexible API, with a solid adapter (Google Sheets). For the next milestone my plan is to focus on adding more adapters and improving performance.