Spice v0.17-beta (July 29, 2024)
Categories:
Announcing the first beta release of Spice.ai OSS! 🎉
The core Spice runtime has graduated from alpha to beta! Components, such as Data Connectors and Models, follow independent release milestones. Data Connectors graduating from alpha
to beta
include databricks
, spiceai
, postgres
, s3
, odbc
, and mysql
. From beta to 1.0, project will be to on improving performance and scaling to larger datasets.
This release also includes enhanced security with Transport Layer Security (TLS) secured APIs, a new spice install
CLI command, and several performance and stability improvements.
Highlights in v0.17-beta
- Encryption in transit with TLS: The HTTP, gRPC, Metrics, and OpenTelemetry (OTEL) API endpoints can be secured with TLS by specifying a certificate and private key in PEM format.
Enable TLS using the --tls-certificate-file
and --tls-key-file
command-line flags:
spice run -- --tls-certificate-file /path/to/cert.pem --tls-key-file /path/to/key.pem
Or configure in the spicepod.yml
:
runtime:
tls:
certificate_file: /path/to/cert.pem
key_file: /path/to/key.pem
Get started with TLS by following the TLS Sample. For more details see the TLS Documentation.
spice install
: Running thespice install
CLI command will download and install the latest version of the runtime.
spice install
Improved SQLite and DuckDB compatibility: The SQLite and DuckDB accelerators support more complex queries and additional data types.
Pass through arguments from
spice run
to runtime: Arguments passed tospice run
are now passed through to the runtime.Secrets replacement within connection strings: Secrets are now replaced within connection strings:
datasets:
- from: mysql:my_table
name: my_table
params:
mysql_connection_string: mysql://user:${secrets:mysql_pw}@localhost:3306/db
Breaking Changes
The odbc
data connector is now optional and has been removed from the released binaries. To use the odbc
data connector, use the official Spice Docker image or build the Spice runtime from source.
To build Spice from source with the odbc
feature:
cargo build --release --features odbc
To use the official Spice Docker image from DockerHub:
# Pull the latest official Spice image
docker pull spiceai/spiceai:latest
# Pull the official v0.17-beta Spice image
docker pull spiceai/spiceai:0.17.0-beta
Contributors
- @y-f-u
- @peasee
- @digadeesh
- @phillipleblanc
- @ewgenius
- @sgrebnov
- @Sevenannn
- @lukekim
What’s Changed
Dependencies
- Upgraded delta-kernel-rs to v0.2.0.
Commits
- update helm chart versions for v0.16.0-alpha by @y-f-u in https://github.com/spiceai/spiceai/pull/2057
- Update spicepod.schema.json by @github-actions in https://github.com/spiceai/spiceai/pull/2060
- fix: Install
unixodbc
for E2E test release installation by @peasee in https://github.com/spiceai/spiceai/pull/2063 - update next release to 0.16.1-beta by @digadeesh in https://github.com/spiceai/spiceai/pull/2065
- update version to 0.17.0-beta by @digadeesh in https://github.com/spiceai/spiceai/pull/2068
- Update ROADMAP.md - removing delivered features and updating Beta timeline. by @digadeesh in https://github.com/spiceai/spiceai/pull/2066
- make bench works for more connectors by @y-f-u in https://github.com/spiceai/spiceai/pull/2042
- enable spark benchmark by @y-f-u in https://github.com/spiceai/spiceai/pull/2069
- Make the
json_pointer
param optional for the GraphQL connector by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2072 - Fix secrets init to not bail if a secret store can’t load by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2073
- Update end_game.md by @ewgenius in https://github.com/spiceai/spiceai/pull/2059
- Fix time predicate with timezone info casting for Dremio by @sgrebnov in https://github.com/spiceai/spiceai/pull/2058
- Add benchmark tests for S3 data connector by @sgrebnov in https://github.com/spiceai/spiceai/pull/2049
- Add benchmark tests for MySQL data connector by @sgrebnov in https://github.com/spiceai/spiceai/pull/2048
- fix: Add Athena dialect for ODBC by @peasee in https://github.com/spiceai/spiceai/pull/2084
- Workflow to build MySQL image with TPCH benchmark data by @sgrebnov in https://github.com/spiceai/spiceai/pull/2070
- Fix secrets replacement within connection strings by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2086
- fix: Correctly prefix missing required parameters by @peasee in https://github.com/spiceai/spiceai/pull/2088
- Add Postgres Data Connector TPCH Benchmark Tests by @Sevenannn in https://github.com/spiceai/spiceai/pull/2009
- Add
spice install
CLI command by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2090 - Use MySQL service container for benchmark tests by @sgrebnov in https://github.com/spiceai/spiceai/pull/2089
- Remove ODBC from default released binaries by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2092
- Add cfg flag to properly support build w / wo feature in benchmark tests by @Sevenannn in https://github.com/spiceai/spiceai/pull/2095
- Move Prometheus metrics server to runtime by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2093
- fix: Remove unixodbc from test release install by @peasee in https://github.com/spiceai/spiceai/pull/2103
- Upgrade
delta_kernel
to 0.2.0 by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2102 - Allow DuckDB to load extensions in Docker by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2104
- Spawn the metrics server in the background. by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2105
- fix: suffix delta kernel table location with slash if none by @y-f-u in https://github.com/spiceai/spiceai/pull/2107
- Bump object_store from 0.10.1 to 0.10.2 by @dependabot in https://github.com/spiceai/spiceai/pull/2094
- Decision Record: Default HTTP and GRPC ports for Spice.ai OSS by @digadeesh in https://github.com/spiceai/spiceai/pull/2091
- Enable TLS for metrics endpoint by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2108
- Use Postgres container for tpch bench by @Sevenannn in https://github.com/spiceai/spiceai/pull/2112
- Add workflow to build Postgres Docker image using tpch data by @Sevenannn in https://github.com/spiceai/spiceai/pull/2101
- Enable TLS for HTTP endpoint by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2109
- Enable TLS on the Flight GRPC endpoint by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2110
- add timeout parameters for object store client options by @y-f-u in https://github.com/spiceai/spiceai/pull/2114
- Enable TLS on the OpenTelemetry GRPC endpoint by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2111
- feat: Add ODBC Databricks Benches by @peasee in https://github.com/spiceai/spiceai/pull/2113
- Support configuring TLS in the spicepod by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2118
- add broken tpch simple queries by @y-f-u in https://github.com/spiceai/spiceai/pull/2116
- Add integration test for TLS by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2121
- Improve SQLite and DuckDB compatibility by @sgrebnov in https://github.com/spiceai/spiceai/pull/2122
- Pass through arguments from
spice run
andspice sql
to runtime by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2123 - Handle TLS in the spice CLI when connecting to the runtime by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2124
- Handle connecting over TLS for
spice sql
by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2125 - Remove
--tls
flag by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2128 - fix: Handle SQLResult error instead of unwrapping by @peasee in https://github.com/spiceai/spiceai/pull/2127
- Add delta bench by @y-f-u in https://github.com/spiceai/spiceai/pull/2120
- feat: Add Athena ODBC benches by @peasee in https://github.com/spiceai/spiceai/pull/2129
- fix: Use odbc-api fork for decimal conversion fix by @peasee in https://github.com/spiceai/spiceai/pull/2133
- Update benchmarks job env for delta testing by @y-f-u in https://github.com/spiceai/spiceai/pull/2134
- Use forked dotenvy to disable variable substitution by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2135
- Remove unnecessary memory allocations in the query path by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2136
- upgrade spiceai df for tpch simple 6 and 7 by @y-f-u in https://github.com/spiceai/spiceai/pull/2137
- Avoid more unnecessary allocations in the query path by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2138
Full Changelog: https://github.com/spiceai/spiceai/compare/v0.16.0-alpha...v0.17-beta
Resources
Community
Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Discord or by email to get involved.
- Twitter: @spice_ai
- Discord: https://discord.gg/kZnTfneP5u
- Telegram: Spice AI Discussion
- Reddit: https://www.reddit.com/r/spiceai
- Email: [email protected]