Spice v0.17.1-beta (August 5, 2024)

By Spice AI (@spice_ai) | Monday, August 05, 2024

Tags:

The v0.17.1-beta minor release focuses on enhancing stability, performance, and usability. The Flight interface now supports the GetSchema API and s3, ftp, sftp, http, https, and databricks data connectors have added support for a client_timeout parameter.

Highlights in v0.17.1-beta

Flight API GetSchema: The GetSchema API is now supported by the Flight interface. The schema of a dataset can be retrieved using GetSchema with the PATH or CMD FlightDescriptor types. The CMD FlightDescriptor type is used to get the schema of an arbitrary SQL query as the CMD bytes. The PATH FlightDescriptor type is used to retrieve the schema of a dataset.

Client Timeout: A client_timeout parameter has been added for Data Connectors: ftp, sftp, http, https, and databricks. When defined, the client timeout configures Spice to stop waiting for a response from the data source after the specified duration. The default timeout is 30 seconds.

datasets:
  - from: ftp://remote-ftp-server.com/path/to/folder/
    name: my_dataset
    params:
      file_format: csv
      # Example client timeout
      client_timeout: 30s
      ftp_user: my-ftp-user
      ftp_pass: ${secrets:my_ftp_password}

Breaking Changes

TLS is now required to be explicitly enabled. Enable TLS on the command line using --tls-enabled true:

spice run -- --tls-enabled true --tls-certificate-file /path/to/cert.pem --tls-key-file /path/to/key.pem

Or in the spicepod.yml with enabled: true:

runtime:
  tls:
    # TLS explicitly enabled
    enabled: true
    certificate_file: /path/to/cert.pem
    key_file: /path/to/key.pem

Contributors

@Jeadie
@y-f-u
@phillipleblanc
@sgrebnov
@peasee
@Sevenannn

What’s Changed

Dependencies

Rust: Upgraded from v1.79.0 to v1.80.0

Commits

Update README.md by @Jeadie in https://github.com/spiceai/spiceai/pull/2142
update helm chart to 0.17.0-beta by @y-f-u in https://github.com/spiceai/spiceai/pull/2144
Update spicepod.schema.json by @github-actions in https://github.com/spiceai/spiceai/pull/2143
Update acknowledgements by @github-actions in https://github.com/spiceai/spiceai/pull/2141
Update Spice runtime to require explicit enablement for TLS by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2148
Update next version, ROADMAP, End Game template, move alpha release notes by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2145
Update EXTENSIBILITY to be correct, update README.md with Beta connectors by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2146
Add benchmark tests for duckdb acceleration by @sgrebnov in https://github.com/spiceai/spiceai/pull/2151
fix: Increase benchmark dataset setup timeout for Databricks by @peasee in https://github.com/spiceai/spiceai/pull/2149
Add LLMs to v1/models by @Jeadie in https://github.com/spiceai/spiceai/pull/2152
Dataset with acceleration enabled = false shouldn’t go through accelerated dataset hot reload by @Sevenannn in https://github.com/spiceai/spiceai/pull/2155
Show single error string in Spice SQL REPL command line by @Sevenannn in https://github.com/spiceai/spiceai/pull/2150
Add CI to build makefile install targets by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2157
Make the FlightClient struct cheap to clone by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2162
Fix bugs with local Unity Catalog server by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2160
Benchmark: data connector tests should continue on query error (s3) by @sgrebnov in https://github.com/spiceai/spiceai/pull/2161
fix hanging spiced when odbc loading data and received a cancel signal by @y-f-u in https://github.com/spiceai/spiceai/pull/2156
Improve MySql schema extraction and add InList and ScalarFunction expr support by @sgrebnov in https://github.com/spiceai/spiceai/pull/2158
Fix issue with use of EmbeddingConnector by @Jeadie in https://github.com/spiceai/spiceai/pull/2165
add client timeout for all object store providers by @y-f-u in https://github.com/spiceai/spiceai/pull/2168
Benchmark: include sqlite acceleration and enable more tests by @sgrebnov in https://github.com/spiceai/spiceai/pull/2172
feat: Use datafusion SQLite streaming updates by @peasee in https://github.com/spiceai/spiceai/pull/2171
Benchmark: include arrow acceleration and enable more tests (tpch_q22) by @sgrebnov in https://github.com/spiceai/spiceai/pull/2173
Localhost -> Sink; Fix Sink connector to not require schema via CREATE TABLE... and infer on first write by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2167
Fix misspelled acceleration engine name in benchmark tests by @sgrebnov in https://github.com/spiceai/spiceai/pull/2175
update spark bench catalog by @y-f-u in https://github.com/spiceai/spiceai/pull/2178
Benchmark: Discard first measurement of sql query, disable result caching by @Sevenannn in https://github.com/spiceai/spiceai/pull/2179
clear message when invalid params configured for accelerator by @y-f-u in https://github.com/spiceai/spiceai/pull/2177
Implement the Flight GetSchema API by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2169
Support AppendStream for SpiceAI data connector by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2181
Support MySQL BINARY, VARBINARY, Postgres BYTEA and improve MySQL auth error message by @sgrebnov in https://github.com/spiceai/spiceai/pull/2184
Benchmark: use SF1 for MySQL TPC-H tests by @sgrebnov in https://github.com/spiceai/spiceai/pull/2183
fix windows build broken by adding tokio unix signal by @y-f-u in https://github.com/spiceai/spiceai/pull/2193
Adds TLS support for flightsubscriber/flightpublisher tools by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2194
Update README output samples by @ewgenius in https://github.com/spiceai/spiceai/pull/2195
Update acknowledgements by @github-actions in https://github.com/spiceai/spiceai/pull/2197

Full Changelog: https://github.com/spiceai/spiceai/compare/v0.17.0-beta...v0.17.1-beta

Resources

Community

Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Discord or by email to get involved.

Twitter: @spice_ai
Discord: https://discord.gg/kZnTfneP5u
Telegram: Spice AI Discussion
Reddit: https://www.reddit.com/r/spiceai
Email: [email protected]