Spice v0.12.2-alpha (May 13, 2024)

The v0.12.2-alpha release introduces data streaming and key-pair authentication for the Snowflake data connector, enables general append mode data refreshes for time-series data, improves connectivity error messages, adds nested folders support for the S3 data connector, and exposes nodeSelector and affinity keys in the Helm chart for better Kubernetes management.

Highlights

• Improved Connectivity Error Messages: Error messages provide clearer, actionable guidance for misconfigured settings or unreachable data connectors.

• Snowflake Data Connector Improvements: Enables data streaming by default and adds support for key-pair authentication in addition to passwords.

• API for Refresh SQL Updates: Update dataset Refresh SQL via API.

• Append Data Refresh: Append mode data refreshes for time-series data are now supported for all data connectors. Specify a dataset time_column with refresh_mode: append to only fetch data more recent than the latest local data.

• Docker Image Update: The spiceai/spiceai:latest Docker image now includes the ODBC data connector. For a smaller footprint, use spiceai/spiceai:latest-slim.

• Helm Chart Improvements: nodeSelector and affinity keys are now supported in the Helm chart for improved Kubernetes deployment management.

Breaking Changes

• API to trigger accelerated dataset refreshes has changed from POST /v1/datasets/:name/refresh to POST /v1/datasets/:name/acceleration/refresh to be consistent with the Spicepod.yaml structure.

Contributors

• @mach-kernel
• @y-f-u
• @sgrebnov
• @ewgenius
• @Jeadie
• @Sevenannn
• @digadeesh
• @phillipleblanc
• @lukekim

What’s Changed

• Fix list type support in spark connect by @y-f-u in https://github.com/spiceai/spiceai/pull/1341
• Add nested folder support in S3 Parquet connector by @y-f-u in https://github.com/spiceai/spiceai/pull/1342
• Improves S3 connector using DataFusion ListingTable table provider by @y-f-u in https://github.com/spiceai/spiceai/pull/1326
• Update ROADMAP May 6, 2024 by @lukekim in https://github.com/spiceai/spiceai/pull/1315
• List flightsql and snowflake as supported connectors in README.md by @sgrebnov in https://github.com/spiceai/spiceai/pull/1317
• Helm chart for v0.12.1-alpha by @ewgenius in https://github.com/spiceai/spiceai/pull/1323
• Read sqlite_file param and use it as path by @Sevenannn in https://github.com/spiceai/spiceai/pull/1309
• Compile spiced with release feature in docker image by @ewgenius in https://github.com/spiceai/spiceai/pull/1324
• Add support for Snowflake key-pair authentication by @sgrebnov in https://github.com/spiceai/spiceai/pull/1314
• Wrap postgres errors in common DataConnectorError by @ewgenius in https://github.com/spiceai/spiceai/pull/1327
• Fix TPCH tests runner by @sgrebnov in https://github.com/spiceai/spiceai/pull/1330
• Spice CLI support for Snowflake key-pair auth by @sgrebnov in https://github.com/spiceai/spiceai/pull/1325
• sql_provider_datafusion: Support TimestampMicrosecond, Date32, Date64 by @mach-kernel in https://github.com/spiceai/spiceai/pull/1329
• Resolve dangling reference for SQLite by @Sevenannn in https://github.com/spiceai/spiceai/pull/1312
• Select columns from Spark Dataframe according to projected_schema by @Sevenannn in https://github.com/spiceai/spiceai/pull/1336
• Expose nodeselector and affinity keys in Helm chart by @mach-kernel in https://github.com/spiceai/spiceai/pull/1338
• Use streaming for Snowflake queries by @sgrebnov in https://github.com/spiceai/spiceai/pull/1337
• Publish ODBC images by @mach-kernel in https://github.com/spiceai/spiceai/pull/1271
• Include Postgres acceleration engine to types support tests by @sgrebnov in https://github.com/spiceai/spiceai/pull/1343
• Refactor dataconnector providers getters to return common DataConnectorResult and DataConnectorError by @ewgenius in https://github.com/spiceai/spiceai/pull/1339
• s3 csv support to validate the listing table extensibility by @y-f-u in https://github.com/spiceai/spiceai/pull/1344
• Move model code into separate, feature-flagged crate by @Jeadie in https://github.com/spiceai/spiceai/pull/1335
• Initial setup for federated queries by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1350
• Refactor dbconnection errors, and catch invalid postgres table name case by @ewgenius in https://github.com/spiceai/spiceai/pull/1353
• Rename default datafusion catalog to “spice”, add internal “spice.runtime” schema by @ewgenius in https://github.com/spiceai/spiceai/pull/1359
• Add API to set Refresh SQL for accelerated table by @sgrebnov in https://github.com/spiceai/spiceai/pull/1356
• Set next version to v0.12.2 by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1367
• Upgrade to DataFusion 38 by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1368
• Incremental append based on time column by @y-f-u in https://github.com/spiceai/spiceai/pull/1360
• Update README.md to include correct output when running show tables from quickstart by @digadeesh in https://github.com/spiceai/spiceai/pull/1371

Full Changelog: https://github.com/spiceai/spiceai/compare/v0.12.1-alpha...v0.12.2-alpha

Resources

• Getting started with Spice.ai
• Documentation

Community

Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Discord or by email to get involved.

• Twitter: @spice_ai
• Discord: https://discord.gg/kZnTfneP5u
• Telegram: Spice AI Discussion
• Reddit: https://www.reddit.com/r/spiceai
• Email: [email protected]

• ←Previous
• Next→