Spice v0.19.1-beta (Oct 14, 2024)

Announcing the release of Spice v0.19.1-beta 🔥

Spice v0.19.1 brings further performance and stability improvements to data connectors, including improved query push-down for file-based connectors (s3, abfs, file, ftp, sftp) that use Hive-style partitioning.

Highlights in v0.19.1

TPC-H and TPC-DS Coverage: Expanded coverage for TPC-H and TPC-DS benchmarking suites across accelerators and connectors.

GitHub Connector Array Filter: The GitHub connector now supports filter push down for the array_contains function in SQL queries using search query mode.

NSQL CLI Command: A new spice nsql CLI command has been added to easily query datasets with natural language from the command line.

Breaking changes

None

Contributors

• @peasee
• @Sevenannn
• @sgrebnov
• @karifabri
• @phillipleblanc
• @lukekim
• @Jeadie
• @slyons

Dependencies

• DataFusion Table Providers: Upgraded to rev f22b96601891856e02a73d482cca4f6100137df8.

What’s Changed

• release: Update helm chart for v0.19.0-beta by @peasee in https://github.com/spiceai/spiceai/pull/3024
• Set fail-fast = true for benchmark test by @Sevenannn in https://github.com/spiceai/spiceai/pull/2997
• release: Update next version and ROADMAP by @peasee in https://github.com/spiceai/spiceai/pull/3033
• Verify TPCH benchmark query results for Spark connector by @sgrebnov in https://github.com/spiceai/spiceai/pull/2993
• feat: Add x-spice-user-agent header to Spice REPL by @peasee in https://github.com/spiceai/spiceai/pull/2979
• Update to object store file formats documentation link by @karifabri in https://github.com/spiceai/spiceai/pull/3036
• Use teraswitch-runners for Linux x64 workflows + builds by @phillipleblanc in https://github.com/spiceai/spiceai/pull/3042
• feat: Support array contains in GitHub pushdown by @peasee in https://github.com/spiceai/spiceai/pull/2983
• Bump text-splitter from 0.16.1 to 0.17.0 by @dependabot in https://github.com/spiceai/spiceai/pull/2987
• Revert integration tests back to hosted runner by @phillipleblanc in https://github.com/spiceai/spiceai/pull/3046
• Tune Github runner resources to allow in memory TPCDS benchmark to run by @Sevenannn in https://github.com/spiceai/spiceai/pull/3025
• fix: add winver by @peasee in https://github.com/spiceai/spiceai/pull/3054
• refactor: Use is modifier for checking GitHub state filter by @peasee in https://github.com/spiceai/spiceai/pull/3056
• Enable merge_group checks for PR workflows by @phillipleblanc in https://github.com/spiceai/spiceai/pull/3058
• Fix issues with merge group by @phillipleblanc in https://github.com/spiceai/spiceai/pull/3059
• Validate in-memory arrow accelertion TPCDS result correctness by @Sevenannn in https://github.com/spiceai/spiceai/pull/3044
• Fix rev parsing for PR checks by @phillipleblanc in https://github.com/spiceai/spiceai/pull/3060
• Use ‘Accept’ header for /v1/sql/ and /v1/nsql by @Jeadie in https://github.com/spiceai/spiceai/pull/3032
• Verify Postgres acceleration TPCDS result correctness by @Sevenannn in https://github.com/spiceai/spiceai/pull/3043
• Add NSQL CLI REPL command by @lukekim in https://github.com/spiceai/spiceai/pull/2856
• Preserve query results order and add TPCH benchmark results verification for duckdb:file mode by @sgrebnov in https://github.com/spiceai/spiceai/pull/3034
• Refactor benchmark to include MySQL tpcds bench, tweaks to makefile target for generating mysql tpcds data by @Sevenannn in https://github.com/spiceai/spiceai/pull/2967
• Support runtime parameter for sql_query_keep_partition_by_columns & enable by default by @phillipleblanc in https://github.com/spiceai/spiceai/pull/3065
• Document TPC-DS limitations: EXCEPT, INTERSECT, duplicate names by @sgrebnov in https://github.com/spiceai/spiceai/pull/3069
• Adding ABFS benchmark by @slyons in https://github.com/spiceai/spiceai/pull/3062
• Add support for GitHub app installation auth for GitHub connector by @ewgenius in https://github.com/spiceai/spiceai/pull/3063
• docs: Document stack overflow workaround, add helper script by @peasee in https://github.com/spiceai/spiceai/pull/3070
• Tune MySQL TPCDS image to allow for successful benchmark test run by @Sevenannn in https://github.com/spiceai/spiceai/pull/3067
• Automatically infer partitions for hive-style partitioned files for object store based connectors by @phillipleblanc in https://github.com/spiceai/spiceai/pull/3073
• Support hf_token from params/secrets by @Jeadie in https://github.com/spiceai/spiceai/pull/3071
• Inherit embedding columns from source, when available. by @Jeadie in https://github.com/spiceai/spiceai/pull/3045
• Validate identifiers for component names by @phillipleblanc in https://github.com/spiceai/spiceai/pull/3079
• docs: Add workaround for TPC-DS Q97 in MySQL by @peasee in https://github.com/spiceai/spiceai/pull/3080
• Document TPC-DS Postgres column alias in a CASE statement limitation by @sgrebnov in https://github.com/spiceai/spiceai/pull/3083
• Update plan snapshots for TPC-H bench queries by @sgrebnov in https://github.com/spiceai/spiceai/pull/3088
• Update Datafusion crate to include recent unparsing fixes by @sgrebnov in https://github.com/spiceai/spiceai/pull/3089
• Sample SQL table data tool and API by @Jeadie in https://github.com/spiceai/spiceai/pull/3081
• chore: Update datafusion-table-providers by @peasee in https://github.com/spiceai/spiceai/pull/3090
• Add hive_infer_partitions to remaining object store connectors by @phillipleblanc in https://github.com/spiceai/spiceai/pull/3086
• deps: Update datafusion-table-providers by @peasee in https://github.com/spiceai/spiceai/pull/3093
• For local embedding models, return usage input tokens. by @Jeadie in https://github.com/spiceai/spiceai/pull/3095
• Update end_game.md with Accelerator/Connector criteria check by @slyons in https://github.com/spiceai/spiceai/pull/3092
• Update TPC-DS Q90 by @sgrebnov in https://github.com/spiceai/spiceai/pull/3094
• docs: Add RC connector criteria by @peasee in https://github.com/spiceai/spiceai/pull/3026
• Update version to 0.19.1-beta by @sgrebnov in https://github.com/spiceai/spiceai/pull/3101

Full Changelog: https://github.com/spiceai/spiceai/compare/v0.19.0-beta...v0.19.1-beta

Resources

• Getting started with Spice.ai
• Documentation

Community

Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Discord or by email to get involved.

• Twitter: @spice_ai
• Discord: https://discord.gg/kZnTfneP5u
• Telegram: Spice AI Discussion
• Reddit: https://www.reddit.com/r/spiceai
• Email: [email protected]

• ←Previous
• Next→