Spice v0.14-alpha (June 17, 2024)

By Spice AI (@spice_ai) |
Categories:
Tags:

The v0.14-alpha release focuses on enhancing accelerated dataset performance and data integrity, with support for configuring primary keys and indexes. Additionally, the GraphQL data connector been introduced, along with improved dataset registration and loading error information.

Highlights

  • Accelerated Datasets: Ensure data integrity using primary key and unique index constraints. Configure conflict handling to either upsert new data or drop it. Create indexes on frequently filtered columns for faster queries on larger datasets.

  • GraphQL Data Connector: Initial support for using GraphQL as a data source.

Example Spicepod showing how to use primary keys and indexes with accelerated datasets:

datasets:
  - from: eth.blocks
    name: blocks
    acceleration:
      engine: duckdb # Use DuckDB acceleration engine
      primary_key: '(hash, timestamp)'
      indexes:
        number: enabled # same as `CREATE INDEX ON blocks (number);`
        '(number, hash)': unique # same as `CREATE UNIQUE INDEX ON blocks (number, hash);`
      on_conflict:
        '(hash, number)': drop # possible values: drop (default), upsert
        '(hash, timestamp)': upsert

Primary Keys, constraints, and indexes are currently supported when using SQLite, DuckDB, and PostgreSQL acceleration engines.

Learn more with the indexing quickstart and the primary key sample.

Read the Local Acceleration documentation.

Breaking Changes

None.

Contributors

  • @phillipleblanc
  • @ewgenius
  • @sgrebnov
  • @Jeadie
  • @digadeesh
  • @gloomweaver
  • @y-f-u
  • @lukekim
  • @edmondop

What’s Changed

Dependencies

  • Apache DataFusion: Upgraded from 38.0.0 to 39.0.0
  • Apache Arrow/Parquet: Upgraded from 51.0.0 to 52.0.0
  • Rust: Upgraded from 1.78.0 to 1.79.0

Commits

Full Changelog: https://github.com/spiceai/spiceai/compare/v0.13.3-alpha...v0.14.0-alpha

Resources

Community

Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Discord or by email to get involved.