Weaviate

This is the implementation of the Weaviate for MindsDB.

Weaviate is an open-source vector database. It allows you to store data objects and vector embeddings from your favorite ML-models, and scale seamlessly into billions of data objects.

Prerequisites

Before proceeding, ensure the following prerequisites are met:

1. Install MindsDB locally via Docker or Docker Desktop.
2. To connect Weaviate to MindsDB, install the required dependencies following this instruction.
3. Install or ensure access to Weaviate.

Implementation

This handler uses weaviate-client python library connect to a weaviate instance.

The required arguments to establish a connection are:

• weaviate_url: url of the weaviate database
• weaviate_api_key: API key to authenticate with weaviate (in case of cloud instance).
• persistence_directory: directory to be used in case of local storage

Creating connection

In order to make use of this handler and connect to a Weaviate server in MindsDB, the following syntax can be used:

CREATE DATABASE weaviate_datasource
            WITH ENGINE = "weaviate",
            PARAMETERS = {
                "weaviate_url" : "https://sample.weaviate.network",
                "weaviate_api_key": "api-key"
};

CREATE DATABASE weaviate_datasource
            WITH ENGINE = "weaviate",
            PARAMETERS = {
                "weaviate_url" : "https://localhost:8080",
};

CREATE DATABASE weaviate_datasource
            WITH ENGINE = "weaviate",
            PARAMETERS = {
                "persistence_directory" : "db_path",
};

Dropping connection

To drop the connection, use this command

DROP DATABASE weaviate_datasource;

Creating tables

To insert data from a pre-existing table, use CREATE

CREATE TABLE weaviate_datascource.test
(SELECT * FROM sqlitedb.test);

As weaviate currently doesn’t support json field. So, this creates another table for the “metadata” field and a reference is created in the original table which points to its metadata entry.

Weaviate follows GraphQL conventions where classes (which are table schemas) start with a capital letter and properties start with a lowercase letter.

So whenever we create a table, the table’s name gets capitalized.

Dropping collections

To drop a Weaviate table use this command

DROP TABLE weaviate_datasource.tablename;

Querying and selecting

To query database using a search vector, you can use search_vector or embeddings in WHERE clause

SELECT * from weaviate_datasource.test
WHERE search_vector = '[3.0, 1.0, 2.0, 4.5]'
LIMIT 10;

Basic query

SELECT * from weaviate_datasource.test

You can use WHERE clause on dynamic fields like normal SQL

SELECT * FROM weaviate_datasource.createtest
WHERE category = "science";

Deleting records

You can delete entries using DELETE just like in SQL.

DELETE FROM weaviate_datasource.test
WHERE id IN (1, 2, 3);

Update is not supported by mindsdb vector database