MindsDB integrates with numerous AI and ML frameworks that are made available via the AI/ML engines. The AI/ML engines are used to create models based on the particular AI/ML framework.
Description
The CREATE ML_ENGINE command creates an ML engine that uses one of the available AI/ML handlers.
Syntax
Before creating an AI/ML engine, make sure that the AI/ML handler of your interest is available by querying for the ML handlers.
SELECT *
FROM information_schema.handlers;
-- or
SHOW HANDLERS;
If you can’t find the AI/ML handler of your interest, you can contribute by building a new AI/ML handler.
Please note that in the process of contributing new AI.ML engines, ML engines and/or their tests will only run correctly if all dependencies listed in the requirements.txt file are installed beforehand.
If you find the AI/ML handler of your interest, then you can create an AI/ML engine using this command:
CREATE ML_ENGINE [IF NOT EXISTS] ml_engine_name
FROM handler_name
[USING argument_key = argument_value];
Please replace ml_engine_name, handler_name, and optionally, argument_key and argument_value with the real values.
Please do not use the same ml_engine_name as the handler_name to avoid issue while dropping the ML engine.
To verify that your AI/ML engine was successfully created, run the command below:
SELECT *
FROM information_schema.ml_engines;
-- or
SHOW ML_ENGINES;
If you want to drop an ML engine, run the command below:
DROP ML_ENGINE ml_engine_name;
Example
Let’s check what AI/ML handlers are currently available:
On execution, we get:
+-------------------+--------------------+-------------------------------------------------------+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------+----------------+-----------------------------------------------------------------------------+
| NAME | TITLE | DESCRIPTION | VERSION | CONNECTION_ARGS | IMPORT_SUCCESS | IMPORT_ERROR |
+-------------------+--------------------+-------------------------------------------------------+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------+----------------+-----------------------------------------------------------------------------+
| "ray_serve" | "RayServe" | "MindsDB handler for Ray Serve" | "0.0.1" | "[NULL]" | "true" | "[NULL]" |
| "neuralforecast" | "NeuralForecast" | "MindsDB handler for Nixtla's NeuralForecast package" | "0.0.1" | "[NULL]" | "true" | "[NULL]" |
| "autosklearn" | "Auto-Sklearn" | "MindsDB handler for Auto-Sklearn" | "0.0.2" | "[NULL]" | "false" | "No module named 'autosklearn'" |
| "mlflow" | "MLFlow" | "MindsDB handler for MLflow" | "0.0.2" | "[NULL]" | "false" | "No module named 'mlflow'" |
| "openai" | "OpenAI" | "MindsDB handler for OpenAI" | "0.0.1" | "[NULL]" | "true" | "[NULL]" |
| "merlion" | "Merlion" | "MindsDB handler for Merlion" | "0.0.1" | "[NULL]" | "false" | "object.__init__() takes exactly one argument (the instance to initialize)" |
| "byom" | "BYOM" | "MindsDB handler for BYOM" | "0.0.1" | "{'code': {'type': 'path', 'description': 'The path to model code'}, 'modules': {'type': 'path', 'description': 'The path to model requirements'}}" | "true" | "[NULL]" |
| "ludwig" | "Ludwig" | "MindsDB handler for Ludwig AutoML" | "0.0.2" | "[NULL]" | "false" | "No module named 'dask'" |
| "lightwood" | "Lightwood" | "[NULL]" | "1.0.0" | "[NULL]" | "true" | "[NULL]" |
| "huggingface_api" | "Hugging Face API" | "MindsDB handler for Auto-Sklearn" | "0.0.2" | "[NULL]" | "false" | "No module named 'hugging_py_face'" |
| "statsforecast" | "StatsForecast" | "MindsDB handler for Nixtla's StatsForecast package" | "0.0.0" | "[NULL]" | "true" | "[NULL]" |
| "huggingface" | "Hugging Face" | "MindsDB handler for Higging Face" | "0.0.1" | "[NULL]" | "true" | "[NULL]" |
| "TPOT" | "Tpot" | "MindsDB handler for TPOT " | "0.0.2" | "[NULL]" | "false" | "No module named 'tpot'" |
| "langchain" | "LangChain" | "MindsDB handler for LangChain" | "0.0.1" | "[NULL]" | "true" | "[NULL]" |
| "autokeras" | "Autokeras" | "MindsDB handler for Autokeras AutoML" | "0.0.1" | "[NULL]" | "false" | "No module named 'autokeras'" |
+-------------------+--------------------+-------------------------------------------------------+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------+----------------+-----------------------------------------------------------------------------+
Here we create an AI/ML engine using the OpenAI handler and providing an OpenAI API key in the USING clause.
CREATE ML_ENGINE my_openai_engine
FROM openai
USING
openai_api_key = '<your opanai api key>';
On execution, we get:
Query successfully completed
Now let’s verify that our ML engine exists.
On execution, we get:
+-------------------+------------+------------------------------------------------------+
|NAME |HANDLER |CONNECTION_DATA |
+-------------------+------------+------------------------------------------------------+
|lightwood |lightwood |{"key":["password"],"value":[""]} |
|huggingface |huggingface |{"key":["password"],"value":[""]} |
|openai |openai |{"key":["password"],"value":[""]} |
|my_openai_engine |openai |{"key":["openai_api_key","password"],"value":["",""]} |
+-------------------+------------+------------------------------------------------------+
Please note that the USING clause is optional, as it depends on the AI/ML handler whether it requires some arguments or not. Here, we created an OpenAI engine and provided own API key.
After creating your ML engine, you can create a model like this:
CREATE MODEL my_model
PREDICT answer
USING
engine = 'my_openai_engine',
prompt_template = 'ask a question to a model'
The USING clause specifies the ML engine to be used for creating a new model.
