The LLM() Function

MindsDB provides the LLM() function that lets users incorporate the LLM-generated output directly into the data queries.

Prerequisites

The LLM() function requires a large language model, which can be defined in the following ways:

• By setting the default_llm parameter in the MindsDB configuration file.

• By saving the default model in the MindsDB Editor under Settings.

• By defining the environment variables as below, choosing one of the available model providers.

  OpenAI

  Here are the environment variables for the OpenAI provider:

  LLM_FUNCTION_MODEL_NAME
  LLM_FUNCTION_TEMPERATURE
  LLM_FUNCTION_MAX_RETRIES
  LLM_FUNCTION_MAX_TOKENS
  LLM_FUNCTION_BASE_URL
  OPENAI_API_KEY
  LLM_FUNCTION_API_ORGANIZATION
  LLM_FUNCTION_REQUEST_TIMEOUT
  

  Note that the values stored in the environment variables are specific for each provider.

  Anthropic

  Here are the environment variables for the Anthropic provider:

  LLM_FUNCTION_MODEL_NAME
  LLM_FUNCTION_TEMPERATURE
  LLM_FUNCTION_MAX_TOKENS
  LLM_FUNCTION_TOP_P
  LLM_FUNCTION_TOP_K
  LLM_FUNCTION_DEFAULT_REQUEST_TIMEOUT
  LLM_FUNCTION_API_KEY
  LLM_FUNCTION_BASE_URL
  

  Note that the values stored in the environment variables are specific for each provider.

  LiteLLM

  Here are the environment variables for the LiteLLM provider:

  LLM_FUNCTION_MODEL_NAME
  LLM_FUNCTION_TEMPERATURE
  LLM_FUNCTION_API_BASE
  LLM_FUNCTION_MAX_RETRIES
  LLM_FUNCTION_MAX_TOKENS
  LLM_FUNCTION_TOP_P
  LLM_FUNCTION_TOP_K
  

  Note that the values stored in the environment variables are specific for each provider.

  Ollama

  Here are the environment variables for the Ollama provider:

  LLM_FUNCTION_BASE_URL
  LLM_FUNCTION_MODEL_NAME
  LLM_FUNCTION_TEMPERATURE
  LLM_FUNCTION_TOP_P
  LLM_FUNCTION_TOP_K
  LLM_FUNCTION_REQUEST_TIMEOUT
  LLM_FUNCTION_FORMAT
  LLM_FUNCTION_HEADERS
  LLM_FUNCTION_NUM_PREDICT
  LLM_FUNCTION_NUM_CTX
  LLM_FUNCTION_NUM_GPU
  LLM_FUNCTION_REPEAT_PENALTY
  LLM_FUNCTION_STOP
  LLM_FUNCTION_TEMPLATE
  

  Note that the values stored in the environment variables are specific for each provider.

  Nvidia NIMs

  Here are the environment variables for the Nvidia NIMs provider:

  LLM_FUNCTION_BASE_URL
  LLM_FUNCTION_MODEL_NAME
  LLM_FUNCTION_TEMPERATURE
  LLM_FUNCTION_TOP_P
  LLM_FUNCTION_REQUEST_TIMEOUT
  LLM_FUNCTION_FORMAT
  LLM_FUNCTION_HEADERS
  LLM_FUNCTION_NUM_PREDICT
  LLM_FUNCTION_NUM_CTX
  LLM_FUNCTION_NUM_GPU
  LLM_FUNCTION_REPEAT_PENALTY
  LLM_FUNCTION_STOP
  LLM_FUNCTION_TEMPLATE
  LLM_FUNCTION_NVIDIA_API_KEY
  

  Note that the values stored in the environment variables are specific for each provider.

OpenAI-compatible model providers can be used like OpenAI models.

There is a number of OpenAI-compatible model providers including OpenRouter or vLLM. To use models via these providers, users need to define the base URL and the API key of the provider.

Here is an example of using OpenRouter.

LLM_FUNCTION_MODEL_NAME = "mistralai/devstral-small-2505"
LLM_FUNCTION_BASE_URL = "https://openrouter.ai/api/v1"
OPENAI_API_KEY = "openrouter-api-key"

Usage

You can use the LLM() function to simply ask a question and get an answer.

SELECT LLM('How many planets are there in the solar system?');

Here is the output:

+------------------------------------------+
| llm                                      |
+------------------------------------------+
| There are 8 planets in the solar system. |
+------------------------------------------+

Moreover, you can use the LLM() function with your data to swiftly complete tasks such as text generation or summarization.

SELECT
    comment,
    LLM('Describe the comment''s category in one word: ' || comment) AS category
FROM example_db.user_comments;

Here is the output:

+--------------------------+----------+
| comment                  | category |
+--------------------------+----------+
| I hate tacos             | Dislike  |
| I want to dance          | Desire   |
| Baking is not a big deal | Opinion  |
+--------------------------+----------+