LLM Component

Available since SDK 1.14.0

The LLM component requires cereal-sdk 1.14.0 or later. Set your manifest's sdk_version to 1.14.0 (or higher). Nothing fills it in for you, and without it a user on an older client gets a runtime failure instead of a prompt to update.

Also declare the llm Script feature with "features": ["llm"] in the same manifest. Without it, provider.llm() throws and the task fails with an error telling you to add it. You need both keys: features doesn't replace sdk_version, because older clients ignore features.

The LLM component lets your script call the large language model the user has set up in the Cereal application under Settings → AI (Anthropic, or any OpenAI-compatible provider such as OpenAI, OpenRouter, Ollama or LM Studio).

The user brings their own API key and chooses the provider and model. Your script can't pick or override the model.
The Cereal application makes the HTTP call, so the API key never reaches your script, and you need no HTTP client or JSON library of your own.
JSON goes in and comes out as Strings, so you can parse it with any library you like.

`complete()`

The component has one method: suspend fun complete(request: LlmRequest): LlmResponse. Each call makes exactly one model call and keeps no state between calls: your script holds the conversation.

val response = provider.llm().complete(
    LlmRequest(
        system = "You summarise product pages in one sentence.",
        turns = listOf(LlmTurn.User(listOf(UserPart.Text(pageText)))),
    ),
)
provider.logger().info(response.text)

Request

Field	Type	Description
`system`	`String?`	System prompt, kept separate from the turns.
`turns`	`List<LlmTurn>`	The conversation in order: your `LlmTurn.User` turns and the model's `response.assistantTurn`s.
`tools`	`List<LlmTool>`	Tools the model may call: `LlmTool(name, description, inputSchemaJson)`.
`toolChoice`	`ToolChoice`	`Auto` (default) lets the model decide; `None` forbids tool calls on this turn.
`responseSchemaJson`	`String?`	JSON Schema for a structured answer. Can be combined with `tools`.
`maxTokens`	`Int`	Upper bound on output tokens. Defaults to `LlmRequest.DEFAULT_MAX_TOKENS` (4096).

A user turn holds UserParts: Text(text), Image(bytes, mediaType) (ImageMediaType.Jpeg, Png, Gif or Webp) and ToolResult(toolCallId, text, isError = false).

Response

Field	Description
`text`	All text of the answer, concatenated.
`toolCalls`	`LlmToolCall(id, name, argumentsJson)` for each tool the model wants run. Arguments are a JSON object string.
`structuredJson`	The validated structured answer. Set only when you passed `responseSchemaJson` and `stopReason` is `EndTurn`.
`stopReason`	`EndTurn`, `ToolUse`, `MaxTokens` (cut off at `maxTokens`) or `Refusal` (the model or a content filter declined).
`usage`	`LlmUsage(inputUncached, cacheRead, cacheWrite, output)` in tokens. The SDK never reports cost.
`model`	The model that actually answered.
`assistantTurn`	This answer as a turn. Append it to `turns` as is to continue the conversation.

assistantTurn carries provider data your script can't see (thinking blocks, thought signatures) that must go back unchanged. Scripts can't construct assistant turns or responses themselves; the compiler rejects it.

A refusal is a normal outcome reported through stopReason, not an exception.

Tool loop

The model can't run tools itself. When it wants one, complete() returns stopReason == ToolUse with the calls in toolCalls; your script runs them and calls complete() again with the model's turn plus the results. Repeat until the model stops asking. Send tools and responseSchemaJson on every call, so the loop can end in a structured answer.

val getPrice = LlmTool(
    name = "get_price",
    description = "Returns the current price of a SKU.",
    inputSchemaJson = """{"type":"object","properties":{"sku":{"type":"string"}},"required":["sku"],"additionalProperties":false}""",
)
var request = LlmRequest(
    turns = listOf(LlmTurn.User(listOf(UserPart.Text("Which of ABC-1, ABC-2 is cheapest?")))),
    tools = listOf(getPrice),
    responseSchemaJson = """{"type":"object","properties":{"sku":{"type":"string"}},"required":["sku"],"additionalProperties":false}""",
)

repeat(5) { // Bound the loop: every call costs the user tokens.
    val response = provider.llm().complete(request)
    if (response.stopReason != StopReason.ToolUse) {
        return response.structuredJson // null on MaxTokens or Refusal
    }
    // Answer every call of this turn in ONE user turn, right after the model's own turn.
    val results = response.toolCalls.map { call ->
        val price = lookUpPrice(call.argumentsJson)
        if (price != null) UserPart.ToolResult(call.id, price) else UserPart.ToolResult(call.id, "Unknown SKU.", isError = true)
    }
    request = request.copy(turns = request.turns + response.assistantTurn + LlmTurn.User(results))
}

Mark a failed tool with isError = true and describe the failure in text; the model can then try something else.

The sample script contains a complete, tested version of this loop (CheapestSkuPicker).

Schemas

Tool inputSchemaJson and responseSchemaJson must stay within the JSON Schema subset that Anthropic supports, so the same schema works on every provider:

every object sets "additionalProperties": false;
no numeric constraints (minimum, maximum, multipleOf, ...);
no length constraints (minLength, maxLength, minItems, ...).

The Cereal application validates every tool call's arguments and the structured answer against your schemas, also on providers that don't enforce schemas themselves, so you can trust their shape. A call to a tool you didn't declare fails validation too. A schema that isn't valid JSON Schema is rejected before anything is sent (LlmException.InvalidRequest).

Errors

complete() throws a sealed LlmException, so you can handle every kind exhaustively:

Exception	When
`NotConfigured`	The user hasn't set up an AI provider. No call is made, so falling back costs nothing.
`Authentication`	The provider rejected the user's API key.
`RateLimited(retryAfter)`	Still rate-limited after the client's retries. `retryAfter` is the provider's requested wait, if given.
`InvalidRequest`	The provider rejected the request (bad schema, oversized image, too-long context, model lacks a capability, model not found). The message is the provider's explanation.
`InvalidOutput(raw, assistantTurn)`	The structured answer or tool arguments don't match your schema. `raw` is the offending output.
`Unavailable`	Network errors, timeouts and provider server errors, after the client's retries.

The client retries rate limits and server errors a few times with backoff before throwing. It never retries InvalidOutput, so a retry never spends the user's tokens without your say: to retry, append e.assistantTurn and a user turn explaining what was wrong, and call again.

An uncaught LlmException fails the task and shows its message as the task status, for example "No AI provider is set up. Choose one in Settings → AI." Cancelling the task cancels a call in flight.

Falling back when no provider is set up

Many users won't configure a provider. Catch NotConfigured and do something useful without the model, or tell the user where to set it up:

val pick = try {
    askModelForCheapest(provider)
} catch (e: LlmException.NotConfigured) {
    provider.logger().warn("${e.message} Picking the lowest price without AI.")
    prices.minBy { it.value }.key
}

Privacy and usage

Prompts, images, tool arguments and answers are never written to the task log, the app log or crash reports.
After each call the task log shows one line with the answering model and its token usage. Don't log prompt or response content yourself if it may contain personal data.
The user pays their provider directly. Point them to their provider's spend limits if your script makes many calls.

Testing

cereal-test-utils ships a FakeLlmComponent, so you can test your LLM logic without a network or an API key. See Testing.