Model client

What the client covers

• Requests and conversations: the OpenAI-compatible request styles for a single response and for a chat of messages.
• Realtime sessions: a session of events that stays open for continuous exchanges, run as a session of its own and not as a one-off request.
• Streaming: output read as it is produced, with backpressure, and input sent as a stream where the endpoint accepts it.
• Tool calls: the model asks for a tool, your agent decides whether to run it, and the result goes back, each call matched to its request.
• Typed output: structured output decoded into your own types and checked before your code sees it, or the raw output if you choose it.
• More than text: text, structured data, images, audio and files, where the endpoint and model accept them.

Model bindings

A node names the model it uses through a binding:

# nodes/summarize.node.ael
node summarize(input: Request) -> Summary {
    config {
        prompt: file("prompts/summarize.md");
        model: binding("primary_model");
        max_attempts: 2;
    }
    return write_summary(input);
}

The binding primary_model is set by the deployment or by configuration, not by the node. It records:

• the endpoint's address and which request style it speaks;
• the model;
• a reference to the credential, never the credential itself;
• a timeout;
• default parameters, and which parameters a single call may override.

Moving to another endpoint changes the binding, not your code. A binding without its model or credential is reported as a configuration error before any request is sent.

Parameters

A binding uses one of two modes:

• A parameter the endpoint or model does not accept fails with an explicit error. It is never dropped silently.
• Leaving parameters to their defaults never removes AEL's own limits: the output ceiling and the run's budget still apply.
• Where the endpoint does not say which value it used, the record says unknown; AEL does not invent one.

Compatibility

An address that calls itself OpenAI-compatible is not taken to promise every feature. AEL checks what each binding's endpoint and model can do. A feature that is not there fails with an explicit error, or uses a fallback that is documented for it; it is never guessed.

Where models run

You choose local or cloud models: local model servers, optional packages for major cloud providers, and clients for inference endpoints you run yourself.

• An endpoint that speaks the OpenAI-compatible interface is called directly, with no package.
• A cloud provider with its own sign-in or request format is reached through an optional official package; see the Package catalog.
• No model and no inference server is bundled into your program or installed for you.
• Prompt-driven agents run on small devices through AEL Gateway, and each device keeps its safe local behaviour when offline. See Microcontroller boards.

Usage and budgets

• Every call records its input, output and total tokens. Each count says where it came from: reported by the provider, estimated, known to be zero, or unknown. Unknown usage is never shown as zero.
• Retries, output repair, timeouts and cancellation draw on one shared budget, so nested retries cannot multiply cost. Routing, fallback, fan-out and verifier calls draw on it too.
• A run's time and token usage is reported by default when it ends; see Console and logging.

Failures

A call returns a typed result that keeps these failures apart:

• a parameter, input kind or capability the endpoint does not accept;
• an input over the endpoint's limit;
• a missing binding or credential;
• a request the endpoint refused as unauthorized;
• a timeout or a cancellation;
• an output that ended early or could not be decoded into your type.

An output that ended early never counts as a success, even if what arrived looks complete. Rate limits are respected, with waits that never pass the deadline.

Security

• Remote endpoints are reached over HTTPS, authenticated with TLS. A plain connection is only for a local endpoint that you explicitly allow.
• Credentials stay references: never in your source, your logs or a command-line argument.
• A model's output is data. It cannot grant a permission, call a tool your agent did not allow, or raise a limit.

Model policies

Model policies cover exactly one call, a bounded loop, routing, ordered fallback, parallel fan-out and an independent verifier model. They belong to agents and nodes: see Models and providers.

Testing

Tests run agents against mock and recorded model responses, with no network and no credentials. See Testing.