Model client
A built-in client for OpenAI-compatible APIs, including realtime sessions, comes with the Open Engineering Language, with explicit or default parameters and no server or SDK to add. A console agent that calls a model is one self-contained program.
What the client covers
- Requests and conversations: the OpenAI-compatible request styles for a single response and for a chat of messages.
- Realtime sessions: a session of events that stays open for continuous exchanges, run as a session of its own and not as a one-off request.
- Streaming: output read as it is produced, with backpressure, and input sent as a stream where the endpoint accepts it.
- Tool calls: the model asks for a tool, your agent decides whether to run it, and the result goes back, each call matched to its request.
- Typed output: structured output decoded into your own types and checked before your code sees it, or the raw output if you choose it.
- More than text: text, structured data, images, audio and files, where the endpoint and model accept them.
Model bindings
A node names the model it uses through a binding:
# nodes/summarize.node.oel
node summarize(input: Str<4000>) -> Str<1000> {
config {
id: "summarize"; version: "1.0.0";
policy: {
kind: "model"; entry: "summarize"; model_binding: "primary_model";
mode: "bounded_loop"; max_attempts: 2; max_repairs: 0;
};
ports: [
{ id: "request"; direction: "input"; schema_type: "String<4000>";
modalities: ["text"]; min_count: 1; max_count: 1;
max_bytes: 4000; join: "all"; },
{ id: "summary"; direction: "output"; schema_type: "String<1000>";
modalities: ["text"]; min_count: 1; max_count: 1;
max_bytes: 1000; join: "all"; }
];
limits: {
ram_bytes: 16777216; cpu_millicores: 1000; cpu_millis: 1000;
duration_ms: 120000; input_tokens: 16384; output_tokens: 1024;
model_calls: 2; tool_calls: 0; steps: 32; concurrency: 1;
max_children: 0; max_depth: 1; queue_items: 1; queued_bytes: 65536;
stream_bytes_per_second: 0; storage_bytes: 0; connections: 0;
cost_microunits: 0;
};
authority: []; profile: "host"; state_domain: "request";
capabilities: []; tools: []; assets: [];
prompt: {
schema_version: 0;
sources: [{ kind: "project_file"; ref: "prompts/summarize.md"; }];
input_renderer: "render_input";
require_system: false; allow_run_override: false;
};
prompt_variables: [];
binding: {
schema_version: 0; binding_id: "primary_model";
adapter_id: "openai_core"; adapter_version: "1.0.0";
family: "openai_chat"; endpoint: "http://127.0.0.1:8080/v1";
model_id: "primary-model"; auth: { kind: "env"; ref: "MODEL_API_KEY"; };
max_attempts: 2; timeout_ms: 60000;
defaults: {}; allowed_overrides: [];
capabilities: {
input_modalities: ["text"]; output_modalities: ["text"];
instructions: true; structured_output: false; tools: false;
streaming: false; max_context_tokens: 32768; max_output_tokens: 1024;
parameters: []; usage: "bounded_estimate";
};
};
binding_capability_revision: "1";
}
return write_summary(input);
}
fn write_summary(request: Str<4000>) -> Str<1000> {
# Prepares the request for the model; this placeholder returns an empty summary.
return "";
}
The node's binding record describes primary_model. It records:
- the endpoint's address and which request style it speaks;
- the model;
- a reference to the credential, never the credential itself;
- a timeout;
- default parameters, and which parameters a single call may override.
Moving to another endpoint changes the binding, not your code. A binding without its model or credential is reported as a configuration error before any request is sent.
Parameters
A binding uses one of two modes:
| Mode | What is sent |
|---|---|
| Explicit | The parameters you set, each checked against the list a call may override. |
| Defaults | No parameters of your own: the endpoint's defaults apply. |
- A parameter the endpoint or model does not accept fails with an explicit error. It is never dropped silently.
- Leaving parameters to their defaults never removes the built-in limits: the output ceiling and the run's budget still apply.
- Where the endpoint does not say which value it used, the record says unknown; the runtime does not invent one.
Compatibility
An address that calls itself OpenAI-compatible is not taken to promise every feature. The model client checks what each binding's endpoint and model can do. A feature that is not there fails with an explicit error, or uses a fallback that is documented for it; it is never guessed.
Where models run
You choose local or cloud models: local model servers, optional packages for major cloud providers, and clients for inference endpoints you run yourself.
- An endpoint that speaks the OpenAI-compatible interface is called directly, with no package.
- A cloud provider with its own sign-in or request format is reached through an optional official package; see the Package catalog.
- No model and no inference server is bundled into your program or installed for you.
- Prompt-driven agents run on small devices through OEL Gateway, and each device keeps its safe local behavior when offline. See Microcontroller boards.
Usage and budgets
- Every call records its input, output and total tokens. Each count says where it came from: reported by the provider, estimated, known to be zero, or unknown. Unknown usage is never shown as zero.
- Retries, output repair, timeouts and cancellation draw on one shared budget, so nested retries cannot multiply cost. Routing, fallback, fan-out and verifier calls draw on it too.
- A run's time and token usage is reported by default when it ends; see Console and logging.
Failures
A call returns a typed result that keeps these failures apart:
- a parameter, input kind or capability the endpoint does not accept;
- an input over the endpoint's limit;
- a missing binding or credential;
- a request the endpoint refused as unauthorized;
- a timeout or a cancellation;
- an output that ended early or could not be decoded into your type.
An output that ended early never counts as a success, even if what arrived looks complete. Rate limits are respected, with waits that never pass the deadline.
Security
- Remote endpoints are reached over HTTPS, authenticated with TLS. A plain connection is only for a local endpoint that you explicitly allow.
- Credentials stay references: never in your source, your logs or a command-line argument.
- A model's output is data. It cannot grant a permission, call a tool your agent did not allow, or raise a limit.
Model policies
Model policies cover exactly one call, a bounded loop, routing, ordered fallback, parallel fan-out and an independent verifier model. They belong to agents and nodes: see Models and providers.
Testing
Tests run agents against mock and recorded model responses, with no network and no credentials. See Testing.