Browse the documentation

Early preview

Model client

A built-in client for OpenAI-compatible APIs, including realtime sessions, comes with the Open Engineering Language, with explicit or default parameters and no server or SDK to add. A console agent that calls a model is one self-contained program.

What the client covers

  • Requests and conversations: the OpenAI-compatible request styles for a single response and for a chat of messages.
  • Realtime sessions: a session of events that stays open for continuous exchanges, run as a session of its own and not as a one-off request.
  • Streaming: output read as it is produced, with backpressure, and input sent as a stream where the endpoint accepts it.
  • Tool calls: the model asks for a tool, your agent decides whether to run it, and the result goes back, each call matched to its request.
  • Typed output: structured output decoded into your own types and checked before your code sees it, or the raw output if you choose it.
  • More than text: text, structured data, images, audio and files, where the endpoint and model accept them.

Model bindings

A node names the model it uses through a binding:

# nodes/summarize.node.oel
node summarize(input: Str<4000>) -> Str<1000> {
    config {
        id: "summarize"; version: "1.0.0";
        policy: {
            kind: "model"; entry: "summarize"; model_binding: "primary_model";
            mode: "bounded_loop"; max_attempts: 2; max_repairs: 0;
        };
        ports: [
            { id: "request"; direction: "input"; schema_type: "String<4000>";
              modalities: ["text"]; min_count: 1; max_count: 1;
              max_bytes: 4000; join: "all"; },
            { id: "summary"; direction: "output"; schema_type: "String<1000>";
              modalities: ["text"]; min_count: 1; max_count: 1;
              max_bytes: 1000; join: "all"; }
        ];
        limits: {
            ram_bytes: 16777216; cpu_millicores: 1000; cpu_millis: 1000;
            duration_ms: 120000; input_tokens: 16384; output_tokens: 1024;
            model_calls: 2; tool_calls: 0; steps: 32; concurrency: 1;
            max_children: 0; max_depth: 1; queue_items: 1; queued_bytes: 65536;
            stream_bytes_per_second: 0; storage_bytes: 0; connections: 0;
            cost_microunits: 0;
        };
        authority: []; profile: "host"; state_domain: "request";
        capabilities: []; tools: []; assets: [];
        prompt: {
            schema_version: 0;
            sources: [{ kind: "project_file"; ref: "prompts/summarize.md"; }];
            input_renderer: "render_input";
            require_system: false; allow_run_override: false;
        };
        prompt_variables: [];
        binding: {
            schema_version: 0; binding_id: "primary_model";
            adapter_id: "openai_core"; adapter_version: "1.0.0";
            family: "openai_chat"; endpoint: "http://127.0.0.1:8080/v1";
            model_id: "primary-model"; auth: { kind: "env"; ref: "MODEL_API_KEY"; };
            max_attempts: 2; timeout_ms: 60000;
            defaults: {}; allowed_overrides: [];
            capabilities: {
                input_modalities: ["text"]; output_modalities: ["text"];
                instructions: true; structured_output: false; tools: false;
                streaming: false; max_context_tokens: 32768; max_output_tokens: 1024;
                parameters: []; usage: "bounded_estimate";
            };
        };
        binding_capability_revision: "1";
    }
    return write_summary(input);
}

fn write_summary(request: Str<4000>) -> Str<1000> {
    # Prepares the request for the model; this placeholder returns an empty summary.
    return "";
}

The node's binding record describes primary_model. It records:

  • the endpoint's address and which request style it speaks;
  • the model;
  • a reference to the credential, never the credential itself;
  • a timeout;
  • default parameters, and which parameters a single call may override.

Moving to another endpoint changes the binding, not your code. A binding without its model or credential is reported as a configuration error before any request is sent.

Parameters

A binding uses one of two modes:

ModeWhat is sent
ExplicitThe parameters you set, each checked against the list a call may override.
DefaultsNo parameters of your own: the endpoint's defaults apply.
  • A parameter the endpoint or model does not accept fails with an explicit error. It is never dropped silently.
  • Leaving parameters to their defaults never removes the built-in limits: the output ceiling and the run's budget still apply.
  • Where the endpoint does not say which value it used, the record says unknown; the runtime does not invent one.

Compatibility

An address that calls itself OpenAI-compatible is not taken to promise every feature. The model client checks what each binding's endpoint and model can do. A feature that is not there fails with an explicit error, or uses a fallback that is documented for it; it is never guessed.

Where models run

You choose local or cloud models: local model servers, optional packages for major cloud providers, and clients for inference endpoints you run yourself.

  • An endpoint that speaks the OpenAI-compatible interface is called directly, with no package.
  • A cloud provider with its own sign-in or request format is reached through an optional official package; see the Package catalog.
  • No model and no inference server is bundled into your program or installed for you.
  • Prompt-driven agents run on small devices through OEL Gateway, and each device keeps its safe local behavior when offline. See Microcontroller boards.

Usage and budgets

  • Every call records its input, output and total tokens. Each count says where it came from: reported by the provider, estimated, known to be zero, or unknown. Unknown usage is never shown as zero.
  • Retries, output repair, timeouts and cancellation draw on one shared budget, so nested retries cannot multiply cost. Routing, fallback, fan-out and verifier calls draw on it too.
  • A run's time and token usage is reported by default when it ends; see Console and logging.

Failures

A call returns a typed result that keeps these failures apart:

  • a parameter, input kind or capability the endpoint does not accept;
  • an input over the endpoint's limit;
  • a missing binding or credential;
  • a request the endpoint refused as unauthorized;
  • a timeout or a cancellation;
  • an output that ended early or could not be decoded into your type.

An output that ended early never counts as a success, even if what arrived looks complete. Rate limits are respected, with waits that never pass the deadline.

Security

  • Remote endpoints are reached over HTTPS, authenticated with TLS. A plain connection is only for a local endpoint that you explicitly allow.
  • Credentials stay references: never in your source, your logs or a command-line argument.
  • A model's output is data. It cannot grant a permission, call a tool your agent did not allow, or raise a limit.

Model policies

Model policies cover exactly one call, a bounded loop, routing, ordered fallback, parallel fan-out and an independent verifier model. They belong to agents and nodes: see Models and providers.

Testing

Tests run agents against mock and recorded model responses, with no network and no credentials. See Testing.