probeJsonWithTools

function probeJsonWithTools(provider, opts?): Promise<JsonWithToolsProbe>;

Defined in: core/src/capability.ts:132

Ask a model, on the wire, which JsonWithTools shape it actually serves. Then configure the provider with probe.use.

Both shapes are always tried, because a result you can read beats a result you have to trust: calls is the table to log, and “0/3 and 3/3” is what makes a later regression obvious. It costs samples × 2 short calls, made one after another, once, at whatever moment the app decides to ask. Boot is the usual one.

Probe each provider on its own, never a fallback chain: the shape is a fact about one model on one endpoint, and a chain answers with whichever member is up.

Only the openai wire and Gemini 3 read jsonWithTools. The anthropic wire (Anthropic, and coding plans like zai and kimi) always puts the schema in the prompt, responses always sends it as the response format, and Gemini 2.x always uses the prompt. There both shapes are one request and the probe measures it twice, so skip it.

Errors propagate. A probe that swallowed a bad key or a dead endpoint would report “this model cannot call tools”, which is a much worse thing to believe than “the call failed”.

It catches a structural failure: a model that cannot emit the call, every time, on the easiest question there is. A model that is merely unreliable will pass: gemini-3.8-flash answers this probe 3/3 and a real sales turn 3/10. That is the right line to draw. A harder probe would start failing good models for being terse, and “sometimes forgets its tools” is a prompt and eval problem, not a wire shape one.

const probe = await probeJsonWithTools(provider);
if (!probe.use) throw new Error(`${model} cannot use tools with a schema`);

Parameters

Parameter	Type
provider	Provider
opts	ProbeOptions

Returns

Promise<JsonWithToolsProbe>