9 releases
Uses new Rust 2024
| 0.11.0-beta.9 | May 27, 2026 |
|---|---|
| 0.11.0-beta.8 | May 18, 2026 |
| 0.11.0-beta.6 | Mar 25, 2026 |
| 0.11.0-beta.5 | Jan 14, 2026 |
| 0.11.0-beta.1 | Oct 28, 2025 |
#2679 in Artificial intelligence
1.5MB
32K
SLoC
Siumai Extras
Optional utilities for the siumai LLM library.
Features
This crate provides optional functionality that extends siumai without adding heavy dependencies to the core library:
schema- JSON Schema validation for structured outputstelemetry- Advanced tracing and logging withtracing-subscriberserver- Server adapters for Axum and other web frameworksmcp- MCP (Model Context Protocol) integration for dynamic tool discoveryall- Enable all features
Installation
[dependencies]
siumai = "0.11.0-beta.8"
siumai-extras = { version = "0.11.0-beta.8", features = ["schema", "telemetry", "mcp"] }
Usage
Orchestrator and high-level object helpers do not require any extra features. Schema validation and tracing are opt-in via the
schemaandtelemetryfeatures.
High-level structured objects
Provider-agnostic helpers for generating typed JSON objects:
use serde::Deserialize;
use siumai::prelude::unified::*;
use siumai_extras::highlevel::object::{generate_object, GenerateObjectOptions};
#[derive(Deserialize, Debug)]
struct Post { title: String }
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let reg = registry::global();
let model = reg.language_model("openai:gpt-4o-mini")?;
let (post, _resp) = generate_object::<Post>(
&model,
vec![user!("Return JSON: {\"title\":\"hi\"}")],
None,
GenerateObjectOptions::default(),
)
.await?;
println!("{}", post.title);
Ok(())
}
If you enable the schema feature, GenerateObjectOptions::schema is
validated via siumai_extras::schema before deserializing into T.
Orchestrator & agents
Multi-step tool calling, agents, and stop conditions:
use serde_json::json;
use siumai::prelude::unified::*;
use siumai_extras::orchestrator::{
ToolLoopAgent, ToolResolver, ToolChoice, step_count_is,
};
struct SimpleResolver;
#[async_trait::async_trait]
impl ToolResolver for SimpleResolver {
async fn call_tool(
&self,
name: &str,
args: serde_json::Value,
) -> Result<serde_json::Value, siumai::error::LlmError> {
match name {
"get_weather" => {
let city = args.get("city").and_then(|v| v.as_str()).unwrap_or("Unknown");
Ok(json!({ "city": city, "temperature": 72, "condition": "sunny" }))
}
_ => Err(siumai::error::LlmError::InternalError(format!(
"Unknown tool: {}",
name
))),
}
}
}
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let reg = registry::global();
let client = reg.language_model("openai:gpt-4o-mini")?;
let weather_tool = Tool::function(
"get_weather",
"Get weather for a city",
json!({
"type": "object",
"properties": { "city": { "type": "string" } },
"required": ["city"]
}),
);
let agent = ToolLoopAgent::new(client, vec![weather_tool], vec![step_count_is(10)])
.with_system("You are a helpful assistant.")
.with_tool_choice(ToolChoice::Required);
let messages = vec![ChatMessage::user("What's the weather in Tokyo?").build()];
let resolver = SimpleResolver;
let result = agent.generate(messages, &resolver).await?;
println!("Answer: {}", result.text().unwrap_or_default());
Ok(())
}
You can attach telemetry to the agent or orchestrator using
siumai::experimental::observability::telemetry::TelemetryConfig:
use siumai::experimental::observability::telemetry::TelemetryConfig;
use siumai_extras::orchestrator::OrchestratorBuilder;
let telemetry = TelemetryConfig::builder()
.record_inputs(false)
.record_outputs(false)
.record_usage(true)
.build();
let builder = OrchestratorBuilder::new().telemetry(telemetry);
Schema Validation
use siumai_extras::schema::SchemaValidator;
// Validate JSON against a schema
let validator = SchemaValidator::new(schema)?;
validator.validate(&json_value)?;
Telemetry
use siumai_extras::telemetry::init_subscriber;
// Initialize tracing subscriber
init_subscriber(config)?;
Server Adapters
use siumai_extras::server::axum::to_sse_response;
// Convert ChatStream to Axum SSE response
let sse = to_sse_response(stream, options);
If you are building an OpenAI-compatible gateway and need to output OpenAI Responses SSE,
siumai-extras also provides a helper that:
- bridges provider-specific
ChatStreamEvent::Customparts intoopenai:*stream parts, and - serializes the stream into OpenAI Responses SSE frames.
use axum::response::Response;
use axum::body::Body;
use siumai_extras::server::axum::to_openai_responses_sse_response;
use siumai::prelude::unified::ChatStream;
fn handler(stream: ChatStream) -> Response<Body> {
to_openai_responses_sse_response(stream)
}
See the runnable example: siumai-extras/examples/openai-responses-gateway.rs (streaming + non-streaming).
For custom conversion hooks, see: siumai-extras/examples/gateway-custom-transform.rs.
For request-normalization bridge demos, see:
siumai-extras/examples/anthropic-to-openai-responses-gateway.rssiumai-extras/examples/openai-responses-to-anthropic-gateway.rs
For custom lossy-policy handling, see:
siumai-extras/examples/gateway-loss-policy.rs
If you need to expose multiple downstream protocol surfaces from the same upstream stream, use the transcoder helper:
use axum::{body::Body, response::Response};
use siumai::experimental::bridge::BridgeTarget;
use siumai::prelude::unified::ChatStream;
use siumai_extras::server::axum::{
TargetSseFormat, TranscodeSseOptions, to_transcoded_sse_response,
};
fn handler(stream: ChatStream) -> Response<Body> {
to_transcoded_sse_response(
stream,
TargetSseFormat::OpenAiResponses,
TranscodeSseOptions::strict().with_bridge_source(BridgeTarget::AnthropicMessages),
)
}
When a streaming gateway route is cross-protocol and you want strict inspected rejection or a
custom BridgeLossPolicy, declare the upstream protocol with
TranscodeSseOptions::with_bridge_source(...). That enables the same source-aware loss-policy
decision path used by the lower-level core stream bridge helpers while keeping the Axum helper
surface.
If your gateway route also needs to read downstream request bodies or buffered upstream bodies
under GatewayBridgePolicy, use the Axum runtime helpers instead of open-coding to_bytes(...):
use axum::body::Body;
use siumai_extras::server::{GatewayBridgePolicy};
use siumai_extras::server::axum::read_request_json_with_policy;
let policy = GatewayBridgePolicy::default().with_request_body_limit_bytes(128 * 1024);
let request_json: serde_json::Value =
read_request_json_with_policy(Body::from(r#"{"input":"hello"}"#), &policy).await?;
If you need to customize the conversion logic, the recommended path is
GatewayBridgePolicy + BridgeOptions + typed bridge hooks as demonstrated in
siumai-extras/examples/gateway-custom-transform.rs.
If the request-side requirement is hosted-tool compatibility across protocols, prefer
ProviderToolRewriteCustomization attached through GatewayBridgePolicy::with_customization(...)
or NormalizeRequestOptions::with_bridge_customization(...) instead of patching raw downstream
JSON. The Anthropic -> OpenAI gateway example demonstrates that path for
anthropic.web_fetch_20250910 -> openai.web_search.
The two request-normalization bridge demos intentionally show a different path:
- source protocol request JSON -> explicit request normalizer ->
ChatRequest - execute on a fixed upstream model handle
- transcode the resulting unified response/stream back into the chosen target protocol
That is useful when you want the bridge surface to stay explicit and testable instead of hiding protocol translation inside route-local JSON glue.
Migration guidance for gateway routes now lives at:
docs/workstreams/protocol-bridge-gateway/migration.md
Recommended route shapes now live at:
docs/workstreams/protocol-bridge-gateway/route-recipes.md
That recipes note covers the currently recommended and test-backed gateway compositions for:
- provider-native ingress -> normalized runtime -> downstream JSON/SSE
- buffered upstream proxy/runtime routes
- cross-protocol SSE with inspected strict rejection
- hosted-tool compatibility via typed request customization
The raw event-transform helper is still available as an escape hatch:
use axum::{body::Body, response::Response};
use siumai::prelude::unified::{ChatStream, ChatStreamEvent};
use siumai_extras::server::axum::{
TargetSseFormat, TranscodeSseOptions, to_transcoded_sse_response_with_transform,
};
fn handler(stream: ChatStream) -> Response<Body> {
to_transcoded_sse_response_with_transform(
stream,
TargetSseFormat::OpenAiResponses,
TranscodeSseOptions::strict(),
|ev: ChatStreamEvent| vec![ev],
)
}
When mutating streamed text, handle stable Part(TextDelta) /
PartWithReplay(TextDelta) events or use ChatStreamEvent::text_delta() for read-only text
access.
For non-streaming gateways, you can also transcode a ChatResponse into a provider-native
JSON response body:
use axum::{body::Body, response::Response};
use siumai::prelude::*;
use siumai_extras::server::axum::{
TargetJsonFormat, TranscodeJsonOptions, to_transcoded_json_response,
};
fn handler(resp: ChatResponse) -> Response<Body> {
to_transcoded_json_response(resp, TargetJsonFormat::OpenAiResponses, TranscodeJsonOptions::default())
}
If you want to customize conversion for non-streaming responses, prefer the response-level transform hook (no JSON parse/round-trip):
use axum::{body::Body, response::Response};
use siumai::prelude::*;
use siumai_extras::server::axum::{
TargetJsonFormat, TranscodeJsonOptions, to_transcoded_json_response_with_response_transform,
};
fn handler(resp: ChatResponse) -> Response<Body> {
to_transcoded_json_response_with_response_transform(
resp,
TargetJsonFormat::OpenAiResponses,
TranscodeJsonOptions::default(),
|r| {
r.content = MessageContent::Text("[REDACTED]".to_string());
},
)
}
MCP Integration
use siumai::prelude::unified::*;
use siumai_extras::mcp::mcp_tools_from_stdio;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let reg = registry::global();
let model = reg.language_model("openai:gpt-4o-mini")?;
let messages = vec![user!("Use the available tools to help me")];
// Connect to MCP server and get tools
let (tools, resolver) = mcp_tools_from_stdio("node mcp-server.js").await?;
// Use with Siumai orchestrator (from siumai-extras)
let stop_condition = siumai_extras::orchestrator::step_count_is(10);
let (response, _) = siumai_extras::orchestrator::generate(
&model,
messages,
Some(tools),
Some(&resolver),
&[&*stop_condition],
Default::default(),
)
.await?;
println!("{}", response.content_text().unwrap_or_default());
Ok(())
}
Documentation
License
Licensed under either of:
- Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
- MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)
at your option.
Dependencies
~8–32MB
~460K SLoC