AI

Model roles and the gateway

Role-based model selection, cloud and local models.

Applications in Converge do not hard-code a model. They ask the AI gateway for a role, and the deployment decides which model fills it.

Roles

A caller asks for a role with the model name role:<name>, for example role:agentic. The model catalog row with that role and the highest priority serves it; other rows with the same role act as ordered fallbacks for chat. A request may still name a specific catalog model.

Administrators manage the catalog in Models. A role can only be assigned to a model of a matching kind.

Cloud models through OpenRouter

Cloud models are reached through OpenRouter. Every request asks for zero data retention and denies data collection by default, so providers that keep prompts are not used.

Local models for air-gapped deployments

A deployment can serve every role from models it hosts itself, through any OpenAI-compatible endpoint such as Ollama. Local providers need no outside network access, which makes fully air-gapped installations possible.

Gateway rules

• Data classification. Requests carrying data classified as restricted are refused unless the deployment allows it for that purpose.
• Rate limits. Requests are limited per user and per model over a one-minute window. Over the limit, the gateway answers 429 with Retry-After.
• Usage. Every call is attributed to a user and model for usage reporting.

OpenAI-compatible endpoint

The gateway exposes an OpenAI-compatible API for tools that expect one:

curl https://cnvrg.si/ai/v2/llm/openai/v1/chat/completions \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"model": "role:agentic", "messages": [{"role": "user", "content": "Summarise open orders"}]}'