Start using Tunnex AI

Start with one working model and one authorized caller. Then add the permissions your application needs. You do not need to configure every AI feature to make your first model call.

This guide describes the updated AI console. Use matching control-plane, runtime and CLI builds; a website guide does not upgrade an installed release. Workload identity remains a preview with the limits listed in its guide. Licensed features must also be explicitly enabled for the organization.

User groups contain people. Agent groups contain managed machines. A model grant, a network rule and an MCP tool rule control different actions: creating one does not automatically create the others.

1. Confirm that the installation has a healthy private AI engine and matching provider-testing services. Follow AI Gateway installation prerequisites.
2. Open AI Gateway → Settings and enable AI access for this organization.
3. Open LLM credentials → Add Credentials. Choose your provider, the exact deployed model name, its supported mode, endpoint and provider credential.
4. Select Test Connect, then Create credentials after success. Testing makes an upstream request and may incur a charge; it does not grant access.
5. Check Models & endpoints for the model and its applied state. To add another model with the same credentials, use Add Model → Existing Credentials.
6. Under Users & Groups → Groups, add the intended people to a group.
7. Under AI Gateway → Access, grant that group the exact model. Wait for the access configuration to be applied.

For Azure, use the deployment’s endpoint and deployment name, not just a catalog suggestion. See Azure endpoint examples. Provider credentials stay in the managed provider configuration; customers calling Tunnex do not need a copy of the Azure or other provider key.

1. Sign in and select the intended organization.
2. Open AI Gateway → My models. Only your permitted models appear.
3. For a chat model, select Call model, enter a short prompt and send it.
4. Read the answer and token count. Expand Response details if you need the original response for troubleshooting.
5. Copy the base URL, API model ID and, when needed, organization ID from the model’s connection details. Do not substitute the upstream model name for the gateway’s API model ID.

For other modes, use the operation-specific example shown for that model. A chat example cannot test an embedding, image, audio, rerank or video deployment.

Your administrator must first configure the VPN AI ingress, private DNS/routing and HTTPS for the control-plane hostname. Connect the Tunnex desktop client and confirm a recent handshake. Signing in with tunnex login alone does not create a VPN tunnel.

When your user has one organization membership, the configured VPN base URL can be https://your-control-plane.example.com/ai/v1. With multiple memberships, copy the organization-specific base URL displayed in My models:

https://your-control-plane.example.com/api/v1/organizations/<organization-id>/ai-gateway/inference/v1

The UI supplies the ID: you do not need to look it up in a database. Use the URL for the selected organization even when several gateways exist. A gateway’s public IP is not a replacement for this HTTPS base URL.

Create an isolated Python environment:

mkdir tunnex-ai-demo
cd tunnex-ai-demo
python3 -m venv .venv
source .venv/bin/activate
python -m pip install openai

Save this as demo.py, replacing the two marked values with My models values:

from openai import OpenAI

client = OpenAI(
    base_url="https://your-control-plane.example.com/ai/v1",  # Copy base URL
    api_key="unused",  # SDK requires a value; configured VPN identity authorizes it
    timeout=60.0,
)
response = client.chat.completions.create(
    model="<copy-the-exact-API-model-ID>",
    messages=[{"role": "user", "content": "Say hello in one sentence."}],
)
print(response.choices[0].message.content)

Run it while connected:

python demo.py

The placeholder key works only through the configured, identity-aware VPN path. It does not authenticate public HTTPS requests. For authenticated CLI access, use the CLI example; the CLI uses your saved Tunnex login. Never replace the placeholder with the upstream provider key.

Use AI Gateway → Access → Workloads for a service or bot that should not depend on a person’s VPN or browser session.

1. Create a workload and select the exact models and modes it may use.
2. Wait for Applied, then open Connect application.
3. Create a single-instance or reusable enrollment key with an appropriate expiry and use limit. Save the one-time secret securely.
4. Download the configuration. Provision its key file and private state directory for the application user using the workload setup steps.
5. Enroll and launch with the provisioned configuration path:

tunnex workload enroll --config /run/secrets/tunnex/workload.json
tunnex workload run --config /run/secrets/tunnex/workload.json -- python app.py

The wrapper supplies OPENAI_BASE_URL and OPENAI_API_KEY to the child process, so a compatible application can use OpenAI() with the exact permitted model ID. These settings belong to the local authenticated proxy; this is different from the laptop VPN placeholder-key example.

6. Check the workload’s Instances and usage. Preserve each instance’s private state across restarts; give replacement replicas independent state.
7. To stop one instance, revoke it. Revoking only an enrollment key stops new joins; existing instances continue unless you select the accompanying instance revocation option. Disabling the workload blocks new calls and revokes its enrollment keys; re-enabling requires new keys for new instances.

See Workload identity for macOS paths, file permissions, retirement and preview limitations. Workload enrollment itself creates no VPN.

1. Prepare a supported Linux host and an eligible gateway with a usable endpoint.
2. An owner enables Settings → AI Agents → Enable runtime synchronization before issuing the managed bootstrap command. Enable agent groups separately when you need group assignments or templates.
3. Open AI Agents → Add agent, choose the name and gateway, and review.
4. Issue the one-time command and run it on the intended host. Protect that command as a credential; do not paste it into tickets or shared logs.
5. Open the agent and confirm owner, runtime status and recent handshake.
6. Assign the model team and agent assignment under AI Agents → Model access. Wait for matching desired/applied state before calling a model.
7. Add MCP and private-network access only as needed below.

Connected proves connectivity; it does not mean every model or tool is allowed. People-group grants do not replace an agent’s model assignment. Use AI Agents for bootstrap and lifecycle details. Check Activity and audit records when investigating behavior; workflow provenance is trusted only when its signature is verified, not because a caller supplied a workflow name.

1. In MCP, create a profile with a recognizable server name and its supported endpoint. Keep secrets out of the URL and ordinary metadata.
2. Assign that profile to the intended agent group, then confirm the agent is in that group and its runtime is connected and synchronized.
3. Ensure the runtime can reach the server. A private server also needs working DNS, routing and the applicable network permission.
4. If OAuth is required, use the profile’s OAuth connection flow and authorize the intended account. Refresh tokens/client secrets stay in the control plane; transient access leases are delivered to runtime memory.
5. Open the agent’s MCP controls and refresh discovery. Inspect the observed tools, source agent and freshness. The profile name alone does not establish a tool list.
6. Configure MCP tool policy: allow only the required tools. Add argument constraints, rate limits or step-up approval where appropriate.
7. Point the agent application at its runtime MCP proxy. Test one permitted tool and one denied tool. For step-up, approve the pending invocation and retry that exact invocation; approval is not a permanent tool grant.
8. Review decisions in the agent’s activity and access/audit views.

Tool policy applies to calls through the managed MCP proxy. A direct call to the upstream server bypasses that proxy, so restrict alternative network paths where this boundary is required. See MCP security for the network and tool-control boundaries.

Check, in order: profile assignment, runtime freshness, DNS/reachability, OAuth connection, then discovery errors. An added profile is saved configuration; observed inventory arrives from a runtime that can actually reach and discover the server. Do not treat an old snapshot as current availability. Reconnect OAuth when revoked or expired authorization prevents discovery, then refresh again.

A policy template is reusable destination-and-port access for an agent group. For example, build agents may need one private database on its service port. Templates do not grant AI models or MCP tools.

1. Enable Agent groups & policy templates and create the agent group.
2. Define the private destination/resource under Access Policies.
3. Open AI Agents → Policy templates, create a version and select the destination and allowed service ports.
4. Preview the assignment impact, then apply it to the intended group.
5. Check the generated managed rules and the gateway’s applied policy state.
6. Verify the allowed destination/port and a destination/port that should fail.
7. Withdraw the assignment when access is no longer needed, and verify the corresponding managed rules disappear.

Network restrictions require organization enforcement to be enabled. With network enforcement off, a template is not evidence that other traffic is blocked. Review the full allow policy before enabling enforcement on a live organization; follow Test and enforce policy.

Just-in-time agent access is a separate licensed, organization-level opt-in. It starts off. An owner/admin enables it in organization settings when ready.

1. Open the agent’s Access → Manage temporary access, or Access Policies → Just-in-time agent access.
2. Select the agent and destination, give a reason and choose 15 minutes, 1 hour, 4 hours or 24 hours. Submit Request access.
3. A pending request changes no policy. An authorized approver reviews it and approves or rejects it with a reason. A requester may cancel their pending request.
4. After approval, check the exact expiry time and generated rule. The approved window starts when approved, not when requested.
5. Use agent/state filters and Load more to find older requests. The agent link opens the relevant filtered view.
6. Revoke approved access early if needed. At expiry the temporary grant is removed; inspect refreshed rules and history to confirm the change.

As with templates, actual network restriction requires enforcement. Other applicable permanent grants can still allow the same destination after temporary access expires. JIT network access does not replace MCP tool step-up approval.

In Usage & cost, review model, team, agent or workload attribution as available. Estimates are not provider invoices. Daily USD thresholds use observed spending and can be exceeded by concurrent or already accepted requests; they are not hard spend caps. Do not interpret missing price data as zero cost.

When requesting support, include the operation, timestamp, HTTP status, request ID and relevant applied/runtime state. Keep provider keys, enrollment commands, OAuth secrets and private instance files out of shared logs.