AI Models & Chat

OpenChat uses OpenRouter as a unified gateway to access 100+ AI models from providers like OpenAI, Anthropic, Google, Meta, and more.

Overview

OpenChat supports two modes for AI access:

Mode	Description	Cost
OSSChat Cloud	Uses server’s OpenRouter key	Free (10¢/day limit)
BYOK	Bring Your Own Key	Your OpenRouter credits

Dual Provider System

OSSChat Cloud (Default)

The free tier uses the server’s OpenRouter API key with daily limits:

// Automatically used when no personal key is set
const provider = "osschat";
const apiKey = process.env.OPENROUTER_API_KEY; // Server's key

Daily limit: ~10¢ worth of usage
Resets: Daily at midnight UTC
Models: All OpenRouter models available

BYOK (Bring Your Own Key)

Users can connect their own OpenRouter account for unlimited access:

// User's personal API key
const provider = "openrouter";
const apiKey = userStore.openrouterApiKey; // User's key

No limits: Uses your own OpenRouter credits
All models: Including premium models
Privacy: API key stored locally (encrypted if saved to Convex)

Connecting OpenRouter (BYOK)

Open Settings

Click your profile in the sidebar, then click Settings.

Go to Providers Tab

Select the Providers tab to see connection options.

Connect OpenRouter

Click Connect OpenRouter. You’ll be redirected to OpenRouter to authorize.

OpenChat uses OAuth PKCE for secure key exchange. Your key is never exposed in URLs.

Authorize

On OpenRouter, authorize the application. You’ll be redirected back to OpenChat.

Start Using

Your models are now available! The provider switches to “Personal OpenRouter” automatically.

Model Selection

Using the Model Selector

Click the model name in the chat input to open the model selector:

Search: Type to filter models
Favorites: Star models for quick access
Details: See context window, pricing, capabilities

Model Capabilities

Models vary in their capabilities:

Capability	Description	Example Models
Reasoning	Extended thinking for complex tasks	Claude 3.5, o1, DeepSeek R1
Vision	Process images	Claude 3.5, GPT-4o, Gemini
Long Context	Large context windows (100K+)	Claude 3.5, Gemini Pro
Fast	Low latency responses	GPT-4o-mini, Claude Haiku

Reasoning Mode

For supported models, enable reasoning mode to see the AI’s thinking process:

// Configure reasoning effort
const reasoningEffort = "medium"; // none, low, medium, high

// Sent to OpenRouter
providerOptions: {
  openrouter: {
    reasoning: {
      effort: reasoningEffort
    }
  }
}

Reasoning appears in a collapsible “Chain of Thought” section above the response.

Chat API

Endpoint

POST /api/chat

Request Body

{
  messages: UIMessage[],      // Conversation history
  model: string,              // e.g., "anthropic/claude-3.5-sonnet"
  provider: "osschat" | "openrouter",
  apiKey?: string,            // Required for "openrouter" provider
  enableWebSearch?: boolean,  // Enable web search tool
  reasoningEffort?: "none" | "low" | "medium" | "high",
  maxSteps?: number           // Max tool call iterations (default: 5)
}

Response

Returns a Server-Sent Events stream compatible with AI SDK 5’s useChat hook:

// Stream events
data: {"type":"text","content":"Hello"}
data: {"type":"text","content":" there!"}
data: {"type":"finish","usage":{"promptTokens":10,"completionTokens":5}}

Example

import { useChat } from "ai/react";

function ChatComponent() {
  const { messages, input, handleInputChange, handleSubmit } = useChat({
    api: "/api/chat",
    body: {
      model: "anthropic/claude-3.5-sonnet",
      provider: "osschat",
    },
  });
  
  return (
    <form onSubmit={handleSubmit}>
      <input value={input} onChange={handleInputChange} />
      <button type="submit">Send</button>
    </form>
  );
}

Streaming Architecture

OpenChat uses AI SDK 5 for streaming:

Cancellation Support

Users can cancel in-progress responses:

// Client
const abortController = new AbortController();
fetch("/api/chat", { signal: abortController.signal });

// To cancel
abortController.abort();

// Server
const abortSignal = request.signal;
streamText({ model, messages, abortSignal });

Web Search

OpenChat can search the web for current information:

// Enable web search
const response = await fetch("/api/chat", {
  method: "POST",
  body: JSON.stringify({
    messages,
    model: "anthropic/claude-3.5-sonnet",
    enableWebSearch: true,  // Enable
    maxSteps: 5             // Allow tool iterations
  })
});

Web search uses the Valyu AI SDK and requires VALYU_API_KEY to be configured.

Token Usage & Cost Tracking

Usage Display

Token usage is tracked and displayed per message:

// Response includes usage metadata
{
  inputTokens: 1234,
  outputTokens: 567,
  totalTokens: 1801
}

Cost Calculation

For OSSChat Cloud, costs are tracked against the daily limit:

// Provider store tracks usage
const { dailyUsageCents, dailyLimitCents } = useProviderStore();

// Display remaining budget
const remaining = dailyLimitCents - dailyUsageCents; // e.g., 7¢ remaining

File Attachments

Upload images and documents to include in messages:

// Generate upload URL
const uploadUrl = await convex.mutation(api.files.generateUploadUrl, {
  userId,
  chatId
});

// Upload file
await fetch(uploadUrl, {
  method: "POST",
  body: file
});

// Save metadata
const { fileId, url } = await convex.mutation(api.files.saveFileMetadata, {
  userId,
  chatId,
  storageId,
  filename: file.name,
  contentType: file.type,
  size: file.size
});

Limits

Limit	Value
Max file size	10 MB
Files per user	150 total
Supported types	Images (jpg, png, gif, webp), PDF

Prompt Templates

Create custom slash commands for frequently used prompts:

// Create template
await convex.mutation(api.promptTemplates.create, {
  name: "Code Review",
  command: "/review",
  template: "Review this code for bugs and improvements:\n\n$ARGUMENTS",
  category: "coding"
});

// Use in chat
/review function add(a, b) { return a + b; }

Template Variables

Variable	Description
`$ARGUMENTS`	Everything after the command
`$1`, `$2`, …	Positional arguments

Configuration

Environment Variables

# Server-side
OPENROUTER_API_KEY=sk-or-v1-...  # For OSSChat Cloud
VALYU_API_KEY=...                 # For web search

# Client-side (if needed)
# None - API keys should never be on client

Rate Limits

The chat API is rate-limited to prevent abuse:

Operation	Rate	Burst
Message send	30/min	10
Stream upsert	200/min	50

Troubleshooting

Model not responding

Check:

Model is available (some are intermittently down)
API key is valid (try regenerating)
You have credits (for BYOK) or daily budget (for OSSChat)

Streaming stops mid-response

This can happen if:

Network connection drops
Model reaches max tokens
Rate limit exceeded

Try sending a shorter message or switching models.

Web search not working

Verify:

VALYU_API_KEY is set on the server
enableWebSearch: true in request
Model supports tool use

Daily limit reached

For OSSChat Cloud, the 10¢ daily limit resets at midnight UTC. Options:

Wait for reset
Connect your own OpenRouter key (BYOK)

Next Steps

Self-Hosting

Deploy with your own API keys

Contributing

Add new model integrations

Getting Started

Core Features

Guides

Self-Hosting

AI Models & Chat

AI Models & Chat

Overview

Dual Provider System

OSSChat Cloud (Default)

BYOK (Bring Your Own Key)

Connecting OpenRouter (BYOK)

Model Selection

Using the Model Selector

Model Capabilities

Reasoning Mode

Chat API

Endpoint

Request Body

Response

Example

Streaming Architecture

Cancellation Support

Web Search

Token Usage & Cost Tracking

Usage Display

Cost Calculation

File Attachments

Limits

Prompt Templates

Template Variables

Configuration

Environment Variables

Rate Limits

Troubleshooting

Next Steps

Self-Hosting

Contributing

​AI Models & Chat

​Overview

​Dual Provider System

​OSSChat Cloud (Default)

​BYOK (Bring Your Own Key)

​Connecting OpenRouter (BYOK)

​Model Selection

​Using the Model Selector

​Model Capabilities

​Reasoning Mode

​Chat API

​Endpoint

​Request Body

​Response

​Example

​Streaming Architecture

​Cancellation Support

​Web Search

​Token Usage & Cost Tracking

​Usage Display

​Cost Calculation

​File Attachments

​Limits

​Prompt Templates

​Template Variables

​Configuration

​Environment Variables

​Rate Limits

​Troubleshooting

​Next Steps

Self-Hosting

Contributing

AI Models & Chat

Overview

Dual Provider System

OSSChat Cloud (Default)

BYOK (Bring Your Own Key)

Connecting OpenRouter (BYOK)

Model Selection

Using the Model Selector

Model Capabilities

Reasoning Mode

Chat API

Endpoint

Request Body

Response

Example

Streaming Architecture

Cancellation Support

Web Search

Token Usage & Cost Tracking

Usage Display

Cost Calculation

File Attachments

Limits

Prompt Templates

Template Variables

Configuration

Environment Variables

Rate Limits

Troubleshooting

Next Steps