AI Models & Chat
OpenChat uses OpenRouter as a unified gateway to access 100+ AI models from providers like OpenAI, Anthropic, Google, Meta, and more.Overview
OpenChat supports two modes for AI access:| Mode | Description | Cost |
|---|---|---|
| OSSChat Cloud | Uses server’s OpenRouter key | Free (10¢/day limit) |
| BYOK | Bring Your Own Key | Your OpenRouter credits |
Dual Provider System
OSSChat Cloud (Default)
The free tier uses the server’s OpenRouter API key with daily limits:- Daily limit: ~10¢ worth of usage
- Resets: Daily at midnight UTC
- Models: All OpenRouter models available
BYOK (Bring Your Own Key)
Users can connect their own OpenRouter account for unlimited access:- No limits: Uses your own OpenRouter credits
- All models: Including premium models
- Privacy: API key stored locally (encrypted if saved to Convex)
Connecting OpenRouter (BYOK)
Connect OpenRouter
Click Connect OpenRouter. You’ll be redirected to OpenRouter to authorize.
OpenChat uses OAuth PKCE for secure key exchange. Your key is never exposed in URLs.
Model Selection
Using the Model Selector
Click the model name in the chat input to open the model selector:- Search: Type to filter models
- Favorites: Star models for quick access
- Details: See context window, pricing, capabilities
Model Capabilities
Models vary in their capabilities:| Capability | Description | Example Models |
|---|---|---|
| Reasoning | Extended thinking for complex tasks | Claude 3.5, o1, DeepSeek R1 |
| Vision | Process images | Claude 3.5, GPT-4o, Gemini |
| Long Context | Large context windows (100K+) | Claude 3.5, Gemini Pro |
| Fast | Low latency responses | GPT-4o-mini, Claude Haiku |
Reasoning Mode
For supported models, enable reasoning mode to see the AI’s thinking process:Chat API
Endpoint
Request Body
Response
Returns a Server-Sent Events stream compatible with AI SDK 5’suseChat hook:
Example
Streaming Architecture
OpenChat uses AI SDK 5 for streaming:Cancellation Support
Users can cancel in-progress responses:Web Search
OpenChat can search the web for current information:VALYU_API_KEY to be configured.
Token Usage & Cost Tracking
Usage Display
Token usage is tracked and displayed per message:Cost Calculation
For OSSChat Cloud, costs are tracked against the daily limit:File Attachments
Upload images and documents to include in messages:Limits
| Limit | Value |
|---|---|
| Max file size | 10 MB |
| Files per user | 150 total |
| Supported types | Images (jpg, png, gif, webp), PDF |
Prompt Templates
Create custom slash commands for frequently used prompts:Template Variables
| Variable | Description |
|---|---|
$ARGUMENTS | Everything after the command |
$1, $2, … | Positional arguments |
Configuration
Environment Variables
Rate Limits
The chat API is rate-limited to prevent abuse:| Operation | Rate | Burst |
|---|---|---|
| Message send | 30/min | 10 |
| Stream upsert | 200/min | 50 |
Troubleshooting
Model not responding
Model not responding
Check:
- Model is available (some are intermittently down)
- API key is valid (try regenerating)
- You have credits (for BYOK) or daily budget (for OSSChat)
Streaming stops mid-response
Streaming stops mid-response
This can happen if:
- Network connection drops
- Model reaches max tokens
- Rate limit exceeded
Web search not working
Web search not working
Verify:
VALYU_API_KEYis set on the serverenableWebSearch: truein request- Model supports tool use
Daily limit reached
Daily limit reached
For OSSChat Cloud, the 10¢ daily limit resets at midnight UTC. Options:
- Wait for reset
- Connect your own OpenRouter key (BYOK)