How Smart AI Comparison Handles Provider API Keys
A transparent overview of how Smart AI Comparison stores and uses BYOK credentials for OpenAI, Anthropic, and Google during side-by-side model comparisons.
Users evaluating AI models need to know exactly how their API keys are used. This article describes Smart AI Comparison's provider key handling as implemented in the product architecture—without marketing claims about security certifications beyond what the codebase and documentation support.
Overview
Smart AI Comparison uses a Bring Your Own Key (BYOK) model. You connect provider credentials in Settings; the platform uses them server-side when you run comparisons. You are billed by providers according to their pricing; the app applies its own usage tiers (2 comparisons per day on Free, unlimited on Pro).
Supported Providers and Categories
The compare edge function routes requests based on model category and provider:
| Category | Live providers (edge function) |
|---|---|
| Text | OpenAI, Anthropic, Google |
| Image | OpenAI, Google |
| Video | OpenAI (partial support) |
| Audio | OpenAI (partial support) |
"Partial support" means not every model or endpoint variant may be exposed in the UI yet; consult in-app provider lists for current availability.
Key Storage Flow
- Authentication required — Only signed-in users add or update keys.
- Submission via settings — Keys are sent to the backend over HTTPS; they are not embedded in client-side source code.
- Server-side storage — Keys persist in Supabase KV storage associated with your user account, not in browser localStorage as long-term secrets.
- Retrieval at compare time — The compare edge function loads the appropriate key when processing a comparison request.
Screens and React hooks do not call OpenAI, Anthropic, or Google directly—consistent with the project's MVC architecture (repositories and services mediate data access).
Comparison Request Flow
When you submit a comparison:
- The client sends the prompt, selected models, and category to the compare edge function with your auth token.
- The server validates usage limits (free daily quota or Pro unlimited).
- For each selected model, the server resolves the provider, fetches your stored key, and calls the provider API.
- Responses (text, image URLs, or media references depending on category) return to the client for side-by-side display.
- The comparison may be saved to history (KV-backed) for later review.
Latency and errors reflect live provider behaviour—including rate limits tied to your key.
Per-Provider API Differences
Smart AI Comparison normalises the user experience, but underlying APIs differ:
OpenAI
Chat completions for text; separate endpoints for image, audio, and some video models. Parameters include temperature, max tokens, and optional tools. See OpenAI API reference.
Anthropic
Messages API with distinct system prompt handling and content block structure. Text comparisons use your Anthropic key; image/video/audio via Anthropic are not in the current live provider set for this platform.
Google (Gemini)
Gemini API with multimodal content parts. Text and image generation comparisons may use Google keys depending on selected models.
The compare function adapts request shapes per provider; identical UI prompts may map to slightly different API payloads.
Usage Limits: App vs Provider
Two separate limit systems apply:
Smart AI Comparison tiers
- Free: 2 comparison runs per day
- Pro: unlimited comparison runs
Provider accounts
- Rate limits, spending caps, and model access defined by OpenAI, Anthropic, and Google
Hitting a provider rate limit during a multi-model comparison may show partial results or errors for affected models even if you have Pro unlimited app access.
Key Rotation and Removal
You can update or delete keys in Settings. After deletion, comparisons requiring that provider fail until a new key is added. Rotate keys on the provider console first, then update stored values—avoid leaving a window where old keys remain active.
What the Platform Does Not Do
Based on documented architecture:
- It does not resell inference as opaque credits at a markup
- It does not expose raw keys in comparison results or history exports
- It does not guarantee identical sampling across providers (non-determinism and parameter differences remain)
Verify current behaviour in the application's privacy policy and terms for data retention specifics.
Recommendations for Teams
- Use project-scoped keys where providers allow it
- Set provider-side spending alerts before large evaluation batches
- Test with Free tier spot checks, then upgrade to Pro for systematic multi-model runs
- Document which keys are used for evaluation vs production services
Troubleshooting Failed Comparisons
If a comparison returns errors for one provider while others succeed, check in order: key validity on the provider console, account rate limits, model availability for your region, and whether the selected category matches the model type (text vs image). Partial failures are common in multi-model runs—log them as availability metrics, not only as user inconvenience.
Rotate keys after personnel changes. Stale keys produce auth errors that can be mistaken for model quality issues if not diagnosed promptly.
Extensibility as Providers Are Added
The platform's compare edge function may expand provider coverage over time. Architecture separates key storage from compare routing so new adapters can ship without changing how users submit prompts. Re-run baseline comparisons when new providers appear in your settings UI—do not assume prior rankings transfer.
Limitations
- Third-party key storage requires organisational trust and legal review
- Partial provider support in some categories means not every advertised model industry-wide is available in-app
- Provider API changes may require platform updates; comparison parity is best-effort