Automations
Guided setup for manual, nightly, and PR-triggered evaluations
Required Environment Variables
`TRIGGER_SECRET` to protect PR/Nightly trigger endpoints.
Provider key(s): `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`.
Optional: `EVAL_DEFAULT_MODEL` for trigger routes.
Can it run live now?
Yes. If credentials are valid, runs execute against live provider APIs. If credentials are missing and live-required is ON, runs fail explicitly. If live-required is OFF, simulation fallback is used.
Production Data Durability
Current runtime file persistence is fine for demos. For production on Vercel, move run/suite storage to Vercel Postgres or KV so data survives scale events and cold starts.