Changelog

New updates, improvements, and bug fixes for Observyze.

New versions are rolled out gradually and may take a few hours to reach all edge nodes.

1.0.20
August 2, 2026

Production-Grade Hallucination Detection & Trace Inspector

Improvements (7)
  • Hallucination scoring is now three-class and production-grade: unverifiable claims (missing evidence) are no longer marked as hallucinations — only genuine contradictions and fabrications fail, dramatically reducing false alarms.
  • Every flagged trace now ships with a per-claim Hallucination Report: the exact type (contradiction, fabrication, wrong entity, wrong number, unsupported detail) and the exact location (which JSON field or sentence) with the supporting evidence and a plain-English explanation.
  • Claims that cannot be verified from search snippets are now checked against the actual cited source pages server-side (securely, with a hard bound) — many previously "unverifiable" claims are confirmed automatically.
  • Structured JSON answers now get automatic quality checks: invalid emails, phone numbers, or URLs are flagged with suggested fixes.
  • Works across every major model provider — Gemini, OpenAI, Anthropic, Cohere, Mistral, and Perplexity-style APIs are all supported out of the box.
  • The trace list now shows HALLUCINATION / NEEDS REVIEW / CLEAN badges with server-side filters that search across all your traces, not just the current page.
  • Redesigned the trace Inspector: a clean summary view shows exactly what was sent (chat-style prompts) and what the model returned (formatted answers), with a one-click Raw JSON view for deep debugging.
Fixes (1)
  • Fixed the root cause of false hallucination FAILs — mis-indexed NLI labels that treated "no evidence" as a contradiction. Unverifiable claims are now reported transparently instead of being scored as hallucinations.
1.0.19
July 23, 2026

Hallucination Calibration Dashboard & Evaluation Feedback Loop

Improvements (7)
  • Introduced a new Calibration Dashboard showing bias, MAE, accuracy-by-source breakdown, and recommended threshold adjustments for your hallucination evaluations.
  • Added Auto-Tune — one-click threshold optimization that analyzes human correction history and automatically adjusts evaluation sensitivity.
  • Added a Tune History timeline so you can see every past auto-tune run and how thresholds have evolved over time.
  • Added a Last Tune Results card showing exactly what changed after each auto-tune: which sources were re-weighted, what thresholds were adjusted, and by how much.
  • Implemented PII redaction on evaluation prompts before sending span data to evaluators, ensuring sensitive data never leaves your infrastructure.
  • Added a periodic calibration job that compares human feedback against automated scores and adjusts thresholds or re-weights evaluation models.
  • Added "Correct This Evaluation" buttons throughout the trace detail view, letting you submit corrected scores that feed directly into the calibration loop.
Fixes (3)
  • Fixed service-to-service authentication between the API gateway and evaluation service to use dual-layer auth, resolving persistent "Eval service HTTP 401" errors in production.
  • Fixed the "score hallucination" action in traces that was returning 401 despite having valid credentials.
  • Resolved an issue where the calibration job would run with stale thresholds right after a correction was submitted — auto-tune now fires immediately on new correction data.
1.0.18
July 12, 2026

Batch Evaluation & Production Performance

Improvements (7)
  • Added batch evaluation — evaluate up to 100 traces in a single API call for faster AI safety and quality checks at scale.
  • Evaluation results are now persisted, surviving service restarts and providing a complete audit trail of every check.
  • Rate limiting now persists across deployments, providing consistent API protection without interruption.
  • Added multi-worker support for higher throughput evaluation processing — evaluations complete faster under load.
  • Added payload size limits and improved request validation across all services for better API reliability.
  • All services now output structured JSON logs for easier integration with log management and observability tools.
  • Evaluation service health endpoint now monitors Redis, MongoDB, OpenRouter, and NLI model connectivity for full operational visibility.
Fixes (4)
  • Fixed per-organization rate limiting to ensure fair resource allocation across teams.
  • Fixed health endpoints that reported "connected" without actually verifying database connectivity.
  • Fixed password hash handling for OAuth and SSO accounts, preventing authentication errors for social login users.
  • Fixed HTTP status codes returned by project creation endpoints to match API standards.
1.0.17
June 28, 2026

Real-Time Policy Enforcement & Dashboard Overhaul

Improvements (6)
  • Added real-time policy enforcement in the proxy gateway — prompt injection blocking, toxicity detection, and PII scrubbing are now enforced server-side based on your active policies.
  • Policy checks are cached for sub-100ms enforcement and automatically refresh when you update a policy.
  • Completely redesigned the Policies dashboard with URL-persisted selection, accessible modals, delete confirmation, and distinct empty/error/loading states.
  • Overhauled the landing page with updated messaging, integration logos (OpenAI, Anthropic, LangChain, Vercel AI SDK, Python SDK, TypeScript SDK), and a real incident cost-breakdown story.
  • Added an Architecture Flow diagram showing the full stack flow from your app through SDK/Proxy to LLM and into the dashboard.
  • New organizations now get 4 default policies (PII, Hallucination, Toxicity, Injection) auto-configured on first load.
Fixes (5)
  • Fixed PII scrubbing being counted on every proxy request regardless of whether a PII policy was active — metrics are now accurate.
  • Fixed PII scrubbing metadata being hardcoded to true even when the PII policy was disabled.
  • Added toxicity detection for streaming responses (audit-only, since content is already delivered to the user).
  • Fixed audit-logs page pagination, export functionality, and filter labels — no more raw keys like `api_key` showing in the UI.
  • Added safe date formatting throughout the app to prevent crashes on null timestamps.
1.0.16
June 28, 2026

Zero Data Leak Compliance Mode & Local PII Redaction

Improvements (4)
  • Introduced Compliance Mode in the Node.js SDK to completely prevent raw prompt data from leaving your VPC.
  • Added Direct Routing Bypass — you can force LLM calls straight to OpenAI/Anthropic without going through the proxy when needed.
  • Added local, in-memory client-side PII Scrubbing for emails, credit cards, SSNs, API keys, and phone numbers.
  • Updated developer documentation with code snippets and guides for the new two-layer privacy architecture.
Fixes (1)
  • Added server-side PII redaction fallback to the API gateway as a secondary safety shield.
1.0.15
June 11, 2026

Autonomous Alerts & Circuit Breakers

Improvements (3)
  • Introduced a real-time Autonomous Alert Engine that monitors trace latency, costs, and error rates — get notified instantly when something goes wrong.
  • Built-in cooldown periods prevent alert fatigue during high-volume traffic spikes.
  • Alerts integrate with existing webhooks and Slack for immediate dispatch when conditions trip.
Fixes (1)
  • Migrated to a self-hosted Redis instance, improving overall system reliability and eliminating timeout issues that affected alert delivery.
1.0.14
June 11, 2026

Trace Waterfall Analysis & Tool Execution Console

Improvements (2)
  • Overhauled the trace detail page with an interactive Waterfall Analysis (Gantt Timeline) layout — see exactly how your LLM calls, tools, and retrievals overlap in time.
  • Introduced a Terminal-style Tool Execution Console to inspect raw shell commands and JSON payload outputs for tool spans.
Fixes (1)
  • Fixed deep nested span structures that were incorrectly scaled in the detail list view.
1.0.13
June 5, 2026

3D Interactive Trace Graph & Immersive Experience

Improvements (4)
  • Introduced an immersive 3D WebGL background with a rotating torus-knot and particle constellation that responds to mouse movement.
  • Redesigned the trace graph with sequential staggered alignment that matches exact trace execution sequences.
  • Added dedicated "Explore Interactive Demo" entry points and "New" feature badges throughout the app.
  • Custom edge paths with animated dashed lines and glowing drop-shadows make trace flow easier to follow.
Fixes (2)
  • Fixed a cursor conflict where the custom dashboard pointer overlapped text inputs and textareas.
  • Added live client-side trace list filtering for instant search by trace ID, name, or tags.
1.0.12
June 5, 2026

Precise Cost Attribution & Settings Reliability

Improvements (1)
  • Dashboard KPIs are now more accurate with improved real-time cost calculations.
Fixes (4)
  • Fixed extremely low-cost traces (e.g. gpt-4o-mini) showing as $0.0000 — they now display up to 6 decimal places correctly.
  • Resolved an authentication bug that affected some OAuth login flows.
  • Fixed a memory leak on the Settings page that could slow down the dashboard over time.
  • Fixed duplicate entries appearing when updating Webhook URLs and Alert integrations.
1.0.11
June 2, 2026

Hallucination Scoring & Circuit Breakers

Improvements (3)
  • Introduced automated Hallucination Scoring using LLM-as-a-judge with OpenRouter fallbacks for reliability.
  • Added the ability to set Circuit Breakers based on hallucination thresholds to automatically prevent downstream errors.
  • Overhauled the trace timeline UI to better visualize nested agent spans and sub-calls.
Fixes (2)
  • Resolved intermittent latency spikes in the proxy gateway under high concurrent load.
  • Fixed a visual alignment bug in the project selector dropdown.