Changelog

New updates, improvements, and bug fixes for Observyze.

New versions are rolled out gradually and may take a few hours to reach all edge nodes.

1.0.23
September 4, 2026

RAG Grounding Attribution, Context Match Metrics & Trace Inspector Expansion

Improvements (5)
  • Claim-Level Grounding Attribution Audit Trail: Added an interactive, expandable Grounding Attribution card to trace evaluation views. Teams can now audit every sentence in a model response with exact verification verdicts (Supported in Context, Weak Context Match, or Contradicts Context) and match scores.
  • Context Match Strength Breakdown: Every evaluated claim now features a tri-color Context Match Strength meter that breaks down Entailment (supported by docs), Neutral (missing from uploaded documents), and Contradiction percentages. This makes it effortless to distinguish between missing retrieval chunks and active model fabrications.
  • Retrieved Document Evidence Snippets: Flagged claims now surface the exact retrieved document chunk directly alongside the claim text, providing instant side-by-side comparison without opening raw JSON payloads.
  • Clean Trace Verification Cards: Clean executions with zero hallucinations now render a dedicated Verification & Compliance card detailing evaluator model, task type, and full alignment reasoning.
  • Trace Inspector In-Place Expansion: Added a full-width inspector toggle button with hotkey support (Escape to collapse) to expand complex multi-turn payloads and code outputs across the entire screen.
Fixes (2)
  • Resolved an issue where long evaluator reasoning strings could be truncated on smaller viewport widths in the horizontal insights bar.
  • Fixed viewport overflow and responsive layout transitions when toggling between waterfall execution views and full-width span inspection.
1.0.22
August 22, 2026

Extended 90-Day Early Access Pro, Resilient AI Proxy & Key Management UX

Improvements (4)
  • Extended Early Access Program: All active beta users and new signups receive 90 days of complimentary Pro tier with 100,000 monthly traces, full evaluation suites, and 10 GB telemetry storage.
  • Universal AI Proxy Resilience: Upgraded the Proxy Gateway with smart path normalization and multi-header authentication support for Google Gemini, OpenAI, and Anthropic SDKs.
  • Redesigned API Key Generation: Added explicit project/environment selection and interactive multi-language quickstart code snippets (Python Google GenAI, OpenAI, Node.js, and cURL) pre-filled upon key creation.
  • Accurate Workspace Badging: Added live Early Access countdown and plan status indicators in the workspace navigation switcher.
Fixes (2)
  • Resolved an issue where 10-day trial timers prematurely restricted API gateway proxy requests for active early access testers.
  • Corrected documentation endpoint URLs for Google Gemini proxy integration to prevent SDK version conflicts.
1.0.21
August 19, 2026

Hallucination Engine Path-Mix Dashboards & Deterministic Code Checks

Improvements (5)
  • Added a new Evaluation Path Distribution chart to the evaluators dashboard, providing complete transparency into whether scores came from NLI, deterministic code validation, or LLM consensus.
  • Upgraded the Code Evaluation engine: hallucinatory code imports now fail with pinpoint line-number accuracy and exact source snippets, making it effortless to trace fabricated APIs.
  • Added per-task-type threshold calibration. The Hallucination Engine now automatically calculates and applies independent, optimal thresholds tailored specifically for RAG, Code generation, and Chat tasks.
  • Added evaluation provenance fields for prompt hashes, evaluator model IDs, and bounded raw model responses to improve reproducibility and audit review.
  • Added Slack webhook delivery for Evaluation Drift Alerts. Get notified instantly when your hallucination rates drift week-over-week or when human agreement degrades.
1.0.20
August 2, 2026

Production-Grade Hallucination Detection & Trace Inspector

Improvements (7)
  • Hallucination scoring is now three-class and production-grade: unverifiable claims (missing evidence) are no longer marked as hallucinations — only genuine contradictions and fabrications fail, dramatically reducing false alarms.
  • Every flagged trace now ships with a per-claim Hallucination Report: the exact type (contradiction, fabrication, wrong entity, wrong number, unsupported detail) and the exact location (which JSON field or sentence) with the supporting evidence and a plain-English explanation.
  • Claims that cannot be verified from search snippets are now checked against the actual cited source pages server-side (securely, with a hard bound) — many previously "unverifiable" claims are confirmed automatically.
  • Structured JSON answers now get automatic quality checks: invalid emails, phone numbers, or URLs are flagged with suggested fixes.
  • Works across every major model provider — Gemini, OpenAI, Anthropic, Cohere, Mistral, and Perplexity-style APIs are all supported out of the box.
  • The trace list now shows HALLUCINATION / NEEDS REVIEW / CLEAN badges with server-side filters that search across all your traces, not just the current page.
  • Redesigned the trace Inspector: a clean summary view shows exactly what was sent (chat-style prompts) and what the model returned (formatted answers), with a one-click Raw JSON view for deep debugging.
Fixes (1)
  • Fixed the root cause of false hallucination FAILs — mis-indexed NLI labels that treated "no evidence" as a contradiction. Unverifiable claims are now reported transparently instead of being scored as hallucinations.
1.0.19
July 23, 2026

Hallucination Calibration Dashboard & Evaluation Feedback Loop

Improvements (7)
  • Introduced a new Calibration Dashboard showing bias, MAE, accuracy-by-source breakdown, and recommended threshold adjustments for your hallucination evaluations.
  • Added Auto-Tune — one-click threshold optimization that analyzes human correction history and automatically adjusts evaluation sensitivity.
  • Added a Tune History timeline so you can see every past auto-tune run and how thresholds have evolved over time.
  • Added a Last Tune Results card showing exactly what changed after each auto-tune: which sources were re-weighted, what thresholds were adjusted, and by how much.
  • Implemented deterministic PII redaction on evaluation prompts before evaluator dispatch; unsupported patterns and arbitrary names still require metadata-only capture or additional controls.
  • Added a periodic calibration job that compares human feedback against automated scores and adjusts thresholds or re-weights evaluation models.
  • Added "Correct This Evaluation" buttons throughout the trace detail view, letting you submit corrected scores that feed directly into the calibration loop.
Fixes (3)
  • Fixed service-to-service authentication between the API gateway and evaluation service to use dual-layer auth, resolving persistent "Eval service HTTP 401" errors in production.
  • Fixed the "score hallucination" action in traces that was returning 401 despite having valid credentials.
  • Resolved an issue where the calibration job would run with stale thresholds right after a correction was submitted — auto-tune now fires immediately on new correction data.
1.0.18
July 12, 2026

Batch Evaluation & Production Performance

Improvements (7)
  • Added batch evaluation — evaluate up to 100 traces in a single API call for faster AI safety and quality checks at scale.
  • Evaluation results are now persisted, surviving service restarts and providing a complete audit trail of every check.
  • Rate limiting now persists across deployments, providing consistent API protection without interruption.
  • Added multi-worker support for higher throughput evaluation processing — evaluations complete faster under load.
  • Added payload size limits and improved request validation across all services for better API reliability.
  • All services now output structured JSON logs for easier integration with log management and observability tools.
  • Evaluation service health endpoint now monitors Redis, MongoDB, OpenRouter, and NLI model connectivity for full operational visibility.
Fixes (4)
  • Fixed per-organization rate limiting to ensure fair resource allocation across teams.
  • Fixed health endpoints that reported "connected" without actually verifying database connectivity.
  • Fixed password hash handling for OAuth and SSO accounts, preventing authentication errors for social login users.
  • Fixed HTTP status codes returned by project creation endpoints to match API standards.
1.0.17
June 28, 2026

Real-Time Policy Enforcement & Dashboard Overhaul

Improvements (6)
  • Added real-time policy enforcement in the proxy gateway — prompt injection blocking, toxicity detection, and PII scrubbing are now enforced server-side based on your active policies.
  • Policy checks are cached and automatically refresh when you update a policy; latency depends on deployment conditions.
  • Completely redesigned the Policies dashboard with URL-persisted selection, accessible modals, delete confirmation, and distinct empty/error/loading states.
  • Overhauled the landing page with updated messaging and integration references for OpenAI, Anthropic, LangChain, Vercel AI SDK, and TypeScript.
  • Added an Architecture Flow diagram showing the full stack flow from your app through SDK/Proxy to LLM and into the dashboard.
  • Added policy templates for PII, toxicity, and prompt-injection controls; administrators explicitly choose which policies become active.
Fixes (5)
  • Fixed PII scrubbing being counted on every proxy request regardless of whether a PII policy was active — metrics are now accurate.
  • Fixed PII scrubbing metadata being hardcoded to true even when the PII policy was disabled.
  • Streaming requests covered by an active toxicity policy are rejected before provider dispatch because streamed output cannot be safely recalled after delivery begins.
  • Fixed audit-logs page pagination, export functionality, and filter labels — no more raw keys like `api_key` showing in the UI.
  • Added safe date formatting throughout the app to prevent crashes on null timestamps.
1.0.16
June 28, 2026

Metadata-Only Capture & Local PII Redaction

Improvements (4)
  • Introduced SDK content omission so prompts and completions can be removed from SDK telemetry before buffering.
  • Added Direct Routing Bypass — you can force LLM calls straight to OpenAI/Anthropic without going through the proxy when needed.
  • Added local, in-memory client-side PII Scrubbing for emails, credit cards, SSNs, API keys, and phone numbers.
  • Updated developer documentation with code snippets and guides for the new two-layer privacy architecture.
Fixes (1)
  • Added server-side PII redaction fallback to the API gateway as a secondary safety shield.
1.0.15
June 11, 2026

Autonomous Alerts & Circuit Breakers

Improvements (3)
  • Introduced a real-time Autonomous Alert Engine that monitors trace latency, costs, and error rates — get notified instantly when something goes wrong.
  • Built-in cooldown periods prevent alert fatigue during high-volume traffic spikes.
  • Alerts integrate with existing webhooks and Slack for immediate dispatch when conditions trip.
Fixes (1)
  • Migrated to a self-hosted Redis instance, improving overall system reliability and eliminating timeout issues that affected alert delivery.
1.0.14
June 11, 2026

Trace Waterfall Analysis & Tool Execution Console

Improvements (2)
  • Overhauled the trace detail page with an interactive Waterfall Analysis (Gantt Timeline) layout — see exactly how your LLM calls, tools, and retrievals overlap in time.
  • Introduced a Terminal-style Tool Execution Console to inspect raw shell commands and JSON payload outputs for tool spans.
Fixes (1)
  • Fixed deep nested span structures that were incorrectly scaled in the detail list view.
1.0.13
June 5, 2026

3D Interactive Trace Graph & Immersive Experience

Improvements (4)
  • Introduced an immersive 3D WebGL background with a rotating torus-knot and particle constellation that responds to mouse movement.
  • Redesigned the trace graph with sequential staggered alignment that matches exact trace execution sequences.
  • Added dedicated "Explore Interactive Demo" entry points and "New" feature badges throughout the app.
  • Custom edge paths with animated dashed lines and glowing drop-shadows make trace flow easier to follow.
Fixes (2)
  • Fixed a cursor conflict where the custom dashboard pointer overlapped text inputs and textareas.
  • Added live client-side trace list filtering for instant search by trace ID, name, or tags.
1.0.12
June 5, 2026

Precise Cost Attribution & Settings Reliability

Improvements (1)
  • Dashboard KPIs are now more accurate with improved real-time cost calculations.
Fixes (4)
  • Fixed extremely low-cost traces (e.g. gpt-4o-mini) showing as $0.0000 — they now display up to 6 decimal places correctly.
  • Resolved an authentication bug that affected some OAuth login flows.
  • Fixed a memory leak on the Settings page that could slow down the dashboard over time.
  • Fixed duplicate entries appearing when updating Webhook URLs and Alert integrations.
1.0.11
June 2, 2026

Hallucination Scoring & Circuit Breakers

Improvements (3)
  • Introduced automated Hallucination Scoring using LLM-as-a-judge with OpenRouter fallbacks for reliability.
  • Added the ability to set Circuit Breakers based on hallucination thresholds to automatically prevent downstream errors.
  • Overhauled the trace timeline UI to better visualize nested agent spans and sub-calls.
Fixes (2)
  • Resolved intermittent latency spikes in the proxy gateway under high concurrent load.
  • Fixed a visual alignment bug in the project selector dropdown.