Skip to content

Release Notes

Version 2.0

Version 2.0 is a major platform update that simplifies the ruleset model, introduces AI-powered rule creation, and redesigns the user interface around a focused document verification workflow.

Simplified Ruleset Model

The platform now focuses exclusively on rulesets powered by DSAIL formal logic. The previous guardrail types (Consensus, Consistency Checking, Critique & Revise, Entailment, LLM-as-a-Judge, and Human Review as standalone) have been replaced by a unified Ruleset/Rule model. Every ruleset contains one or more rules, each encoding a single policy requirement as DSAIL code evaluated by an SMT solver for mathematically provable results.

Ruleset Creation Wizard

A new wizard extracts rules from policy documents automatically. Paste a compliance document or policy summary, and the wizard analyzes the text, identifies individual requirements, and generates both natural language rule descriptions and DSAIL formal logic code. This replaces the manual rule-by-rule configuration workflow.

Ruleset Studio

The integrated Ruleset Studio provides a unified environment for viewing and editing rules, DSAIL code, and extraction questions. Each rule's components are visible in context -- natural language text, formal logic, and the questions the platform uses to extract relevant information from documents.

REST API

A new REST API provides full programmatic access to all platform operations -- managing projects, documents, rulesets, datasets, and runs. Interactive API documentation is available at /api/docs (Swagger UI) and /api/redoc (ReDoc), generated automatically from the OpenAPI specification.

Redesigned Navigation

The interface is now sidebar-driven with four primary sections:

  • Documents -- First-class document management with versioning and text content support
  • Rulesets -- Create, view, and manage rulesets with the creation wizard and Ruleset Studio
  • Datasets -- Organize test data and link documents for accuracy measurement
  • Runs -- Unified testing interface with accuracy and variance testing capabilities

Documents

Documents are now a first-class entity with full versioning support. Create documents by pasting text directly or uploading files. Every edit creates a new immutable version, providing a complete audit trail.

Runs

The Runs interface replaces the previous Playground as the unified testing environment. Run rulesets against individual documents or entire datasets. Test types include:

  • Single document tests -- Quick compliance verification against one document
  • Accuracy tests -- Measure ruleset performance against gold-standard datasets with F1, Precision, and Recall metrics
  • Variance tests -- Statistical analysis across multiple iterations to assess ruleset consistency

Previous Releases

Version 1.3.0

Dataset Management & Testing

  • Dataset Creation & Management: Comprehensive dataset system for organizing test data. Create datasets from scratch, import from CSV, or export to CSV for external analysis. Manual record editing with full CRUD operations.

  • Synthetic Data Generation: Parallel bulk generation of synthetic test records (1/10/100 at a time) using configurable LLMs. Advanced candidate scoring with D-score diversity metrics ensures high-quality, diverse training data. Background job processing with real-time progress tracking.

  • Test Execution Framework: Background test job system for running comprehensive accuracy and variance tests against datasets. Real-time progress monitoring with cancellation support. Jobs persist across navigation for long-running test suites.

Quality Assurance & Validation

  • Accuracy Testing: Evaluate ruleset performance against gold-standard datasets with detailed per-rule metrics including F1 score, Precision, and Recall. Overall ruleset scoring with confusion matrix analysis to identify systematic errors.

  • Variance Testing: Statistical variance analysis across multiple test iterations to measure ruleset consistency. Identify non-deterministic behavior patterns and evaluate reliability across different inputs and conditions.

  • Test Results Visualization: Interactive test result displays with sortable tables, per-rule performance breakdowns, and detailed accuracy metrics. Export test results for reporting and quality assurance workflows.

Enhanced User Experience

  • Record Management UI: Intuitive interface for viewing, adding, editing, and deleting dataset records with appropriate field validation.

  • Background Job Monitoring: Real-time status cards for generation and test jobs with progress bars, completion metrics, and cancel/done actions. Jobs continue running during navigation with persistent state.

  • Data Quality Tracking: Automatic labeling of records as Gold (uploaded/manual) or Silver (synthetic) quality for downstream filtering and weighted evaluation strategies.

Version 1.2.0

Authentication & Authorization

  • Authentik Integration: Enterprise-grade authentication system with SSO support and comprehensive user management.
  • Group-Based Access Control: Security groups function as teams that own assets at a project level, with flexible assignment based on organizational structure.

Configuration Management

  • Version Control for Rulesets: Track changes to ruleset configurations with complete version history and ability to revert to previous versions.

Policy Rules Enhancements

  • Verification Rulesets for Questions: Add secondary verification layer to Policy Rules questions, enabling configurable rulesets to validate LLM-generated answers for improved accuracy and reliability.

Version 1.1.2

Enhanced User Experience

  • Sorting for Rulesets lists: Quickly organize items in the rulesets view.

Platform & Infrastructure

  • Kafka Topic Creation Flow: Adjusted creation process to avoid topic creation bootstrapping issues.
  • Kafka Image Update: Switched to the Apache-maintained image due to Bitnami deprecation.

Version 1.1.1

Enhanced User Experience

  • Speech-to-Text Inputs for Textareas: Microphone-powered STT for textareas across the UI for faster, hands-free input.

AI-Powered Automation

  • RAG-Based Pipeline for Policy to Rule Extraction: Retrieval-augmented option for converting policy documents into structured rules.

Expanded LLM Support

  • Support for GPT-5: Configure and run rulesets with GPT-5.

Reliability & Quality

  • Bug Fixes & UI/Usability Improvements: General stability enhancements and minor UI polish.

Version 1.1.0

Core Features

  • Batch Processing: Enhanced Python client with support for batch request processing, enabling efficient bulk verification operations.

Client Library Improvements

  • Pre-Flight Validation: Jaxon client validates target rulesets before execution, preventing wasted resources on invalid configurations.
  • Enhanced Batch Processing: Improved batch operations that continue processing valid items while clearly identifying and logging validation failures.

Platform & Infrastructure

  • Kubernetes Support: Full Kubernetes deployment support with Helm charts for production-scale orchestration and container management.
  • Enhanced Python Client: Improved client library with batch processing capabilities.

Version 1.0.0

Core Features

  • Rulesets: Verify LLM outputs with Policy Rules (Neurosymbolic) rulesets.
  • Integrated Logging & Dashboard: Real-time dashboards for monitoring inference.
  • LLM Generalization: LLM proxy for decoupled usage of major LLM providers.

APIs and Integrations

  • Python Client: Installable Python client for direct integration into customer applications.