Skip to content

Best Practices

This page provides guidance for getting the most out of the Jaxon platform -- writing effective rules, building accurate rulesets, and developing a systematic testing strategy.

Writing Effective Rules

Each rule in a ruleset should encode one specific, testable requirement. The quality of the natural language rule text directly affects the quality of the generated DSAIL code and data extraction questions.

Be specific and unambiguous. A rule that says "financial reports must be accurate" is too broad to verify meaningfully. A rule that says "the CEO must certify the accuracy of quarterly financial reports" is specific enough for the platform to generate targeted questions and assertions.

Write testable statements. Each rule should be something that can clearly be evaluated as true or false given a piece of input text. Avoid subjective language like "adequate" or "reasonable" unless those terms are explicitly defined in the rule text.

One requirement per rule. If a policy paragraph covers three distinct requirements (e.g., record retention, access controls, and audit trails), create three separate rules rather than combining them. Atomic rules produce clearer results and make it easier to identify which specific requirement failed.

Use positive framing. Write rules that describe the desired state rather than the violation. "The CEO signed the financial report certification" is clearer than "The CEO did not fail to sign the certification." Positive framing reduces double-negative logic in the generated DSAIL code.

DSAIL Code Quality

DSAIL Language is the formal logic language that powers every rule. While the Ruleset Wizard generates DSAIL code automatically, reviewing and refining it improves accuracy.

Choose the right completion policy. Every assertion includes a policy annotation that determines how unknown values are handled:

  • [pessimistic] -- treats unknowns as failures. Use for compliance checking where missing information should be flagged. This is the recommended default for regulatory scenarios.
  • [neutral] -- leaves unknowns as uncertain. Useful for exploratory analysis where visibility into missing information is needed.
  • [optimistic] -- treats unknowns as passing. Use sparingly, only when missing information should be interpreted favorably.

Keep assertions focused. Each assertion should check one logical condition. If a rule has multiple requirements, use multiple assertions rather than a complex compound expression.

Use clear variable names. DSAIL variables become the basis for data extraction questions. A variable named ceoSignedCertification produces a better question than one named x or condition1. Prefer positive, descriptive names that naturally translate to yes/no questions.

declare ceoSignedCertification as boolean;
declare reportFollowsGAAP as boolean;

assert ceoCertification [pessimistic] { ceoSignedCertification };
assert gaapCompliance [pessimistic] { reportFollowsGAAP };

Review generated code. The wizard uses an LLM to generate DSAIL code, which means the output should always be reviewed for logical correctness. Verify that the assertions actually encode the intent of the rule text, and that the variable names will produce meaningful extraction questions.

Building Accurate Rulesets

Ruleset accuracy depends on the quality of the rules, the clarity of extraction questions, and iterative refinement of results.

Start with a clear policy source. The Ruleset Wizard extracts rules from uploaded policy Documents. Well-structured policy documents with explicit requirements produce better rule extractions than vague or poorly organized source material.

Review extraction questions. Each DSAIL variable has an associated question that the platform asks of the input text. These questions drive the accuracy of data extraction. Review them to ensure they are specific, unambiguous, and answerable from the type of text being evaluated.

Test iteratively. After creating a ruleset, run it against a few known examples before building a full test dataset. This quick feedback loop helps catch obvious issues with rule logic or question phrasing early.

Review per-rule results. When running batch tests against a Dataset, examine which individual rules are underperforming. A ruleset with 10 rules where 9 perform well and 1 performs poorly needs targeted refinement of that one rule, not a wholesale rewrite.

Refine questions before adding complexity. If a rule produces incorrect results, the most common fix is improving the extraction question. A more specific or differently-worded question often resolves accuracy issues without changing the DSAIL code.

Document Preparation

Text evaluated by a ruleset should give the platform enough information to answer the extraction questions for each rule.

Ensure sufficient context. If a rule checks whether the CEO signed a certification, the input document needs to contain information about the CEO's actions. Documents that are too brief or off-topic will produce UNKNOWN results for many rules.

Use clear structure. Documents with headings, sections, and explicit statements are easier for the platform to analyze than unstructured prose. When preparing test documents, favor clarity over brevity.

Match the document type to the use case. If a ruleset is designed to evaluate chat transcripts, test it with chat transcripts. If it is designed for financial reports, test it with financial reports. The platform's extraction accuracy depends on the input text matching the type of content the questions were designed for.

Testing Strategy

Systematic testing is the key to confident ruleset deployment.

Start with interactive tests. Before building a full dataset, run the ruleset against 2-3 known examples using interactive mode on the Runs page. This validates that the basic setup is working and the rules are producing sensible results.

Build a dataset with diverse examples. Create a Dataset with a mix of documents: some that should pass, some that should fail specific rules, and some edge cases. This gives a realistic picture of ruleset behavior.

Use variance testing for consistency. Create a batch run with the "Variance Test" option selected to evaluate the same Dataset of test documents multiple times and measure result stability. The dataset can include synthetic documents as long as they are representative of real inputs. High variance on specific claims indicates that the extraction question is ambiguous or the document content is unclear. See Variance Testing for details on reading and acting on variance results.

Iterate on the weakest rules. After a batch run, identify the rules that produce the most unexpected results. Focus refinement effort there rather than making broad changes.