Tutorial: Test your rule documents

Rule documents change more often than code, and they decide real outcomes, so they deserve tests. Build a small runner that validates a document and then checks it against a table of cases, each a fact and the outputs it must produce. Then watch it catch a one-character edit that would have charged customers for shipping.

RuleWright 15 minutes

You will learn to:

keep test cases next to the rules, as data
validate before evaluating, and report every problem
compare outputs across numeric types

Step 1: The cases

Each case is a fact and the outputs it expects. A null expectation means the output must not be written. These test the checkout policy (download rule-tests.json):

[
  {
    "name": "adult VIP with a big basket",
    "fact": {
      "Customer": { "Name": "Aroha", "Age": 34, "IsVip": true, "LoyaltyYears": 1 },
      "Order": { "Total": 240, "Lines": [] }
    },
    "expect": { "DiscountPercent": 15, "Shipping": 0 }
  },
  {
    "name": "loyal customer, small basket",
    "fact": {
      "Customer": { "Name": "Mere", "Age": 61, "IsVip": false, "LoyaltyYears": 9 },
      "Order": { "Total": 40, "Lines": [] }
    },
    "expect": { "DiscountPercent": 10, "Shipping": 5.95 }
  },
  {
    "name": "teenager buying wine gets no discount and an ID check",
    "fact": {
      "Customer": { "Name": "Ben", "Age": 17, "IsVip": true, "LoyaltyYears": 0 },
      "Order": { "Total": 60, "Lines": [ { "Category": "alcohol" } ] }
    },
    "expect": { "DiscountPercent": null, "IdRequired": true, "Shipping": 0 }
  },
  {
    "name": "shipping is free from exactly 50",
    "fact": {
      "Customer": { "Name": "Sam", "Age": 30, "IsVip": false, "LoyaltyYears": 0 },
      "Order": { "Total": 50, "Lines": [] }
    },
    "expect": { "Shipping": 0, "Points": 5 }
  }
]

Facts are JSON, so the cases don't depend on your C# classes and anyone who can edit the rules can add a case.

Step 2: The runner

using System.Text.Json;
using RuleWright.Core;
using RuleWright.Execution;
using RuleWright.Json.SystemText;
using RuleWright.Serialization;

/// <summary>
/// Runs table-driven tests against a rule document: each case is a fact and the outputs it
/// must produce. A null expectation means "this output must not be written".
/// </summary>
public static class RuleTests
{
    public static int Run(RuleWrightEngine engine, string rulesJson, string testsJson)
    {
        // 1. The document must be valid, with every problem listed, not just the first.
        RuleSetValidationResult validation = engine.Validate(rulesJson);
        foreach (RuleValidationError error in validation.Errors)
        {
            Console.WriteLine($"INVALID {error.Path}: {error.Message}");
        }
        if (!validation.IsValid)
        {
            return validation.Errors.Count;
        }

        LoadedRuleSet rules = engine.LoadRuleSet(rulesJson);

        // 2. Every case must produce exactly the outputs it expects.
        int failures = 0;
        using JsonDocument tests = JsonDocument.Parse(testsJson);
        foreach (JsonElement test in tests.RootElement.EnumerateArray())
        {
            Dictionary<string, object?> fact = SystemTextJsonFacts.ToDictionary(test.GetProperty("fact"));
            RuleEvaluationResult result = engine.Evaluate(rules, fact);

            var problems = new List<string>();
            foreach (JsonProperty expected in test.GetProperty("expect").EnumerateObject())
            {
                result.Outputs.TryGetValue(expected.Name, out object? actual);
                if (!Matches(expected.Value, actual))
                {
                    problems.Add($"{expected.Name}: expected {expected.Value}, got {actual ?? "nothing"}");
                }
            }

            string name = test.GetProperty("name").GetString()!;
            Console.WriteLine(problems.Count == 0 ? $"PASS  {name}" : $"FAIL  {name}\n        {string.Join("\n        ", problems)}");
            failures += problems.Count == 0 ? 0 : 1;
        }

        return failures;
    }

    // Numbers compare by value (10 == 10.0), everything else by its JSON text.
    private static bool Matches(JsonElement expected, object? actual) => expected.ValueKind switch
    {
        JsonValueKind.Null => actual is null,
        JsonValueKind.Number => actual is long or int or decimal or double && expected.GetDecimal() == Convert.ToDecimal(actual),
        JsonValueKind.True or JsonValueKind.False => actual is bool b && b == expected.GetBoolean(),
        JsonValueKind.String => actual is string s && s == expected.GetString(),
        _ => false,
    };
}

Why it's written this way

Validate first. A document that fails validation reports every problem, each with its JSON pointer, before any case runs.
Compare numbers by value. A rule writes 10 as a long and addToOutput produces a decimal. The test cares that the discount is 10, not which numeric type carried it.
Only check what's listed. A case states the outputs it cares about. New outputs don't break old cases.

Step 3: Run it

using System.Text.Json;
using RuleWright.Core;
using RuleWright.Execution;
using RuleWright.Json.SystemText;
using RuleWright.Serialization;

RuleWrightEngine engine = new RuleWrightBuilder()
    .UseJsonReader(new SystemTextJsonReader())
    .Build();

int failures = RuleTests.Run(engine, File.ReadAllText("checkout-policy.json"), File.ReadAllText("rule-tests.json"));
Console.WriteLine($"{failures} failure(s)");

Output

PASS  adult VIP with a big basket
PASS  loyal customer, small basket
PASS  teenager buying wine gets no discount and an ID check
PASS  shipping is free from exactly 50
0 failure(s)

Step 4: Catch a regression

Someone tidies the shipping rule and turns GreaterThanOrEqual 50 into GreaterThan 50. It still validates, it still loads, and it's wrong for exactly one total:

using System.Text.Json;
using RuleWright.Core;
using RuleWright.Execution;
using RuleWright.Json.SystemText;
using RuleWright.Serialization;

// Someone "tidies" the shipping threshold from >= 50 to > 50, and the tests catch it.
string edited = File.ReadAllText("checkout-policy.json")
    .Replace("""{ "field": "Order.Total", "operator": "GreaterThanOrEqual", "value": 50 }""",
             """{ "field": "Order.Total", "operator": "GreaterThan", "value": 50 }""");

int failures = RuleTests.Run(engine, edited, File.ReadAllText("rule-tests.json"));
Console.WriteLine($"{failures} failure(s)");

Output

PASS  adult VIP with a big basket
PASS  loyal customer, small basket
PASS  teenager buying wine gets no discount and an ID check
FAIL  shipping is free from exactly 50
        Shipping: expected 0, got 5.95
1 failure(s)

The boundary case was written for exactly this. Put a case on every threshold in your rules.

Run it in CI

The runner returns the number of failures, so a console app that calls it for each rule file can exit with that number and fail the build. Or call it from a test in your test framework and assert it returns zero. Either way, a rule change gets the same review and the same gate as a code change.

Going further

Test typed facts too. If production evaluates your C# classes, also evaluate each case as a typed fact. A typo in a field path is a compile error there, where a dictionary would just read null.
Test with traces. Evaluate with EnableTrace and assert which rules fired, not only what they wrote. See Trace why a rule fired.
Validate every file. Even a file with no cases yet should pass Validate in CI.

Table of Contents