Simulation Harness

Simulation Harness

The ultradag-sim crate provides a deterministic consensus simulation for testing DAG-BFT logic without any network I/O. It uses the real BlockDag, FinalityTracker, StateEngine, and Mempool from ultradag-coin — only the network layer is replaced with a virtual network.


Architecture

Architecture diagram: The SimHarness orchestrates a VirtualNetwork and Invariant Checkers. The VirtualNetwork delivers messages between SimValidator instances (1 through N), each containing a BlockDag, FinalityTracker, StateEngine, and Mempool. A Transaction Generator feeds transactions into each validator’s Mempool.

Key Design Decisions

  • No TCP, no Tokio, no async: purely synchronous, deterministic execution
  • Real consensus logic: uses production BlockDag, FinalityTracker, StateEngine, Mempool
  • Virtual network: message delivery controlled by the harness
  • Deterministic seeding: all randomness from ChaCha8Rng with configurable seed
  • Master invariant: all honest validators that finalize the same round produce identical compute_state_root() output

Components

VirtualNetwork

Controls message delivery between simulated validators:

Delivery ModeBehavior
PerfectAll messages delivered immediately in order
RandomOrderMessages delivered but in random order
Drop { probability }Messages dropped with the given probability (0.0-1.0)
Partition { split, heal_after_rounds }Messages between groups dropped; validators split at index split, healing after N rounds
Lossy { drop_probability }Combined reorder + drop at the specified rate
let network = VirtualNetwork::new(num_validators, DeliveryPolicy::Lossy { drop_probability: 0.05 }, seed);

SimValidator

A lightweight wrapper around real consensus components:

struct SimValidator {
    index: usize,
    sk: SecretKey,
    address: Address,
    dag: BlockDag,
    finality: FinalityTracker,
    state: StateEngine,
    mempool: Mempool,
    honest: bool,
    finality_history: Vec<(u64, [u8; 32])>,
}

Each SimValidator instance uses identical code to a production node, differing only in that messages are delivered through the VirtualNetwork instead of TCP.

ByzantineStrategy

Strategies for Byzantine validators:

StrategyBehavior
EquivocatorProduces two different vertices per round (conflicting content)
WithholderProduces vertices but withholds them from targeted peers
CrashStops producing entirely
TimestampManipulatorProduces vertices with manipulated timestamps
RewardGamblerAttempts to game reward distribution using a puppet address
GovernanceTakeoverAttempts to take over governance via malicious proposals
DuplicateTxFlooderFloods the network with duplicate transactions
FinalityStallerAttempts to stall finality progress
SelectiveEquivocatorSelectively equivocates to target specific rounds
let strategy = ByzantineStrategy::Equivocator;

SimHarness

The driver that orchestrates simulation rounds:

  1. For each round, each honest validator produces a vertex
  2. Byzantine validators execute their strategy
  3. Messages are delivered through the VirtualNetwork
  4. Each validator processes received vertices (insert, finality check, state apply)
  5. Invariants are checked after each round

Invariant Checkers

Automated checks run after every round:

InvariantDescription
State convergenceAll honest validators produce identical compute_state_root() for the same finalized round
Supply consistencyliquid + staked + delegated + treasury == total_supply on all validators
Round monotonicityFinalized round never decreases
Stake consistencytotal_staked and total_delegated match across all validators
Governance consistencyGovernance params and proposal IDs match across all validators
Council consistencyCouncil member count and set match across all validators

Transaction Generator

TxGen produces deterministic random transactions for stress testing:

  • Transfer transactions with random amounts and recipients
  • Stake and delegation transactions
  • Governance proposals and votes
  • All deterministically seeded from ChaCha8Rng

Test Suite

Base Consensus Tests (sample)

TestConfigurationRoundsValidates
4-validator perfectPerfect delivery100Basic consensus convergence
4-validator with transactionsPerfect + 20 tx/round200Tx processing under consensus
Single validator1 validator50Solo finality works
Random message reorderRandomOrder delivery200Order-independent convergence
100-seed sweepPerfect, 100 different seeds50 eachDeterminism across seeds
2-2 partition healPartition for 100 rounds, heal200Partition recovery
Equivocator detection1 Byzantine/4100Equivocation detected + supply correct
21-validator stress5% loss, 50 tx/round1000Large-scale convergence
Mixed Byzantine (2/7)2 Byzantine, 5 honest200BFT tolerance
Late-joiner convergence1 node joins at round 50200Late join converges
Governance with reorderRandomOrder + governance200tick_governance deterministic under reorder

Scenario Tests (sample)

ScenarioDescriptionRounds
StakingLifecycleStake, earn rewards, set commission, unstake500
DelegationRewardsDelegate, earn split rewards, undelegate300
GovernanceParameterChangePropose, vote, execute ParameterChange200
CrossFeatureStake + delegate + governance + equivocation simultaneously500
EpochTransitionForce active set recalculation250
StakeWithReorderStaking under random message reordering300
DelegationWithLossDelegation under 5% message loss400
GovernanceStressMultiple proposals + votes under adversarial conditions300

Total: 80+ simulation tests, all passing, all deterministic.


Master Invariant

The simulation’s primary correctness check:

Master Invariant
All honest validators that finalize the same round must produce identical compute_state_root() output.

This invariant has been verified under:

  • Normal operation (perfect delivery)
  • Random message reordering
  • Message loss (5%)
  • Network partitions with healing
  • Equivocation with slashing
  • Staking, delegation, and commission splits
  • Governance parameter change execution
  • Epoch transitions with validator set changes
  • Combined adverse conditions

Determinism

All simulation tests are fully deterministic:

  1. Each test has a seed value (u64)
  2. ChaCha8Rng::seed_from_u64(seed) initializes all randomness
  3. Message delivery order, transaction generation, and Byzantine behavior all derive from the seeded RNG
  4. Running the same test with the same seed always produces the same result

The 100-seed sweep test verifies this by running 100 different seeds and confirming all converge correctly.


Running the Tests

# Run all simulation tests
cargo test -p ultradag-sim

# Run a specific test
cargo test -p ultradag-sim -- test_4_validator_perfect

# Run with output
cargo test -p ultradag-sim -- --nocapture

Adding New Tests

To add a new simulation test:

  1. Define the scenario configuration (validators, rounds, delivery mode, byzantine strategies)
  2. Optionally configure transaction generation
  3. Run the harness
  4. The master invariant is checked automatically after each round
#[test]
fn test_my_scenario() {
    let config = SimConfig {
        validators: 4,
        byzantine: vec![(2, ByzantineStrategy::Withholder)],
        rounds: 200,
        delivery: DeliveryPolicy::Lossy(0.03),
        seed: 42,
        tx_per_round: 10,
    };
    let result = SimHarness::run(config);
    assert!(result.all_invariants_passed());
}

Relationship to Other Testing

LayerWhat It TestsHow
Unit testsIndividual functions and types#[cfg(test)] inline
Integration testsCross-module interactionstests/ directory
Simulation (this)Full consensus with virtual networkultradag-sim crate
Jepsen testsConsensus under fault injectionReal BlockDag + fault injector
TestnetFull stack including real TCP5-node Fly.io deployment

The simulation harness fills the gap between unit/integration tests (too narrow) and testnet (too slow, non-deterministic) by providing fast, deterministic, full-consensus testing.


Next Steps