Integration Tests

Open in Claude

This guide provides comprehensive information for running and writing integration tests for the OpenZeppelin Relayer. These tests validate multi-network transaction processing, API functionality, and end-to-end system behavior. This documentation serves both end users validating their relayer setup and contributors developing new tests.

Running Integration Tests

Prerequisites

Integration tests require specific dependencies and configuration files based on your testing mode.

Requirement	Description
Docker and Docker Compose	Required for Docker-based testing (recommended)
Rust 1.88+	Required for local testing without Docker
Redis instance	Required for local testing without Docker
Test configuration	Mode-specific `config.json` and `registry.json` files

Docker-based testing is recommended as it handles all dependencies automatically, including Redis, Anvil, and the Relayer service.

Quick Start

Local Mode (Anvil) - Recommended

Uses a local Anvil node with no testnet funds required.

# 1. One-time setup: Create Anvil keystore
cast wallet import anvil-test \
  --private-key PK - from anvil account already funded \
  --keystore-dir tests/integration/config/local/keys \
  --unsafe-password "test"

mv tests/integration/config/local/keys/anvil-test \
   tests/integration/config/local/keys/anvil-test.json

# 2. Copy and configure environment
cp .env.integration.example .env.integration
# Edit .env.integration with your API key (any value works for local mode)

# 3. Run tests via Docker
./scripts/run-integration-docker.sh

The Docker mode uses the local-anvil-integration network configuration with the Anvil RPC URL automatically configured.

Standalone Mode (Development)

For faster iteration during development with cargo run and cargo test:

Add Anvil relayer to your config/config.json. Add this relayer entry to the relayers array:

{
  "id": "anvil-relayer",
  "name": "Standalone Anvil Relayer",
  "network": "localhost",
  "paused": false,
  "signer_id": "anvil-signer",
  "network_type": "evm",
  "policies": {
    "min_balance": 0
  }
}

Add this signer entry to the signers array:

{
  "id": "anvil-signer",
  "type": "local",
  "config": {
    "path": "tests/integration/config/local/keys/anvil-test.json",
    "passphrase": {
      "type": "plain",
      "value": "test"
    }
  }
}

Start Anvil and run tests:

# Start Anvil and deploy contracts
./scripts/anvil-local.sh start

# In another terminal, run relayer
cargo run

# In another terminal, run tests
TEST_REGISTRY_PATH=tests/integration/config/local-standalone/registry.json \
cargo test --features integration-tests --test integration

# When done, stop Anvil
./scripts/anvil-local.sh stop

Standalone mode uses the localhost network pointing to http://localhost:8545, while Docker integration tests use localhost-integration pointing to http://anvil:8545.

Testnet Mode

For testing against live testnet networks:

Testnet mode requires real testnet funds. Ensure your test wallet is funded on all networks you plan to test.

# 1. Copy and configure environment
cp .env.integration.example .env.integration
# Edit .env.integration with your API key and passphrase

# 2. Copy and configure the testnet config
cp tests/integration/config/config.example.json tests/integration/config/testnet/config.json
cp tests/integration/config/registry.example.json tests/integration/config/testnet/registry.json
# Edit registry.json to enable the networks you want to test

# 3. Run tests via Docker
MODE=testnet ./scripts/run-integration-docker.sh

Test Configuration

Environment Variables

The .env.integration file stores API keys and secrets:

Variable	Description	Example
`API_KEY`	Relayer API authentication key	`ecaa0daa-f87e-4044-96b8-986638bf92d5`
`KEYSTORE_PASSPHRASE`	Password for local signer keystore	`your-secure-passphrase`
`WEBHOOK_SIGNING_KEY`	Webhook signing key (UUID)	`your-webhook-signing-key-here`
`LOG_LEVEL`	Logging verbosity	`info`

Create your environment file from the example:

cp .env.integration.example .env.integration

Registry Configuration

The registry.json file stores network-specific test metadata including contract addresses, minimum balances, and network selection. Create mode-specific registry files:

# For Local Mode (Anvil with Docker)
cp tests/integration/config/registry.example.json tests/integration/config/local/registry.json

# For Testnet Mode
cp tests/integration/config/registry.example.json tests/integration/config/testnet/registry.json

Example registry entry:

{
  "networks": {
    "sepolia": {
      "network_name": "sepolia",
      "network_type": "evm",
      "contracts": {
        "simple_storage": "0x5379E27d181a94550318d4A44124eCd056678879"
      },
      "min_balance": "0.1",
      "enabled": true
    }
  }
}

Network selection is controlled by the enabled flag. Only networks with "enabled": true will be included in test runs.

Relayer Discovery

Tests automatically discover relayers by querying the running relayer's API (GET /api/v1/relayers). This approach:

Provides a single source of truth by discovering what's actually running
Eliminates duplication by removing the need for separate test-specific configuration
Works identically in both Docker and standalone modes

The relayer service must be running before tests start. The config.json file is only used to start the relayer service, not by the tests themselves.

Running Specific Tests

Via Docker (Recommended)

# Run all tests (default MODE=local)
./scripts/run-integration-docker.sh

# Run tests in testnet mode
MODE=testnet ./scripts/run-integration-docker.sh

# Build images only
./scripts/run-integration-docker.sh build

# Stop services
./scripts/run-integration-docker.sh down

# View logs
./scripts/run-integration-docker.sh logs

# Open shell in test container
./scripts/run-integration-docker.sh shell

# Clean up everything
./scripts/run-integration-docker.sh clean

Via Cargo

# Run all integration tests
cargo test --features integration-tests --test integration

# Run specific test
cargo test --features integration-tests --test integration test_evm_basic_transfer

# Run with verbose output
RUST_LOG=debug cargo test --features integration-tests --test integration -- --nocapture

# Run with different registry path
TEST_REGISTRY_PATH=tests/integration/config/testnet/registry.json \
cargo test --features integration-tests --test integration

Test Architecture

Directory Structure

tests/integration/
├── README.md                    # Integration testing guide
├── tests/                       # All test files
│   ├── mod.rs
│   ├── authorization.rs         # API authorization tests
│   └── evm/                     # EVM network tests
│       ├── mod.rs
│       ├── basic_transfer.rs    # Basic ETH transfer tests
│       └── contract_interaction.rs
├── common/                      # Shared utilities and helpers
│   ├── mod.rs
│   ├── client.rs                # RelayerClient for API calls
│   ├── confirmation.rs          # Transaction confirmation helpers
│   ├── context.rs               # Multi-network test runner
│   ├── evm_helpers.rs           # EVM-specific utilities
│   ├── network_selection.rs     # Network filtering
│   └── registry.rs              # Test registry utilities
├── config/                      # Configuration files
│   ├── config.example.json
│   ├── registry.example.json
│   ├── local/                   # Local mode configs (gitignored)
│   ├── local-standalone/        # Standalone mode configs (gitignored)
│   └── testnet/                 # Testnet mode configs (gitignored)
└── contracts/                   # Smart contracts (Foundry)
    ├── README.md
    ├── foundry.toml
    └── src/

The structure separates test files (tests/) from shared utilities (common/) and configuration (config/).

Test Categories

Category	Location	Description	Example Tests
API Tests	`tests/integration/tests/`	REST endpoint validation	Authorization, CRUD operations
EVM Tests	`tests/integration/tests/evm/`	EVM chain operations	Basic transfers, contract interactions

Integration tests for Solana and Stellar networks are planned for future releases.

Test Registry System

The test registry (registry.json) centralizes network-specific test data, eliminating hardcoded values and simplifying network addition.

Schema

{
  "networks": {
    "<network-key>": {
      "network_name": "string", // Network identifier used by relayer
      "network_type": "string", // "evm", "solana", or "stellar"
      "contracts": {
        "<contract_name>": "address" // Deployed contract addresses
      },
      "min_balance": "string", // Minimum balance required (native token)
      "enabled": true // Whether network is active for testing
    }
  }
}

Adding a New Network

Add network entry to the appropriate mode-specific registry.json:

{
  "networks": {
    "arbitrum-sepolia": {
      "network_name": "arbitrum-sepolia",
      "network_type": "evm",
      "contracts": {
        "simple_storage": "0x..."
      },
      "min_balance": "0.01",
      "enabled": true
    }
  }
}

Add corresponding relayer entry to config.json with appropriate signer configuration
Deploy test contracts and update addresses in registry
Fund the signer wallet on the new network
Run tests: MODE=testnet ./scripts/run-integration-docker.sh

For detailed network configuration, see the Network Configuration documentation.

Test Execution Flow

Integration tests follow a standardized execution flow:

Test Initialization: Logging is initialized using the tracing crate with configurable log levels
Network Discovery: Enabled networks are loaded from the mode-specific registry.json file
Relayer Discovery: Active relayers are discovered via API endpoint GET /api/v1/relayers
Multi-Network Execution: Tests run across all eligible network and relayer combinations using the run_multi_network_test function
Transaction Confirmation: Transactions are submitted and confirmed using network-specific timeout configurations via wait_for_receipt

The run_multi_network_test function in tests/integration/common/context.rs:57 encapsulates this pattern for consistent test execution across all integration tests.

Writing New Integration Tests

Test Structure and Conventions

Integration tests follow standardized naming and structure conventions:

Test files: snake_case.rs
Test functions: test_<feature>_<scenario>
All integration tests require the integration-tests feature flag
Use #[tokio::test] for async tests

Minimal Test Example

use crate::integration::common::{
    client::RelayerClient,
    context::run_multi_network_test,
};
use openzeppelin_relayer::models::relayer::RelayerResponse;

async fn run_my_test(
    network: String,
    relayer_info: RelayerResponse,
) -> eyre::Result<()> {
    let client = RelayerClient::from_env()?;
    // Test logic here
    Ok(())
}

#[tokio::test]
async fn test_my_feature() {
    run_multi_network_test(
        "my_feature",
        is_evm_network,
        run_my_test
    ).await;
}

Using Test Utilities

The common/ directory provides essential utilities for integration testing.

RelayerClient

The RelayerClient in tests/integration/common/client.rs provides methods for API interaction:

send_transaction(&self, relayer_id: &str, tx_request: Value) - Submit transactions
get_transaction(&self, relayer_id: &str, tx_id: &str) - Retrieve transaction status
get_relayer_balance(&self, relayer_id: &str) - Check relayer balance

Example from tests/integration/tests/evm/basic_transfer.rs:53:

let client = RelayerClient::from_env()?;

let tx_request = serde_json::json!({
    "to": "0x000000000000000000000000000000000000dEaD",
    "value": "1000000000000",
    "data": "0x",
    "gas_limit": 21000,
    "speed": "fast"
});

let tx_response = client.send_transaction(&relayer.id, tx_request).await?;

Multi-Network Runner

The run_multi_network_test function in tests/integration/common/context.rs:57 handles multi-network test execution with network filtering:

run_multi_network_test(
    "test_name",
    is_evm_network,          // Network filter predicate
    run_test_function         // Async test function
).await;

Available network filters:

is_evm_network - Filters for EVM networks
evm_with_contract("contract_name") - Filters for EVM networks with specific contract deployed

Transaction Confirmation

The wait_for_receipt function in tests/integration/common/confirmation.rs handles transaction confirmation with network-specific timeouts:

use crate::integration::common::confirmation::{wait_for_receipt, ReceiptConfig};

let receipt_config = ReceiptConfig::from_network(&network)?;
wait_for_receipt(&client, &relayer_id, &tx_id, &receipt_config).await?;

Test Registry

Load the test registry to access network configuration and contract addresses:

use crate::integration::common::registry::TestRegistry;

let registry = TestRegistry::load()?;
let network_config = registry.get_network(&network)?;
let contract_address = registry.get_contract(&network, "simple_storage")?;

Working with Test Contracts

Test contracts are located in tests/integration/contracts/ and managed using Foundry.

Deploying New Contracts

cd tests/integration/contracts
forge build
forge create src/YourContract.sol:YourContract \
  --rpc-url <RPC_URL> \
  --private-key <PRIVATE_KEY>

After deployment, add the contract address to the appropriate registry.json file:

{
  "networks": {
    "sepolia": {
      "contracts": {
        "your_contract": "0x..."
      }
    }
  }
}

Best Practices

When writing integration tests, follow these guidelines:

Use structured logging: Leverage the tracing crate for informative, filterable logs with info!, debug!, and error! macros
Test isolation: Each test should be independent and not rely on state from other tests
Error handling: Use eyre::Result for clear error propagation and context
Timeouts: Configure appropriate timeouts based on network characteristics using ReceiptConfig

Example structured logging from tests/integration/tests/evm/basic_transfer.rs:25:

use tracing::{info, debug, info_span};

async fn run_basic_transfer_test(
    network: String,
    relayer_info: RelayerResponse,
) -> eyre::Result<()> {
    let _span = info_span!("basic_transfer",
        network = %network,
        relayer = %relayer_info.id
    ).entered();

    info!("Starting basic transfer test");
    debug!(relayer = ?relayer_info, "Full relayer details");

    // Test implementation

    info!("Test completed successfully");
    Ok(())
}

Error: MacMismatch

Solution: Ensure KEYSTORE_PASSPHRASE in .env.integration matches the password used when creating the keystore file.

Connection Refused

Cannot connect to required services (Redis, Relayer, or Anvil).

Error message:

Error: Connection refused (os error 111)

Solution: Ensure all required services are running. For Docker mode:

docker-compose -f docker-compose.integration.yml ps

For standalone mode, verify Redis and the Relayer service are running.

Insufficient Funds

Test wallet does not have sufficient funds for transactions.

Error message:

Error: insufficient funds for transfer

Solution: Fund the test wallet. Check the relayer logs for the signer address:

docker-compose -f docker-compose.integration.yml logs relayer | grep address

Then use a testnet faucet to fund the address.

Network Timeout

Transaction confirmation timeout exceeded.

Error message:

Error: Transaction confirmation timeout

Solution:

Verify the network RPC endpoint is accessible
Check if the network is experiencing congestion
Review network-specific timeout configurations in ReceiptConfig

Debugging Techniques

View Container Logs

# All services
./scripts/run-integration-docker.sh logs

# Specific service
docker-compose -f docker-compose.integration.yml logs integration-relayer
docker-compose -f docker-compose.integration.yml logs integration-tests

Interactive Shell Access

Access the test container for debugging:

./scripts/run-integration-docker.sh shell

Run Single Test with Verbose Output

RUST_LOG=debug \
  cargo test --features integration-tests --test integration test_name -- --nocapture

The --nocapture flag shows real-time log output during test execution.

Cleanup and Maintenance

Remove all Docker resources if tests leave behind resources:

# Using helper script
./scripts/run-integration-docker.sh clean

# Or manually
docker-compose -f docker-compose.integration.yml down -v --remove-orphans

Additional Resources

Quick Start Guide - Initial setup and configuration
Project Structure - Overview of project organization
API Reference - Detailed API documentation
Network Configuration - Network setup guide
GitHub Repository - Source code and examples

Integration Tests

On this page