Skip to content
TradeX Documentation

Data Flow

This document describes the data flows for key operations in the TradeX platform.

Order Processing Flow

Section titled “Order Processing Flow”
sequenceDiagram participant Client participant OrderService participant MetadataService participant RiskService participant WalletService participant Kafka participant MatchingEngine participant PositionService participant MarketData Client->>OrderService: POST /orders OrderService->>MetadataService: GetInstrument (gRPC) OrderService->>RiskService: PreTradeCheck (gRPC) RiskService-->>OrderService: APPROVED + margin details OrderService->>WalletService: LockFunds (gRPC) WalletService-->>OrderService: funds locked OrderService->>Kafka: order.command.v1 Kafka->>MatchingEngine: order.command.v1 MatchingEngine->>MatchingEngine: Match Orders MatchingEngine->>Kafka: engine.event.v1 Kafka->>OrderService: engine.event.v1 (update status) Kafka->>PositionService: engine.event.v1 (apply trade) Kafka->>MarketData: engine.event.v1 (update book) OrderService->>WalletService: SettleTrade (gRPC) MarketData->>Client: WebSocket update

Steps

Section titled “Steps”
  1. Client submits order via Order Service REST API
  2. Instrument validated — tick size, lot size, min notional checked via Metadata Service gRPC
  3. Risk pre-trade check — Risk Service validates leverage, margin sufficiency, exposure limits, price bands, and account status
  4. Funds locked — Wallet Service reserves initial margin via LockFunds
  5. Order published to Kafka (order.command.v1)
  6. Matching Engine consumes and matches against the order book
  7. Trade executed — engine.event.v1 published to Kafka
  8. Order Service updates order status (filled / partially filled) and calls WalletService.SettleTrade for the order-side wallet posting
  9. Position Service also consumes engine.event.v1, appends to position_ledger, and emits:
    • position.updated.v1 / position.closed.v1
    • settlement.v1 (canonical settlement signal — consumed by Wallet Service)
    • risk.exposure.v2 (per-user aggregate, debounced)
  10. Market Data Service updates order book and broadcasts via WebSocket

Market Data Flow

Section titled “Market Data Flow”
sequenceDiagram participant MatchingEngine participant Kafka participant MarketData participant Redis participant PostgreSQL participant Client MatchingEngine->>Kafka: engine.event.v1 (trades + book deltas) MatchingEngine->>Kafka: engine.snapshot.v1 (full book state) MatchingEngine->>Kafka: engine.bookhash.v1 (integrity check) Kafka->>MarketData: Consume events MarketData->>MarketData: Apply delta / verify hash MarketData->>Redis: Cache order book MarketData->>PostgreSQL: Persist trades (TimescaleDB) MarketData->>Kafka: md.trades.v1 + md.orderbook.delta.v1 MarketData->>Client: WebSocket update

Steps

Section titled “Steps”
  1. Matching Engine publishes trade events, book deltas, periodic full snapshots, and book hashes to Kafka
  2. Market Data Service consumes all three event types — deltas are applied incrementally, snapshots reset local state, hashes verify integrity
  3. Order book cached in Redis for low-latency REST queries
  4. Trades persisted to PostgreSQL (TimescaleDB) for OHLCV aggregation and history
  5. Normalized events re-published to md.* topics for downstream consumers
  6. Updates broadcast to WebSocket subscribers

User Onboarding Flow

Section titled “User Onboarding Flow”
sequenceDiagram participant Client participant AuthService participant Kafka participant UserService participant WalletService Client->>AuthService: POST /signup AuthService->>AuthService: Hash password, create account AuthService->>Kafka: auth.account.created.v1 Kafka->>UserService: auth.account.created.v1 UserService->>UserService: Create profile (KYC tier 0) UserService->>Kafka: user.created.v1 Kafka->>WalletService: user.created.v1 WalletService->>WalletService: Initialize balance (INR + USDT) AuthService->>Client: Return JWT tokens

Steps

Section titled “Steps”
  1. Client signs up — Auth Service hashes password and creates auth_accounts record
  2. Auth event published — auth.account.created.v1 triggers user profile creation
  3. User Service creates profile at KYC tier 0 (limited trading until verified)
  4. User event published — user.created.v1 triggers wallet initialization
  5. Wallet Service creates zero-balance ledger entries for INR and USDT
  6. JWT tokens returned to client — account is immediately usable within KYC tier 0 limits

Authentication Flow

Section titled “Authentication Flow”
sequenceDiagram participant Client participant AuthService participant PostgreSQL participant Kafka Client->>AuthService: POST /login AuthService->>PostgreSQL: Validate credentials AuthService->>AuthService: Generate JWT (access + refresh) AuthService->>PostgreSQL: Store refresh token AuthService->>Kafka: auth.login.v1 AuthService->>Client: Return tokens

Steps

Section titled “Steps”
  1. Client submits credentials
  2. Auth Service validates credentials against database (bcrypt comparison)
  3. JWT tokens generated — short-lived access token + long-lived refresh token
  4. Refresh token stored in database
  5. Login event published to Kafka for audit trail
  6. Tokens returned to client

Wallet Operations Flow

Section titled “Wallet Operations Flow”
sequenceDiagram participant Client participant WalletService participant PostgreSQL participant Kafka participant External Client->>WalletService: POST /v1/deposit WalletService->>PostgreSQL: Record deposit (pending) WalletService->>External: Generate deposit address / initiate payment External->>External: Confirm deposit External->>WalletService: Webhook / callback WalletService->>PostgreSQL: Credit balance (NUMERIC, optimistic lock) WalletService->>Kafka: wallet.deposit_confirmed.v1 WalletService->>Client: Return status

Steps

Section titled “Steps”
  1. Client initiates deposit request
  2. Wallet Service creates pending deposit record
  3. External system generates deposit address (crypto) or initiates payment rail (fiat)
  4. External system confirms the deposit
  5. Balance credited atomically with optimistic locking (version check prevents double-credit)
  6. Deposit confirmed event published to Kafka
  7. Status returned to client

Funding Settlement Flow

Section titled “Funding Settlement Flow”

Runs automatically every 8 hours (00

, 08
, 16
UTC). Scheduler-driven — md.funding.v1 is consumed only for rate-cache warming, not as a settlement trigger.

sequenceDiagram participant MarketData participant FundingService participant PositionService participant Kafka participant WalletService MarketData->>FundingService: md.funding.v1 (rate cache warm) FundingService->>FundingService: Phase 1 — open cycle (SCHEDULED) Note over FundingService: Wait SNAPSHOT_GRACE_PERIOD (30s) FundingService->>PositionService: ListPositionsBySymbol(as_of_unix_ms = cycle_ts) PositionService-->>FundingService: Positions as observed at boundary FundingService->>FundingService: Phase 2 — insert funding_settlements (PENDING) loop Worker pool (SKIP LOCKED, exp. backoff) FundingService->>Kafka: position.funding.v1 (idempotent fanout) FundingService->>Kafka: funding.payment.settled.v1 (audit) end Kafka->>PositionService: position.funding.v1 PositionService->>PositionService: Fold funding delta into ledger PositionService->>Kafka: settlement.v1 (kind = FUNDING) Kafka->>WalletService: settlement.v1 → ledger posting FundingService->>FundingService: Sealer — zero-sum check → SEALED or NEEDS_REVIEW

Steps

Section titled “Steps”
  1. Market Data Service publishes md.funding.v1; Funding Service tails it into the in-memory rate cache (does not trigger settlement).
  2. Scheduler Phase 1 (leader-elected): INSERT … ON CONFLICT DO NOTHING into funding_cycles at the boundary, freezing rate / mark / index. Status → SCHEDULED.
  3. Snapshot grace (30s) — wait for all in-flight trades for the cycle to land.
  4. Scheduler Phase 2: call PositionService.ListPositionsBySymbol with as_of_unix_ms = cycle_timestamp. Position Service replays its ledger at exactly that instant, so the snapshot is deterministic and retry-safe.
  5. Settlement work queue: one funding_settlements row per non-flat position with status PENDING and idempotency key funding:{cycle_ts_unix}:{user_id}:{symbol}.
  6. Worker pool (every pod, SELECT … FOR UPDATE SKIP LOCKED) drains the queue: publishes position.funding.v1 (fanout to Position Service) and funding.payment.settled.v1 (public audit). On terminal failure → DEAD_LETTER.
  7. Position Service consumes position.funding.v1, folds the funding delta into the ledger, and emits settlement.v1 (kind = FUNDING) — the canonical wallet posting signal.
  8. Sealer runs the zero-sum check once all settlements reach a terminal state. Pass → SEALED. DLQ rows present or |paid − received| > tolerance → NEEDS_REVIEW; admin overrides via POST /internal/funding/cycles/:id/seal.

Configuration Update Flow

Section titled “Configuration Update Flow”
sequenceDiagram participant Admin participant MetadataService participant PostgreSQL participant Outbox participant Kafka participant Consumer Admin->>MetadataService: POST /v1/instruments (Maker) MetadataService->>PostgreSQL: Save Instrument (pending) MetadataService->>PostgreSQL: Write to Outbox Admin->>MetadataService: POST /v1/instruments/:symbol/approve (Checker) MetadataService->>PostgreSQL: Update status (active) MetadataService->>PostgreSQL: Write to Outbox MetadataService->>Outbox: Process Outbox Outbox->>Kafka: md.instrument.created.v1 Kafka->>Consumer: Instrument event

Steps

Section titled “Steps”
  1. Admin creates instrument (maker role, status: pending)
  2. Instrument saved to database with outbox entry
  3. Admin approves instrument (checker role, status: active)
  4. Status updated and second outbox entry written
  5. Outbox worker polls and publishes events to Kafka
  6. Downstream consumers receive events and update local caches

Liquidation Flow

Section titled “Liquidation Flow”
sequenceDiagram participant RiskService participant Kafka participant SettlementService participant OrderService participant MatchingEngine participant WalletService RiskService->>Kafka: risk.liquidation.v1 Kafka->>SettlementService: risk.liquidation.v1 SettlementService->>SettlementService: Create settlement_record (initiated) SettlementService->>SettlementService: Compute bankruptcy price SettlementService->>OrderService: Submit IOC liquidation order (gRPC) OrderService->>Kafka: order.command.v1 Kafka->>MatchingEngine: order.command.v1 MatchingEngine->>Kafka: engine.event.v1 (fill) Kafka->>SettlementService: engine.event.v1 SettlementService->>SettlementService: Compute shortfall alt No shortfall SettlementService->>Kafka: settlement.completed.v1 else Shortfall covered by insurance fund SettlementService->>WalletService: Draw from insurance fund SettlementService->>Kafka: settlement.insurance.used.v1 SettlementService->>Kafka: settlement.completed.v1 else Shortfall requires ADL SettlementService->>Kafka: settlement.adl.triggered.v1 SettlementService->>Kafka: settlement.completed.v1 end

Steps

Section titled “Steps”
  1. Risk Service emits risk.liquidation.v1 when a user’s margin ratio breaches the maintenance threshold
  2. Settlement Service creates a settlement_records row and computes the bankruptcy price
  3. Liquidation order submitted — IOC limit order at bankruptcy price routed through the normal Order Service → Matching Engine pipeline
  4. Matching Engine fills the order (partially or fully)
  5. Settlement Service monitors engine.event.v1 for the fill
  6. Shortfall handled — if the fill price is worse than the bankruptcy price, the insurance fund covers the gap; if the insurance fund is insufficient, ADL proportionally force-reduces the most profitable counterparty positions
  7. Settlement events published — consuming services update positions, wallets, and compliance records

Index Price Calculation Flow

Section titled “Index Price Calculation Flow”
sequenceDiagram participant ExternalExchanges participant MarketData participant PostgreSQL participant Redis participant Kafka participant Client loop Every interval MarketData->>ExternalExchanges: Fetch prices (ccxt — Binance, Coinbase, Kraken, OKX) ExternalExchanges->>MarketData: Return prices MarketData->>MarketData: Weighted average MarketData->>PostgreSQL: Store index price MarketData->>Redis: Cache index price MarketData->>Kafka: md.index.v1 MarketData->>Client: WebSocket update end

Steps

Section titled “Steps”
  1. Index Price Worker fetches spot prices from configured external exchanges via ccxt
  2. Weighted average computed using per-exchange weights from Metadata Service config
  3. Index price stored in PostgreSQL and cached in Redis
  4. md.index.v1 published to Kafka for downstream consumers (mark price, funding rate computation)
  5. Update broadcast to WebSocket subscribers
Section titled “Related Documentation”