Funding Service

The Funding Service is a Go microservice that runs the perpetual-futures funding settlement loop. A leader-elected scheduler snapshots open positions at fixed cycle boundaries (00

/ 08

/ 16

UTC by default), inserts per-user settlement work rows, and a pod-wide worker pool drains the queue — calling Wallet Service or publishing the settlement to the Position Service depending on the configured cutover phase.

Architecture note: The service is scheduler-driven, not Kafka-driven. The md.funding.v1 topic is tailed only to keep the in-memory rate cache warm; it does not trigger settlements.

Responsibilities

• Cycle creation: Open a funding_cycles row at each boundary, freezing rate/mark/index for the cycle.
• Position snapshotting: Call Position Service with as_of_unix_ms = cycle_timestamp to reconstruct positions at the boundary deterministically.
• Settlement queue: Insert one funding_settlements row per (cycle, user) pair.
• Settlement application: Apply each settlement idempotently — either through WalletService.ApplyFunding (gRPC) or by publishing position.funding.v1 for Position Service to fold and emit as settlement.v1 (cutover-dependent).
• Audit publication: Emit funding.payment.settled.v1 for every applied settlement.
• Dead-letter management: Move repeatedly-failing settlements to DEAD_LETTER; expose admin retry / cancel endpoints.
• Zero-sum sealing: When all settlements in a cycle reach a terminal state, run a zero-sum check (total_paid ≈ total_received) before sealing.

Architecture

graph TB MDS[Market Data Service] -->|md.funding.v1| RateTailer[Rate Tailer
cache warmer] Meta[Metadata Service] -.gRPC ListAllInstruments.-> Scheduler[Scheduler
leader-elected] RateTailer --> Cache[(In-Memory Rate Cache)] Scheduler -->|Phase 1: create cycle| PG[(PostgreSQL)] Scheduler -.gRPC ListPositionsBySymbol
as_of_unix_ms=cycle_ts.-> Pos[Position Service] Scheduler -->|Phase 2: insert settlements| PG PG --> Workers[Worker Pool
SELECT FOR UPDATE SKIP LOCKED] Workers -.gRPC ApplyFunding.-> Wallet[Wallet Service] Workers -->|funding.payment.settled.v1| Kafka[Kafka] Workers -->|position.funding.v1| Kafka Sealer[Sealer] -->|aggregate + zero-sum check| PG

Cycle & Settlement State Machines

funding_cycles.status:

SCHEDULED ──Phase 2 inserts settlements──▶ IN_PROGRESS
                                              │
                       all settlements terminal + zero-sum OK
                                              ▼
                                            SEALED
                                              │
                    DLQ rows present OR zero-sum violation
                                              ▼
                                         NEEDS_REVIEW
                                              │
                       admin POST .../seal with acknowledgement
                                              ▼
                                            SEALED

funding_settlements.status:

PENDING ──ApplyFunding OK──▶ APPLIED ──Kafka publish OK──▶ APPLIED_PUBLISHED
   │                            │
   │ position size = 0          │
   └────▶ SKIPPED ◀─────────────┘ (also when wallet returns "no-op")
   │
   │ attempts >= WORKER_MAX_ATTEMPTS
   ▼
DEAD_LETTER ──admin retry──▶ PENDING
   │
   │ admin cancel
   ▼
CANCELLED

Terminal statuses: APPLIED_PUBLISHED, SKIPPED, DEAD_LETTER, CANCELLED.

Funding Math

The funding rate itself is computed upstream by the Market Data Service and arrives on md.funding.v1. Funding Service only freezes it into the cycle row.

Per-user amount:

amount = sign(side) × size × mark_price × funding_rate
         where sign(LONG) = -1 and sign(SHORT) = +1

Long  + positive rate  → amount < 0  (user pays)
Long  + negative rate  → amount > 0  (user receives)
Short + positive rate  → amount > 0  (user receives)
Short + negative rate  → amount < 0  (user pays)

Net of all users in a cycle should be ≈ 0 (modulo ZERO_SUM_TOLERANCE_USDT).

Scheduler Phases

A leader is elected via a Postgres advisory lock; only the leader runs the scheduler. On leader gain, the scheduler also catches up any missed boundaries.

Phase 1 — Create Cycle (idempotent on (symbol, cycle_timestamp))

1. Fetch instrument set (cached for 5 min) and current rate from the in-memory rate cache.
2. INSERT … ON CONFLICT DO NOTHING into funding_cycles with status SCHEDULED, freezing funding_rate, mark_price, index_price, premium_rate_twap, interest_rate, funding_interval_hours.

Phase 2 — Snapshot + Enqueue Settlements

1. Wait SNAPSHOT_GRACE_PERIOD (30s default) after the boundary so all upstream trades for the cycle are visible.
2. Call:

   positions, err := positionCli.GetPositionsBySymbol(
       ctx, cycle.Symbol,
       cycle.CycleTimestamp, // → as_of_unix_ms
       cycle.TraceID,
   )

   Position Service replays its ledger at exactly cycle_timestamp — fills happening after the boundary do not contaminate the snapshot, and the snapshot is deterministic across retries.
3. For each non-flat position compute amount and INSERT a funding_settlements row with status PENDING and idempotency key:

   wallet_idempotency_key = "funding:{cycle_ts_unix}:{user_id}:{symbol}"

4. Stamp position_snapshot_taken_at and move the cycle to IN_PROGRESS.

Worker Loop

Runs on every pod (not just the leader). The claim query keeps the work queue off the heap by using a partial covering index:

SELECT id FROM funding_settlements
WHERE status IN ('PENDING','APPLIED')
  AND next_attempt_at <= now()
ORDER BY next_attempt_at, attempt_count
FOR UPDATE SKIP LOCKED
LIMIT $batch;

The visibility timeout (next_attempt_at += 60s on claim) means a crashed worker’s rows auto-reappear.

For each claimed row:

1. If status = PENDING and Wallet ApplyFunding is enabled (WALLET_GRPC_APPLY_FUNDING_ENABLED=true): gRPC call to WalletService.ApplyFunding with the deterministic idem key. On success → APPLIED. On terminal “no-op” reply → SKIPPED. Otherwise increment attempt_count and back off.
2. If status = PENDING and Wallet ApplyFunding is disabled (the steady-state after Phase-4 cutover): Skip the wallet call; flip directly to APPLIED. The settlement reaches the user via Position Service’s settlement.v1 (downstream of the position.funding.v1 publish below).
3. If status = APPLIED: publish funding.payment.settled.v1 and, when enabled, position.funding.v1 (fanout). On success → APPLIED_PUBLISHED.

Backoff: BackoffBase × 2^attempt with ±20% jitter, capped at BackoffMax (5m). After WORKER_MAX_ATTEMPTS (5 default) → DEAD_LETTER.

Sealer

Polls IN_PROGRESS cycles where every settlement is terminal. Aggregates:

SELECT
  SUM(CASE WHEN funding_amount > 0 THEN funding_amount ELSE 0 END) AS total_received,
  SUM(CASE WHEN funding_amount < 0 THEN ABS(funding_amount) ELSE 0 END) AS total_paid
FROM funding_settlements
WHERE cycle_id = $1
  AND status IN ('APPLIED_PUBLISHED','SKIPPED','DEAD_LETTER','CANCELLED');

• If |total_paid - total_received| <= ZERO_SUM_TOLERANCE_USDT and zero DLQ rows → SEALED.
• Otherwise → NEEDS_REVIEW; admin must POST /internal/funding/cycles/:id/seal with an acknowledgement payload to override.

Mid-cycle visibility: While IN_PROGRESS, totals can be derived live from funding_settlements rows (the sealer just persists the final aggregate to funding_cycles).

Messaging (Kafka Topics)

All topic names are environment-suffixed at runtime.

Consumed

Topic	Purpose
md.funding.v1	Rate snapshots from Market Data Service. Cache-warming only — does not trigger settlements.

Published

Topic	When
funding.payment.settled.v1	After every APPLIED → APPLIED_PUBLISHED transition. Public audit / analytics.
position.funding.v1	Optional fanout (gated by env). Consumed by Position Service, which folds the funding delta and emits settlement.v1 for the Wallet Service. Idempotent on (event_source, event_id).

gRPC Clients (Outbound)

Service	RPC	Use
Wallet Service	ApplyFunding	Settlement application when WALLET_GRPC_APPLY_FUNDING_ENABLED=true. Token-bucket rate-limited (200 RPS default).
Position Service	ListPositionsBySymbol(symbol, as_of_unix_ms, …)	Phase 2 snapshot — deterministic positions at the exact cycle boundary.
Metadata Service	ListAllInstruments	Drives the scheduler’s symbol set. 5-minute in-memory cache. Bootstrap call at startup (30s timeout).

Funding Service exposes no gRPC server of its own.

REST API

Public (authenticated)

Method	Endpoint	Description
GET	/api/v1/funding/history?limit=&offset=&symbol=	User’s settlement history
GET	/api/v1/funding/summary?since=<RFC3339>	Per-symbol totals: total_funding, total_paid, total_received

Admin / Internal

Method	Endpoint	Description
POST	/internal/funding/trigger	Force-create a SCHEDULED cycle (body: symbol, funding_rate, mark_price, index_price)
GET	/internal/funding/cycles?limit=&offset=&symbol=	List cycles with aggregated stats
GET	/internal/funding/cycles/:id	Cycle detail
GET	/internal/funding/cycles/:id/settlements	Cycle settlements (alias /payments)
POST	/internal/funding/cycles/:id/seal	Force-seal a NEEDS_REVIEW cycle (requires acknowledgement payload)
GET	/internal/funding/dead-letters?...	List DEAD_LETTER rows
POST	/internal/funding/settlements/:id/retry	Reset a DEAD_LETTER row to PENDING
POST	/internal/funding/settlements/:id/cancel	Move DEAD_LETTER → CANCELLED (terminal)
POST	/internal/funding/dead-letters/retry-batch	Bulk retry matching DLQ rows
GET	/internal/funding/stats	Lightweight stats

Health

• GET /health, /health/live, /health/ready — aggregated checks: postgres, kafka_consumer, kafka_producer, wallet_grpc, metadata_grpc, position_grpc, scheduler_leader.

Data Models (PostgreSQL / SQLC)

funding_cycles — one row per (symbol, cycle_timestamp)

Field	Type	Description
id	UUID PK	Cycle ID
symbol	VARCHAR(32)	Perpetual symbol
cycle_timestamp	TIMESTAMPTZ	Boundary (used as as_of to Position Service)
funding_interval_hours	INT	Typically 8
funding_rate, premium_rate_twap, interest_rate	NUMERIC(20,12)	Frozen at cycle creation
mark_price, index_price	NUMERIC(30,8)	Frozen at cycle creation
position_snapshot_taken_at	TIMESTAMPTZ	When Phase 2 fetched positions
status	TEXT	SCHEDULED / IN_PROGRESS / SEALED / NEEDS_REVIEW
total_settlements, terminal_settlements	INT	Sealer counts
total_paid, total_received	NUMERIC(30,8)	Sealer aggregates (zero-sum check)
trace_id, created_at, updated_at	—	—

Indexes:

• idx_funding_cycles_symbol_timestamp (UNIQUE) — Phase 1 idempotency
• idx_funding_cycles_status_partial WHERE status IN ('SCHEDULED','IN_PROGRESS','NEEDS_REVIEW') — scheduler/sealer scans
• idx_funding_cycles_created_at — admin dashboards

funding_settlements — one row per (cycle, user, symbol), reusable work queue

Field	Type	Description
id	UUID PK	Settlement ID
cycle_id	UUID (FK)	→ funding_cycles.id
user_id	UUID	Position owner
symbol	VARCHAR(32)
position_side	TEXT	LONG / SHORT
position_size	NUMERIC(30,8)	Snapshot at cycle_timestamp
funding_amount	NUMERIC(30,8)	Signed (negative = user pays)
wallet_idempotency_key	TEXT	funding:{cycle_ts_unix}:{user_id}:{symbol}
status	TEXT	See state machine
attempt_count, last_attempt_at, next_attempt_at	—	Retry state
wallet_balance_version	BIGINT	From wallet response
kafka_published_at	TIMESTAMPTZ	Set on APPLIED_PUBLISHED
trace_id, created_at	—	—

Critical indexes:

• idx_funding_settlements_workqueue on (next_attempt_at, attempt_count, status) WHERE status IN ('PENDING','APPLIED') — keeps the SKIP LOCKED claim query off the heap.
• idx_funding_settlements_dlq partial on WHERE status = 'DEAD_LETTER'.
• Per-user/cycle/symbol indexes for history and admin queries.

Cutover Notes (Wallet vs. Position-Settlement Path)

Phase	WALLET_GRPC_APPLY_FUNDING_ENABLED	position.funding.v1 published	Who debits/credits the wallet
Pre-cutover	true	optional	Funding Service via WalletService.ApplyFunding
Phase 4+ (current default)	false	true	Position Service consumes position.funding.v1, applies the funding delta to its ledger, emits settlement.v1 → Wallet Service applies it

Idempotency is preserved across modes — the same (cycle, user, symbol) triple cannot double-credit.

Observability

Prometheus Metrics

• funding_cycle_seal_seconds — sealer duration
• funding_zero_sum_violation_total{symbol} — sealer mismatches
• funding_dlq_settlements_total{symbol} — current DLQ size
• funding_wallet_apply_funding_duration_seconds — gRPC latency
• funding_settlement_state_transitions_total{from,to} — worker activity
• funding_settlement_attempt_count_bucket — retry distribution

Tracing

OpenTelemetry: funding.scheduler.phase1, funding.scheduler.phase2, funding.worker.apply, funding.worker.publish, funding.sealer.run. trace_id flows from cycle creation through every settlement → wallet call → Kafka header.

Logging

Structured JSON via zap: cycle_id, settlement_id, symbol, user_id, attempt_count, funding_amount, trace_id.

Configuration

Variable	Description	Default
POSTGRES_URL	PostgreSQL connection	required
KAFKA_BROKERS	Brokers	required
KAFKA_SCHEMA_REGISTRY_URL	Schema Registry	required
WALLET_SERVICE_GRPC_URL	Wallet gRPC endpoint	required
POSITION_SERVICE_GRPC_URL	Position gRPC endpoint	required
METADATA_SERVICE_GRPC_URL	Metadata gRPC endpoint	required
WALLET_GRPC_APPLY_FUNDING_ENABLED	Toggle wallet gRPC vs position-settlement path	false
FUNDING_INTERVAL_HOURS	Cycle length	8
SCHEDULER_TICK_INTERVAL	Scheduler poll	5s
SNAPSHOT_GRACE_PERIOD	Wait after boundary before snapshot	30s
WORKER_CONCURRENCY	Worker pool size per pod	8
WORKER_BATCH_SIZE	Claim batch	16
WORKER_MAX_ATTEMPTS	Retries before DLQ	5
BACKOFF_BASE / BACKOFF_MAX	Retry backoff	2s / 5m
WALLET_RATE_LIMIT_RPS	Token-bucket cap on ApplyFunding	200
ZERO_SUM_TOLERANCE_USDT	Sealer zero-sum check tolerance (absolute USDT). Cycles where `	total_paid − total_received

Technology Stack

• Language: Go 1.21+
• Storage: PostgreSQL, SQLC, Atlas migrations
• Messaging: confluent-kafka-go with Avro
• HTTP: Fiber
• gRPC: google.golang.org/grpc
• Metrics & Tracing: Prometheus client_golang, OpenTelemetry

Related Documentation

• Position Service — provides as_of_unix_ms snapshots and is the steady-state owner of funding settlement publication
• Wallet Service
• Market Data Service — publishes md.funding.v1
• Kafka Events