Skip to content

Concepts

The vocabulary used across the platform. Each section is a survey — the formal definition lives in the OpenSpec.

gRPC (mirrored invocation)

gRPC is the inter-service wire protocol. Every backend service exposes a gRPC interface; the API gateway discovers that interface via gRPC reflection and proxies calls to it. Clients do not need the backend's .proto files at compile time — they call by service and method name through the gateway.

  • Why Virtufin uses it: one wire protocol for all backends, one client library per language, streaming for free, native integration with the reflection-based gateway.
  • See Cross-cutting spec for the platform-wide rules.

Dapr

Dapr is the sidecar that Virtufin uses for pub/sub, state, actors, and service-to-service invocation. The Virtufin services do not talk to Redis directly — they talk to Dapr (pub/sub and state through the API gateway), and Dapr talks to the backend. This is why the API gateway can swap brokers without changes to the calling code.

  • Why Virtufin uses it: portable infra (Dapr runs on Kubernetes and on a laptop via dapr run), one API for pub/sub, state, actors and service invocation, and no bespoke SDK per language.
  • See Cross-cutting spec.

Ownership (Dapr actors)

In a multi-instance deployment, an RPC that targets a specific resource (a worker, and so a WebSocket connection, which is a WebSocketFeed worker) must reach the instance that holds it. Virtufin gives each worker a Dapr actor in the WorkManager Runtime, and a started worker holds a lease (the gateway's publication fence), so a moved worker's old activation stops publishing. Dapr's placement service decides which pod hosts an actor and routes every call to it, and moves the actor to another pod when its host goes away.

  • Why Virtufin uses it: no hand-rolled peer discovery or forwarding; Dapr owns placement, routing and failover.
  • See worker-management spec.

Pub/Sub and the CloudEvents envelope

All inter-service events are CloudEvents 1.0 envelopes. Producers publish via Pubsub.PublishEvent on the API gateway; consumers subscribe via Pubsub.Subscribe (server-streaming). The gateway is the only service that talks to Dapr pub/sub directly.

On top of the CloudEvents base attributes, Virtufin events carry extensions such as correlationid, scenarioid, clocktype, eventtime, and the world and universe legs (marketworld, marketuniverse, …). The pubsub-topics spec lists them all, with which events must carry which, and defines the topic naming scheme.

Worlds & scenarios (event routing)

Two orthogonal axes split the event stream into addressable topics:

  • Market data world — act.exchange.<venue>.<entity>.<event>[.<id>] for live data, hyp.<name>.exchange.<venue>.<entity>.<event>[.<id>] for hypothetical data such as a backtest (<name> identifies which hypothetical world, e.g. hyp.BACKTEST).
  • Scenario world — sc.<scenarioId>.<domain>.<entity>.<event>[.<id>] for strategy / position / P&L / risk events that depend on a triplet of (market data, portfolio, strategy) state.

The reserved scenario id LIVE is the production scenario and is auto-seeded at service start with the {act, act, act} triplet. A backtest or paper run registers its own scenario id, with hypothetical legs where it needs them.

  • Why Virtufin uses it: the same producer code can target production and backtest by changing one address. The data universe is not part of the topic: it travels in the marketuniverse envelope extension.
  • See pubsub-topics spec and scenarios spec.

State (distributed KV)

State is a flat key-value store, partitioned by service name. The State.* RPCs on the API gateway are the only interface; each service sees its own namespace (State.SaveState(service: "workmanager", ...)). The gateway handles ETag-based conflict detection and forwards to Dapr's state API.

  • Why Virtufin uses it: services do not own storage. The gateway's state is a generic KV; the Dapr sidecar talks to whatever backing store is configured (Redis, Cosmos, Postgres, in-memory for tests).
  • See Cross-cutting spec.

Workers

A worker is a piece of code that subscribes to events (or accepts commands) and produces events back. The WorkManager is the worker catalog and router; the WorkManager Runtime hosts each worker in a Dapr actor, loads its code through a pluggable engine and dispatches incoming events to it. The wire protocol is fixed by the worker-management spec.

  • Why Virtufin uses it: strategy code is a unit of deployment, not a service. A new strategy does not require a new gRPC service, a new proto, or a new deploy. It is a file in a package that the WorkManager loads on CreateWorker.
  • See the Worker DevKit for the writing side.

Engines

An engine is the loader. The WorkManager Runtime supports four:

MIME type Engine Process model
application/x-dotnet-dll DotNetDllEngine inside the Runtime process (embedded CoreCLR via hostfxr)
application/x-native-dll NativeDllEngine inside the Runtime process (a shared library implementing the native C ABI)
text/x-python, application/x-python PythonEngine python3 child process
text/x-csharp CSharpSourceEngine child process (Roslyn compile in CSharpSource.Bridge)
  • Why Virtufin uses it: the worker language is a choice for the author, not a constraint of the platform. Any language that can build a shared library against the C ABI (C, C++, Rust) runs on the native engine.
  • See the engines spec.

NativeAOT

The .NET services (api, workmanager) are compiled to native binaries with NativeAOT. No JIT, no runtime install, no warm-up.

  • Why Virtufin uses it: consistent resource profile with the Python and native workers, sub-100ms cold start for autoscaling, fewer moving parts in production.
  • See Cross-cutting spec.