Most software developers have built a REST API or a database-backed service. Far fewer have built the core of a marketplace: the order matching engine (OME).
As a systems programming exercise, I built a simplified betting exchange in Rust. The project explores order matching, event-driven architectures, and asynchronous processing pipelines. At its core is an in-memory matching engine that emits trade events into Redis Streams, where independent Python services consume them for analytics and persistence.
This article provides a high-level overview of the architecture. Later posts will dive into the matching engine, order book design, and event processing pipeline.
The following flowchart shows how the data flows within the system:
flowchart TD IO[Incoming Orders] LB["Load Balancers (Optional)"] IA[Ingestion API] Engine[Rust Order Matching Engine] MQ[Redis Streams] DS[Asynchronous Python Services] A[Analytics] S[Storage] DB[Database] WWW[Clients: Web, Mobile, etc.] IO -.-> |HTTPS / WebSockets / RPC| LB LB -.-> |HTTP / HTTPS / Unix sockets| IA IA -.-> Engine Engine --> |Asynchronous channel communication| MQ MQ --> |Event Streaming| DS A -.-> WWW DS --> A DS --> S S --> DB style IO fill:#F28C28,stroke:#FFFFFF,stroke-dasharray: 5 5,color:#000000 style LB fill:#F28C28,stroke:#FFFFFF,stroke-dasharray: 5 5,color:#000000 style IA fill:#F28C28,stroke:#FFFFFF,stroke-dasharray: 5 5,color:#000000 style WWW fill:#F28C28,stroke:#FFFFFF,stroke-dasharray: 5 5,color:#000000
Note: the orange rectangles are out of scope for this project.
The system is divided into two execution paths:
- Hot path: order matching and trade generation
- Cold path: analytics, persistence, and downstream processing
The Rust matching engine generates trade events which are published to Redis Streams via a background worker. Downstream services consume these events independently and process them according to their own requirements.
Design decisions#
Pros#
The architecture is built around a separation between the hot path and the cold path.
Hot path:
- In-memory Rust matching engine
- Deterministic order processing
- No external I/O
- Trade generation
Cold path:
- Redis Streams
- Analytics services
- Persistence services
- Reporting and dashboards
Benefits:
- Low-latency matching
- Failure isolation
- Independent service evolution
- Replayable event stream
Although simplified compared to production exchange systems, this architecture captures the essential idea of modern event-driven design: keep the critical decision-making path minimal and deterministic, and move everything else into asynchronous, replayable processing pipelines.
Cons#
This architecture introduces several trade-offs:
- Eventual consistency between trade execution and downstream consumers
- Dependence on Redis as the event backbone
- No distributed matching or order book replication
- No explicit latency or throughput guarantees
These trade-offs were intentional and allow the project to focus on event-driven architecture rather than production-scale exchange infrastructure.
Tech. stack#
| Component | Technology | Reason |
|---|---|---|
| Matching Engine | Rust | Performance, memory safety, deterministic execution |
| Event Backbone | Redis Streams | Replayability, consumer groups, operational simplicity |
| Downstream Services | Python | Fast development and strong analytics ecosystem |
Together, these technologies form a layered architecture: a high-performance deterministic core in Rust, an event distribution layer via Redis Streams, and flexible consumer services in Python for secondary processing tasks. This separation allows each layer to optimise for different constraints without compromising the others.
This architecture intentionally separates order matching from downstream processing through an event-driven pipeline. The result is a small but representative example of how modern trading systems isolate latency-sensitive execution from analytics and persistence concerns.
In the next article, we’ll dive into the matching engine itself and examine how orders are matched and trade events are generated.
NOTE: as we progress through this series, if confused, it helps to return to the diagram above, to keep track of what layer is being discussed.
Feel free to share the article on socials: