Every product eventually wants to show customers their own data — dashboards, analytics, insights. And almost every team’s first instinct is to run those queries against the live database. Razorpay’s data highway exists because that instinct quietly kills your payments.
Razorpay's microservices emit roughly half a billion events a day, and merchants want real-time analytics built from them. The naive path — point dashboard queries at the transactional database — means heavy analytical reads competing with the payment writes that must stay fast. Two workloads with opposite needs, fighting over one database.
A database tuned for transactions (recording payments quickly and correctly) is bad at analytics (scanning millions of rows to compute trends), and vice-versa. They want different layouts, indexes, and hardware. Force both onto one database and each makes the other slow — your dashboards drag, and worse, your payments drag.
The fix in spirit: stop making one store do two jobs. Continuously copy the data into a second store shaped for reading and analytics, and serve all the dashboards from there. The transactional database goes back to doing only what it's good at.
A highway, not a query
Razorpay built a streaming pipeline — their “real-time data highway.” Events flow via windowed reads and PCollection-style transforms, get enriched and classified in the stream, and land in Elasticsearch/Kibana, a store optimized for fast analytical reads. Crucially, they wrapped it in an in-house SQL-to-Elasticsearch API framework so merchants (and internal teams) query analytics in familiar SQL — which executes against the search store, never against the live payments database.
This is the CQRS idea in practice — Command Query Responsibility Segregation: separate the write model from the read model. Writes hit the transactional store; a stream propagates changes into one or more read-optimized stores; queries hit those. The SQL-to-Elasticsearch layer is the thoughtful touch — it preserves a familiar query interface while completely changing what executes underneath, so consumers don't pay a learning tax for the architecture.
Worth knowing
The general principle outlives the specific tools (Elasticsearch, Kibana, the in-house framework): when one datastore is asked to serve both fast transactional writes and heavy analytical reads, separate the read path early. Streaming changes into a purpose-built read store is how you give customers fresh analytics without ever letting a dashboard query threaten a payment.
The gap it reveals
Many engineers will happily bolt analytics onto the primary database — it's the path of least resistance — and only discover the workload clash when dashboards start slowing down checkout. The senior realization is that transactional and analytical workloads are fundamentally incompatible on one store, and that the answer is a separate, stream-fed read model (CQRS). Seeing that before it bites is the gap.
In the interview room
When a prompt adds “…and merchants can view analytics,” the trap is to query the main DB. The strong move: “I'd stream events into a read-optimized store and serve analytics from there, so analytical load never competes with transactional writes.” Naming CQRS and the workload incompatibility shows you anticipate the second-order problem instead of discovering it in production.
The reframe
The convenient architecture and the correct one diverge the moment a single database is asked to serve two masters. Razorpay's data highway is really a refusal to let the easy path — “just query the prod DB” — endanger the one workload that can never be slow: taking people's payments. Building a separate road for analytics is more work upfront and far less work the day a dashboard goes viral.
Don't make your payments database moonlight as an analytics engine.
Primary source →
razorpay.com — Data Engineering at Scale: Building a Real-Time Data Highway