Dear Customers,
Some of you experienced slowness at the start of the market today when using our mobile app. You experienced that rate refresh was not at the usual speed. Rate refresh is an important function and is used to render charts, trigger price alerts, place orders, and many more capabilities.
To explain the cause I will have to talk a little about how we are spread across multiple servers to service you from the closest and best locations.
As part of de-risking, a portion of our rate refresh solution is hosted on the Microsoft cloud. Especially the one used by mobile apps to display watchlists. This solution uses cloud capability to grow with increased usage, even during trading hours. Unfortunately, this service refused to grow today. This was when you started noticing the slowness in the rate refresh.
This triggered our fail-safe mechanism and some of our core trading components started stepping in to serve these failed requests. The increased pressure on the core system caused order update messages to be delivered slowly to the customers. We immediately knew this was not a good situation to be in.
In parallel, our cloud team was still attempting to stand up fresh services for rate refresh. The customers on fast rate refresh and website started noticing the impact much later. While their rate refresh was not impacted, they started noticing order update delays.
Unfortunately, it took us some time to route the traffic away from our core systems to an isolated component that served the rate refresh to the mobile app. By 10:30 am you started noticing that the mobile app was not sluggish anymore and rates started to be close to real-time.
We will revisit all our services and fail-safe mechanisms to ensure that our technology is highly available even in the times of duress, capable enough to sustain the traffic volatility, and do not interfere with each other.
Regards
R. Balaji
CTO
5paisa Capital Ltd.