It wasn’t the leap year, a coding blip, or a hack that caused Robinhood’s massive outages yesterday and today that left customers unable to trade stocks. Instead, the co-CEOs write that “the cause of the outage was stress on our infrastructure — which struggled with unprecedented load. That in turn led to a “thundering herd” effect — triggering a failure of our DNS system.”
Robinhood was offline from Monday at 6:30am Pacific to 11pm Pacific, then had another outage this morning from 6:30am Pacific until just before 9am Pacific.
The $912 million-funded fintech giant will provide compensation to all customers of its Robinhood Gold premium subscription for borrowing money to trade plus access to Morningstar research reports, Nasdaq data, and bigger instant deposits. It’s offering them three months of service.
A month of Robinhood Gold costs $5 plus 5% yearly interest on borrowing above $1,000, charged daily. Before a pricing change, the flat fee per month could range as high as $200. However, compensated users will only get the $5 off per month, for a total of $15. That could seem woefully insufficient if Robinhood users missed out on buying back into stocks like Apple that went up over 9% on Monday. Robinhood is calling it a “first step”.
Impacted Robinhood users can contact the company here to ask for compensation. Below you can see the email Robinhood sent to custoemrs late last night.
Robinhood’s email to customers late last night
Robinhood is also working to contact impacted customers on a individual basis, and it’s looking into other forms of compensation on a case by case basis, company spokesperson Jack Randall tells me. It’s unclear if that might include cash to offset what traders might have lost by having their money locked in inaccessible Robinhood accounts during the outage.
Compensation could become a significant cost if the startup assesses that many of its 10 million users were impacted. The markets gained a record $1.1 trillion yesterday, but some Robinhood traders may not have been able to buy back in as the rebound occurred following mass selloffs due to fears of coronavirus.
Now the startup, valued at $7.6 billion, will have to try to regain users’ trust. “When it comes to your money, we know how important it is for you to have answers. The outages you have experienced over the last two days are not acceptable and we want to share an update on the current situation . . . We worked as quickly as possible to restore service, but it took us a while. Too long” wrote co-founders and co-CEO Baiju Bhatt and Vlad Tenev [disclosure: who I know from college].
As for exactly what triggered the downtime, the founders write that “Multiple factors contributed to the unprecedented load that ultimately led to the outages. The factors included, among others, highly volatile and historic market conditions; record volume; and record account sign-ups.” There’s been a frenzy of retail trading activity in the wake of coronavirus. There’s also been sudden spikes in stocks like Tesla amidst mainstream media attention.
Going forward, Robinhood promises to “work to improve the resilience of our infrastructure to meet the heightened load we have been experiencing. We’re simultaneously working to reduce the interdependencies in our overall infrastructure. We’re also investing in additional redundancies in our infrastructure.” However, they warn that “we may experience additional brief outages, but we’re now better positioned to more quickly resolve them.”
The outage comes at a vulnerable time for Robinhood as oldschool brokerages like Charles Schwab, Ameritrade, and Etrade all recently moved to eliminate per-trade fees to match Robinhood’s pioneering zero-comission trades. Though some of those brokerages experienced infrastructure troubles recently, Robinhood massive outages could push users towards those incumbents that they might perceive as more stable.
March 3, 2020 at 07:59PM
via https//www.brucedayne.com/
Josh Constine, Khareem Sudlow
