We’re back!
by Yori Nelken on January 25, 2010Hi Folks! As you may have seen we had some network issues this morning. By 12:30pm Pacific Time all was clear and all users were up and running. Apologies to anyone who experienced an inconvenience here – we had our support team working hard to correct the issue quickly.
Some more details – at about 2am Pacific Time we pushed a new version to production. A version with some cool new features, but more on that later. by 9am we were pretty much done with production tests – all was clear and running smoothly. At 9:30 we started noticing intermittent errors for some users. Our support team was alerted right away, and was on the case less than a minute after the problems started.
It took us about two hours to recover completely and get everybody back up and running. During this time some users could work, and some were running into system errors. We are now analyzing all logs and activities that led to the outage, and also analyzing our own response to it. We will learn not only how to minimize the chances of such an incident in the future, but also how to optimize our own response to system errors and outages in general. This is now work in progress.
While this sort of thing can happen when you’re pushing out new features and upgrading your product at the clip that we do, this morning’s outage is not something we accept, or take lighty in any way.
Again, so sorry if you were inconvenienced and we greatly appreciate your patience. For a limited time, we enabled Live Chat support (a premium feature) for all our users. Contact us or email support@timebridge.com if you run into any issue.
Yori and the TimeBridge team.
Categories: General