Weekly Byte: Colocation and Risk Management
I joined pMD when it was a charge capture company based out of New York City. During my interviews, the team revealed that in a few months time, the whole company was relocating to San Francisco. I'm originally from southern California and knew I wanted to move back, so this moved up my time table nicely. At the time, not only was our headquarters and the majority of our customers on the east coast, but our primary data center was nearby as well, in Chicago. The move to San Francisco, other than eliminating my winter wardrobe, sparked some discussions and a desire to geographically distribute our production data center.
Over the last few years, our customer base has grown by a few orders of magnitude and our customers are more uniformly distributed across the country. Network latency due to geography hasn't really been an issue as we've tried to keep our backend code snappy enough to absorb the occasional bottlenecks, in addition to offloading non-sensitive assets to content distribution networks. Having a second live production data center certainly doesn't hurt network latency, but a better argument was needed. For us, it comes down to risk management. We already have several layers of redundancy in hardware and software in addition to off-site data backups, but having redundancy across colocations centers and their respective organizations is a new level of risk diversification that is very attractive. Having two centers serving live requests is something we felt would help us and our customers sleep easier. As a result, early this year we’ve begun the build out of a new San Francisco data center which we hope to complete in the coming weeks.
Risk reduction, however, is only half the story. As developers, the ROI of trips to distant data centers also reduces the frequency of experimenting with newer hardware setups and technologies. Having a nearby site where we can try new things and in essence take greater risks without the fear of negatively affecting our customers is also a huge added bonus for our team culture. In the larger picture, reducing our tolerance for customer facing risk while increasing the internal appetite for risk and innovation is a win-win formula we expect will pay dividends in the years to come.
Over the last few years, our customer base has grown by a few orders of magnitude and our customers are more uniformly distributed across the country. Network latency due to geography hasn't really been an issue as we've tried to keep our backend code snappy enough to absorb the occasional bottlenecks, in addition to offloading non-sensitive assets to content distribution networks. Having a second live production data center certainly doesn't hurt network latency, but a better argument was needed. For us, it comes down to risk management. We already have several layers of redundancy in hardware and software in addition to off-site data backups, but having redundancy across colocations centers and their respective organizations is a new level of risk diversification that is very attractive. Having two centers serving live requests is something we felt would help us and our customers sleep easier. As a result, early this year we’ve begun the build out of a new San Francisco data center which we hope to complete in the coming weeks.
Risk reduction, however, is only half the story. As developers, the ROI of trips to distant data centers also reduces the frequency of experimenting with newer hardware setups and technologies. Having a nearby site where we can try new things and in essence take greater risks without the fear of negatively affecting our customers is also a huge added bonus for our team culture. In the larger picture, reducing our tolerance for customer facing risk while increasing the internal appetite for risk and innovation is a win-win formula we expect will pay dividends in the years to come.
