I attended GigaOm’s Structure 2009 conference today. It was a very well run conference; like the Swiss train system, all events started on time.
There there three key themes at this year’s conference:
- Cloud is ambiguous term – let’s define it
- Bandwidth is now the constraint, not compute
- Horizontal scalability for databases is essential for an increasing number of companies
Defining the cloud
Paul Sagan, cousin of Carl Sagan and CEO of Akamai, defined cloud services very eloquently. Cloud services provide elastic, on-demand, web accessed, metered, multi-tenant systems governed by reporting and security. They come in three flavors: Infrastructure-as-a-Service (EC2), Platform-as-a-Service (App Engine), Software-as-a-Service (Zuora). It’s a clean and succinct definition.
Bandwidth is the constraint
With new mulitcore architectures and the cost reduction of memory, data centers have become powerful to a point that the network and the bus are the limiting factors.
What’s the implication? Minimize the use of the network by intelligent buffering, caching and pre-fetching in addition to compression. Technologies like memcached, developed by LiveJournal, offer this type of functionality.
Horizontal scalability is essential
Technologies like Hadoop, CouchDB, HyperTable among others allow data stores to grow simply by adding machines. These non-relational data stores don’t yet have the properties of relational databases such as the ability to run joins in a SQL like syntax, but bring advantages in reducing the cost and complexity of scaling.
Many web companies like Google, Facebook, Yahoo, Baidu and Gilt Group (a fashion retailer) are turning to these technologies to handle the huge amounts of data they’re generating. The real need for these storage systems comes from computer generated data: log files, transaction events, user profiling and the analysis that must be run on top of this data.
Expect to see many more companies moving to horizontal data stores in the near future.
Cloud computing seems to be taking the enterprise world by storm with large corporations looking to understand the technology, develop strategies and develop systems faster than most previous waves of ERP, CRM and KM. Exciting stuff!