Infrastructure for Modern Web Sites « random($foo)
http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites

leonard lin's weblog. topical geek news.

list of backend and frontend requirements

Who'd have thought it.. Google runs of thousands of 12-volt batteries :o) Each Google server has its own 12-volt battery to supply power if there's a problem with the main source of electricity. The company also revealed for the first time that since 2005, its data centers have been composed of standard shipping containers--each with 1,160 servers and a power consumption that can reach 250 kilowatts. ... Most people buy computers one at a time, but Google thinks on a very different scale. The cores of the company's data centers are composed of standard 1AAA shipping containers packed with 1,160 servers each, with many containers in each data center.

Google's big surprise: each server has its own 12-volt battery to supply power if there's a problem with the main source of electricity. The company also revealed for the first time that since 2005, its data centers have been composed of standard shipping containers--each with 1,160 servers and a power consumption that can reach 250 kilowatts.

Google muestra por primera vez sus servidores, de diseño propio.

I love the approach and the the fact it's unconventional and better - google servers revealed

good content - horrible design makes me not want to read it

by Bruce Perens

googleのサーバーは一台ごとにバッテリーがついていて、1160台１セットでコンテナに収まっている。

Unresolvable

Googleサーバで非常に驚くのは、サーバ1台1台が、それぞれ12Vのバッテリを備えていて、メイン電源に問題がある場合には電力を供給することだ。Googleはまた、2005年以来、同社のデータセンターが標準規格の運送用コンテナで構成されていることを初めて明らかにした。1つのコンテナには1160台のサーバが搭載され、その電力消費は250KWに達する。

To summarize: build the smart grid on top of the Internet, and put wireless mesh routers in power meters and cars.

So obvious once heard; everything can be reduced to software eventually.

The intelligent network we need for electricity can also turn cars into nodes. Interoperability is a multiplier.

how it works

""There is no time for celebration when we fix a cable," Rennie says. "There is lots of pressure from cable owners to move quickly. They are losing revenue."" そんなに頻繁に切れてるのかぁ… (いや、確かにそうだけど)

"For the past five years, John Rennie has braved the towering waves of the North Atlantic Ocean to keep your e-mail coming to you. As chief submersible engineer aboard the Wave Sentinel, part of the fleet operated by U.K.-based undersea installation and maintenance firm Global Marine Systems, Rennie--a congenial, 6'4", 57-year-old Scotsman--patrols the seas, dispatching a remotely operated submarine deep below the surface to repair undersea cables."

The Beast

Como ufuncionan los cables submarinos por internet

Pull up the wrong undersea cable, and the Internet goes dark in Berlin or Dubai. See our animated infographics of how the web works!

to be read

Stack Overflow Architecture | High Scalability

Stack Overflow is a much loved programmer question and answer site written by two guys nobody has ever heard of before. Well, not exactly. The site was created by top programmer and blog stars Jeff Atwood and Joel Spolsky. In that sense Stack Overflow is like a celebrity owned restaurant, only it should be around for a while. Joel estimates 1/3 of all the programmers in the world have used the site so they must be serving up something good.

それは「月を追いかける（follow the moon）データセンター」というコンセプトです。 夜間は外気温も低く、また電気料金も安くなっています。そこで、世界中のデータセンターのうち、夜になっている地域のデータセンターだけを稼働させれば、低い外気温を活用でき、しかも夜間の安い電気料金を利用できます。これはクラウド技術者のあいだで議論されている構想ですが、グローバルにデータセンターを展開し、その負荷をダイナミックに切り替えられるグーグルであればそれを実現可能かもしれない、とこの記事「Google's Chiller-less Data Center 」では解説されています。

[cloud]

Casey: We’ve got 430,000 registered users, in a month we’ll see 200,000 of those, about 135,000 in a week and about 70,000 in a day. We peak at 3.6 million pageviews per day. That’s registered users only (doesn’t include the very few pages that are Google accessible) and does not include the usual API calls, RSS feeds, AJAX. Actual requests that hit Rails per day is 10 million. 900 new users sign up per day. The forums are very active with about 50,000 new posts being written each day. Some various numbers — 2.3 million knitting/crochet projects, 19 million forum posts, 13 million private messages, 8 million photos (the majority are hosted by Flickr).

Some various numbers — 2.3 million knitting/crochet projects, 19 million forum posts, 13 million private messages, 8 million photos (the majority are hosted by Flickr).

YouTube may pay less to be online than you do, a new report on internet connectivity suggests, calling into question a recent analysis arguing Google’s popular video service is bleeding money and demonstrating how the internet has continued to morph to fit user’s behavior.

Youtube's bandwidth bill may be almost zero because it connects directly with the ISPs through it's Dark Fiber.

"YouTube’s low or nonexistent bandwidth bill points to a very important shift in the structure of the internet...top 30 websites serving up 30 percent of net traffic, either from their own sets of pipes or from data centers around the world that connect much closer to your computer — and for much cheaper — than ever before."

YouTube may pay less to be online than you do, a new report on internet connectivity suggests, calling into question a recent analysis arguing Google’s popular video service is bleeding money and demonstrating how the internet has continued to morph to fit user’s behavior. In fact, with YouTube’s help, Google is now responsible for at least 6 percent of the internet’s traffic, and likely more — and may not be paying an ISP at all to serve up all that content and attached ads. Credit Suisse made headlines this summer when it estimated that YouTube was binging on bandwidth, losing Google a half a billion dollars in 2009 as it streams 75 billion videos. But a new report from Arbor Networks suggests that Google’s traffic is approaching 10 percent of the net’s traffic, and that it’s got so much fiber optic cable, it is simply trading traffic, with no payment involved, with the net’s largest ISPs.

"But a new report from Arbor Networks suggests that Google’s traffic is approaching 10 percent of the net’s traffic, and that it’s got so much fiber optic cable, it is simply trading traffic, with no payment involved, with the net’s largest ISPs"

Google Fellow Jeff Dean gave an excellent keynote talk at the recent WSDM 2009 conference that had tidbits on Google I had not heard before. Particularly impressive is Google's attention to detail on performance and their agility in deployment over the last decade.

how many data centers does Google use, and where are they

Salesforce - incredible efficiency! and other metrics

all of Salesforce.com runs on only about 1,000 servers. And that is mirrored, so it is really only 500

The goal of OpenStack is to allow any organization to create and offer cloud computing capabilities using open source software running on standard hardware. OpenStack Compute is software for automatically creating and managing large groups of virtual private servers. OpenStack Storage is software for creating redundant, scalable object storage using clusters of commodity servers to store terabytes or even petabytes of data.

via: http://news.ycombinator.com/item?id=1527572 What the software does: The goal of OpenStack is to allow any organization to create and offer cloud computing capabilities using open source software running on standard hardware. OpenStack Compute is software for automatically creating and managing large groups of virtual private servers. OpenStack Storage is software for creating redundant, scalable object storage using clusters of commodity servers to store terabytes or even petabytes of data.

The OpenStack Open Source Cloud Mission: to produce the ubiquitous Open Source Cloud Computing platform that will meet the needs of public and private cloud providers regardless of size, by being simple to implement and massively scalable.

The goal of OpenStack is to allow any organization to create and offer cloud computing capabilities using open source software running on standard hardware. OpenStack Compute is software for automatically creating and managing large groups of virtual private servers. OpenStack Storage is software for creating redundant, scalable object storage using clusters of commodity servers to store terabytes or even petabytes of data.

Open source hosting and storage software. database ?