Scaling Digg and Other Web Applications | High Scalability
http://highscalability.com/scaling-digg-and-other-web-applications

Joe Stump, Lead Architect at Digg, gave this presentation at the Web 2.0 Expo. I couldn't find the actual presentation, but fortunately Kris Jordan took some great notes. That's how key moments in history are accidentally captured forever. Joe was also kind enough to respond to my email questions with a phone call.

Scaling Strategies

InfoQ: Rockstar Memcaching http://www.infoq.com/presentations/lutke-rockstar-memcaching performance presentation

About the conference RubyFringe is an avant-garde conference for developers that are excited about emerging Ruby projects and technologies. They're mounting a unique and eccentric gathering of the people and projects that are driving things forward in our community.

Memcached tutorial video from the RubyFringe conference.

Video of memcached best-practices

RubyFringe presentation on Memcached

tobias lutke rubyfringe talk about memcached

CollegeHumor

CollegeHumor memcache use

A software-based distributed caching system such as memcached is an important piece of today's largest Internet sites that support millions of concurrent users and deliver user-friendly response times. The distributed nature of memcached design transforms 1000s of servers into one large caching pool with gigabytes of memory per node. This blog entry explores single-instance memcached scalability for a few usage patterns.

"A software-based distributed caching system such as memcached is an important piece of today's largest Internet sites that support millions of concurrent users and deliver user-friendly response times. The distributed nature of memcached design transforms 1000s of servers into one large caching pool with gigabytes of memory per node. This blog entry explores single-instance memcached scalability for a few usage patterns."

I wanted a simple command-line tool to be able to grab real-time stats from memcache (memcached, I know, I know), and output it in a view something like top. I couldn't find anything like it, so I wrote one myself in perl. When writing it, I tried to keep it simple, portable, and lightweight. (No memcached perl modules required! I tried to keep it to modules I thought would be preinstalled on almost any modern system. It's also fairly polite - non-critical modules get checked, and if they aren't installed, the functionality is disabled without spewing errors or dying.) I realize it's not written well. But, hey, at least it exists, right? Until the day I released it, there wasn't any comparable tool like it for memcached. It gives you the basic stats, and not too much else. (You can specify thresholds, for instance, and it'll change color to red if you exceed the thresholds. You can also choose the refresh/ sleep time, and whether to show immediate (per second) stats, or lifetime stats.

Note about Code-State-Cache-Data (CSCD) pattern in scalable web applications.

Short Article propounding the use of a "Code-State-Cache-Data-Architecture" (CSCD) instead of just CD or CCD applications. Basically saying that you should forget about stateful apps if you wan't maximum performance...

Application state - Data you can restore from the database or afford to lose if server is restarted (logged in users). He recommends storing this in-memory. "Application state goes into an in-memory key-value store like Tokyo Tyrant. Cache data goes into Memcached. Persistent data goes into a database"

"What he needs is the insight to identify state, cached data and persistent data in his application. Application state goes into an in-memory key-value store like Tokyo Tyrant. Cache data goes into Memcached. Persistent data goes into a database. Note that the seperation of code and application state may be beneficial later, because it allows you to scale easily by adding new memory servers. ... Let's call this the Code-State-Cache-Data (CSCD) pattern. What Damian originally had was a Code-Data (CD) pattern, and later he optimized to get a Code-Cache-Data (CCD) pattern"