sccache - Google Code
http://code.google.com/p/sccache/

The SHOP.COM Cache System is an object cache system that...

Joe Stump, Lead Architect at Digg, gave this presentation at the Web 2.0 Expo. I couldn't find the actual presentation, but fortunately Kris Jordan took some great notes. That's how key moments in history are accidentally captured forever. Joe was also kind enough to respond to my email questions with a phone call.

Scaling Strategies

Rails Performance Resources - Expert advice on tuning and optimizing your Rails app.

A pro-caching technique for presenting cachable relative times (e.g., "15 minutes ago")

I put together a new mini app for our new status site yesterday that needed exactly this technique. I wanted the content of the application to be entirely page cached, so it would withstand the onslaught if the terrible should happen and we need to redirect all trafic to the status site.

Active record memory cache.

A Write-Through Cacheing Library for ActiveRecord

class Message < ActiveRecord::Base

CollegeHumor

CollegeHumor memcache use

Redis is a key-value database. It is similar to memcached but the dataset is not volatile, and values can be strings, exactly like in memcached, but also lists and sets with atomic operations to push/pop elements.

“Redis is a key-value database. It is similar to memcached but the dataset is not volatile, and values can be strings, exactly like in memcached, but also lists and sets with atomic operations to push/pop elements. “In order to be very fast but at the same time persistent the whole dataset is taken in memory and from time to time and/or when a number of changes to the dataset are performed it is written asynchronously on disk. You may lost the last few queries that is acceptable in many applications but it is as fast as an in memory DB (beta 6 of Redis includes initial support for master-slave replication in order to solve this problem by redundancy).”

A nice fast K/V data store, with some nice list/set features.

Interesting comparison of JSON and homebrewed control-char delimited data.

This widget downloads a list of all of your contacts, in JavaScript, in under 200ms (this is true even for members with 10,000+ contacts).

We are on the edge of two potent technological changes: Clouds and Memory Based Architectures. This evolution will rip open a chasm where new players can enter and prosper. Google is the master of disk. You can't beat them at a game they perfected. Disk based databases like SimpleDB and BigTable are complicated beasts, typical last gasp products of any aging technology before a change. The next era is the age of Memory and Cloud which will allow for new players to succeed. The tipping point is soon. Let's take a short trip down web architecture lane: # It's 1993: Yahoo runs on FreeBSD, Apache, Perl scripts and a SQL database # It's 1995: Scale-up the database. # It's 1998: LAMP # It's 1999: Stateless + Load Balanced + Database + SAN # It's 2001: In-memory data-grid. # It's 2003: Add a caching layer. # It's 2004: Add scale-out and partitioning. # It's 2005: Add asynchronous job scheduling and maybe a distributed file system. # It's 2007: Move it all into the cloud. # It's 2008: Cloud +

What makes Memory Based Architectures different from traditional architectures is that memory is the system of record. Also discussed Jim Starkey NimbusDB

Plus a nice comment from Zed.

Cachr

"Facebook will complete its roll-out of a new photo storage system designed to reduce the social network's reliance on expensive proprietary solutions from NetApp and Akamai."

画像はえー。

A method of using XHR requests to get chrome from the server rather than fetching it as the inital HTTP GET.

Django tip: Caching and two-phased template rendering

It's a clever solution because you end up defining what doesn't get cached instead of what does get cached.

RED (Resource Expert Droid) checks HTTP resources to see how they use HTTP, makes suggestions, and finds common protocol mistakes

a thin wrapper around Rack middleware intended for application-specific end points that need the extra speed

In the last post I wrote on caching in Hibernate in general as well as on the behavior of the session cache. In this post we will have a closer look at the QueryCache. I will not explain the query cache in details as there are very good articles like Hibernate: Truly Understanding the Second-Level and Query Caches.

s: we want choice. We believe that Apple has m

74.208.105.171 gs.apple.com

A user named Apathy in this thread on how Reddit scales some of their features, shares some advice he learned while working at Google and other major companies. To be fair, I [Apathy] was working at Google at the time, and every job I held between 1995 and 2005 involved at least one of the largest websites on the planet. I didn't come up with any of these ideas, just watched other smart people I worked with who knew what they were doing and found (or wrote) tools that did the same things. But the theme is always the same: # Cache everything you can and store the rest in some sort of database (not necessarily relational and not necessarily centralized). How do you go about applying this strategy?

ing caches is a clasisc strategy for milking your servers as much as possilbe. First look for an exact match. If that's not foun

W3 Total Cache

A software-based distributed caching system such as memcached is an important piece of today's largest Internet sites that support millions of concurrent users and deliver user-friendly response times. The distributed nature of memcached design transforms 1000s of servers into one large caching pool with gigabytes of memory per node. This blog entry explores single-instance memcached scalability for a few usage patterns.

"A software-based distributed caching system such as memcached is an important piece of today's largest Internet sites that support millions of concurrent users and deliver user-friendly response times. The distributed nature of memcached design transforms 1000s of servers into one large caching pool with gigabytes of memory per node. This blog entry explores single-instance memcached scalability for a few usage patterns."

Article summarizing presentation by Facebook on some of their scaling challenges and solutions.

Understanding Caching in Hibernate – Part Three : The Second Level Cache Performance, Scalability and Architecture – Java and .NET Application Performance Management (dynaTrace Blog)

In particular I read a whitepaper several years ago a

In the last posts I already covered the session cache as well as the query cache. In this post I will focus on the second-level cache. The Hibernate Documentation provides a good entry point reading on the second-level cache. The key characteristi

In this article, I’ll show you a few methods for controlling how your site’s files are cached by browsers so you can achieve the best of both worlds: maintaining optimal performance while ensuring that any updates are seen immediately, without a hitch by all of your users.

As

Alex Miller's technical blog on Java, concurrency, programming, design, languages, and more

Hibernate et la gestion du cache

Good overview of Django Caching Techniques

denormalization

5 Ways to Speed Up Your Rails App

Cómo jugar con los refrescos de caché para diferentes tipos de archivos en un proxy squid.

Will look into implementing this at the office some time...

refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern -i \.(gif|png|jpg|jpeg|ico)$ 10080 90% 43200 override-expire ignore-no-cache ignore-no-store ignore-private refresh_pattern -i \.(iso|avi|wav|mp3|mp4|mpeg|swf|flv|x-flv)$ 43200 90% 432000 override-expire ignore-no-cache ignore-no-store ignore-private refresh_pattern -i \.(deb|rpm|exe|zip|tar|tgz|ram|rar|bin|ppt|doc|tiff)$ 10080 90% 43200 override-expire ignore-no-cache ignore-no-store ignore-private refresh_pattern -i \.index.(html|htm)$ 0 40% 10080 refresh_pattern -i \.(html|htm|css|js)$ 1440 40% 40320 refresh_pattern . 0 40% 40320

Caching is easy to screw up. Usually it's a manual process which is error-prone and tedious. It's actually quite easy to cache, but knowing when to invalidate which caches becomes a lot harder. There is a subset of caching the caching problem that, with Django, can be done quite easily. The underlying idea is that every Django model has a primary key, which makes for an excellent key to a cache. Using this basic idea, we can cover a fairly large use case for caching, automatically, in a much more deterministic way. Let's begin.

some sample caching code

They are still re

t Level Cache (aka Transaction layer level cache)

net.sf.ehcache.hibernate.Provider

performance tuning tips for hibernate.Best article

Caching paginated results

One of the many benefits of side projects is that you get to try out new things. In my job I can’t screw around too much—I’ve got a site to run. But with side projects, I can play with new APIs and try out ideas. Lately, Twistr has been my playground.

"Mike Malone shares code used by Pownce to add QuerySet level caching to Django. It’s a smart implementation—a CachingQuerySet class inspects the arguments passed to get(), and if they’re just a straight forward exact PK lookup hits memcache for the object before hitting the database. Signals are used to invalidate the cache."

Some examples of transparently caching things in Django. An example Django app that uses custom managers, fields, and QuerySets to transparently cache objects.

Some examples of transparently caching things in Django.

mmalone's django-caching app

The jQuery offline plugin provides an easy mechanism for retrieving JSON data from a remote server, and then caching it. Subsequent requests for the same URL will retrieve the data from the cache, rather than the remote server. If the user is online, the plugin will transparently request new content from the remote server, firing the callback again if the content has changed. If the user is offline, the plugin will request the data from the remote server for the most recent request when the user comes back online.

A jQuery plugin to facilitate conveniently working with local storage

Shared: JQuery offline released http://bit.ly/b2q8xI

Web applications that wish to work robustly in flaky or offline scenarios can use client-side persistence to serve stale data while transparently trying to reconnect for more up-to-date data if possible. In a mobile scenario, the user may consider himself “connected” when in fact he has dropped out of connectivity for a moment (for instance, he may have gone under a tunnel). Because of this, and because latency on mobile devices can be quite high, a well-behaved mobile web application (or even simple website) will serve up content out of a local cache, so the user can see it quickly, before trying to make a connection to retrieve new content

The jQuery offline plugin provides an easy mechanism for retrieving JSON data from a remote server, and then caching it. Subsequent requests for the same URL will retrieve the data from the cache, rather than the remote server. If the user is online, the plugin will transparently request new content from the remote server, firing the callback again if the content has changed. If the user is offline, the plugin will request the data from the remote server for the most recent request when the user comes back online.

An assessment of mobile brower cache behaviour and recommendations for improving the chances that your content can be cached for improved performance.

An assessment of mobile brower cache behaviour and recommendations for improving the chances that your content can be cached for improved performance.