optimize site media serving

general = { about, articles, links, projects } meta = { date-posted: 2008-02-22 }

I wanted to optimize my site media serving, while providing as much of an adaptable environment for myself as I could.

I have a few sites that share images (logos) and CSS files. Each site had its own media directory, and the media was served from that particular domain. It was simple, and it worked.

The problem? It wasn't very cache-friendly. A user would visit siteA, and get imageA. They would visit siteB, and get imageB. It didn't matter that imageA and imageB were the same image, just in different locations. The URLs for each image were different, so they were cached by browsers and proxies as separate objects.

So I created a separate subdomain, and named it cdn.cactuswax.net.
It isn't really a CDN, more like a femto-cdn.
I moved my static media that doesn't change into this subdomain.
I changed my links and references in my normal sites to point to media in these locations.
I added some nginx stanzas to increase the cacheability of the media files for the new CDN subdomain.
```
location / {
    expires max;
    add_header Cache-Control public;
}
```
This stanza sets the expires header to 2037, and cache-control to public. This tells proxies that they can cache the objects.. for a long time.

The fourth change above, requires a bit of rethinking of my references. I want things to cache indefinitely in the CDN subdomain. However, my normal domains may want to update some media. With the above, a client may never see an update to a CSS file for instance.

There are two ways to deal with this. The first way I have seen is to append a query string to media references, such as the following:

<link rel="stylesheet" href="http://cdn.domain.com/style.css?ver=1" type="text/css" />

When a new version of the CSS file is posted, the ver number is incremented. The query string is discarded by the webserver, and the media is returned. A cache should incorporate the query string portion of the URL as part of its determination of cache object uniqueness. However, I have heard of some caches not caching things with query strings, or caching them in unexpected ways. Be aware of potential issues when using this method.

The other method is to simply change the name of the media reference:

<link rel="stylesheet" href="http://cdn.domain.com/style01.css" type="text/css" />

This is the method that I chose. When I make a change to the style file, I rename the media file (or make my change to a copy of the original media file), and increment the number. Then I change any references in sites that I want to use the new media file.

This may seem burdensome, but a recursive grep and sed operation is pretty painless. Add to this the fact that most of my sites have well defined templates, and the change becomes inconsequential. No more burdensome to me than the previous method, but having the benefits of greater transparency and being slightly more cache friendly.

Besides the greater cacheability of using a your own media service domain, your femto-cdn if you will, you will be primed for a move to a 'real' CDN should the need arise -- be it one of your making, or through a provided service.