Regular readers will know that for the last month and a bit, we’ve been in a never ending battle of wills against servers and load times.

Our first major change was implementing Amazon’s S3 service to serve static files from the site. It helped, but in the end we ended up moving to Rackspace Cloud Sites to finally get a grip on the server issues we were having.

Since moving last Monday, the site has mostly purred, and since I’ve been looking for ways to improve load time again. One thing that did surprise me that as much as the files served from Amazon S3 were being served well, there was some latency there, and it wasn’t as quick as I’d hoped for, even if it did help decrease the load on our core hosting setup.

If you know about CDN’s and how to set them up, you should probably stop reading from this point, because the following isn’t going to be a detailed, or perhaps completely accurate technical description.

What is a CDN

A Content Delivery Network (CDN) is a network of server locations that deliver your content in multiple locations. Traditionally, if someone visited your site, they’d pull all your locally hosted static files (images, js, css) off your server directly. With a CDN setup, they pull those static files from their closest CDN, so for example if you were in Europe, the static files would be served from a local node of the CDN network. What this means is that your static files are quicker to load because they are served from a closer location, and although you’re still serving some files (php etc) from your original server, the end user gets the page quicker.

Each node in the CDN takes a copy and cache’s that for local traffic. The bigger the CDN network, the quicker load times are for most because there are more closer nodes to serve the static files.

Amazon Cloudfront

The reason we’ve implemented Amazon Cloudfront was one of ease of use: we already had Amazon S3 set up, so it was simply a matter of 5 minutes and bingo: CDN. A number of people have pointed out to me that given the site is on Rackspace, we have access to Rackspace Files, which uses the Limelight CDN network. The Limelight CDN network is superior as a CDN, not only because they have an Australian presence where as Amazon doesn’t, but sheer global presence.

However, the WordPress tools to set up Rackspace Files aren’t as easy to use as those available for Amazon, and we already have files on S3, which cut out the need to transfer a ton of image files across as well. Ultimately Rackspace Files (with Limelight) will offer a better solution, but for now we implement what we can with the tools at hand.

Setup

To setup Amazon Cloudfront with WordPress, you need S3 to begin with: read Using Amazon S3 for image hosting with a WordPress blog. The cool thing about Cloudfront is that it pulls data from pre-existing S3 files, so there’s no transfers required, it’s just a matter of setting things up.

Paul Stamatiou has the best guide for setup here, and I won’t attempt to replicate it all (he has pictures as well). The key points though:

- Go into Amazon Web Services and enable Cloudfront
- Using S3Fox, select the bucket (folder) on S3 you want to add to the CDN, right click, and pick manage distributions
- Add a name for the distro (Amazon Cloudfront), tick the “enable the distribution” box, pick a CName (in our case, turbo.inquisitr.com) and click create distribution
- you then get a resource URL. Using the instructions in our S3 guide (or via your host on how to setup a CName), using the details given, setup a Cname for your URL (turbo.yourname.com) pointing at the Cloudfront URL.

This is where I did it differently. Presuming you’re on S3 and you’ve got something like images.yourname.com for your S3 content, download the CDN Rewrites plugin here and install it. Once that’s setup, you simply need to tell it to redirect any URL from images.yourname.com to turbo.yourname.com (and this presumes you’ve got your files on S3 to start with, if you haven’t, this won’t work.) Select the options as fit, for example images, css, js etc…

Besides some little bugs with a thumbnail plugin we’re using (not related at all to Cloudfront), our page load times have been cut by between 40% and 80% (the former, only when we’re loading a full image to then make the thumbnail.) On one test, Pingomatic went from 8.7 seconds before to 2.9 seconds after…and that’s staggering. I don’t get the full effect in Australia because there’s no local CDN, but even I can see the difference here.

If you’re under load stress, and what to make you site load quicker, a CDN might be right for you.

Related posts:

1. Using Amazon S3 for image hosting with a WordPress blog
2. Review: Our First Month On The Rackspace Cloud
3. It was the best of times, it was the worst of times (or why we’re with Rackspace)

The promise of Amazon S3 cloud hosting has been discussed extensively across the tech blogosphere, but for those of us who aren’t coders by trade, tapping into that potential does appear to be a serious challenge from the outside.

Regular readers would know that at times The Inquisitr isn’t as quick as it should be. Despite spending a small fortune on a Media Temple Nitro Server, our peak loads slow the site down, and more recently that’s meant the occasional database error as well. The next stage was a load server, but that would be a pretty penny on top of what we’re paying already. Thanks to some friendly advice from Wayne Robinson, there was another alternative: Amazon S3 for image hosting.

The key on our end is to bring the CPU load down (the memory has never once been taxed). Most of the load is SQL related, but add to that posts with larger images, and the constant bane of late of people hot linking our images, and combined we’ve managed to get CPU load out to as high as 70x what it should be.

As of yesterday, all new images being uploaded for posts, and a sizable portion of the template images (the ones I’ve dragged across) are served by Amazon. Early days yet, but I saw the CPU load hit 0.03x at one stage today (be it briefly) on a reasonable live traffic count, and I don’t remember ever seeing it go so low.

What you’ll need

If you’re using WordPress, head straight to the Amazon S3 plugin (here). At the time of writing it claims it’s only compatible up to WordPress 2.7, but it’s working fine on my 2.8.4 install. There’s also a WordPress Amazon S3 backup plugin available (here), you don’t need it for image hosting, but if you’ve ever been slack with backups, it’s a handy plugin to have.

You’ll also need Firefox plugin S3Fox (here) as well. I do most of my browsing on Safari, but I use Firefox for dev work. There’s probably an IE plugin/ add on out there as well, you’d have to look. The Firefox plugin gives you quick and easy access to your S3 account; if you’ve used an ftp program before you’ll immediately be able to use S3Fox.

Naturally you’ll need an Amazon S3 account. You can sign up here. If you’ve got an existing Amazon account, you use that to log in, and if you have a credit card on file with that account, you use that. Everyone raves about how cheap S3 is but I won’t profess to be able to explain how much it costs. What I do know is that 24 hours after signing up, we’ve racked up a bill of 56c. As we’ll be pushing more and more images onto the server this will go up, but if it ends up at $5-$10 a day, I won’t be the least bit concerned.

Set Up Round One

Once you’ve signed up to Amazon S3, you need to set it up to work with WordPress. First you need to create what Amazon calls a “bucket.” In layman’s terms, think of it as a folder on your Amazon S3 account. That’s where S3Fox comes in.

You have two options with S3Fox, you can go to a full view access via Tools in the Firefox menu, or you can click on the S3Fox icon at the bottom of Firefox which presents a smaller, limited version. It doesn’t matter at this point which one you pick, as long as you get access to your S3 account.

S3Fox requires three things for access: your Account Name, Access Key and Secret Key. All of those are available from S3 via the “security credentials option.” Cut and paste.

Once you’re in, it’s simply a matter of clicking on the “create directory” button which is a blue icon with a gold star on it, and naming your bucket. Note that you should name the bucket what you’d like your directory to be We’ll get to that part in a minute, but if you want to go with images.yoururl.com, name your bucket images.yoururl.com.

Setup Round 2: WP plugin settings

Once you’ve gotten to this point, you’re nearly there. You’ll have your S3 settings handy from setting up S3Fox, and you’ll need them again.

In WordPress, navigate to “Amazon S3″ under the settings option. The first two options are easy: access key and secret code. After that you pick your bucket by drop down. In my experience, if you haven’t got a bucket set up, this just won’t show, so make sure you’ve set one up per the above instructions. After that I’ll go through step by step:

- Host name settings: if you want your images to appear at images.yoururl.com and not images.yoururl.com.s3.amazonsomething.com, click this option. However if you go this way, it requires you to setup a Cname entry on your domain. We’ll get to that part soon.
- File Uploads: check this if you want all WordPress uploads to go to your S3 account. The plugin gives you the option, and you also get a new button above the add new entry box if you’d rather pick between your server and S3 on a post by post basis.
- Expires Header:something to do with caching images. I could find no advice saying don’t check this, so I did. If you want to read more, there’s a link next to this option so you can find out more.
- File Permissions: you absolutely should check this otherwise your images won’t be publicly available. Uploads to S3 are defaulted to private…which is a pain in the neck if you’re uploading directly to S3 because you have to change file permissions each time. This option negates the need to do this with WordPress uploads.

Hit save, and that’s all there is to it.

Cname
You have a choice here, and this could be the most challenging point in the process if you want to go down this path. It’s easy to not go the cname route, although the problem there is that all your image uploads a linked into Amazon for the future, where as if you go cname, you can easily export out of Amazon at a later date.

Details on the cname settings are available on Amazon here. Depending on your setup, you may need to set this up with your domain name registrar or host. In my case, it was via MediaTemple’s control panel. Check with your host on how to set up.

And that’s all there is to it.

24 hours later and we haven’t had a hiccup yet. Mind you, if S3 goes down (which is extremely rare, but it has happened in the past) The Inquisitr is going to look mighty weird. But if it saves us money by not having to buy a new server, it’s a win all round.

Related posts:

1. Amazon Cloudfront CDN with a WordPress Blog
2. Amazon.com Goes Down (Now Up)
3. Setting up your own blog is easier than you think [Blogging 101]

Amazon’s S3 storage solution crashed earlier today, causing widespread issues with a range of web 2.0 applications. The service is popular among startups due to its cheap pricing and usual reliability.

Allen Stern broke the news on Centernetworks, noting that the service went down at around midday US EDT. His latest update indicates (per the screenshot above) that some 6 hours later the service was slowly coming back up. Affected services include the iPhone Twitter client Twinkle and photo hosting service SmugMug. Other services affected include the AIR Twitter client Twhirl, which relies on S3 for avatar hosting.

WebWorker Daily notes that Amazon offers a 99.99% uptime guarantee on S3, which equates to no more than 45 minutes of downtime a month, with refunds available for longer downtime.

For those affected by the downtime, the Amazon status page can be followed here.

Update: outage seems to have take Plurk completely offline as well.

Related posts:

1. Amazon.com Goes Down (Now Up)
2. Twitter users get a dose of the old days as Twitter API fails
3. Watch out Amazon Marketplace, Amazon Basics launches