/clientscripts over to Amazon S3/Cloudfront?

[RCA]

move completed:




is easy code a php script to capture files from one server and move to others (i do in original upload process), vb 3.x only needs avatars and profilepicture if remember correctly, all other files and images are full static.

[imported_silkroad]

Originally Posted by RCA

is easy code a php script to capture files from one server and move to others (i do in original upload process), vb 3.x only needs avatars and profilepicture if remember correctly, all other files and images are full static.

That is not what I asked about.

I specifically said in my post "capture a 404 error on one web server and the retrieve the missing file from another web server...." (and then serve it on the first server, of course).

This is non-trivial.

[Kaelon]

This is the first time I have heard about SimpleCDN and its game-changing "mirror bucket" feature. I definitely am eager to try this out, as I have just switched our Big Board (1.5 million posts, 40,000 users, around 200-300 concurrent a peak) from a dedicated colo over to the Cloud, and would love to migrate all of our static content off of the server.

[imported_silkroad]

Originally Posted by Kaelon

This is the first time I have heard about SimpleCDN and its game-changing "mirror bucket" feature. I definitely am eager to try this out, as I have just switched our Big Board (1.5 million posts, 40,000 users, around 200-300 concurrent a peak) from a dedicated colo over to the Cloud, and would love to migrate all of our static content off of the server.

Give it a try! We get around 4,000 to 4,200 peak users these days, and are reasonably happy with SimpleCDN, but they do "hiccup" from time to time. In the beginning, their team told me it was because of some DDOS attacks against their EU nodes. They told me all that would be resolved in Sept.

September has come to pass, SimpleCDN will not respond alas!

Wake me up, when November ends :-)

[Kaelon]

I am definitely going to try an upload bucket and see how things go in the short-term before delving in too deeply.

Do you have a back-up plan for when SimpleCDN is on the fritz? Or does your site suffer, and then basically you're stuck with what has come to pass?

[imported_silkroad]

Originally Posted by Kaelon

Do you have a back-up plan for when SimpleCDN is on the fritz? Or does your site suffer, and then basically you're stuck with what has come to pass?

We CNAME alias our Simple CDN mirror buckets and use a short TTL in the DNS. So, if there seems to be a prolonged hiccup with Simple CDN we just repoint the CNAME to our origin server and bypass SimpleCDN until the problem is fixed.

If you host your own nameserver, you could easily automate this.

[Kaelon]

Originally Posted by imported_silkroad

We CNAME alias our Simple CDN mirror buckets and use a short TTL in the DNS. So, if there seems to be a prolonged hiccup with Simple CDN we just repoint the CNAME to our origin server and bypass SimpleCDN until the problem is fixed.

If you host your own nameserver, you could easily automate this.

This is fascinating. I've already gotten a single Upload Bucket and created a CNAME alias to point to it, for the specific static stuff that I want to serve. I'm thinking of creating a Mirror Bucket for the dynamic-CDN, but I have a lot of directories that have dynamic content in them:

/gallery/files <-- all of my user uploaded galleries
/forums/customavatars <-- custom avatars
/forums/customprofilepics <-- custom profilepics
/forums/signaturepics <-- custom signaturepics
/forums/thumbs <-- generated thumbnails
/forums/albums <-- uploaded albums
/links/banners <-- uploaded banner images
/forums/images <-- all of my other static forum images

What do you think the best way to create a dynamic-CDN "Mirror Bucket" for these would be? Should I change my hierarchies? So far, SimpleCDN is working like a charm.

[imported_silkroad]

Originally Posted by Kaelon

What do you think the best way to create a dynamic-CDN "Mirror Bucket" for these would be? Should I change my hierarchies? So far, SimpleCDN is working like a charm.

We have to be "speaking the same language" first. I don't know what you mean by "Dynamic CDN" when discussing in terms of Mirror Bucket. Every file can be updated based on the cache configuration of the CDN, and any file not found in the CDN cache will go to the original server to get it.

So, your directory for the CDN mirror will MIRROR your origin site.

We use two Mirror Buckets, one for images and the other for clientscript and CSS files, but that is not necessary and we paid for an extra CNAME alias. That way we could track the usage for the clientscript differently from the images.

It is simply a Mirror, so you can Mirror in as many virtual buckets as you like.

Also, if you need to update some javascript, you can delete the script from the CDN cache in the user admin control panel.....

[Kaelon]

Okay, I think I understand this. So, for example, I could have one mirror bucket that basically mirrors my entire domain, but it will only fetch the content that is requested of it. So, does this mean if it is asked to produce /forums/images/21.gif, it will create the forums and images directories for me, and handle everything on its own? If so, then, I suppoes I should be fine with just one mirror for my entire domain given that I have so much user-generated content spread throughout all of my services.

1. Is there any security with SimpleCDN to prevent hot-linking (and thus, raising your bandwidth bills)?

2. How often do Mirror Buckets refresh or update images? For example, if a user uploads a new signature that overwrites the old one, will it be updated?

3. How do you recommend using the no-cache header linking options for your buckets?

[imported_silkroad]

Originally Posted by Kaelon

1. Is there any security with SimpleCDN to prevent hot-linking (and thus, raising your bandwidth bills)?

No, not at this time.

2. How often do Mirror Buckets refresh or update images? For example, if a user uploads a new signature that overwrites the old one, will it be updated?

If a user uploads a signature, the signature resides in the database, so that is not a CDN issue. The CDN does not work for PHP files that access the database. If you are talking about images in the signature, I think you will find that new signatures, like avatars, get a new (version) number appended in the name of the gif.

3. How do you recommend using the no-cache header linking options for your buckets?

We cache all our image and client content to the maximum number of days. This is configurable in the CDN admin panel. This overrides the no-cache header information in your files, as I recall; I suggest you open a ticket with SimpleCDN and ask them questions about how they deal with headers, cache, etc. That way you get the info directly from them.