We’ve have to take FTP offline to do some unexpected maintenance – we will update this post when it’s back up.
UPDATE – 10:40AM EST – FTP is back online and quickcast functionality should be working again.
We’re getting reports of assets not loading for some users and we’ve been able to replicate this issue ourselves. It does not appear to be affecting all users so it might just be an issue with a particular node on the content delivery network. We are escalating the issue with them and will report back when we know more.
UPDATE – August 4th 2011 – 12:00AM EST - Our content delivery network provider has tracked this down to being a single node in their Washington, DC area cluster that’s causing the problem. We expect this issue to be resolved shortly and will update this post once we recieve confirmation from our provider that the issue is fixed.
UPDATE – August 4th 2011 – We received notice at approx 12:30AM EST this morning that this issue has been resolved.
UPDATE – August 4th 2011 – 9:25AM EST – We’re seeing different errors with reports of users getting 401 unauthorized errors when trying to load assets. We’re escalating this with our provider immediately and we apologize for the trouble.
UPDATE – August 4th 2011 – 4:45PM EST – Assets on blog pages and external sites should be working again – we have failed them over to our backup system while the content delivery network gets their act together. Unfortunately assets in RSS feeds are still not loading – we may fail those over to the backup systems soon as well. We sincerely apologize for these troubles today and will plan in the future to fail-over assets to backup systems much faster.
Note: having assets failed over to the backup system means newly uploaded assets may take a while before they begin to load properly.
UPDATE – August 4th 2011 – 7:30PM EST – All assets should now be serving correctly – we apologize for the problem and now have systems and procedures in place to prevent this sort of issue from occurring again.