Hey guys, this we are still working on this thing. We still do not have a handle on what exactly is happening, but basically we are being hammered by connections that are overloading a specific part of the network. We worked on this all yesterday, banning bad ips and clients and are contining to work on this today. Sorry for the lack of communication. Thats my fault as I have been running around all day but marty and the guys are glued to their command lines. We'll keep you updated and get this resolved asap.

dave mansueto

Update: We've made some major changes to the network that we feel will balance things out much better. The distribution and balancing technique we were using needed some updating. We're going to need to see tomorrow at peek times to see if we really have things tweaked perfectly.

As luck would have it, while doing some other general maintanence over the weekend, where we were adding some redundency to other key parts of the system, the back kinda broke on a segment of the network. Most new users (who've signed up in the last month or so) were affected, so we're not really starting out with our best foot forward with a lot of you. I hope you can understand that these kinds of things happen to everyone, and hope you stick with us- as you can be assured we're going to stick with you- through thick and thin. We're going to keep killing ourselves (and hopefully not each other) to ensure the best, most reliable, most stable system for all you content creators.
-Dave Chekan

It looks like we are still experiencing overload on parts of our network. The connections are swamping the media nodes and this is causing the 404s you may be experiencing.  We have been working on this all this evening, and are implementing  a fix.
We'll keep you posted.

Update:  Issues are back with the 1 partition. We are working to recitify the problem by moving resources around while also trying to stop the bleeding.  We understand the problems this is causing many of you and we're working hard to smooth things out.

Thank you for your patience.

Things are back to operating normally.  We are checking the logs for any strange behavior.  There were some misbehaving ips, but it is not yet clear if this was the cause of the network overload.  We will continue to analyze what happened.
Sorry for the interruption to those affected users.

Something is causing a branch of our distribution network to overload.  We are looking into this now, and will report back soon.

We know that stats are temporarily not accessible.  We are working on the issue, and assure you that all numbers are being collected and tabulated.  The connection to the servers that deliver the XML back to the display is hosed, and we're working to get it back ASAP!

Thanks for your patience.

We are moving some services around tonight, putting things on other servers, databases, etc.  Downtime for logging in should be minimal if at all, and for the most part, there should be no noticable impact at all.

Just wanted to give everyone the heads up incase there's any weird blips that happen.  Rest assured the team is on top of things and working to keep everything running smooth.

We know many of you have been receiving emails from outside organizations regarding a show hosted on libsyn.  We are taking the matter very seriously, and apologize that you are stuck in the middle of it.

If you wish to read about the actions we are taking, please check out this forum post: http://forum.libsyn.com/viewtopic.php?p=10964#10964

Thank you

We have made 2 updates to our terms of service.  The following has been added to section II:

The opinions expressed in Your Content are not necessarily (and in some cases not at all) those of libsyn.  libsyn is not responsible for any Content provided by You, which is deemed harmful or offensive to others.  Furthermore, libsyn is not responsible in any way for any injury, loss or damage, which occurs as a result of an opinion expressed or information, provided in Your Content or through your use of the Site.

And in section V this:

You may not use the Site or any Content for any purpose that is unlawful or prohibited by these Terms of Use, or to solicit the performance of any illegal activity or other activity which infringes the rights of libsyn or others.

Has been replaced by:

You may not use the Site or any Content for any purpose that is unlawful or prohibited by these Terms of Use, or to solicit the performance of any illegal activity or other activity which infringes the rights of libsyn or others.  If we suspect that your use of the Site or any Content is unlawful or prohibited by these Terms of Use, or is soliciting performance of any illegal activity or other activity which infringes the rights of libsyn or others, we reserve the right, at our sole discretion or at the request of law enforcement officials, to indefinitely suspend your use of the Site or any Content while we and/or law enforcement officials investigate the potentially unlawful or prohibited activity.

Hey guys, the interim stats box fell over. We are working to bring it back up. The v3 stats engine is nearing completion and will be ready to roll by the top of next week. Please bear with us, promise to have this thing licked very soon.

Hey everyone, there's been a ton of requests for the option to allow comments to be moderated before going live onto your blog page. We came up with a quick and dirty implementation of such a feature which, like much of the rest of the publishing system, will be improved in the upcoming months as we revise the whole publishing system

For now here's how it works. Login, under the Settings / Advanced section you'll see a check box to enable comment moderation. Once you turn that on, all comments left on your blog will be hidden until you go in and enable them. To enable the hidden comments, under the publish / previous posts screen, if you click on the comments viewer, you will see a button to enable the comments. When you have new comments, you will see a link like this when you first login (see screenshot).

We're sure you're going to want additional features to support handling multiple comments at the same time, but for now we hope this holds you over. Everything has been tested and shouldn't cause any negative side effects, but please let us know right away if there are any problems.

Stats are back online.

(1) Whilst I was at a conference, our failing stats hardware gave up the ghost for good.
(2) I debated about the value of even trying to resuscitate the old system, since we are literally days away from the V3 server.
(3) The rest of LibSyn convinced me that it was necessary, and special hardware was obtained and put in place to help us through the remaining days to V3.
(4) This hardware is now online and serving stats.

Anomalies?

Shouldn't be too many, but on some of these kinds of awkward recoveries, you may see date buckets split or duplicated, esp around the time of our collapse, which was Sept 1.

The future:

I am in the final stages of development for the V3 server. This will not initially bring new features online, but rather new speed and (most importantly) stability. It establishes a new architecture, however, which will be far more adaptable to new features. Depending on company priorities, I would like geo-location to be the first of these features.

The timeline:

Sept 7 : Stats server with core functionality in stress testing mode;
Sept 8 : V3 stats server ready for emergency failover if V2 dies again.
Sept 10 : Extended functionality complete (referers, podcatcher info, etc.)
Sept 12 : V3 stats server online.

I understand this has been a rough month for stats; and ironically it was just about a year ago that I came onboard with LibSyn because V1 stats were hitting their limits. Since that time, stats have grown 150x as large as they were then, and now the V2 system, which is actually backwards compatible to V1, has hit its limit. The V3 system is redesigned from the ground up and is expected to directly support 100,000x growth and to scale without limit. Obviously, until we see all this in production it's just more words, but fortunately, we'll see it in production quite soon.

We have moved the forums to a new address: http://forum.libsyn.com

We experienced some unplanned downtime due to a hard drive failure. The user accounts, blogs and rss feeds were unavailable from 2:30pm - 5:00pm EST.  Podcast media was available during the outage. 
Sorry for the inconvenience

We realize the continued flow of comment spam is an annoying problem.  We want to let everyone know, this is not a problem we are ignoring... but battling the spammers is a full time effort, no doubt. There have been some great suggestions on how to best deal with the comment spam and while we will be working towards a more permanent solution, right now our development capital is being placed in other areas.

As a short term solution, however, we have put in a hard limit on the number of links allowed in a comment post. The limit is currently set to three. Any comments with more than three links will be blocked and a message will be displayed to that user.

Hopefully this will, at least for a short time, give everyone a breather from this annoyance, until we can get a more sophisticated solution in place.