Stats processing is currently behind due to an error that caused processing to be halted. We’ve restarted the stats processing and expect it to be fully caught up in the next few hours.
Stats processing is currently stopped, this appears to stem from the database issues we had last night. We’re investigating and hope to have processing up and running again soon. We’ll update this post with details as we get them.
UPDATE – 3:10PM EST – We have rectified the problem that caused stats processing to stop. We’ve begun processing through the backlog which will take approximately 8 hours or so. We’ll update this post once processing is completely caught up.
UPDATE – 8:15PM EST – Stats processing is now fully caught up and has finished going through the backlog from earlier today.
At approximately 11PM tonight we received alerts from our monitoring systems and quickly discovered it to be an issue with our main database cluster. A fail-over had been triggered and while this did cause a brief interruption in services everything was still running smoothly. During the next hour we put the Libsyn platform into maintenance mode for approximately two separate periods of 10 minutes each. This was done to allow us to switch back to the main database cluster after ensuring that it was functioning properly. As of 12:05 AM EST we are running on our main database cluster again and all systems are functioning normally.
We have recently completed making some improvements to our network and hardware infrastructure. If you are using a custom domain with us and you are not using our Libsyn nameservers, you will need to make a change to the IP address you are pointing to.
If you aren’t sure if you are using our name servers or not, you can check by visiting: http://intodns.com and entering your domain and clicking the “Report” button:
On the next screen look at the very top section, this will list the nameservers that your domain are pointed to:
If your results show at least two of the above nameservers then you can stop reading this article. If you see any nameservers other than the libsyn ones please continue reading this article.
Since your still reading that means you aren’t using our nameservers which is completely fine. While we do recommend using our nameservers we understand there are situations where you might not be able to do so. What you need to do now is login to the website of whoever you are using for DNS hosting. This is often times the same site where you initially registered the domain. Once you have logged in, please select and browse to the records for the domain that you are using with us. Then look for the following records:
|Record Type||Record Name||Current Value||New Value|
|A||blank or yourdomain.com||18.104.22.168||22.214.171.124|
The goal here is to change any record you have pointing to: 126.96.36.199 and update it to point to: 188.8.131.52. If you are using a CNAME for the www or any other sub-domains you don’t need to change anything.
The old IP address will continue to work through the end of June. Starting July 1st it will stop working, so you must have your DNS records updated by then.
If you have any questions please contact firstname.lastname@example.org
We received notification today from our datacenter provider that they will be performing a software upgrade on their core routing equipment at 5AM EST tomorrow Sunday June 16th. They are telling us to expect at most a 5-10 minute outage. Our admin staff will be online during this time and will be in contact with the datacenter and kept up-to-date as the work is done. We’ll update this post when the work begins and when it is completed.
UPDATE 11:30PM EST – June 15th:
We just received noticed that the scheduled network maintenance planned for tomorrow morning has been postponed and is now scheduled for next Sunday June 23rd
Stats processing is currently running about 9 hours behind due to an issue overnight that caused processing to stop. We have resumed the stats processing and will update this post when it is completely caught up.
UPDATE 2PM EST – At this time stats processing is completely caught up and is continuing to process normally.
Some listeners may notice a ‘504’ error when attempting to access content. As of this time, it appears to be happening only on rare occasions. We are aware of our issue and have contacted our provider where the issue is stemming. We hope to have this resolved as quickly as possible.
We will update this notice once the issue has been considered resolved.
UPDATE 10PM EST – At this time we believe the issue has resolved, it’s been several hours since we’ve seen any reports of the 504 timeout occurring. If you do still see this error please open a support ticket by e-mailing email@example.com.
There appears to be a delay in new episode content being entered into the iTunes store. We have been in contact with iTunes and expect episode updates will be operating smoothly again soon. This will only affect new content being entered into the iTunes store itself, content in feeds will continue to be syndicated out to subscribers immediately.
Stats processing was halted sometime yesterday evening due to an error, we’ve restarted processing and will keep an eye on the system to see if the error occurs again. At this point the error appears to stem from invalid data in a single log file that wasn’t properly stripped out. No data has been lost and we expect processing to catch up over the next several hours.
UPDATE 2:30 PM EST – Stats processing is fully caught up now.