Stats ETA

Another update: based on the rate things are going, we expect the system to take about another 36 hours to become fully current.  That puts us at around 7pm Eastern, Tuesday evening.  Hold tight everyone, we're almost there.

---

[updated 8/15 7:28am ]

As some have noticed, our 7pm Tuesday night ETA was a little optimistic.  As we speak the whole system is catching up and is currently processing numbers for August 7th.   The speed of this catchup process is dependent on a number of of factors, so rather than giving another estimate which may or may not be right, I'll say that all pistons are firing, the stats engines are chewing through all of the raw data, and we'll post here again when we are completely up to date. 

In the meantime, you can check out your numbers for everything up until the 7th by looking under the stats tab of your libsyn dashboard.  Again: there will be inflated numbers for July 23rd and for some of you, July 24th.  These spikes represent downloads of your show that were fed into the system "late" so they were counted as having happened on the 23rd/24th when in fact they happened in the days and weeks previous to that. In total this was a few hundred thousand hits that were added after the fact, and this will affect some users more than others.

If you feel that something about your numbers isn't right, and you send our support team a report of it, please help us out by being as specific as possible.  Include your user name, the file name(s) affected, and the specific numbers that seem to be wrong.  This helps us greatly to figure out what, if anything, is going on.

---

[update 8/21 12:26pm]

Sorry folks, been neglecting updates on the blog.  There is a running thread on the state of the stats system here on the forums.

The latest:

Hi folks. Your daily update: the rollback is still catching up. We've made less progress than I had hoped in the last 24 hours and I'm sorry to say it's my fault. I "tripped over the cord" so to speak while trying to tweak things to speed up the process, which resulted in a number of hours without forward movement yesterday. I apologize for that, I was really trying to gain us a day or so in this catchup process. The misstep was corrected and we started plugging ahead again early last evening.

We're now working on 8/18 logs. I will keep my hands off the thing and let it run it's course. In response to the questions about files not showing up in stats, you are correct, if we haven't processed the logs yet for the period in time when downloads for your newest files started, then your files won't show up in the stats display.

For those who are interested, throughout this process we have also identified some bottlenecks that could be resolved by a different hardware configuration. While it would not buy us much time to move to new hardware at the moment, we are putting together a plan to upgrade the hardware the new collection service runs on, which will keep things nice and snappy during normal operation, and will help make a rollback such as the current one (god forbid we ever have to do this again) less excruciatingly slow.