Last night the author and many users around Australia experienced a network outage which severed their connection to the outside world for a small window of 1-4 hours, based upon my reading of the Twitter hash-tag #tpg. Fellow Tweeter @samotage suggested that Senator Conroy was “installing disk 1 of his cleen (sic) feed!” which I found amusing.
Whilst an outage of this time-frame isn’t overly dramatic, especially considering the fairly good up-times I’ve had throughout my many years as a TPG ADSL customer, the fact that it was a large-scale outage effecting users in at least three states/territories (VIC, ACT and NSW, according the location profiles of Twitter users who reported the problem last night), I would have thought this raised some alarms at TPG HQ in North Ryde, prompting them to make a service disruption notice of some kind.
After I conducted my initial local troubleshooting last night, which showed that although I had a sound DSL connection, my PPPoE connection had dropped, I attempted to call TPG customer service but their number didn’t even ring. This experience was reported by some other Twitter users, as was the inability to access the TPG website via various wireless internet connections (in my case via my iPhone connected to the Hutch3g network).
My service resumed some time between 10-11pm, and this-morning I checked various news sources assuming there’d be some report about a wide-scale outage last night – nothing. No mention of it on NEWS.com.au, theage.com.au, AustralianIT.com.au or ZDNet.com.au.
Curiosity aroused I checked TPG’s service status page showing outages over the last 24 hours and saw only two entries, neither of which matched the time period in question.
Turning then to Whirlpool.net.au I found the following post in which this outage was discussed, and user ‘Karen C’, apparently a TPG representative confirmed that TPG weren’t tracking any large outages at the time.
I have emailed TPG asking for some more details which I shall post back here if I receive a response. In the mean-time, do you know anything about last night’s outage? Were you effected? Was Twitter your only source of information at the time?
Codenix.
=====
UPDATE 2010/06/03 12:58PM
A TPG representative replied via email saying they are aware of the outage and it was apparently posted to the service status page, but I can’t see any mention of it there, unless there’s a mistake in the times, locations and effected service (I’m on ADSL1), of the outages listed for the last 24hours.
I’ve asked for clarification and await a reply.
=====
UPDATE 2010/06/03 2:09PM
The TPG rep responded with:
The only reported cases on the outage are the ones posted last night. I can’t really check what had caused it yesterday for there are no conclusive report provided to us other than the ones posted.
I take from this that TPG either don’t know about the outage, which seems very strange indeed considering how far spread it was, or they don’t want to discuss it just yet.
Either way, very interesting. I wonder if any businesses were effected? I wonder how you would prove the outage effected your business if you had a Service Level Agreement with TPG and they claimed no knowledge of the outage?
=====
UPDATE 2010/06/04 5:22PM
With no further replies from TPG, Whirlpool remains the only source of information about this outage.
A very long thread here followed the outage from it’s beginnings at around 7PM on Wednesday 2nd June, to around 11PM that night.
This thread grew rapidly and currently sits at round 45 pages. At one point someone pointed out there were approximately 5 posts every 2 seconds, and many commented that this was the fastest growing thread in Whirlpool history.
There were many theories as to the cause, and at 11:58PM, Whirlpool User ‘Jared.’ posted the folllowing;
Click on the images to enlarge
So according to Jared, the issue was caused by an issue with faulty routes from one carrier (he didn’t state which), which effected the Peering Interconnections, which in turn would have effected any ISPs using those Interconnections.
In the meantime, there were some status updates posted to the TPG status page which didn’t show up in the history once the author’s internet connection was restored, some samples below;
Who knows why TPG were cagey about the outage when I asked them – perhaps the rep I communicated with wasn’t aware, but it seems unlikely for such a large-scale outage. Either way it still comes as a surprise to me that this still hasn’t been reported in any mainstream media at all.
Codenix.
codenix | blog 