Network Maintenance Last Night

[FliesLikeABrick]

Sorry I forgot to announce this to everyone beforehand, but if anyone is wondering why the forums, sites, lobby and everything else was offline for 1-3 hours last night, this is why:

Dear Customer,

In our continuous efforts to provide you to better services, We are pleased to inform you that we will be deploying a new Cisco Catalyst 6500 series routers and switches to  provide redundant internet connectivity to Your Server/services in our Chicago Datacenter.

We have been testing the new hardware and its configurations for several weeks and expect the upgrades to go well. After this upgrade all our Customers will have BGP based redundancy from multiple carriers to provide you extremely high uptime as well as greater performance afforded by multiple routing paths.

The upgrade can take up to three hours (but will likely be shorter) and will occur between 11pm on April 14,2008 and 6 am on April 15,2008  Central Time.

During this upgrade, Your Servers will go offline. You may find it useful to inform your end users about unavailability of your server/services  during that time.

If you have any questions or concerns before or after this process, please don't hesitate to contact us by email at helpdesk@chinetworks.com  or by phone at 312-756-1500 x 2


Sincerely,
Chi Networks Helpdesk

There appear to be some problems with the new connectivity right now.  I'm seeing extremely low throughput and high packet loss on everything.  I have contacted them and will be following up with them later.

Besides those problems, the new routing setup looks really nice.  The traceroutes to/from my machine to elsewhere are much shorter and cleaner than they were before.  Once the issues get cleared up, almost everyone's connectivity to my hardware should be much better and lower latency.

[Shoozza]

I didn't notice the downtime but what I noticed were the connectivity problems. Good to know that it's 'just' your host.
Already was wondering what has happened .

Better late info then no info.

[FliesLikeABrick]

A reply from my host:

Yes, we are aware of this, and working with the service provider to get to the resolution ASAP.  This has been escalated already, and we hope to wrap this up pretty soon.

[echo_trail]

Argh, the slowness is killing me. It was pretty good in the morning for me, but now it's back to being intollerably slow again. I was hoping to see a downtime estimate

I can't believe I'll have to interfere with 'the real world' now.

[FliesLikeABrick]

I would hope they have it fixed within 12 hours since it is such a major issue.  I will ask them for an ETR.

[FliesLikeABrick]

They are going to revert to the old circuit within the next 2-3 hours and troubleshoot the issue offline.  Once they switch back to the old circuit, things should return to normal.

[echo_trail]

That's nice, good. Though it's actually been pretty fast for me since the last post I wrote.

[Catatonic`]

Well, I'm really relieved that it was just a hardware maintenance issue. I was sorta phreaking out at 3AM this morning, thinking that your datacenter burned down or something as nothing you hosted worked at all.

P.S.: Page created in 0.059 seconds with 22 queries.. Page loaded instantly for me.

[echo_trail]

Page created in 0.065 seconds with 25 queries. - Seems to work fine now for me also.

[Laser Guy]

0.066 seconds with 23 queries

[FliesLikeABrick]

They're back on the old connection for now until they can figure out what is wrong with the new one.  I'll keep you guys updated in this thread.