Whole server crashed? DDoS atack?
Kinda. No.
The server has had two instances of an odd problem in the past month where the vmware server runs off into the weeds and another custom tool running on it starts acting weird. Within 20 minutes the machine is grinding to a halt.
That's not what happened today. today all of apache's processes were in the Z state and most or all other services on the machine weren't responding (fortunately ssh was fine). The load average on the machine was 10+ even though it wasn't really doing anything.
I'm not sure what was going on, couldn't find any way to recover from it (unlike the first problem), so I rebooted it hoping it'd clear up these issues. The machine hasn't been rebooted since it was moved to the new datacenter 6-8 months ago. Hopefully booting into a new kernel will help things, we'll see.
If problems like this keep happening I'll probably look at upgrading the server from Ubuntu 8.04 to 10.04 (easy enough to do)
