Win2ksp4 Dell NAS shuts down. Pick fellow ITgoons brains?

[AJ Dual]

Hi.

Just a shot in the dark.

I've got an "old" Dell Powervault 725 NAS/Raid5 running Win2k sp4. Damn thing just shuts off whenever it feels like it.  My first major chore here at the new job. It's been down since last October (this is also their backup server/solution) and the other IT guy here doesn't even really remember what was wrong with it.  They've been running manual backups to a 1TB external USB drive from a workstation.

Event log is of no help. It's simply telling me:

Source: Win32k Event ID: 240 A request to suspend power was denied by svchost.exe

Source: Removeable Storage Event ID: 133 Refused request to hibernate/suspend. One or more applications have active connections to this service... blah blah... Use the MMC. blah, before attempting to shut down etc.

Of course, I don't want to shut down, and don't know where the damn request is coming from. Xp/2003/Vista have a shutdown logging utility through the policy control panel, but it's no help, this is Win 2k server...
There is no MEMORY.DMP, which I guess makes sense because this isn't a BSOD.
Symantec Corporate AV is running and is current as of yesterday.

Any idea where this is coming from? At first I thought hardware, RAM, Raid controller etc. but if there's an actual request "somewhere" to shut-down what's making it?

Any ideas?

[AZRedhawk44]

APC Smart-UPS software on it, perhaps?

That's gotta be the buggiest most gawd-awful software I've ever used from a commercial retailer.

[RevDisk]

Random question, are you using NTBackup as your backup utility?   If so, there's a known issue of it causing that with certain devices.

[AJ Dual]

It does not seem to be running APC software, and the power control panel reports the generic MS UPS service is stopped.

And we're using BackupExec, not NTBackup. I found the articles/help forum entries about that while searching for this problem on Google etc.

[RevDisk]

It does not seem to be running APC software, and the power control panel reports the generic MS UPS service is stopped.

And we're using BackupExec, not NTBackup. I found the articles/help forum entries about that while searching for this problem on Google etc.

I assume you killed BE to see if that was the issue.

Did the Dell hardware diags say anything?

Also, mind posting or PM'ing the services list?

[AJ Dual]

I assume you killed BE to see if that was the issue.

Did the Dell hardware diags say anything?

Also, mind posting or PM'ing the services list?

Yes, and I did finally coax a hardware failure out of it. I forced the disabled drive in the RAID back on, (something I had done 3 times previously to try and force a failure with no results.) And it finally died. "PLINK!"

There may be FUBAR services trying to shut down, but I now think it was the RAID controler and drive manager stepping on itself.

[RevDisk]

Yes, and I did finally coax a hardware failure out of it. I forced the disabled drive in the RAID back on, (something I had done 3 times previously to try and force a failure with no results.) And it finally died. "PLINK!"

There may be FUBAR services trying to shut down, but I now think it was the RAID controler and drive manager stepping on itself.

RAID controller.   Don't shoot me, but try upgrading the firmware.  If that doesn't work (and it probably won't), replace.  They're not that expensive.

[AJ Dual]

RAID controller.   Don't shoot me, but try upgrading the firmware.  If that doesn't work (and it probably won't), replace.  They're not that expensive.

Could be, although in this case, AFAIK, it means a new server. The Dell NAS is a single pizza-box rackmount with four EIDE fast swap chassis on the front face.  It's all one board inside, with the exception of a backplane SCSI adaptor for the tape drive system they used to use, or to bridge it over to a bigger chassis with more drives in it.

At $70 for a new 300gb drive, it's not a big expense to test. It would even be cheaper if this box was new enough to support SATA instead of PATA... These are just a bunch of WD Caviars, with one Maxtor thrown in for excrement and giggles. (I'm guessing a repair no one remembers or has documented...) Drive 1 is what's acting funky, so I've just left it out all day and it's been stable, other than all the warnings in the RAID manager....

And as I'm typing this, it just spontaneously shut itself down...  :rolleyes:

The two things that have me absolutely scratching my head is that the data volumes are all striped RAID 5, but they only mirrored the C volume with the system folder. Was it done for a percieved performance boost? Of course, it's one of the mirrored drives that failed, leaving me flying naked until the new drives come in and it can clone and rebuild itself.

The second is that this damn thing has no removeable media. No CD-Rom, or anything. That's not at all uncommon in a larger datacenter, where you run a central PXE server to bootstrap etc. But it tells me here they were just relying on the instance of Win2k the server shipped with, with no real plan of how to do a full recovery if it needed to be rebuilt.

What fun...

[lee n. field]

It's been down since last October (this is also their backup server/solution) and the other IT guy here doesn't even really remember what was wrong with it.

On a good (dual conversion) UPS? 

Capacity?  Storage is cheap right now, any reason this isn't just replaced?

I've got a server I'm keeping an eye on.  Every once in a while it will just --BOING!-- restart, and all you see in the event log is "last shutdown was unexpected".  I suspect motherboard (onboard NIC is also flakey), and HP will hear about it first chance I get to be on site.

Any ideas?

Microsoft (sysinternals) process explorer, and dig in deeper into what's running that task manager can show you.

[RevDisk]

We just tossed our NAS's, for the most part.   Went with a SAN, and couldn't be happier.  Much easier management.

[ilbob]

Sounds to me like the cost of your time to figure it out will be more than just replacing it with something new.

[Zardozimo Oprah Bannedalas]

Check the 'scheduled tasks' in the control panel.

[41magsnub]

Sounds to me like the cost of your time to figure it out will be more than just replacing it with something new.

That's what I was thinking

[AJ Dual]

That's what I was thinking

Well, I'm running it right now off of the recovery volume and it's been very stable all day. We have $75 invested in a new EIDE drive that's on it's way, if it clones everything for the mirrors, and starts repopulating the other RAID data, I'm going to call it done.

If it fails after that, it gets one rebuild, then if it still acts up it will get replaced IMO.

I might just push for a third-party offsite backup solution instead, such as Iron Mountain or some such. We're only a 20 seat co., including me, just have to see what the costs are. We have a pretty hefty fiber drop with plenty of unused capacity from Time Warner here. We get about 32mbit down and about 12mbit up. And maybe 500gb of mission critical data, and maybe just a few gb of deltas a week. So speed won't be a bottleneck.

[RevDisk]

Well, I'm running it right now off of the recovery volume and it's been very stable all day. We have $75 invested in a new EIDE drive that's on it's way, if it clones everything for the mirrors, and starts repopulating the other RAID data, I'm going to call it done.

If it fails after that, it gets one rebuild, then if it still acts up it will get replaced IMO.

I might just push for a third-party offsite backup solution instead, such as Iron Mountain or some such. We're only a 20 seat co., including me, just have to see what the costs are. We have a pretty hefty fiber drop with plenty of unused capacity from Time Warner here. We get about 32mbit down and about 12mbit up. And maybe 500gb of mission critical data, and maybe just a few gb of deltas a week. So speed won't be a bottleneck.

If you're small enough and cost is a concern, just make weekly trips to a safety deposit box.  Plenty secure for most folks' needs.   If in doubt, encrypt the entire disk.  I've always been leery of off site backup companies.   Not saying you can't make excellent use of them, but it's one more organization in the loop and you need to take extra precautions to deal with it according.

[AJ Dual]

If you're small enough and cost is a concern, just make weekly trips to a safety deposit box.  Plenty secure for most folks' needs.   If in doubt, encrypt the entire disk.  I've always been leery of off site backup companies.   Not saying you can't make excellent use of them, but it's one more organization in the loop and you need to take extra precautions to deal with it according.

Once this NAS back up and running, that's exactly what we'll do for now. Get an external ESATA dock and just stick a large drive in it, then have a few cheap big drives in rotation, and the owner will take them home with him and put it in his safe. I'd suggest some rotation of full and deltas, but we're not that big, the whole enterprise could fit on a $99 1TB drive from TigerDirect or whoever...

[AZRedhawk44]

Is someone maybe screwing with you?

shutdown /m \\NASDeviceHostName /s /t 0

Can be run over the LAN or via scheduled task.  And since your NAS is a windoze box it is succeptible to this.

PS:  Very mean trick to pull on co-workers.

[RevDisk]

Is someone maybe screwing with you?

shutdown /m \\NASDeviceHostName /s /t 0

Can be run over the LAN or via scheduled task.  And since your NAS is a windoze box it is succeptible to this.

PS:  Very mean trick to pull on co-workers.

Not as mean as randomly opening the CD tray.  Only thing better is to do so in a computer lab. 

[41magsnub]

Not as mean as randomly opening the CD tray.  Only thing better is to do so in a computer lab. 

My favorite was back in the day running Microsoft SMS server at my old job.  I found the registry edit to kill notification of remote control sessions.  Several of the floor supervisors were practical jokers.  As revenge I would take control of their PC and make the mouse cursor slowly fall off the screen.  A couple of times I was doing it while they were calling me for support because of it.  I'd say they were crazy and needed to come see it for myself, of course it wasn't happening when I got to their desks :)

One of them finally caught me though...