So.. I’ve spent all day trying to figure out why our backup server lost connection with the storage NAS. At first, I thought the server iSCSI initiator service had died on me since I could connect and access the NAS iSCSI target from my workstation. Ok. Problem solved. Reboot the server (the universal approach) and I should be fine!
Or so I thought.
Still no connection. The backup server locked up entirely every time I tried adding the now removed discovery portal, or a quick connect to the target.. whatever i tried.. crash!
Ok. I tried connecting to a different iSCSI target… which to my surprise worked perfectly. So.. is it the target then? Apparently yes. After numerous reboots I could no longer even access the volume manager in the QNAP device which was pretty odd.. It worked 2 minutes ago. Not even a reboot or shutdown command would do anything. I tried a shell reboot.. Nothing. The NAS was entirely unresponsive, except for the Web UI which would show but gather no data from the device, and SSH which worked (but no commands did, not even nano would do anything).
A cold reboot and 7 hours of disk resyncing later, the NAS was live again. A forced connection through the backup network IP bound to the iSCSI initiator… and… SUCCESS!
What’s the moral of this story?
I don’t know. Some days just aren’t meant to be fun, and hugs from a loved one can be a great source of inspiration (K).