Solution: Preventing damage after a system lockup

Text Links

Search

Calendar

«	July 2009	»
			1	2	3	4
5	6	7	8	9	10	11
12	13	14	15	16	17	18
19	20	21	22	23	24	25
26	27	28	29	30	31

Solution: Preventing damage after a system lockup

rechosen | 02 July, 2007 21:42

No matter how stable the Linux kernel is, things like hardware failures can still lock up your system quite effectively. If you ever encounter a case like that, rebooting is pretty much the only option. However, there is rebooting and rebooting. This solution describes a way to reboot your system that will do things like terminating all processes, syncing the remounting the disks, in order to prevent damage as much as possible. This can save you a lot of fscking and data loss.

What this is and what this isn't

First of all, I'll make it more clear what this solution is and what it isn't. This solution is meant to be used when your system is locked up, e.g. the cursor doesn't even move in X, Ctrl + Alt + Backspace or Ctrl + Alt + F1 and the likes don't seem to work anymore, maybe even Num Lock doesn't turn on and off when you press the button. This solution is not meant to use when a single application doesn't work. In that case, just terminate the application, not the whole system. Explaining how to do that is outside the scope of this solution.

Prerequisites

In order to be able to use the way of rebooting that this solution describes, you will need to have a kernel with 'Magic SysRq key (CONFIG_MAGIC_SYSRQ)' enabled. The easiest way to check this at runtime (without downloading and examining your kernel source package, that is) is to check if the file /proc/sys/kernel/sysrq exists. You can easily do this at the terminal using the following command:

[rechosen@localhost ~]$ ls /proc/sys/kernel/sysrq

If that command returns the name and location of the file ("/proc/sys/kernel/sysrq"), you will most probably be able to use the described solution. If it returns something like "cannot access /proc/sys/kernel/sysrq: No such file or directory", you'll have to get yourself a kernel with 'Magic SysRq key' support or this solution won't work.

The actual keyboard shortcuts

Keyboard shortcuts? Yes, 'Magic SysRq key' allows using the SysRq* key on your keyboard to make the kernel perform various actions, even if it is in serious trouble. We'll use some of these for our purpose. To be exactly, the following ones, in the following order:

Alt + SysRq + r (switch to "Raw" keyboard input)
Alt + SysRq + s ("Sync" mounted filesystems)
Alt + SysRq + e (send the "tErm" signal to all running processes)
Alt + SysRq + i (send the "kIll" signal to all running processes)
Alt + SysRq + u ("remoUnt" all mounted filesystems read-only)
Alt + SysRq + b ("reBoot" the computer)

Okay, now you might think: "I'll never remember that!". Fortunately, mnemonic devices exist. This site suggest remembering the sentence: "Raising Skinny Elephants Is Utterly Boring". Although that one is pretty nice, I though adding some rhyme might make the keys easier to remember: "Red Stripe Ends In Unique Blends". Or, though this one is a bit outdated: "Rebooting So Easy In Ubuntu Breezy". I'm looking forward to all the creative sentences you can think of! If you know a good one, please leave a comment.

*: If you can't find a SysRq key on your keyboard, try using the Print Screen key. In either way, the key is usually located on the right of the F12 key.

How and when to use them

The keyboard shortcuts I mentioned about should be used, as described in What this is and what this isn't, when your system is locked up. Though sometimes the kernel might be in troubles so big that it won't respond, in most cases it will respond. I recommend allowing the kernel to take its time: don't use the shortcuts too fast after each other. Wait at least 4-5 seconds before pressing the next one (and even longer on slower systems). After the last one (Alt + SysRq + b) the computer should reboot. If it doesn't, the kernel probably was unable to accept the commands (it doesn't support 'Magic SysRq key' or it is locked up totally). In that case, you'll just have to reset the computer and hope you won't lose anything.

More information about 'Magic SysRq key'

For more information about this kernel feature (other keyboard shortcuts, how to use it on different architectures etc.), have a look at the kernel documentation about it. The Wikipedia entry about it might also give useful information (though, at the moment of writing, it recommends a slightly different order of keystrokes).