My server has been acting up lately. It would periodically slow to a crawl and stop responding. Checking the server log files I would always see an entry about mysql running out of memory. I assumed it was a mysql performance tuning issue and was reading up on the plethora of performance settings you could tweak. This lead me down various wild goose chases. I should have started with the obvious: What is consuming all the memory on my machine? I did do a "top" of my system and nothing appeared to be using more memory than it should, but I didn't look at the overall memory usage until today. Here is what I saw:
Look at that carefully. First of all 84% of the memory is being used, but that's not a big deal. Linux caches as much as it can into RAM and then swaps out what it doesn't need to the virtual memory on demand. Virtual memory....swap....look at that picture again. 0 bytes of swap!!! There is NO virtual ram available! According to my system it doesn't exist! That's trouble. Looking over my /etc/fstab file (the file that specifies all of your partitions that you are mounting) it looked like it was loading my swap partition.
But executing:
cat /proc/meminfo
Showed that there was 0 bytes of swap.
There is a command in Linux to enabled/disable the swap (virtual memory). The commands are swapon and swapoff. I gave swapon a try:
/sbin/swapon /dev/sda3
Then looked at my memory usage. Ahhhh there is my swap.
No wonder my machine was locking up, it was running out of physical ram and then crashing hard because it had nothing else to do. But the question is why was the swap partition not being mounted? Well that's my stupidity. I was restructuring my partitions the other day and created a new swap partition but never gave it a label. The label is basically a name you can give your partition such as "backup" or "work". In my fstab file I was mounting my swap file using the label rather than specifying what device it is (/dev/hda3 in my case). So to fix this I just needed to update my fstab file to mount my swap using the device name rather than the label name.
Moral of the story: When you have insomnia, don't screw with your partitions late in the evening...compile your kernel instead. ;)