Linux Swap Space and your WordPress Server

Introduction

The following article was created to send to clients that might be having issues with swap on their servers running WordPress.

What is Linux Swap Space?

Linux swap is a space on a hard drive or SSD that is used as virtual memory when the physical RAM is fully utilized. When Linux runs out of RAM, less frequently used data can be moved to the swap space, allowing the system to free up actual RAM for tasks that need it immediately.

This helps in managing memory more efficiently, especially in systems with limited RAM, although accessing data from swap is slower than from RAM. It’s particularly useful for handling temporary spikes in memory usage, running more applications than the RAM can hold, or for systems like servers where stability under load is critical.

Great Articles on Linux Swap

• https://www.redhat.com/sysadmin/clear-swap-linux
• https://www.cyberciti.biz/faq/linux-which-process-is-using-swap/
• https://www.howtogeek.com/449691/what-is-swapiness-on-linux-and-how-to-change-it/

How can I tell if my server is using Swap Space?

If you haven’t already been alerted by monitoring tools such as Netdata or Monit, you can check a servers swap space usage using on of the following methods.

free Command

The free command is a quick way to view the amount of free and used memory on the system, including swap. Running free -h will display the information in an easy-to-read format, showing how much swap is in use. Example:

In the above example you can see the total, used and free amount of Mem and Swap.

top or htop Command:

Both top and htop are interactive system monitors. top comes pre-installed on most Linux distributions, and htop can be installed for a more user-friendly interface.

These tools show the current swap usage along with other vital system statistics. In top, swap usage is displayed in the header section. You can install htop by running.

sudo apt-get install htop

Here’s an example of htop’s output:

As you can see in the top left section, there is a “Swp” graph, which shows the swap usage.

vmstat Command

The vmstat command provides information about system processes, memory, paging, block IO, traps, and CPU activity, including swap. Running vmstat with a few seconds of delay will show you real-time swap usage statistics. Example:

What should I do if my server is utilizing Swap Space?

Ideally this isn’t a good situation, however it’s better than you server going down completely due to lack of memory. Typically swap space is utilized when there isn’t enough free memory, this is caused by a request by a single or many processes asking for memory.

1. Review Running Processes

You will want to review the running processes to see if any one process or if multiple processes are running and taking up a significant amount of memory. You can do so by running the following commands.

You might find the culprit, or you may not find any process using a large portion of memory. This just signifies that the surge in memory was temporary or triggered based on an event. You’ll want to review tools that provide historical data to see if you can pinpoint what caused the spike in memory.

htop and top

The gold standard for understanding what’s going on with your system, htop is an improvement on top allowing for colors and quick filtering of data such as number of processes and filtering by memory.

atop

atop takes a snapshot of top periodically, providing a method to look back at a specific time to see what processes were running and utilizing which resources. For more information on atop, reference my article Dealing with High CPU or Memory Usage on your WordPress Site or Server

ps

You can never go wrong with ps, if you need to know everything about what processes are running on your system then ps is your tool. The following command arguments will provide very detailed usage information as well as show the parent and child relationship.

ps -auxwwf

Netdata

Although you can’t see very much detail, you can see what users or user groups are utilizing memory over time which is helpful. I have an article that goes further in-depth into Netdata when trying to tracking down high cpu or memory usage.

2. Review Linux Processes in Swap

You can view what processes are currently utilizing swap space by running the following command. It’s long, but will provide you with the process ID, process name and the amount in KB currently in swap.

find /proc -maxdepth 2 -path "/proc/[0-9]*/status" -readable -exec awk -v FS=":" '{process[$1]=$2;sub(/^[ \t]+/,"",process[$1]);} END {if(process["VmSwap"] && process["VmSwap"] != "0 kB") printf "%10s %-30s %20s\n",process["Pid"],process["Name"],process["VmSwap"]}' '{}' \; | awk '{print $(NF-1),$0}' | sort -h | cut -d " " -f2-

As I mentioned previously, these programs had shifted some of their memory into swap space due to no memory being available on the system. This memory in swap is memory that is infrequently used.

3. Clear out Swap Space

If you find that your system is no longer running out of memory, or you want to reset your system now that you have setup monitoring (atop) to try and catch what might be triggering memory to go into swap. You can clear out the swap quickly by disabling your swap space and enabling it again.

The following command will turn of your swap space, be patient as it’s not instant and might take a couple of seconds for the kernel to move the processes that are using swap back into memory.

swapoff -a

Once completed, you can then turn your systems swap space back on using the following command

swapon -a

This should be quicker, as this is just enabling the swap space and not doing any work to move memory around.

4. Change Linux swappiness

Linux swappiness is a setting that tells your Linux system how much it should try to use the swap space. Here’s specific explanation of the actually Linux kernel parameter.

The vm.swappiness parameter in Linux controls how aggressively the kernel swaps memory pages out of physical memory to the swap space. This parameter can have a value from 0 to 100, which directly affects the balance between swapping out runtime memory and dropping pages from the system cache.

The swappiness setting can have a value from 0 to 100:

• A low value (close to 0) tells the system to try not to use the swap space unless it really has to. This is good for keeping things fast because using RAM is much faster than using swap space on a hard drive.
• A high value (close to 100) tells the system it’s okay to use the swap space more freely. This might help the system handle more programs at once, but it could slow things down if it uses the swap too much.

ChatGPT

By default vm.swappiness is set to 10 and is perfectly fine to keep this setting, unless you wanted to push the kernel to only use swap space when it really has to. You would then want to set vm.swappiness to 0. You can check what vm.swappiness is set to by running the following command

cat /proc/sys/vm/swappiness

If you want to change vm.swappiness to say 0 until next reboot (it will reset to the default value of 10) , then you would run the following command and then confirm with the previous command that it was set.

sudo echo 0 > /proc/sys/vm/swappiness

If you want to change vm.swappiness to 0 and have it persist between reboots, then you would want to run the following command.

sudo sysctl vm.swappiness=0

5. Reducing Memory Usage

There is no magical method to reduce memory usage, you really have to dig deep to figure out what needs to be optimized or removed to reduce your systems memory footprint. Here’s a couple of ideas.

Locate Processes Utilizing an Abnormal Amount of Memory

• Monitor lsphp or php-fpm processes, specifically the user account for each as this will identify a site that is possibly the culprit due to increased traffic or an attack. You want to ensure that all your sites are under their own system user, otherwise you can’t trace down what might be triggering an increase in memory usage.
• Look out for MySQL and it’s memory usage, it’s quite possible that MySQL is configured to user more memory than is required. You can use a tool called mysqltuner.pl that will provide configuration recommendations for MySQL. There is even a specific section on memory usage.

Historical Performance tracking using atop or Netdata

• Historical performance data is going to be key to find on time memory spikes on your system.
• atop is a linux shell command and is not as user friendly
• Netdata requires installation, but has a graphical user interface.

Stop Resource Spiking Requests

• Look at blocking requests that spike resource usage unnecessarily.
• Review your sites access log for requests to xmlrpc.php, wp-login.php and 404 requests that might be related to an automated scan.
• Employ Cloudflare and setup custom WAF rules like the ones decribed in the article I wrote. Secure, Protect and Lock Down your WordPress site with Cloudflare Custom WAF Rules (was Firewall Rules)
• Turn on any security features that block xmlrpc.php from your hosting control panel.

6. Adding more Memory

Depending on the size of your instance, upgrading your instance to have more memory will help in all cases. However it might simply just band aid the situation for the foreseeable future, only to crop up again in months time. Or it’s quite possible you undersized your instance, to begin with and it’s just now catching up with you.

Conclusion

This article will help you understand how Linux swap works, how to monitor it and actions you can take when you system is currently utilizing swap space for memory storage.