TL;DR The OS is responsible for managing the memory, including allocation, deallocation, defragmentation, and reclaim.
Buddy system
The buddy system reduces fragmentation to some degree. At least smaller blocks can be merged at times to serve larger requests.
Let's first see how memory can get fragmented. Say you are managing 16K of memory
| Initial state | <-----------------------16K-----------------------> |
| Allocate 3 * 4K |
<-----------------------16K-----------------------> <----------8K-----------><-----------8K----------> <----4K----><----4K----><----4K----><----4K---> |
| Release the second block |
<-----------------------16K-----------------------> <----------8K-----------><-----------8K----------> <----4K----><----4K----><----4K----><----4K---> |
| Allocate 1 * 8K | Request can't be fulfilled unless the allocated memory gets defragmented. |
The memory compaction process:
All memory allocated by the processes is supposed to be used; the OS can't simply take them away. What the OS can do is offload some pages onto the swap area of the disk. This is done by a memory reclaim process. When the program requests the data of a swapped page, the OS can swap it back into memory.
But if the program constantly requests them back and fights against the reclaim process, this is called page thrashing which slows down the entire system.
Watermarks
Check out the diagram in https://www.kernel.org/doc/gorman/html/understand/understand005.html
The vm.watermark_boost_factor setting can cause more aggressive memory reclaim behavior. By tuning up the watermarks, it is meant to claim more memory proactively with fewer attempts, which might improve performance.
Problems:
Setting it to 0% disables the feature.
https://www.kernel.org/doc/gorman/html/understand/