I ran into a problem where the default installer for RHEL had configured a server’s partitions in a way that was really suboptimal for how the server was being used. It looks like the default was intended for workstation use or one in which a lot of users would keep their stuff in home directories, but this system is being used as a web application server. The root partition filled up, and the database (with its files in /var) got quite unhappy.
[root@epicwin]# df -h
Filesystem Size Used Avail Use% Mounted on
50G 47G 0 100% /
tmpfs 1.9G 76K 1.9G 1% /dev/shm
/dev/sde1 485M 80M 380M 18% /boot
532G 198M 505G 1% /home
So I had to learn some LVM basics, and move the space around. I found various instructions out there, but many of them were missing steps or glossed over whether or not I’d suffer data loss. Here’s how I solved my problem (without data loss!) and got the system back on its feet.
- Boot into “rescue” mode.
- Decline to mount the existing system or filesystem for repair, instead go straight to a root shell.
- Activate the LVM system:
lvm vgchange -a y
- Make sure the filesystems are clean:
e2fsck -f /dev/mapper/vg_epicwin-lv_home
e2fsck -f /dev/mapper/vg_epicwin-lv_root
- Reduce the size of the lv_home logical volume from 540G to 40G:
resize2fs /dev/mapper/vg_epicwin-lv_home 40g
lvreduce -L 40g /dev/mapper/vg_epicwin-lv_home
- Expand the size of the lv_root logical volume from 50G to 550G:
lvextend -L +500g /dev/mapper/vg_epicwin-lv_root
- Expand the filesystem on the root logical volume:
- Exit shell / reboot.
- Watch SELinux freak out on next boot:
*** Warning -- SELinux targeted policy relabel is required. ***
*** Relabeling could take a very long time, depending on file ***
*** system size and speed of hard drives. ***
- Watch SELinux print asterisks and then the system reboot unexpectedly in a way that looks quite disturbingly bad.
- System comes up clean!
A few notes:
If /home had a lot of stuff in it, this would obviously be a lot more complicated. This approach works because I had the space and it wasn’t being utilized. if /home had been closer to full, the approach would have been to add a new physical drive, and it to the logical volume set, and then do something similar to the above.