I ran into a problem where the default installer for RHEL had configured a server’s partitions in a way that was really suboptimal for how the server was being used. It looks like the default was intended for workstation use or one in which a lot of users would keep their stuff in home directories, but this system is being used as a web application server. The root partition filled up, and the database (with its files in /var) got quite unhappy.

[root@epicwin]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_epicwin-lv_root
50G 47G 0 100% /
tmpfs 1.9G 76K 1.9G 1% /dev/shm
/dev/sde1 485M 80M 380M 18% /boot
/dev/mapper/vg_epicwin-lv_home
532G 198M 505G 1% /home


So I had to learn some LVM basics, and move the space around. I found various instructions out there, but many of them were missing steps or glossed over whether or not I’d suffer data loss. Here’s how I solved my problem (without data loss!) and got the system back on its feet.

• Boot into “rescue” mode.
• Decline to mount the existing system or filesystem for repair, instead go straight to a root shell.
• Activate the LVM system:
  lvm vgchange -a y

• Make sure the filesystems are clean:
  e2fsck -f /dev/mapper/vg_epicwin-lv_home
e2fsck -f /dev/mapper/vg_epicwin-lv_root

• Reduce the size of the lv_home logical volume from 540G to 40G:
  resize2fs /dev/mapper/vg_epicwin-lv_home 40g
lvreduce -L 40g /dev/mapper/vg_epicwin-lv_home
• Expand the size of the lv_root logical volume from 50G to 550G:

  lvextend -L +500g /dev/mapper/vg_epicwin-lv_root

• Expand the filesystem on the root logical volume:

  resize2fs /dev/mapper/vg_epicwin-lv_root

• Exit shell / reboot.
• Watch SELinux freak out on next boot:

  *** Warning -- SELinux targeted policy relabel is required. ***
*** Relabeling could take a very long time, depending on file ***
*** system size and speed of hard drives. ***

• Watch SELinux print asterisks and then the system reboot unexpectedly in a way that looks quite disturbingly bad.
• System comes up clean!

A few notes:
If /home had a lot of stuff in it, this would obviously be a lot more complicated. This approach works because I had the space and it wasn’t being utilized. if /home had been closer to full, the approach would have been to add a new physical drive, and it to the logical volume set, and then do something similar to the above.

I have to admit, this latest presidential election has me perplexed¹. Why is there even a contest? The Democrats love Obama, and the Republicans should too. After all, the Obama administration is the apotheosis of traditional supply-side GOP ideals — a Republican wet dream. “Wait. What?” I hear you say. Turn off the partisan filters for a moment, and consider the following facts:

• Corporate profits are at an all-time high.²
• CEO salaries are the highest ever.³
• Wall Street salaries are close to the highest they’ve ever been.⁴
• The Stock Market is within a few percent of its all-time high.⁵
• Interest rates are the lowest on record.⁶
• Corporate Taxes are at the lowest effective rate ever (recording begun in 1947).⁷
• Tax rate for highest earners is the lowest it’s been since 1931.⁸
• The US Defense budget is at an all-time high.⁹
• There are more deportations of illegal immigrants than ever.¹⁰
• More terrorists / enemy militants killed per year than ever before¹¹

So, given this list, why is the current administration so hated by the American Right and so loved by the American Left? We have, in essence, a pitched battle between two symbols, which we choose to take at face value. Never mind that the actuality is nothing like what it says on the label¹².

¹ OK, not really. But the phrase works better, rhetorically. This entire posting should be considered in that light.

²“Corporate Profits Just Hit An All-Time High, Wages Just Hit An All-Time Low”, Business Insider Magazine, 22 June 2012.
US Department of Commerce, Bureau of Economic Analysis.

³“Historical Trends in Executive Compensation” (PDF), 1936-2003, Carola Frydman and Raven E. Saks, November 15, 2005.
“C.E.O. Pay Is Rising Despite the Din,” NY Times, 16 June 2012.
United States Bureau of Labor Statistics, Occupational Employment Statistics survey for 2001-2010.

⁴“Wall Street Pay: A Record $144 Billion, Financial Overhaul Has Affected Structure but Not Level; Revenue-to-Compensation Ratio Stays Flat”, Wall Street Journal, October 11, 2010.
“The Giant Pay Gap Between Wall Street And Everyone Else Isn’t Going Anywhere”, Business Insider Magazine, October 10, 2012.

⁵The Dow Jones Industrial Average was 13,494.6 on 3 Oct 2012, the all time high was 14,164.53, so it’s only 4.8% down from the highest it’s ever been. The S&P 500 was 1,450.99 on 3 Oct 2012, the all time high was 1,576.09, so that’s a bigger gap, but it’s still only 8% down from the all-time high.

⁶“Mortgage rates now below even lows of early 1950s”, Associated Press, 30 Sept 2011.
National Bureau of Economic Research, Macrohistory: Interest Rates.

⁷St. Louis Federal Reserve Bank, FRED® Economic Data, “Graph: Federal Government: Tax Receipts on Corporate Income (FCTAX)/(Corporate Profits After Tax (CP)+Federal Government: Tax Receipts on Corporate Income (FCTAX))”

⁸Historical Top Tax Rate, Tax Policy Center of the Urban Institute and Brookings Institution.

⁹Government Printing Office, Budgets
US Census Bureau historical records

¹⁰“President’s Approval Rating Drops, but He Leads 2012 Rivals. As Deportations Rise to Record Levels, Most Latinos Oppose Obama’s Policy” PEW research, 28 Dec 2011.
Numbers may have been inflated by changes in how they’re counted; even so, deportations are still higher than Bush numbers. “Deportation statistics said to be inflated”, Washington Times, 23 August 2012

¹¹“The Year of the Drone. An Analysis of U.S. Drone Strikes in Pakistan, 2004-2012” New America Foundation

¹²And just so you don’t think I’m picking on the Democrats alone, I only point out the one side as it’d take a whole additional posting to show that the Republicans are the “spend and borrow” party, etc.