fogbound.net




Fri, 12 Oct 2012

Resizing LVM partitions

— SjG @ 10:30 am

I ran into a problem where the default installer for RHEL had configured a server’s partitions in a way that was really suboptimal for how the server was being used. It looks like the default was intended for workstation use or one in which a lot of users would keep their stuff in home directories, but this system is being used as a web application server. The root partition filled up, and the database (with its files in /var) got quite unhappy.

[root@epicwin]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_epicwin-lv_root
50G 47G 0 100% /
tmpfs 1.9G 76K 1.9G 1% /dev/shm
/dev/sde1 485M 80M 380M 18% /boot
/dev/mapper/vg_epicwin-lv_home
532G 198M 505G 1% /home

So I had to learn some LVM basics, and move the space around. I found various instructions out there, but many of them were missing steps or glossed over whether or not I’d suffer data loss. Here’s how I solved my problem (without data loss!) and got the system back on its feet.

  • Boot into “rescue” mode.
  • Decline to mount the existing system or filesystem for repair, instead go straight to a root shell.
  • Activate the LVM system:
    lvm vgchange -a y
  • Make sure the filesystems are clean:
    e2fsck -f /dev/mapper/vg_epicwin-lv_home
    e2fsck -f /dev/mapper/vg_epicwin-lv_root
  • Reduce the size of the lv_home logical volume from 540G to 40G:
    resize2fs /dev/mapper/vg_epicwin-lv_home 40g
    lvreduce -L 40g /dev/mapper/vg_epicwin-lv_home
  • Expand the size of the lv_root logical volume from 50G to 550G:

    lvextend -L +500g /dev/mapper/vg_epicwin-lv_root

  • Expand the filesystem on the root logical volume:

    resize2fs /dev/mapper/vg_epicwin-lv_root

  • Exit shell / reboot.
  • Watch SELinux freak out on next boot:

    *** Warning -- SELinux targeted policy relabel is required. ***
    *** Relabeling could take a very long time, depending on file ***
    *** system size and speed of hard drives. ***

  • Watch SELinux print asterisks and then the system reboot unexpectedly in a way that looks quite disturbingly bad.
  • System comes up clean!

A few notes:
If /home had a lot of stuff in it, this would obviously be a lot more complicated. This approach works because I had the space and it wasn’t being utilized. if /home had been closer to full, the approach would have been to add a new physical drive, and it to the logical volume set, and then do something similar to the above.


Mon, 6 Aug 2012

Sneaky, sneaky SELinux

— SjG @ 4:38 pm

I was getting a web site up and running on Centos 6.3, and PHP was refusing to talk to the SMTP server on the same box. This, in and of itself, is not unusual. There’s a checklist:

  • Verify SMTP’s running
  • Check that the SMTP server is configured to accept connections from localhost
  • Check the firewall rules
  • Really check the firewall rules by generating an email using telnet localhost 25 or nc localhost 25 from the command line
  • Check that PHP is trying to talk to the SMTP server you think it is
  • Look in the system log for SELinux being a bully and preventing communication

In this case, all of the above checked out cleanly. I was perplexed. But it turns out that, in what is as shocking and surprising as my cat wanting to be fed,
SELinux was indeed the culprit. It just wasn’t logging the problem anywhere.

A quick
setsebool -P httpd_can_network_connect=1

as root solved the problem.

Moral(s) of the Story: The cat is always hungry. SELinux is always fuckin’ shit up.


Sun, 22 Jul 2012

Teensy board example code “usb_debug_only.c” and gcc > 4.6

— SjG @ 9:44 pm

Wow, now there’s a blog title that will have very limited appeal 🙂

I was trying to validate my toolchain worked for building Teensy/ATMEGA32u4 applications. I used the mac port avr-gcc, avr-bintools, avr-libc install, and it failed to build the sample “blinky.c” program. I tried using the recommended AVR CrossPack, just in case it had different versions of the tools, but I got the same results:


samuelg$ make
-------- begin --------
avr-gcc (GCC) 4.7.0
Copyright (C) 2012 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Compiling C: usb_debug_only.c
avr-gcc -c -mmcu=atmega32u4 -I. -gdwarf-2 -DF_CPU=16000000UL -Os -funsigned-char -funsigned-bitfields -ffunction-sections -fpack-struct -fshort-enums -Wall -Wstrict-prototypes -Wa,-adhlns=./usb_debug_only.lst -std=gnu99 -MMD -MP -MF .dep/usb_debug_only.o.d usb_debug_only.c -o usb_debug_only.o
usb_debug_only.c:96:24: error: variable ‘device_descriptor’ must be const in order to be put into read-only section by means of ‘__attribute__((progmem))’
usb_debug_only.c:113:24: error: variable ‘hid_report_descriptor’ must be const in order to be put into read-only section by means of ‘__attribute__((progmem))’
usb_debug_only.c:128:24: error: variable ‘config1_descriptor’ must be const in order to be put into read-only section by means of ‘__attribute__((progmem))’
usb_debug_only.c:175:52: error: variable ‘string0’ must be const in order to be put into read-only section by means of ‘__attribute__((progmem))’
usb_debug_only.c:180:52: error: variable ‘string1’ must be const in order to be put into read-only section by means of ‘__attribute__((progmem))’
usb_debug_only.c:185:52: error: variable ‘string2’ must be const in order to be put into read-only section by means of ‘__attribute__((progmem))’
usb_debug_only.c:198:11: error: variable ‘descriptor_list’ must be const in order to be put into read-only section by means of ‘__attribute__((progmem))’
make: *** [usb_debug_only.o] Error 1

Google didn’t immediately help me with this. Eventually, I actually went ahead and read the error messages, and then tried following directions, i.e., I looked at line 96, and changed


static uint8_t PROGMEM device_descriptor[] = {

to


static const uint8_t PROGMEM device_descriptor[] = {

That’s all it took to make gcc happy with that line. Repeat the process for all of the error lines, and it compiles and builds correctly.

Evidently, the change to gcc happened between 4.5.3 and 4.6, and, once I knew what to look for, I found numerous discussions of why and wherefore (although these discussions were quite frankly beyond my comprehension). I was, however, happy to find the simple solution detailed above.


Wed, 13 Jun 2012

Building Direct IO library for PHP with Mac Ports

— SjG @ 11:47 am

Say you’re developing on a Mac, and want to test some PHP code that has calls to the direct IO library. You may not actually have a physical serial port, but your unit tests will fail in the wrong way if the library isn’t present. You want the unit tests to fail in the right way!

If you do the expected thing, you’ll find that dio is currently beta:

root# pecl install dio
Failed to download pecl/dio within preferred state "stable", latest release is version 0.0.5, stability "beta", use "channel://pecl.php.net/dio-0.0.5" to install
install failed

So you have to try to do it the hard way:

samuel:~ root# pecl install channel://pecl.php.net/dio-0.0.5
downloading dio-0.0.5.tgz ...

Easy-peasy, eh? Not so fast!


/private/tmp/pear/temp/dio/dio.c: In function 'zif_dio_fdopen':
/private/tmp/pear/temp/dio/dio.c:138: error: 'EBADFD' undeclared (first use in this function)
/private/tmp/pear/temp/dio/dio.c:138: error: (Each undeclared identifier is reported only once
/private/tmp/pear/temp/dio/dio.c:138: error: for each function it appears in.)
make: *** [dio.lo] Error 1
ERROR: `make' failed

Fortunately, via this bug report, we can see what to do:

# pecl download channel://pecl.php.net/dio-0.0.5
# tar xzvf dio-0.0.5.tgz
# cd dio-0.0.5
# phpize
# ./configure

Then edit dio.c, and change line 138 to:

if ((fcntl(fd, F_GETFL, 0) == -1) && (errno == EBADF)) {

Then, finish up:

# make
# make install

Then, create a file called “dio.ini” in /opt/local/var/db/php5/ containing:

extension=dio.so

Now you can run your tests!


Tue, 12 Jun 2012

Spelling Suggestions from Solr using Yii-solr

— SjG @ 7:48 pm

So a Yii app wants to query Solr using the Yii-solr extension, and wants to provide spelling suggestions for the provided search term. We need the suggestions to be unique, i.e., no repeated suggesttions, and we don’t want negative suggestions, i.e., if the user’s search wanted to exclude “foo”, we don’t want to suggest “foo” as a possible correction. There’s probably a better way to do all this, but I couldn’t figure it out. Here’s what I did instead (as always, forgive the code style/formatting):

I created a subclass of ASolrConnection that switches to using the “spell” Solr servlet:


class SolrSpellingConnection extends ASolrConnection {
public $servlet_type;
public $servlet_path;
public function resetClient()
{
parent::resetClient();
if (isset($this->servlet_type) && isset($this->servlet_path))
{
$this->_client->setServlet($this->servlet_type,$this->servlet_path);
}
}
}

This is instantiated in the app’s config file:

'solr' => array(
'class'=>'app.components.SolrSpellingConnection',
'clientOptions'=>array(
'hostname'=>'localhost',
'port'=>8983,
),
'servlet_type'=>SolrClient::SEARCH_SERVLET_TYPE,
'servlet_path'=>'spell'
),

Then, my Solr query data-provider looks something like:


$criteria = new ASolrCriteria;
$criteria->query = $queryterm;
$criteria->set('spellcheck','true');
$pages=array('pageSize'=>15);
$dataprovider = new ASolrDataProvider(ASolrDocument::model(),array('criteria'=>$criteria,'pagination'=>$pages));
return $dataprovider;

In my controller, I add some code to process the suggestions:

public function buildSuggestionList($dataprovider, $originalQueryString)
{
$suggestions = array();
$terms = array();
if ($dataprovider != null)
{
$facets = $dataprovider->getQueryFacets();
$resp = $dataprovider->getSolrQueryResponse()->getSolrObject();
if (isset($resp['spellcheck']))
{
if (isset($resp['spellcheck']['suggestions']))
{
foreach ($resp['spellcheck']['suggestions'] as $thisSuggest)
{
if (is_object($thisSuggest) && get_class($thisSuggest) == 'SolrObject')
{
if (isset($thisSuggest['suggestion']) && is_array($thisSuggest['suggestion']))
{
foreach($thisSuggest['suggestion'] as $thisTerm)
{
$terms[$thisTerm]=1;
}
}
}
}
}
}
}
foreach($terms as $termKey=>$val)
{
// Solr adds negated or added terms to suggestions if they don't match case
if (!preg_match('/\b'.$termKey.'\b/i',$originalQueryString))
{
$suggestions[] = CHtml::link($termKey,'/index.php/mysearchcontroller?q='.urlencode($termKey));
}
}
return $suggestions;
}

I can then display the main collection of results using a CWidget. For displaying suggestions, however, a bit of specific code is required:


$suggestions = $this->buildSuggestionList($dataprovider, $originalQueryString);
if (count($suggestions) > 0)
{
echo '<p>Did you mean '.implode(' or ',$suggestions).'?</p>';
}
?>

When this is all in place, you should get decent suggestions from Solr. Of course, you will need to have Solr build a spelling index! The easiest way to do that is simply connect to Solr’s web interface and tell it to build the index:

http://localhost:8983/solr/spell?spellcheck.build=true