fogbound.net




Page 1 of 712345...Last »

Thu, 22 Sep 2016

Checking Solr index with nagios: obsolete versions

— SjG @ 12:33 pm

I needed to check that the index process that populates the Solr index succeeded and didn’t die during the night, leaving an empty index.

To make things more complicated, the versions of Solr and nagios in use are probably not the latest.

The check_solr -o numdocs command doesn’t work with our Solr configuration. But the internet tells me that the Solr query http://localhost:8983/solr/select/?debug=q‌uery&q=*:* includes the size of the result set. Testing it, I found this to be true:

<response>
   <lst name="responseHeader">
      <int name="status">0</int>
      <int name="QTime">0
      <lst name="params">
         <str name="q">*:*</str>
         <str name="debug">q‌uery</str>
      </lst>
   </lst>
   <result name="response" numFound="9832" start="0">
      <doc>
...

I want to use nagios to check that that numFound is never zero (or too small). I thought I’d just be able to use a nagios regex:

check_http -H localhost -p 8983 -u "/solr/select/?debug=query&q=*:*" -lr 'numFound=\"\d{2+}"'

It didn’t work. To make a long story short, there’s regex and then there’s regex. The kind that works for nagios is:

check_http -H localhost -p 8983 -u "/solr/select/?debug=query&q=*:*" -lr 'numFound=\"[1-9][0-9][0-9]'

This guarantees at least a hundred docs are in the index.


Mon, 29 Jun 2015

Sorting lots of files into directories, by date

— SjG @ 10:32 am

A process handles periodic tasks, and each time it does, it spits out some telemetry. The telemetry gets written off into a file each time. This process could be any of a number of interesting things: a procmail script doing something with incoming email, a Twitter-bot responding to searches, an IRC-bot responding to events, or whatever. The only important thing in this case is that it creates a variable number of log files each day, and the log files all get dumped into a single directory.

Before long, the log directory will be filled with thousands of files, and will be unmanageable. But, for some reason, we want to keep all these logs, and maybe actually use them. So the key is a script to move the logs into sub-directories based on date.

It turns out it’s easy to write a bash script to create directories in a YYYY-MM format, and move the files into them appropriately. The key is in the stat command. Conveniently, the implementation of this command is completely different and incompatible between Mac OS/BSD and Linux. Jesus H. Christ.

Linux:

#!/bin/bash
for i in *.log
do
filemonth=`stat --format=%y $i | cut -c 1-7`
mkdir -p $filemonth
mv $i $filemonth/$i
done

In Linux, the stat command will give you the data in an ISO format, and using the cut command, you can extract YYYY-MM information. The -p flag to mkdir makes it silently exit without complaining if a directory by that name already exists.

MacOS (or presumably other BSDs):

#!/bin/bash
for i in *.log
do
filemonth=`stat -f%Sm -t %Y-%m $i`
mkdir -p $filemonth
mv $i $filemonth/$i
done

In this case, we’re telling stat that we want the modified date as a string, and we specify the time format.

Either of these would be easy to modify to single day resolution (changing which columns you cut in the Linux version, or the timestamp format in the Mac version).


Tue, 2 Dec 2014

Compiling Kannel 1.4.4 under Centos 7.0

— SjG @ 4:28 pm

This took me while to get to work. If you follow these steps in order, it should work nicely.


# yum install mariadb-devel
# yum install libxml2-devel
# yum install bison
# yum install byacc
# cd /usr/local/src
# wget http://kannel.org/download/1.4.4/gateway-1.4.4.tar.gz
# tar xzvf gateway-1.4.4.tar.gz
# cd gateway-1.4.4
# ./configure --prefix=/usr/local/kannel --with-mysql --with-mysql-dir=/usr/include/mysql --disable-wap
# make

There are a few tricks here. First, just having libxml2 installed is not enough. You need the libxml development headers, etc. Should be obvious, but tricked me. Next, if you run ./configure before you have some of the dependencies installed (e.g., Bison), you will have modified source files that will still fail even after you install the dependency. Thus it’s important to install all that stuff before you run ./configure.

This stuff isn’t really that hard, but it can be time consuming to track down why it’s not working.


Tue, 22 Apr 2014

Annoying Xcode issue and resolution

— SjG @ 3:08 pm

I’d upgraded by home machine to Mavericks fairly soon after the OS was released, but hesitated in upgrading my main work machine. I didn’t want to have extensive downtime while tracking down odd dependencies and incompatibilities.

Well, time came to upgrade. It seemed safe. Everything was fine on my home machine. So I went ahead and upgraded my work machine.

Suddenly, I couldn’t get MacPorts to build MySQL.

I had followed the migration guide carefully. I tried all the usual tricks. In the port /opt/local/var/macports/build/…/config.log, the error was:

ld: library not found for -lcrt1.10.6.o

Google seemed to think this indicated that my Xcode command-line tools were not installed correctly. That library should be installed with all of the Unixy support that comes with Xcode’s command-line tools. Within the Xcode application, it told me that I had the command-line tools 5.1.1 (5B1008) properly installed. When trying various command-line options, the command-line tools were, in fact, installed. For example, xcrun gave the exact results one would expect.

Other tests also made it look like everything was good:
# xcode-select -p
/Applications/Xcode.app/Contents/Developer

Numerous Googled sites said to use the “xcode-select” command to install the tools if they were not functioning properly. Eventually, I gave in and tried it. Interestingly, this yielded an unexpected result:
# xcode-select --install
xcode-select: error: no developer tools were found, and no install
could be requested (perhaps no UI is present), please install
manually from 'developer.apple.com'.

Since I had originally installed Xcode under Leopard from a downloaded package rather than recently through the App store, I thought perhaps in the various upgrades something had gotten messed up. I decided to completely uninstalled Xcode:

/Developer/Library/uninstall-devtools --mode=all
I also removed the vestigial /Developer directory, and the /Application/Xcode directory.

I re-installed Xcode from the App store. Everything functioned and/or failed exactly as it had before.

In desperation, I downloaded the Mavericks command line tools package from Apple, and installed it. It should be the same thing as what was installed with Xcode. But it evidently is not, because now I can build the MacPort for MySQL.

edit/update: It may not be clear above, but normally doing the “xcode-select –install” is all that’s needed. It’s also not stated above, but I tried that after re-installing Xcode from scratch, and had the same issue. Evidently, whatever was mis-configured on my machine is quite rare.

Also, don’t trust Xcode when it tells you that it’s installed the command-line tools in the preferences panel like this: Locations_and_Macintosh_HD
It’s probably lying to you – it’s installed stubs, but not the actual tools.


Fri, 6 Dec 2013

Another SELinux Lesson

— SjG @ 6:17 pm

So there’s this project that requires a ridiculously complicated communication protocol involving lots of byte-wrangling and formatting and weird transports. For the sake of brevity, I’ll only mention one of the endpoints, which requires decoding an email attachment.

Of course, this means a procmail script sending the email to PHP for processing and much ceremonious mucking about. On first go, it was failing. In debug mode, procmail was telling me that permission was denied. But it wasn’t user permissions: the file was owned by the same user as receiving the email and running the procmail script.

Naturally, when faced with cryptic permission failures, the first thing I did was look at /var/log/audit/audit.log and /var/log/messages for SELinux denials. There I found nothing at all. No errors, no warnings, no ugly “avc: denied” splatters.

Finally, this page here explained it to me. Rebuilding the policies with semodule -DB quickly revealed that my problem was, in fact, SELinux (as it all so often is). Once I could see the policies that were marked “dontaudit,” it was just an hour of building more and more complicated policies for procmail before stuff started working.

Once everything was good, happy, and shiny, a simple semodule -B returned the SELinux logging to the previous state, and I could once again spend my time fighting the convoluted bit-twiddling of the communication protocol.


Page 1 of 712345...Last »