fogbound.net




Wed, 14 Jun 2023

pamd running out of sessions for cron

— SjG @ 6:51 am

I manage a very busy Rocky Linux test-server. For one staging environment, cron is already running five or six maintenance scripts every minute. But when the tests run, the system has to do a lot of additional permissions fixes and filesystem adjustments. I’ve started seeing in the logs the following error message:

pam_systemd(sudo:session): Failed to create session: Maximum number of sessions (8192) reached, refusing further sessions

Now, there is a known older problem with systemd and dbus, that comes up when you search for this error message. I couldn’t find any concrete actions I could take to fix the issue. The other major search results are RedHat pages behind their subscription wall, and, at this point I’m apparently too dumb and out of date to even be able to figure out how to pay for a RedHat subscription.

I think I’ve found at least a temporary solution, however. In /etc/systemd/logind.conf there is the SessionsMax field where you can override the default. I doubled it to 16384, then ran systemctl restart systemd-logind

I’ll have to see if that’s a viable long-term fix rather than just treating the symptoms of a bigger issue.


Thu, 20 Apr 2023

Taming logwatch on Linux*

— SjG @ 6:59 am

*This is actually on Rocky Linux / CENTOS / RHEL, but will likely work on others.

Logwatch can be a nice tool for keeping an eye on your servers. It goes through your logs and creates a nightly aggregate email containing information to keep you apprised of various important details. It can be good to bring things to the attention of lazy / overwhelmed sysadmins like me.

Where it fails, though, is where it overwhelms you with useless information. There are different output level settings, and if you turn the detail levels down far enough, it helps a lot. However, with certain configs and certain OSes, you still get overwhelmed with non-actionable information. Here’s how to fix a few of those.

Crontabs. In Rocky Linux, cron logs a success message that contains a process number, which means the default log is filled with lots and lots of lines like session-685197.scope: Succeeded.: 1 Time(s) which logwatch happily throws into the nightly email. Most searches tell you to edit your /etc/logwatch/conf/ignore.conf file and add the following line:

session-.*scope: Succeeded

This doesn’t work for me. Further research indicates that the ignore.conf file wants a Perl-style regular expression. The recommendation above is sort-of-Perlish, but what ended up working correctly for me was putting the following line in my ignore.conf:

\s*session-(.*?)\.scope: Succeeded\.(.*)

HTTP. For some reason, someone thought having a long list of hostile IP addresses would be helpful. Maybe to manually block them? Seems like a hopeless task. Check out /usr/share/logwatch/scripts/services/http around line 596… and un-comment out the conditional.

$flag = 1;
foreach my $i (sort keys %ban_ip) {
   if ($flag) {
      print "\nA total of ".scalar(keys %ban_ip)." sites probed the server \n";
      $flag = 0;
   }
   #if ($detail > 4) {
      print "   $i\n";
   #}
} 

sshd. I know there are a lot of hackers, script kiddies, and bots out there. I don’t need to see the long list of people who tried and failed to log in with ssh. Unfortunately, the detail level setting for sshd aren’t very helpful. I ended up editing /usr/share/logwatch/scripts/services/sshd and liberally sprinkling my own if ($Detail > 4) {} barriers starting around line 500. Hacky, I know. Also will be clobbered with the next logwatch update. Yuck.

Maybe it’s time for me to submit a bunch of pull requests.


Sun, 9 Apr 2023

Ugh, that Old Yak-shaving Refrain

— SjG @ 12:28 pm

Went to update an image gallery with some new pictures. I wrote the gallery generating code four years ago, but the version of PHP on the server has been updated and one of the libraries I use evidently relies on deprecated syntax ($string{$char_index} for the curious, which now would be $string[$char_index]). So I tried to check out the repo from GitHub, only to find they’ve updated their SSH host keys, so I need to fix that before I can fix the code problem. So I tried to update that, but had packed my Yubikey off in a drawer in the office…


Mon, 30 Jan 2023

Back on my BS again

— SjG @ 8:45 pm

Which BS is that, I hear you ask? With so many possible domains to choose from?

It’s my interactive geometry tiling BS this time. In the simplest description, the program should provide a triangle and allow the user to draw on it, and then expand that into a tiling across a whole plane. Eventual goals include different kinds of triangles and tilings (30-60-90 triangles into hexagons, 45-45-90 triangles into squares, and then later stuff that’s not quite as regular and that will form octagon/square tilings, and so on), weaving of the lines that are drawn, and so on.

It’s sort of embarrassing to see how many times I’ve gone and worked on this same basic idea without ever having really completed it. I remember doing stuff in Turbo-C on a DOS PC for drawing in VGA resolutions (ooh! ahh!) in the early 90s, and then again in a Java AWT applet in the late 90s. I have various residues of Adobe JSX scripts to do similar stuff in Photoshop and Illustrator, trying to use their discontinued ExtendScript interface. I have an attempt written in Processing. I have at least two previous attempts in JavaScript using Canvas.

Well, here I am, doing it in JavaScript and (quelle nouveauté!) SVG.

So far, I have the 30-60-90 triangles working. I can set pen widths and colors. I can save/load works-in-progress, and render and export the SVG files. I don’t yet have editing or removal of lines implemented. I haven’t even started thinking about the weaving problem again.

Watch this space (but not too closely) for updates. Maybe I’ll actually finish this time ’round.


Sun, 1 Jan 2023

New Year New Fear

— SjG @ 11:03 am

I thought I’d update this VPS from Ubuntu 20.04.5 LTS to Ubuntu 22.04.1 LTS. I ran do-release-upgrade which gave all the appropriate warnings and stuff, and proceeded to upgrade. This VPS is hosted on Linode, so there’s a local Ubuntu mirror — the download is blindingly fast.

The do-release-upgrade process appears to run in a Unix screen window. Unfortunately, that interacted poorly with the terminal I was using, so any time I moved my mouse, it spewed control characters as input. That was bad when it was prompting … something? … probably about my nonstandard sshd configuration.

Anyway, Linode provides a web-based console access, so I was eventually able to fix the broken sshd, PHP, and Apache configurations. Only about two hours of downtime. Not bad for New Year’s Day.