Professional Services
Custom Software
Managed Hosting
System Administration
See my CV here.
Send inquiries here.
Open Source:
tCMS
trog-provisioner
Playwright for Perl
Selenium::Client
Audit::Log
rprove
Net::Openssh::More
cPanel & WHM Plugins:
Better Postgres for cPanel
cPanel iContact Plugins
I recently had a problem with a hypervisor where the dom0 was underprovisioned for RAM, and it OOMKiller'd the hypervisor processes for VMs during scheduled rsync backups. After nice-ing the processes appropriately to prevent this I decided to implement a simple cron to email me whenever oomkiller fires. This obviously isn't foolproof, as something could be killed preventing outgoing mails. It is however unlikely, so this will likely be a good enough solution going forward.
oomdetect.sh
#!/bin/bash
touch /root/ooms.log
FSZ=$(stat --printf "%s" /root/ooms.log)
grep -i oom-killer /var/log/messages >> /root/ooms.log
echo $(uniq < /root/ooms.log) > /root/ooms.log
NEWSZ=$(stat --printf "%s" /root/ooms.log)
if [ $FSZ != $NEWSZ ]
then
echo "New OOM detected, investigate /var/log/messages:"
tail -n1 /root/ooms.log
fi
Pretty simple altogether, just make sure to run it once before you install it to root's crontab. Don't forget that you can send crons to multiple recipients by making the relevant updates to /etc/aliases for that cron's user and running newaliases.