RE: https://mementomori.social/@juergen_hubert/115984654197203481
( #WebsiteAdministration stuff)
One of the suggestions (by @ratsnakegames ) for figuring this website problem out was to look at the processes that are active at 23:00 UTC. Sadly, I haven't managed to do so, but I _was_ able to look at the processes with the "ps fax" command last Friday when an outage lasted until the morning hours.
For the sake of comparison, this is what the processes looked today when there wasn't an outage:
hosting-user@[...]/bin$ ps fax
PID TTY STAT TIME COMMAND
1 ? Ss 0:00 /usr/bin/dumb-init -- /sbin/paas-start --db mysql --engine php7
8 ? Ss+ 0:01 /bin/bash -e /sbin/paas-start --db mysql --engine php7
49 ? Sl 6553:45 /opt/mysql-5.7/sbin/mysqld --defaults-file=/srv/data/.config/mysql-5.7/my.cnf --basedir=/opt/mysql-5.7
3405 ? Ss 1:23 php-fpm: master process (/srv/data/.config/php-8.4/php-fpm.conf)
107940 ? S 1:33 \_ php-fpm: pool www
108136 ? S 1:15 \_ php-fpm: pool www
108137 ? S 1:57 \_ php-fpm: pool www
108138 ? S 1:18 \_ php-fpm: pool www
108139 ? S 1:19 \_ php-fpm: pool www
108140 ? S 1:19 \_ php-fpm: pool www
108141 ? S 1:13 \_ php-fpm: pool www
108686 ? S 0:10 \_ php-fpm: pool www
3409 ? Ss 0:55 /opt/apache2-2.4/usr/sbin/apache2 -d /srv/data/.config/apache -f apache2.conf
108896 ? Sl 0:00 \_ /opt/apache2-2.4/usr/sbin/apache2 -d /srv/data/.config/apache -f apache2.conf
57746 pts/0 Ss 0:00 bash /usr/local/bin/restart-shell hosting-user
57751 pts/0 S 0:00 \_ /bin/bash --rcfile /srv/admin/configs/bash/bash.bashrc -i
108963 pts/0 R+ 0:00 \_ ps fax
And this is what they looked during an outage:
hosting-user@[...]:/bin$ ps fax
PID TTY STAT TIME COMMAND
1 ? Ss 0:00 /usr/bin/dumb-init -- /sbin/paas-start --db mysql --engine php7
8 ? Ss+ 0:01 /bin/bash -e /sbin/paas-start --db mysql --engine php7
49 ? Sl 5430:08 /opt/mysql-5.7/sbin/mysqld --defaults-file=/srv/data/.config/mysql-5.7/my.cnf --basedir=/opt/mysql-5.7
3405 ? Ss 1:08 php-fpm: master process (/srv/data/.config/php-8.4/php-fpm.conf)
89733 ? D 0:00 \_ php-fpm: pool www
89734 ? D 0:00 \_ php-fpm: pool www
89735 ? D 0:00 \_ php-fpm: pool www
89736 ? D 0:00 \_ php-fpm: pool www
89737 ? D 0:00 \_ php-fpm: pool www
89738 ? D 0:00 \_ php-fpm: pool www
89739 ? D 0:00 \_ php-fpm: pool www
89740 ? D 0:00 \_ php-fpm: pool www
3409 ? Ss 0:45 /opt/apache2-2.4/usr/sbin/apache2 -d /srv/data/.config/apache -f apache2.conf
89150 ? Sl 0:00 \_ /opt/apache2-2.4/usr/sbin/apache2 -d /srv/data/.config/apache -f apache2.conf
89225 ? Sl 0:00 \_ /opt/apache2-2.4/usr/sbin/apache2 -d /srv/data/.config/apache -f apache2.conf
89505 ? Sl 0:00 \_ /opt/apache2-2.4/usr/sbin/apache2 -d /srv/data/.config/apache -f apache2.conf
89577 ? Sl 0:00 \_ /opt/apache2-2.4/usr/sbin/apache2 -d /srv/data/.config/apache -f apache2.conf
57746 pts/0 Ss 0:00 bash /usr/local/bin/restart-shell hosting-user
57751 pts/0 S 0:00 \_ /bin/bash --rcfile /srv/admin/configs/bash/bash.bashrc -i
89778 pts/0 R+ 0:00 \_ ps fax
Now, to my layperson's eye this is largely gibberish. But even I can see that there are four "apache2" processes running instead of one during the outage. Is this significant? And the timer for the "\_ php-fpm: pool www"-processes is at 0:00, too.
If anyone can help my aging brain figure out what is going on here, I'd much appreciate it. 😅