Again I go unnoticed

I’m awaiting a dedicated database-server for the websites. In order to set up a secure and fast database-connection in a distributed environment, I played around with mysql-connections over an SSH-tunnel. I used this really good tutorial (German though) . I will also set up a watchdog for watching and restarting the link.

The new server will serve as primary database-server and all connections will be tunneled through SSH. The website the servers are for broke through the 90.000 visitor/day barrier on tuesday and it’s still growing …

Last week I switched the websites whose servers I administer from Apache to Lighttpd 1.3.13. The servers ran under really heavy load (about 100 on single-processor x86 machines). The reasons fpr choosing an innovative solution like lighttpd over somethin like mod_backhand were as follows:

• lighttpd has a very small memory footprint. Though we are running on 2 gigs of RAM, the mysql-databases had to swap a lot. Now there’s nearly no more swapping
• Load-balancing with FastCGI. Now it’ll be easy to integrate more hardware just running PHP-FastCGI-processes
• It’s a matter of seconds to compile a current version of PHP for use as FastCGI (it’s easier than compiling it into apache)
• lighttpd is two times as fast in serving static pages and pictures. Those are about 50% of the served content.

The switching process was kind of hard during to the fact that lighttpd does not read .htaccess – files. So all access restricted areas had to be transformed into the lighttpd configuration file. After switching the server load dropped dramatically on both machines (from about 20 to 4). Now, with more visitors attracted by the website, the load has gone up to 8 to 9 again.

The load corresponds directly to the number of spawned FastCGI PHP processes. I started with 15 simultaneous processes, went up to 140 and am now at about 80 per machine: Many processes not serving content comsume way too much system load.

I’ll write a more detailed case-study this month to support spreading this heavy-load http-daemon.

I began experimenting with AWStats on Saturday and installed it on the main machine. I wanted it to parse the logfiles just like Webalizer does and added it to the cronjob to parse the logfiles once a day.

I wanted to create Webalizer like static pages and that once a day. I installed awstats on Debian using a Debian-package from backports.org and thanks to the help of this tutorial AWStats is now producing more detailed statistics than Webalizer does.

I noticed the following.

• AWStats is goddamn slow. OK, the logfiles are huge (about a gig per day) but it takes about on hour to parse one file. Here the Webalizer C-code is way ahead of the Perl-code AWStats is written in
• The total visitor count differs from the one Webalizer created. It differs not by some hundred visitors, using the logs of the website I’m administering Webalizer counted ~80.000 visitors per day, AWStats just ~ 57.000 … huh!

Argh! I’m desperately trying to get a lighttpd running to serve as a proxy in front of an Apache webserver. I just gave up, right now there’s no chance, cause the lighttpd does Proxy-Requests using HTTP/1.0 and not 1.1. On top of that i noted it has a strange behaviour as I began digging deep in the magic IP-box.

I tried to manage to establish different proxy connections on different local IPs (range 127.0.0.x) but lighttpd keeps on connectiing on 127.0.0.1 .. always! I tried to bind different virtual hosts to different ports but realized, that it’s not worth the effort.

Virtual hosting is simply not possible behind an 1.0 proxy, so I’ll set up the whole website on pure lighttpd as soon as the new database-server arrives.