Hostnames in Logwatch reports

Where I work, we have a lot of servers to maintain, and only 2 server admins (me and my colleague). We use Nagios to keep us informed about the server status and Logwatch to analyze to server logs on a daily basis.

We have per server a lot of subdomains/vhosts and these virtual hosts all write into their own log (blog.jachim.be_acces_log, www.jachim.be_error_log, etc…).

The log entries look like this:

192.168.200.6 - - [10/Nov/2009:09:55:41 +0100] "GET /a/i/red_cube.png HTTP/1.0" 200 190
192.168.200.6 - - [10/Nov/2009:09:55:41 +0100] "GET /a/i/search/search_icon.gif HTTP/1.0" 200 428
192.168.200.6 - - [10/Nov/2009:09:55:41 +0100] "GET /index.php HTTP/1.0" 200 6541

When Logwatch merges all the httpd log files, the host information (in the log filename) is lost, resulting in Logwatch reports like this:

Requests with error response codes
    401 Unauthorized
       /: 4 Time(s)
       /a/i/blue_cube.png: 1 Time(s)
       /favicon.ico: 2 Time(s)
       /wp/login: 2 Time(s)
...

We actually want reports like this:

Requests with error response codes
    401 Unauthorized
       www.jachim.be/: 4 Time(s)
       jachim.be/a/i/blue_cube.png: 1 Time(s)
       blog.jachim.be/favicon.ico: 2 Time(s)
       blog.jachim.be/wp/login: 2 Time(s)
...

Now we have all the information we want and are able to fix the possible problems much easier.

Because this is not possible in Logwatch (see mailinglist), I’ve added it in the Apache logs.

I’ve added a new logformat named logwatch in httpd.conf:

LogFormat "%h %l %u %t \"%m %{Host}i%U%q %H\" %>s %b" logwatch

Now the new format is available and can be used in the Virtual Host:

CustomLog logs/www.jachim.be-access_log logwatch

Resources:

Random PHP function explained

file1_Open_Source_PHP_logo_667A few weeks ago, Jamie Bicknell notified me he read my article about the Random PHP function. Because I didn’t publish any code, he wrote a 3 line implementation of how he would fetch the functions for the randomness.

I promised him I would publish the code that weekend, and here we are, a few weeks later…

The script actually parses the http://php.net/quickref.php page, saves the result, and redirects the user.

$cacheFile './functions.tmp';
if (!
is_file($cacheFile)) {
    
$quickref 'http://php.net/quickref.php';
    
$xml DOMDocument::loadHTMLFile($quickref);
    
$links $xml->getElementsByTagName('a');
    
$functions = array();
    foreach (
$links as $link) {
        if (
substr($link->getAttribute('href'), 08) == '/manual/') {
            
$functions[$link->nodeValue] = $link->getAttribute('href');
        }
    }
    
file_put_contents($cacheFileserialize($functions));
} else {
    
$functions unserialize(file_get_contents($cacheFile));
}
$function $functions[array_rand($functions)];
$function 'http://php.net' $function;
header('Location: ' $functiontrue303);
die();

Some improvements:

  • Invalidate the cache file when a new PHP version is announced.
  • Read random lines from the file (instead of loading the complete array in the memory).

Update: Forgot the mention the actual blogpost of Jamie Bicknell…

phpBenelux

phpbenelux_logoAs you can see in my Blogroll panel, on my homepage, profile and LinkedIn, I’m a proud member of phpBenelux.

Tuesday there was another meeting, this time in the offices of Netlog, with talks from Ivo Jansch about “PHP and the Cloud” and Felix De Vliegher about “High gear PHP with Gearman”.

Because it took me quite a while to figure out what user group was/is the real one and what’s the plan for the future, I wanted to share my “knownledge” with you:

  • phpBenelux is the new name after phpBelgium and phpgg merged.
    I’m still waiting for Luxembourg, but they’ll come along :).
  • Meetings are held monthly, one month in the Netherlands, the other month in Belgium.
  • A new website (phpBenelux.eu) is on the way, and I’m offering my help, together with my colleague Jochen.

I will post updates on the list above, because there are still some questions that I haven’t figured out yet:

  • How the website is working.
  • What should have happened, after I payed € 15.
  • Why the “statutes” are still those of phpgg.

That being said: phpBenelux is a great user group, where I’ll offer my help to expand the knowledge of PHP into the benelux.

See you on the next meeting!

php msort() – multidimensional array sort

There have been several moments where I had a rowset from a database as a PHP array, and I had to sort it based on some columns.

$tickets = array(
    array(
        
'id' => 13,
        
'owner' => 'jachim',
        
'time' => '2009-09-25 10:39:42.011612',
        
'project' => 'jachim.be',
        
'title' => 'Some random ticket'
    
),
    array(
        
'id' => 31,
        
'owner' => 'jachim',
        
'time' => '2009-09-24 14:38:47.945020',
        
'project' => 'joggink.be',
        
'title' => 'Some other random ticket'
    
),
    array(
        
'id' => 22,
        
'owner' => 'root',
        
'time' => '2009-09-24 10:58:02.904198',
        
'project' => 'joggink.be',
        
'title' => 'A specific ticket'
    
)
);

I searched for functions for a while, and even try to understand/use the array_multisort function, but I never managed to get something working on a simple array (like the one above). The new function msort() should be a solution for this.

The function works normally when you use a string for the second parameter. That way the $sort_flag works like you would expect it. When using an array of keys however, a string key is built to sort the array on. This part could use some improvement.

Here an example of the usage:


var_dump($tickets);

$tickets msort($tickets, array('owner''time'));

var_dump($tickets);

And here’s the function itself:

/**
 * Sort a 2 dimensional array based on 1 or more indexes.
 * 
 * msort() can be used to sort a rowset like array on one or more
 * 'headers' (keys in the 2th array).
 * 
 * @param array        $array      The array to sort.
 * @param string|array $key        The index(es) to sort the array on.
 * @param int          $sort_flags The optional parameter to modify the sorting 
 *                                 behavior. This parameter does not work when 
 *                                 supplying an array in the $key parameter. 
 * 
 * @return array The sorted array.
 */
function msort($array$key$sort_flags SORT_REGULAR) {
    if (
is_array($array) && count($array) > 0) {
        if (!empty(
$key)) {
            
$mapping = array();
            foreach (
$array as $k => $v) {
                
$sort_key '';
                if (!
is_array($key)) {
                    
$sort_key $v[$key];
                } else {
                    
// @TODO This should be fixed, now it will be sorted as string
                    
foreach ($key as $key_key) {
                        
$sort_key .= $v[$key_key];
                    }
                    
$sort_flags SORT_STRING;
                }
                
$mapping[$k] = $sort_key;
            }
            
asort($mapping$sort_flags);
            
$sorted = array();
            foreach (
$mapping as $k => $v) {
                
$sorted[] = $array[$k];
            }
            return 
$sorted;
        }
    }
    return 
$array;
}

Random PHP function

To explore new things, I sometimes use the ‘Random article‘ link on Wikipedia. It is a simple and basic principle, but it gets you to articles you would never get by just browsing the site.

I was looking for something that did the same job for the php.net site, but could not find anything. As a programmer, you’re most likely to end up in the same chapters of the manual over and over again, and a ‘Random function’ link would be nice to find out new stuff.

So I wrote on myself…

The script parses the http://php.net/quickref.php page, and picks a random link to redirect you to. Anyone heard of HaruDoc? 🙂

Try it yourself: Random PHP Function. Good luck!

ini_restore() and str_replace()

Today I’ve found a new function (new for me, that is). The function is called ini_restore(). It’s a small and simple function call, but it’s just cool to use built-in functions and save some typing and variables.

I use ini_set() from time to time in large scripts where I know it will take a while to execute and need some additional resources. I always try to avoid to set resets on the top of the script, because most of the scripts is doing regular stuff. When it would go wrong there, I want to know about it.

This is where ini_restore() comes in to play:

// old
$origMaxExecutionTime = ini_get('max_execution_time');
ini_set('max_execution_time', 900); // 15 minutes
// do some amazingly long work
ini_set('max_execution_time', $origMaxExecutionTime);

// new
ini_set('max_execution_time', 900); // 15 minutes
// do some amazingly long work
ini_restore('max_execution_time');

Aside from that one, I’ve discovered the fourth parameter in str_replace(): &$count. Nice to know whether the function call did anything or not.

Next step is to beat that annoying multibyte thing in str_replace().