chevybeef.com

GoAccess for web log analysis

Posted on: 13 Jan 2024

GoAccess is an open source real-time web log analyzer and interactive viewer that runs in a terminal or in your browser.

Requirements

This only works with httpd, tested on OpenBSD 7.4, assumes your log style is set to _common_ (default)

To install the GoAccess package run:

$ doas pkg_add goaccess
    

Run from the command line

Read access.log, remove the _domain_ from beginning of each line and save as stats.log

$ grep example.com /var/www/logs/access.log | cut -d' ' -f2- > stats.log
    

Run GoAccess excluding local IP’s and specifing common style

$ goaccess -e 192.168.1.1-192.168.1.255 --log-format=COMMON stats.log
    

Update HTML report every hour

Make a folder in your website for stats.html

$ doas mkdir -p /var/www/example.com/stats
    

Create script to be run by root’s crontab, then paste in the two commands below

$ doas vi /etc/hourly.local
$ zgrep example.com /var/www/logs/access.log | cut -d' ' -f2- > stats.log
$ /usr/local/bin/goaccess >/dev/null 2>&1 stats.log -o \
    /var/www/example.com/stats/stats.html \
    --log-format=COMMON -e 192.168.1.1-192.168.1.255
    

Change to the root user:

su -
    

Edit root’s crontab with:

# crontab -e
    

Paste in the following line at the bottom:

@hourly /bin/sh /etc/hourly.local
    

Save and exit

Now, every hour on the hour the stats report HTML will be updated in:

https://www.example.com/stats/stats.html
    

Secure the stats folder with htpasswd

Create a user with these properties:

stats:*:1002:1002:Stats Viewer:/home/stats:/sbin/nologin
$ doas htpasswd /var/www/htpasswd stats
    

Enter a **strong** password for the stats user and confirm

Change ownership of the htpasswd file:

$ doas chown www:daemon /var/www/htpasswd
    

Modify /etc/httpd.conf to include:

location "/stats*" {
    authenticate with "/htpasswd"
}
    

Reload the configuration:

$ doas rcctl reload httpd