This is the ckbw script I briefly mentioned earlier. Before going any deeper into the subject, realize that this script assumes some things about your system:
- You are using the “combined” Apache2 log format.
- Logs are stored as /home/username.logs/domainname.com.access.log and compatible logrotate (.1, .2, etc) equivalents.
In a general sense, ckbw parses through all log files in the above directory and gets a file size total for all items used. Some things, such as unbuffered PHP scripts, will not return a file size and so will not be counted. Be sure that you at least have a month of logs if using logrotate or data can not be reported accurately.
Onto the script itself:
ckbw script; Perl
Basic usage:
ckbw apr - grab the bandwidth used in April on the current user
sudo -u bob ckbw apr - get the bandwidth used in April by “bob”
The script is hardcoded to grab the bandwidth per user and not per site, but it should be very easy to adapt it for a getting stats of a single site.
If anyone has a more elegant way of grabbing the output of system() or similar without piping to a file, let me know.
It’s pretty inefficient to first copy all the logs in a directory to a file in /tmp, and then to slurp that entire file into memory.
Let me try improving that a bit (and I do hope code remains intact):
#!/usr/bin/perl -wmy $month = shift
or die "Usage: ckbw month\nPrints the bandwidth used in a month.\nUse shorthand notation (ie, Jan, Feb, Mar)\n";
my $cuser = $ENV{"USER"};
my $total = 0;
#find all of the log files for the user
for my $logfile ( glob( "/home/"${cuser}.logs/*.access.log*" ) ) {
open my $fh, $logfile or do {
warn "couldn't open $logfile: $! (skipping)";
next;
};
while(< $fh>) {
my @linesep = split / /;
#check the month
if($linesep[3] =~ m/$month/i) {
#make sure there is an actual value
if($linesep[9] =~ m/\d/) {
$total += $linesep[9];
}
}
}
}
print “Bandwidth totals:\n”;
print $total.” bytes\n”;
$total /= 1024;
print $total.” KB\n”;
$total /= 1024;
print $total.” MB\n”;
$total /= 1024;
print $total.” GB\n”; [fixed formatting -Jacob]
Hmm, sorry about that. How do you do code on this blog?
Anyway, the while() should read while(<$fh>)…
Here’s an even terser solution (skipping the stuff around the loop):
@ARGV = glob(”/home/${cuser}.logs/*.access.log*” );
while() {
# split etc
}
That’s a useful idiom, which I keep forgetting myself
Great little tool.
I thought it would be a good exercise to do something like this in Python, so here it goes:
http://nickpresta.ath.cx/lab/python/bwlog
It doesn’t handle rotating logs like yours does, but its simple enough for me.
Thank you Jacom, and thank you Nick.
I was looking for a script like this for my server. I do not know Perl so I’ll try the Python version.
Now I have another question for you. My log files are in /var/log/apache2/site/ directory. the log files in /var/log are being rotated daily by the system but the apache logs are growing all the time. How can I rotate them automatically like the log files in /var/log ? My server is an ubuntu server edition 7.10.
Thanks!!!