Well, who knew Cloudwatch would be so much fun to tinker with?! Not me!

I have been slowly refining my Cloudwatch dashboard: adding new alarms, expanding the scope of the log watch, all that good stuff. It is very satisfying. Over the last week or so I have also set-up fail2ban because (according to my audit log via Cloudwatch ?) sshd was getting hammered. As previously mentioned, this box is not well resourced, so I wanted to nip that in the bud. But does the cost of running fail2ban outweigh the benefits? Hard to say!

Anyway, I am getting quite a lot of email from fail2ban. This is good because I know it is working but I’d rather not have the email and still be able to easily check it was working… so Cloudwatch!

I added the fail2ban log to the config and used the Logs Insights tool to explore. This is typical line:

2021-04-30 17:58:06.631, "2021-04-30 18:58:06,208 fail2ban.filter [100432]: INFO [sshd] Found 205.185.119.236 - 2021-04-30 18:58:05"

We could use the date/time a few more times, right? I decided this was the time to jump into the parse command in the CloudWatch Logs Insights query language (rolls off the tongue that). I knew I was going to need another regex within about 10 seconds. But, damn, if the examples aren’t thin on the ground. I googled and found virtually nothing although this post did help a bit.

So, to regex101.com I went. I exported a few lines from the log to test and I must be getting quite a lot better because I got the basics working pretty quickly:

\[sshd\]\ (?<action>[a-zA-z]*)\ (?<ip_address>[\d\.]*)

Then this query in Cloudwatch Logs Insights did the job:

parse @message /\[sshd\]\ (?<action>[a-zA-z]*)\ (?<ip_address>[\d\.]*)/

| display @timestamp, action, ip_address

| limit 200

Unfortunately, I find reading timestamp pretty hard so a bit more tinkering:

parse @message /(?<date>\d\d\d\d-\d\d-\d\d)\ (?<time>\d\d:\d\d:\d\d).*\[sshd\]\ (?<action>[a-zA-z]*)\ (?<ip_address>[\d\.]*)/

| display date, time, action, ip_address

| limit 200

Excellent! It was running for about 5 minutes and it suddenly produced a blank line. Of course, [sshd] in the log refers to the jail. I have several set up so…

parse @message /(?<date>\d\d\d\d-\d\d-\d\d)\ (?<time>\d\d:\d\d:\d\d).*\[(?<jail>sshd|recidive|mysqld-auth)\]\ (?<action>[a-zA-z]*)\ (?<ip_address>[\d\.]*)/

| display date, time, jail, action, ip_address

| limit 200

And that does the job nicely at the moment. You can find an explanation of the regex on regex101.

Once I am a bit more confident, I’ll start filtering on the action, so I can just see bans and unbans:

| filter action = "Ban" or action ="Unban"