Month: April 2021

Posted on
by

Well, who knew Cloudwatch would be so much fun to tinker with?! Not me!

I have been slowly refining my Cloudwatch dashboard: adding new alarms, expanding the scope of the log watch, all that good stuff. It is very satisfying. Over the last week or so I have also set-up fail2ban because (according to my audit log via Cloudwatch ?) sshd was getting hammered. As previously mentioned, this box is not well resourced, so I wanted to nip that in the bud. But does the cost of running fail2ban outweigh the benefits? Hard to say!

Anyway, I am getting quite a lot of email from fail2ban. This is good because I know it is working but I’d rather not have the email and still be able to easily check it was working… so Cloudwatch!

I added the fail2ban log to the config and used the Logs Insights tool to explore. This is typical line:

2021-04-30 17:58:06.631, "2021-04-30 18:58:06,208 fail2ban.filter [100432]: INFO [sshd] Found 205.185.119.236 - 2021-04-30 18:58:05"

We could use the date/time a few more times, right? I decided this was the time to jump into the parse command in the CloudWatch Logs Insights query language (rolls off the tongue that). I knew I was going to need another regex within about 10 seconds. But, damn, if the examples aren’t thin on the ground. I googled and found virtually nothing although this post did help a bit.

So, to regex101.com I went. I exported a few lines from the log to test and I must be getting quite a lot better because I got the basics working pretty quickly:

\[sshd\]\ (?<action>[a-zA-z]*)\ (?<ip_address>[\d\.]*)

Then this query in Cloudwatch Logs Insights did the job:

parse @message /\[sshd\]\ (?<action>[a-zA-z]*)\ (?<ip_address>[\d\.]*)/

| display @timestamp, action, ip_address

| limit 200

Unfortunately, I find reading timestamp pretty hard so a bit more tinkering:

parse @message /(?<date>\d\d\d\d-\d\d-\d\d)\ (?<time>\d\d:\d\d:\d\d).*\[sshd\]\ (?<action>[a-zA-z]*)\ (?<ip_address>[\d\.]*)/

| display date, time, action, ip_address

| limit 200

Excellent! It was running for about 5 minutes and it suddenly produced a blank line. Of course, [sshd] in the log refers to the jail. I have several set up so…

parse @message /(?<date>\d\d\d\d-\d\d-\d\d)\ (?<time>\d\d:\d\d:\d\d).*\[(?<jail>sshd|recidive|mysqld-auth)\]\ (?<action>[a-zA-z]*)\ (?<ip_address>[\d\.]*)/

| display date, time, jail, action, ip_address

| limit 200

And that does the job nicely at the moment. You can find an explanation of the regex on regex101.

Once I am a bit more confident, I’ll start filtering on the action, so I can just see bans and unbans:

| filter action = "Ban" or action ="Unban"

Posted on
by

Had a bit of trouble with what should have been a simple regex today.

I removed a whole bunch of Pages from our site today, and they were all sat under the same parent. I wanted to redirect the now missing pages to the parent. I came up with this:

^\/dignity-care-reports\/[a-z-]*(\/?)

It worked fine in that it redirected the sub-pages but the parent was now in a redirect loop. So, I headed over to regex101.com to investigate. I quickly found that, yes, this regex did match the parent URL.

The great thing about regex101 is that it gives an explanation of what each of your tokens is doing and I quickly saw:

* matches the previous token between zero and unlimited times

And there’s the problem! Now I just needed any sort of match after that second / to stop the redirect loop and a ‘+’ does the job:

^\/dignity-care-reports\/[a-z-]+(\/?)

You can see the example here: https://regex101.com/r/IfIPpn/1

Now I just need to remember that * can match zero times!

Replied to https://waterpigs.co.uk/notes/5ASGj8/ by Barnaby WaltersBarnaby Walters (waterpigs.co.uk)
I don’t often check my internet numbers, but I’m happy to see that a music video I made is at least marginally more popular that that one time I 3D printed some minecraft foxes. (And yes, I know that the foxes have a vastly higher Internet Number Per Year, but the music is the more enduring content so that’ll self-correct over time)

Those foxes are remarkably accurate!

“[Facebook] argues that sharing data with advertisers is key to giving users “better experiences”” https://t.co/3XjtR2Z89d

@gnomeweb @MicrosoftToDo nailed it. Because I am using an Outlook/Live MS account, it authenticates through http://login.live.com rather than http://login.microsoftonline.com, so I had to add http://login.live.com to the “Manage Additional URLs” list under Preferences.