Kind: Articles

traditional long form content: a post with an explicit title and body

Posted on
by

In the last 24 hours I’ve swung back into the IndieWeb orbit. I think this was mostly prompted by a general disatisfaction with social media right now.

The last time I was here I was trying to work out how to “scrobble” tracks from Amazon Music to this site. I was trying to do this with Shortcuts on iOS and IFTTT.

This time I have been looking at how I can better share my watches from Letterboxd. I have an IFTTT applet that uses an RSS feed Trigger on my Letterboxd and uses the WordPress Action to post here. It’s not very flexible, the formatting is kind of crappy, and, as a result, all the watches are sat in Drafts, waiting to be beautified.

I have been doing a lot with Microsoft Power Automate at work recently and that got me thinking that maybe I could manipulate the data coming in from the RSS feed. I mostly rely on 3rd party tools as I am not skilled enough to whip up and deploy my own API/handle turning apps.

I started scouting around to see if I could find a tool to convert the RSS into another format that I could manipulate a bit more and then feed that into IFTTT (or w/e). I didn’t have much luck, so I thought I better have a look at IFTTT and see if there was anything I could do on their side. I couldn’t remember there being anything. Turns out IFTTT added this thing called filter code. I think the idea is, as the linked article suggests, that it gives you a bit more control of the flow. But it turns out you can basically re-write the applet. You can read the data from the Trigger and set the parameters for the Action.

I came up with a nice regex to remove the username from the feed url, so you can get a link to the movie page on Letterboxd. So, now I should be able to feed that into the Response properties for a watch. I’ll update with a link to the applet when it does exactly what I want.

At that point I was pretty happy and decided to go to bed. That was about 10pm ish. Obviously my brain didn’t fancy that so I ended up looking at my Shortcuts scrobble again… two hours later I was successful. Learned a lot of “obvious” stuff, which I’ll leave to reflect on another day.

Posted on
by

Don’t know about anyone else but I find Teams REALLY useful. EXCEPT when I am trying to get something done. I’m also starting to find Microsoft products better and better. Maybe it is my old age. Windows 10 has way more stuff built-in than, I think, a lot of people realise. One feature I really like is Focus Assist, which basically switches of notifications. However, Teams does not seem to support Focus Assist and I was getting very annoyed, so I looked into it…

Turns out, it does but you need to toggle what sort of notifications Teams uses: https://support.microsoft.com/en-us/office/change-your-notifications-style-in-teams-0da93820-93d0-4da3-86b6-fc28d19908e3

Posted on
by

So, I had a minor disaster. This is what went wrong and how I fixed it.

We use the amazing WordPress Redirection plugin. We recently started a survey and the best way to contact the participants was by letter. We had to include a URL to the survey. Some people were mistyping the URL. For example:

/direct-payments-servey
/direct-payments-urvey
/directpayment-survery/
/directpaymentsurvey
/directpaymentssurvey

The correct URL is /direct-payments-survey/

To fix this, at 8:10am this morning, I threw up a very hasty regex redirect and went to have my breakfast, slapping myself on the back. It matches all the errors and, I thought, would catch most other typos. Here it is:

^/(directpayment|direct-payment).*

Problem is it also caught the target URL and an endless redirect ensued. The page was down for 9 hours.

After my trials a little while back trying to get to grips with not matching strings in a regex I had a good idea for how to fix it.

^/(?!direct-payments-survey)(directpayment|direct-payment).*

It was that easy. Here is the regex on regex101 as usual.

The silver lining here is that I now have a very reusable fix when I need to match something very close to the target URL. I’ve had this problem in the past and often just created a completely different URL. Even then this was not foolproof as WordPress keeps it’s own records of old URLs and redirects.

Posted on
by

Well, who knew Cloudwatch would be so much fun to tinker with?! Not me!

I have been slowly refining my Cloudwatch dashboard: adding new alarms, expanding the scope of the log watch, all that good stuff. It is very satisfying. Over the last week or so I have also set-up fail2ban because (according to my audit log via Cloudwatch ?) sshd was getting hammered. As previously mentioned, this box is not well resourced, so I wanted to nip that in the bud. But does the cost of running fail2ban outweigh the benefits? Hard to say!

Anyway, I am getting quite a lot of email from fail2ban. This is good because I know it is working but I’d rather not have the email and still be able to easily check it was working… so Cloudwatch!

I added the fail2ban log to the config and used the Logs Insights tool to explore. This is typical line:

2021-04-30 17:58:06.631, "2021-04-30 18:58:06,208 fail2ban.filter [100432]: INFO [sshd] Found 205.185.119.236 - 2021-04-30 18:58:05"

We could use the date/time a few more times, right? I decided this was the time to jump into the parse command in the CloudWatch Logs Insights query language (rolls off the tongue that). I knew I was going to need another regex within about 10 seconds. But, damn, if the examples aren’t thin on the ground. I googled and found virtually nothing although this post did help a bit.

So, to regex101.com I went. I exported a few lines from the log to test and I must be getting quite a lot better because I got the basics working pretty quickly:

\[sshd\]\ (?<action>[a-zA-z]*)\ (?<ip_address>[\d\.]*)

Then this query in Cloudwatch Logs Insights did the job:

parse @message /\[sshd\]\ (?<action>[a-zA-z]*)\ (?<ip_address>[\d\.]*)/

| display @timestamp, action, ip_address

| limit 200

Unfortunately, I find reading timestamp pretty hard so a bit more tinkering:

parse @message /(?<date>\d\d\d\d-\d\d-\d\d)\ (?<time>\d\d:\d\d:\d\d).*\[sshd\]\ (?<action>[a-zA-z]*)\ (?<ip_address>[\d\.]*)/

| display date, time, action, ip_address

| limit 200

Excellent! It was running for about 5 minutes and it suddenly produced a blank line. Of course, [sshd] in the log refers to the jail. I have several set up so…

parse @message /(?<date>\d\d\d\d-\d\d-\d\d)\ (?<time>\d\d:\d\d:\d\d).*\[(?<jail>sshd|recidive|mysqld-auth)\]\ (?<action>[a-zA-z]*)\ (?<ip_address>[\d\.]*)/

| display date, time, jail, action, ip_address

| limit 200

And that does the job nicely at the moment. You can find an explanation of the regex on regex101.

Once I am a bit more confident, I’ll start filtering on the action, so I can just see bans and unbans:

| filter action = "Ban" or action ="Unban"

Posted on
by

Had a bit of trouble with what should have been a simple regex today.

I removed a whole bunch of Pages from our site today, and they were all sat under the same parent. I wanted to redirect the now missing pages to the parent. I came up with this:

^\/dignity-care-reports\/[a-z-]*(\/?)

It worked fine in that it redirected the sub-pages but the parent was now in a redirect loop. So, I headed over to regex101.com to investigate. I quickly found that, yes, this regex did match the parent URL.

The great thing about regex101 is that it gives an explanation of what each of your tokens is doing and I quickly saw:

* matches the previous token between zero and unlimited times

And there’s the problem! Now I just needed any sort of match after that second / to stop the redirect loop and a ‘+’ does the job:

^\/dignity-care-reports\/[a-z-]+(\/?)

You can see the example here: https://regex101.com/r/IfIPpn/1

Now I just need to remember that * can match zero times!

Posted on
by

This site runs on CentOS 8, on an AWS t2.micro EC2 instance. It’s all free tier at the moment as this is very much a trial run/learning experience.

It’s taken several attempts over the last few months to get the instance running nice and stable. I was getting regular out-of-memory errors, with mysqld just killing itself. It would then get caught in an endless suicidal loop because the restart of the service caused another out-of-memory error.

However, I’m not sure that mysqld was really at fault. I don’t think php-fpm was very well configured. When I set the site up I followed a few tutorials that didn’t really explain how to clean-up afterwards and I ended up running way more php-fpm threads/children than needed for such a tiny, low traffic site. Since I did that, I have not had a crash and have not had to restart the instance.

Chart showing EC2 memory used. It averages around 60% and tops out at 62%

Yesterday, I was finally able to get an AWS Cloudwatch dashboard set up that actually showed me what was going on. That part was surprisingly easy compared to setting up Cloudwatch itself, which, even following AWS’ on tutorials, was a bit of shot in the dark.

As I said, it all seems pretty stable now so I am adding new WP plugins etc very slowly so I can see if any one thing causes a problem.

Posted on
by

Found a weird WordPress bug today. I exported ALL from my old WP hosted blog and imported everything here. It all went fine except all the Post counts for imported Tags in the Tags screen were zero. I found this old post with a similar problem but no solution.

I figured it just needed a nudge to force it to recount the Tags, so simply did a Bulk Edit and added a dummy tag to all my Posts. Checked the Tags and bingo! Then I just deleted the dummy tag.