Modern Vespa

I'm currently dealing with a few different issues on the server and wanted to mention it in case anyone is currently being affected.

The first is that there is an unknown crawler originating out of Singapore (via AWS) that (a) does not identify itself as a crawler, and (b) is hammering the site with requests. It appears to be attempting to perform a topic search on every single topic in the forum's database. Because bots are stupid.

The second problem is that, for reasons I have yet to determine, I've managed to un-set the bit on every post that indicates that it has been indexed. Which means the MV background task is currently trying to re-index Every. Single. Post. in the entire database. The vast majority of the index is actually still there, but the current backlog means that current posts (i.e. those from the last day or two) have not been indexed yet, and will not show up in searches.

Sorry for the service interruption. I'll get the issues sorted as fast as I can.

Update: I did some digging into the logs to try to figure out who the crawler is. And while they don't identify themselves as a crawler in the standard way that crawlers are supposed to, they do actually identify themselves: For those that don't know, ByteDance is the parent company of TikTok.

Fuck those guys.

I've determined the problem here. An overly-broad database trigger (simple scripting language that runs in the database itself) un-set the search index bit of every single post when I made an unrelated change to the metadata of... every single post. It really shouldn't have done that, as there was no need to, but the guy that wrote this particular database trigger (aka "past jess") is a big fat jerk.

Thank you for keeping the forum up and running. It is one of the few I frequent because it is moderated so well. And I like Greasy's comments.

Another couple of updates:

• ByteDance has been exorcised from the server. I blocked them coarsely at first (by blacklisting a fairly large range of IP addresses in Amazon AWS's Singapore data center) but now have a more refined approach -- if 'bytedance' appears in their user agent string, I give them the boot immediately. This will work until they figure out that they just have to change their user agent string to something anonymous.

• The server has managed to re-index every single post on the site, including the last couple of days that were actually missing from the search index. So searches should be, errr... as reliable as they were prior to my latest mistakes.

I think this concludes all of the pressing problems, at least the ones that were having an impact. I'm now going back to fixing the insignificant problems that nobody will actually notice.

Is the database using doppler or Euclidean calculations?

It's using geodesics.

at the risk of setting a Lambretta on fire ...

WGS84?

WGS84 is a geodetic datum. Geodesic is more… abstract.

Here's a Stack Exchange discussion on the difference between geodetic and geodesic. In some contexts, they are used interchangeably, because of a historical mistranslation into English. But in this case, they are different.

Oh, unintended consequences. Thanks, Jess, for keeping the place going. And going.

oh, hey, I heard somebody was over here talking bout broken lambrettas? shall I go fetch the gas can?

Nah, I think we've burned down enough topics for the moment.

fair enough. I mean, what with the price of gas today i should be a little more discriminating.

*vanishes in a cloud of two stroke smoke*

What octane?

I'm not going to dignify that with a response.

Oh, wait…

If it's any consolation I didn't see much dignity in that response.

I make it a point to avoid TikTok. I won't even watch video in a post hosted there. The thing is a f'in mean ace.