It's been the case for literally
years that the system by which topic references, e.g.:
[topic1234]
are turned into topic titles, e.g.
...has been incredibly useful, but also incredibly inefficient. For the one or two topics that might be casually pasted into a post, this isn't a big deal. However (and there's always a "however" in these missives) this completely grinds to a halt when used with even a modest number of topic references.
One thing that might not be apparent is the same system that makes topic references work also forms the literal heart of the MV wiki that I created so many years ago. And with many wiki pages having many tens of links on them, this makes the wiki unbearably slow for link-heavy pages.
And it's been that way for years.
The main issue is that the processing of topic (and wiki) references is amateurishly procedural.
1. Find all the references in the given post or article (this, at least, is done with regex).
2. Loop through the list (one by one) and look up the topic or article title for each of them in the database (this is the slow part)
3. Check to see if the user is allowed to see this particular topic (which in the case of the wiki doesn't even make any sense)
4. Build a proper HTML link that points to the topic (or article) and has the correct title as the display text.
5. Substitute the HTML for the topic reference in the post.
I wrote this a long time ago. Probably 15 years, at least. And at the time, it worked well enough, though I do remember distinctly someone creating a topic full of topic references, and it quickly brought the server to its knees.
At some point along the way, I added HTML cacheing of each post to the server's repertoire of tricks, which means that generally only
one person has to wait on a post full of links to be processed. This, plus the fact that the server is running on
much better infrastructure these days, means that the issue has largely been ignorable.
Except for the @!#$ wiki. the main page of the @!#$ is pathetically slow to load.
So last night I spent all of half an hour remedying this long-simmering issue, refactoring the function that processes the wiki-specific version of these references. To my delight, it actually runs pretty well now.
The new procedure goes like this:
1. Find all the article references in the given wiki article (again with regex).
2. Make a list of all of the article references on that page.
3. Ask the database once for the article titles corresponding to the references found in step 2.
4. Mass-change the text with a single php call.
This takes the processing of the wiki main page (which used to take, I kid you not) from 125 separate queries down to only 9 or 10. And instead of taking many seconds to load, it now comes up in about 200ms, if the database is warm. Sometimes a bit longer if the database is cold, but still always under 1s.
I am satisfied with that.