Modern Vespa

Yeah, definitely some kind of bug. I've seen variations on this, but it was usually while I was changing things under the hood (there's some dark voodoo going on under there) and there was a good reason for the errant behavior.

If you have a replicable case (set of steps) I'd be very interested in hearing about it.

And thanks for the tip!

Yes. Sort-of. I tried to do that in the first version, and it got really complicated really quickly. Rather than try to solve all the problems at once, I chose to ship a simplified version. I'll revisit the user selection at some point in the near future.

There are a number of other major structural changes going on, too. But it will happen. Yes, that's a batch of search results. It's rally the same as a page in the old search, but rather than having to click the next page number, you can just scroll and it will reload.

This batching (or the older version, paging) is necessary because it's unreasonable to return all the search results at once. The query time will be much, much, MUCH longer, and the amount of data delivered to your browser would be extraordinary. The scroll-and-reload technique is used quite a bit on modern websites to keep loading the next set of data only as long as the user is actually interested in it. If you stop scrolling, the server doesn't have to do any more work. I am working (like, right this minute) on a member-focused posts and topics search, specifically to replace the "See all posts by such-and-such" link that you see in profiles. It won't be able to search for specific text, and will be locked to a specific user (the one whose link you followed to get there) but it will let you sift through posts AND topics, sort in ascending or descending order, and additionally apply filters like year, year range, recent, and subforum. So basically all of the elements of the XSearch page, minus the text.

I'm making progress, but in the interest of not having a lot of duplicate code, I am breaking the underlying parts of XSearch into smaller pieces so I can mix and match them in new and different ways. I don't think you'll see as many sort options on the member posts search page. Certainly views and replies doesn't make any sense for posts (there's no data there to support such a sort order).

For topics, it had been my intent to only search for topics started, not topics participated. I confess I'm a bit skeptical that searching for topics that a user has participated in is useful, but I'm willing to let someone (anyone) describe a use case and change my mind.

Thanks for the info! To the above point, I personally find it useful to have an easy way to find all posts by a user. Others may have even more reasons (or maybe nobody agrees and I'm the only one who uses that "find all posts" link!). Anyways, here are a few use cases I'd offer:

1. community / social - I wonder what Jess has been up to on MV lately?
2. user background - What's my impression of this person and possible context of what they just wrote? Am I taking their post the wrong way? Basically helping to get a "feel" for user (especially for newer users where you want to give benefit of the doubt).
3. issue background - what other posts has the user made about this bike? If they are asking for help on something now, did they reply to some other thread with info / clues on what might be contributing?
   

Oh, I'm definitely providing a list of posts. That's what "Show all this user's posts" has always done and I'm not taking that away.

What I was contemplating was adding an option to see a list of topics by that user. Specifically, topics that the user has started.

So I'm now not at all sure we're talking about the same things -- you'll have a way to search for posts by that user (as you always have) and also a way to search for topics started by that user.

What I was debating was whether or not there is a third category, which would be TOPICS (not POSTS) that the user has replied to. This is effectively the same as searching for posts, only... with less information, and likely someone else's name on the topic. And that's what does NOT seem useful to me. Again, we're going back and forth between posts and topics here, but it's... confusing.

Am I off track?

I agree we were talking about different things; my fault, as I did not catch the distinction between Topics and Posts. You explained it separately in the other thread. Basically, I was on the same page as jimc in not understanding the nuance / purpose of Topics.

I previously thought Topic was just a collection of individual {post_title + post_text} elements, so filtering should be possible and useful. I better understand the concept of Topics now as the_whole_thing. I guess I lean towards the view that Topic search will be confusing and maybe even too specific for most search cases.

In the time since we first exchanged these posts I can see that you've put a lot more time into this now. To avoid confusion, feel free to just keep posting there and let this one roll off.

No problem! I definitely want to provide something that is useful, if it's within my power to do so. I'm still improving the interface and making subtle changes, which will be wrapped up with Member Posts (I'm having trouble with the name) when it comes out.

Progress has been slow today. Several other distractions.

on the new search clicking a result opens a new tab. on the old it didn't. I prefer the old.

I normally hold down ctrl on windows when clicking a link if I want it in a new tab or long press when on my phone.

That wasn't an accidental decision. When you leave the dynamically-generated search results page, browsers have a terrible habit of restoring (when you hit "back") the page as it was first served, rather than the later modified version of it. This is especially acute with the new XSearch page, as there are a lot of controls that fundamentally alter the content of the page.

You may prefer opening in the same tab (and I agree with you) but you most certainly would not prefer what happens when you press back. Your search results would be gone, and what you would be left with would be out of sync with the search parameters that you can see on the screen.

It was a nightmare. This is not the most elegant solution, but it is very effective.

I fully understand these sort of compromises

I'll revisit it at some point in the near future, when I get some other parts of search nailed down. I've got a tenuous grasp on some of the more advanced tricks that one can play to get it to work in an acceptable way. I'm using a few of those tricks now -- when you first get any search results, the URL in the address bar changes and behind the scenes (in javascript) I "push" a page state. That doesn't get me all the way there, unfortunately, but it allows me to detect when the user does hit the back button to go from search results to a blank search page, and reset everything back to a clean starting point. But just that bit took me a couple days to figure out!

I am really beginning to hate the browser landscape. HTML and CSS are finally pretty predictable across browsers after years of divergent behavior, but some of the things that happen in JavaScript are a hundred times worse than any difference browsers ever had rendering HTML.

Aieeee.

I'm going to be having weird dreams involving Jess Bots fixing Forums now

Okay, that's just weird.

Actually I like that - I can open several links in their own tabs, so I can easily read through all that I think relevant, rather than reading one at a time in the same tab. That's how I read news sites as well - go down the main page, opening tabs to all the articles I think I'll want to read. Very quick and easy - at least, that's how it seems to the way my mind works, YMMV considerably...

As I was changing the link behavior on search results pages (which took some doing) I was certain you would be the one to object to opening in a new tab. Guess I misjudged that.

I'd hate to be too predictable!

Just an FYI I just did a search for vespa diary. My own post and it took me straight there like a lightning ⚡

Excellent!

UPDATE UPDATE UPDATE

A few of you might have noticed that the old search has disappeared, and there's now only one magnifying glass in the menu bar.

I'll likely do some tidying up behind the scenes today -- removing the underlying search code, and (most importantly) deleting the wordmatch index from the database.

The wordmatch table used by the old search system is actually huge. It has over 61 million entries, and is second in sheer size only to the actual posts table.

I will be quite happy to get that @#$% out of my database.

With some of the cleanup complete, I daresay that posting seems snappier, from the point you hit the "submit" button to the point that it shows you your post in context.

Or maybe I'm imagining it.

Previously, every post would have to get dissected and compared to the wordlist and the wordmatch list so that search could continue to function. That's no longer the case. So it stands to reason that, without that computationally-intensive task as part of posting, that posting would be snappier.

Or maybe I'm imagining it.

A quick reply - and it's VERY rapid.

missing sort when searching Topics AND the search field is non-blank

bug?

Not missing. Or, intentionally missing. You can't do a fulltext search and also sort, due to MySQL/MariaDB's insistence on using relevance.

In the narrow case of searching titles, we can do the full search of all titles and then sort afterwards, because the dataset is smaller. But with posts and topics, it's computationally prohibitive.

On the plus side, I've actually added sorting in cases where you're not doing a fulltext search, and are instead searching by various filters. That's why it appears to be missing -- I didn't take anything away, I added something.

testing:

topics + blank text -> can sort (but no results of course)
topics + non-blank text -> fast, no sort
topics + blank text + user filter -> fast, can sort
topics + non-blank text + user filter -> slow, no sort

all feels a bit inconsistent but I do get it that MariaDB's insistence on using relevance does limit you

Yeah, definitely on the slow side, depending on exactly what you're searching and what filters you use. This is the worst-case scenario -- it's doing both a fulltext search and filtering out things that don't match the filter. Agreed. I might give up on MariaDB fulltext search and move to a different search index system, when I get the chance to investigate them. There are some that operate entirely in memory, though I'm not sure we have the memory for that, and bumping up to a system that does will be costly. How costly, not sure. I don't have any hard numbers yet. It's annoying. It's really fast in general cases, but fails miserably in complex ones. I would have preferred a more graceful degradation curve.

when searching Posts the 'click to edit' for the username is missing?

Now that's a bug.

And how is it that you've not been given an entomologist award yet? That will not do.

Gimme a moment.

I hereby bestow upon you the MV Entomologist award for finding and reporting a forum software bug. Wear it with pride, sir.

Now that that is out of the way, can you elaborate on how you got into that state? Doesn't seem to exhibit 100% of the time.

starting on Titles, add a user filter but don't add a username. Then click Posts.

Got it. I can reproduce. Thanks for the report!

This should now be fixed. Thanks again for the report.

no EV titles/posts/topics??

It's too short. You need at least 3 characters.

Another MySQL fulltext-searchism.

EDIT: And a fairly common search restriction among the various non-Google search systems.

🤦🤦🤦

You could at least use the built-in picardism!

Way better.

In any case, I can probably lower it down to 2 letters (somehow -- I've seen it referenced). Not entirely sure it's worth the additional index size.

not all posts showing in Forum Activity?

Try it now.

Well that does now show znomit's recent posts.

So what happened? I wonder if it'll happen again. Also wonder how many accounts are affected

Indexing (which happens in a background process) got switched off somehow. I toggled it back on and it immediately indexed 300 or so posts and 60 or so topics that were waiting around to be indexed.

is there a particular reasons that can't search for a url in a post? eg if I post a link to aliexpress then I'd expect to get results if I search for aliexpress