Modern Vespa

It's not just terrible, it's been terrible forever. Even back in 2005, when this forum was new, the search function was abysmally bad. It only had a small amount of data to search through, but it was doing it on 2005-era servers, which meant that it was slow. And as Modern Vespa has grown, the awfulness of the search function has scaled disproportionately to the capabilities of web servers and modern databases, such that it sucks even more now than it did then. Which is quite the feat.

Over the years, on four different occasions, I have sat down with a serious look on my face (I'm imagining that last part) to remedy the search function, to really take care of it this time. I have failed each time, and the evidence of each failure is still buried in the forum code, taunting me the way unfinished projects always do.

What I discovered through each of these failures (apparently forgetting the previous lessons each time) is that getting the data indexed for search is a relatively straightforward task. Building a web interface that people can use to query that data and get useful results is quite a bit harder.

And building a web interface that doesn't suck is, well… apparently that's impossible.

The main obstacles I've had trying to remedy the MV Search Problem are just the mundane realities of life: job, family, house, and so on. There's always something more important to do, and this task keeps getting pushed to the back of the queue. And search performance just gets worse and worse, to the point that search is almost meaningless on this forum.

I have come to understand, in retrospect, that the amount of time necessary to complete a project like this has never been available to me. It's a huge coding task, and requires an immense amount of focus over an extended period of time. I can focus intently on something for an afternoon. Maybe a whole day. Focus for much longer than that usually comes with a paycheck attached.

But hey, I'm retired now. My wife and I are working on a Portuguese residency visa, which means we're spending a lot of time waiting.

Guess what I can focus on now?

(Keep reading for the answer)

I think the MV Search does a serviceable job most of the time. The one thing I wish it did was point to the specific post in a thread that contains my search terms. I give up on the MV Search when it returns a result pointing to a thread with more than a couple pages. On occasion when the search function returns unacceptable results, I've typed the following search query into my browser (Safari using duck-duck-go): "site:modernvespa.com search terms". It works ok. Between the two, I can usually find what I was looking for pretty quickly. Neither approach is flawless tho

Miguel

So today I am rolling out a new feature: Experimental Search, or XSearch. And, well, it's kind of experimental, in that I don't really know where the breaking points are just yet. So for the time being, it will live alongside the "old" search, until we collectively have enough confidence in the new search system that there can be only one.

(You can get to XSearch by clicking the magnifier with the plus sign in the menu above)

What I do know is that for simple searches, it is much faster -- lightning fast, in most cases. It starts slowing down when you apply filters (searching posts for text posted by a specific person, for instance) but on the whole it is much improved.

It also has a bunch of drawbacks, the biggest one being that you can't sort by anything except relevance when searching posts or searching titles. I've been around and around on this issue, and it can't really be done with MySQL fulltext search. When searching topic titles, you can sort by a few things, because topic titles are (comparatively) a very modest data set. But searching and then sorting posts or topics is... computationally infeasible. At least, with the fulltext search we're using.

Anyway, I'm getting ahead of myself: there are three main modes: topics, titles, and posts. Titles is the default, because it is fastest and generally the most useful. Obviously, not everything you're searching for will show up in a title, but it's usually a good place to start.

You can also search using natural language. For example: You can also choose advanced mode, where you can use various operators to include or exclude certain words. For example: Then you can choose filters to narrow the results. You can search by forum, member, recent activity, single year, or range of years. Those last three, in particular, are there to mitigate the fact that you can't sort by time for topics and titles.

There's also no search button. Start typing and results are shown right away. If you have a filter in place, you'll probably see the "Loading" message, be patient. Scroll down. More results will load when you get to the bottom of the page. Jump back to the top and alter your search parameters, and new results appear.

Rinse, repeat.

Questions? Bugs? There are bound to be many. Put them here in this thread.

Sounds interesting. I've stumbled through forum searches with fair success in the past, but also super grateful for Google consistently pointing me towards MV posts.

Hey, this isn't how Znomit found the hamster post??

This is 100% how Znomit found the hamster post.

I use the search function frequently and I appreciate your hard work. This is truly a great resource.

But any and all improvements are welcome! Thanks so much.

On my iMac using Safari the experience is often frustrating. Yet, with Chrome things work better. Overall, it works and it's very helpful.

Which is ironic, since I develop primarily using Safari and only occasionally look at Chrome and Firefox.

So it's a raging success then! Perhaps the pinnacle of MV development.
Brings a tear to the eye it does.

On that note, I would like to point out that the old search would turn up the hamster-eating-burrito thread as well. And searching only titles, it was fairly fast.

What's interesting is that the old search turned up a bunch of other threads as well -- that didn't obviously have "hamster" in the title. What they did have, though, was "hamster" in a reply title, something the new system deliberately ignores. And that's largely because Bill and Bitsa polluted the database with randomness in the subject of their replies. A feature that has since been removed.

And that, my friends, is why we can't have nice things.

Let me say this about that: No, it's not the most-intuitive Search interface by a long shot. But at least it works, and I sorted out how to make it work for me only a few years into my discovering MV.

And, look on the bright side of this, Jess: If you've ever used the "advanced search" feature on eBay, you'd see how your efforts don't suck quite as much as you imagine...

So are we changing our stance on neco threads?
Because having a search machine that works is like opening up a can of worms. We are going to be faced with a barrage of necros.

Do you mean this one?

Exactly JimC is going to have a heart attack

No, that's not the look when I first sit down to fix something. That's the look when I fail.

The main reason for this -- in both the old search system and the new one I rolled out today -- is that it just doesn't know what page it's on. The information isn't in the search index. All it knows is that a given set of words are associated with a given topic.

What you want is per-post granularity. Which both the old search and the new search support. Just search by posts instead of topics, and it will take you right to the post.

There are some fairly subtle differences between searching by topics and searching by posts -- if the words you are looking for are spread across several posts, rather than all being in one post, then you might want to choose topics. If you're looking for the most extensive discussions on a given subject, you might want to use topics. But if you're looking for a specific word that might only show up in a handful of posts, then searching posts is almost always the right answer.

But did you see the new search? I know I buried the lede and everything, but there's a whole new search system online today.

Dammit! If only I still had access to my photoshop I could raise those little arms in the air and capture the moment perfectly. Fricken Adobe!

D00d! Affinity Photo!

Nope, just commenting on the old setup. Sorry for the confusion. The longer I put up with Adobe, the more tempting Affinity gets. (Still on CS6 here.)

I don't miss Photoshop at all. It did take a little time to figure out Affinity Photo and how to do the things I was used to doing in Photoshop, but everything was there, just rearranged.

I figured out a search here is only going to work if I'm already smart enough to know the answer.
So, being clueless is a double-bind, as it were....
O.S.

The search function is now much better and improved! Thanks Jess

Thanks, Max! I've been working on it for about a month. Happy to finally see it getting used.

Damn Jess, were you in a particularly navel-gazing kind of mood yesterday? lol

My favorite way to get any kind Vespa info is to search site:modernvespa.com (query goes here, no brackets)

That will leverage Google's search engine (assuming that's your default search engine) to show only results for your query from modernvespa.com

Example:

This search
site:modernvespa.com how to replace voltage regulator vespa gts250


Will take you to this page
https://www.google.com/search?q=site%3Amodernvespa.com+how+to+replace+voltage+regulator+vespa+gts250

All the ModernVespa goodness, with Google-quality search results, until Jess does the magic things.

Yes. That's good, but there are (in my mind) three problems with that:
1) It sends people off the site to do something that is basic and should really be handled without ever leaving the site.
2) It gives Google more of people's browsing data. I realize that this is largely a losing argument, and that nobody cares how much information Google has about you. But I care.
3) Google search can't really capture some of the specific parameters of the forum, such as narrowing the search down to a specific user or searching within a specific part of the forum. Granted, these are niceties, but sometimes they make all the difference in the world when you're searching for something specific. Keep reading this thread. The magic has already happened.

Agree with all 3. As soon as I can! Still working my way through that monster first post haha

Tried this new search today, it worked so fast that I didn't realize that the search was completed.

Thanks.

You're in touch with your inner German.

Mine's impossible to shake between Prussian, Pomeranian and Bavarian. I mostly blame the Prussians.

Been playing with the search and so far seems pretty good. Expect I'll be much happier with it.

So that's what it's called!

Thing is, search sucks on every forum I visit, so sucky-search is expected. I'm not sure what I'll think if you fix it, Jess.

HINT: Try the new search. It's the magnifier with the plus in the menu above.

I just did, well done. I have, in researching troubles people have with Beverlies, searched for titles like My BV350, by me. Using the old search, I return everything that includes my bv350, even if I choose titles only.

The new search worked just as I expected. Well done, Jess.

the new search button appears to work and not need advanced education to use. I have found that messages can be sent and appear when one clicks on the little envelope in the upper right. I do not get along well with electronic devices very well, so i I can use it to find an answer it should be a snap for the younger generations that have grown up with computers and the internet.

Aaaaaaw, Daaaaaad


Thanks, yet again, for all of it. Hope you and Mrs jess are enjoying an adult beverage and a peaceful holiday.

Perfect!!

Thanks! Hope you and yours are doing well.

FIVE olives?
Not three?

Five.