Modern Vespa

Way back in early May, I took the forum down for a couple hours to do a major database upgrade. I had practiced the transfer from the old database to the new one several times so that I had some idea what I would encounter when I made the switch, but I didn't have any idea what kind of impact the upgrade would have on the overall performance of the real, production site.

As per the linked thread above, the initial numbers looked good. Now that I've accumulated over a month of data, though, the numbers look even better. The graph below largely tells the story. "Render Time" is the time it takes from the moment the server gets a request for a page to the moment it sends it back to your browser. On your end, you'll see some additional delay as resources (images, etc) are transferred from the server, but the render time is a huge part of what makes a website feel "responsive".

Most of the speed increase can be attributed to switching the database to a solid-state drive, instead of the old spinning magnetic platter that housed the database prior. Some additional improvements were achieved by streamlining a lot of the write operations, postponing them until after the page was rendered or moving the database table into RAM instead of on disk.

All in all, I'm really pleased with the increase in performance. As a bonus, the monthly hosting bill is a little bit less as well.

Jess:

Thanks for taking such good care of us. I noticed that things seemed quicker - but that's a pretty considerable reduction. Amazing what a little SSD can do!

Kudos, Jess.

I've noticed the difference as well.

Nicely done. Thanks.

Whenever I seem to have a problem linking to some site and suspect my wireless connection is not working, I hit the MV bookmark in Safari. MV is the most reliable and fastest site I regularly visit.

In several posts over the last year, you've predicted the slow demise of MV due to people migrating to FB and similar sites and not visiting forums. Do the numbers (daily posts, queries, new memberships) over the last year support that hunch? Just curious.

Thanks for all the work Jess.

Miguel

That's actually really awesome. Happy to be the go-to reliable site. Yep. New registrations are up significantly this month (due to the banner that guests receive) but for the year overall, we're way behind. I was hoping that new topics and new posts would shadow the uptick in new user registrations, but they haven't yet. Those stats are way down year-over-year.

Sorry, Jess... I'm guilty of not doing my part of posting or commenting on this site in the last year or two. I have not ridden my scooters as much either. If you would like, I'd be willing to send $20 for a membership fee if that would help. Just let me know.
I've made a few friends on here, and I was thinking about Trepp, YardSale, Sticky, Bitsy, Belkwinith, CDwise, Route66 Lawdog and others that brought me back..
It was good to see the improved performance, and the Donkey Skok post are still around. .
Rail 50

Well done, Jess. It's clear the site is well cared for.

Too bad about the overall numbers. Hard to get why there isn't a bit more general enthusiasm (outside of the active MV membership). They don't know what they're missing.

There's no blame, really. It's not anyone's responsibility to post here (except maybe mine, heh) and I don't really even blame Facebook. It's just a function of a number of simultaneous factors that all tend to favor Facebook as the go-to place for daily interaction. Those factors used to favor forums, now they favor Facebook. It is the inevitable march of progress. Thanks for the offer, but this site has never really been about money. It's a community service. If the community wants it, I'll run it. If not, I'll do something else.

I'm pretty sure that we want it.

I understand you like doing this for us and I for appreciate what you do.

I've also often wondered if I can donate towards maintenence and advertisement in effort to boost numbers. Would be a shame for MV to go due to lack of traffic

Heh. Let me rephrase that:

If there's enough community to sustain a discussion forum, I'll continue to keep Modern Vespa going. Discussion forums need actual discussions to keep people coming back, though, and when there are no longer a critical mass of people visiting the site and posting to topics, the end comes rather suddenly.

So, in other words, don't be shy about posting. 8)

Definitely.

The performance is great, and for me part of the "attraction" for want of a better word, is that posting here is safe. The moderators do a great job of dealing with the bits that can also hasten the demise of a discussion forum. Lack of knowledge is accepted and help is given freely; there are some great guys (genderless term for people) here who seem willing to help anyone and everyone willing to put their hand up for that help.
For newbies the Search can be pretty intimidating, hell even the experienced people note that, so I see a future so long as we keep the standard, and people want to learn and share about the scoots and the adventures.

I'm wondering if it is not all Facebook. It seems to me that a lot of people are connecting on 'phones rather than computers, but the 'phones don't seem like a comfortable way to browse a forum.

MV is the best forum to browse on an (i)Phone, Jess made sure of that!

Apparently safari emulation in a android browser works well too...

I've noticed the difference, especially when I sign into Modern Buddy, which I do every time I leave here.

The two forums used to operate at similar speeds. No more, though. MV is significantly quicker now.

Could be but I browse the forum several times, everyday. I'd guess 60-70% is on my iPhone 6. The multiple views you get by holding the phone at different orientations provide a lot of flexibility. Holding the phone with the home key at the bottom or right brings up mobile device specific format that size the text so its readable on a phone. Turning the home key to the left brings up a normal, full-mode browser format. I generally only use this mode to zoom in on pictures. The other modes (home key down and right) don't allow zooming.

Best
Miguel

The Dolphin browser works very well as an iPhone emulator on an Android. Recommended.

Thanks for the comments, Gents. I sand corrected about browsing on 'phones. Actually, I was just winging it, having no first hand knowlege. My cell 'phone looks like a hockey puck.

Here's an interesting recent paper about SSD failures in large data centers. This is based on Facebook's extensive experience and it's definitely interesting. It's a scholarly article so there is a lot of very careful defining of terms, but definitely worth a read.

I got the gist..

(yeah right!)

Lol! I figured jess will want to read the source article. Here's a rollup of the article with a little perspective on what it means.

Good article. Read most of the paper also. I've been using ssd in my home computer for years. They have been excellently reliable so far. The oldest ones have over 17,000 hours on them.
Nice to know what issues to look for...

Yep. Interesting stuff. I had been sort-of following the tech news about FB's SSD study, though it has very little practical impact on our usage. The SSD we're using is an anonymous unit in an anonymous data center somewhere, and I'm only using a slice of a larger whole. It's not even clear to me that the SSD I'm using today will still be the same SSD I'll be using next week.

Fortunately, the database is backed up daily.

Hey,

I think you should post your address so we can send
you some donation to keep the effort up. Everyone
can do what they want or don't want. At least get a
few bucks and go out for a nice dinner.

So, don't bashful.

Bob C
Minnesota

Thanks for the sentiment, Bob. As said earlier, though, I run the site as a community service. Money has never been motivation for this site, or we would have closed up shop long ago. As long as our hosting bills are paid, all is well.

The only real way for a member to give directly to the site, in fact, is pretty indirect. We get a cut of anything you buy from Amazon if you follow one of the Amazon links on this site. Alternately, you can click here or on the Amazon ad on the bottom of the main page and we'll similarly get a cut of whatever you buy.

A couple of other forums I use offer a paid membership with some perks - whether that's more PM capacity, the ability to sell stuff on the forum, less adverts or just a nice logo under their name varies from site to site.

Don't know whether that's worth considering here - a few few pounds/dollars a year you might have quite a few people willing to be "Site Supporters" or similar which would help with hosting costs.

I try to remember this, I really, really do...

Jess,
I buy a lot of stuff from Amazon but through smile.amazon.com. Smile.amazon donates 2% of all my purchases to the charity of my choice (Native Animal Rescue here in Santa Cruz is my choice). I was wondering if I can make purchases through MV but do it through smile.amazon.com to get a twofer, a donation to NAR and MV?

When I clicked on your link above, it routed me to: http://www.amazon.com//ref=as_sl_pd_tf_lc?&tag=modernvespa-20

I tried adding "smile." to the link and it routed me to http://smile.amazon.com/?&tag=modernvespa-20 .

Looks like it dropped the ref tag. Not being a web programmer, I'm not sure if MV will get the benefit of my purchases.

Comments and recommendation?

Thanks.
Miguel

The ref tag isn't especially important, AFAIK. The "tag=modernvespa-20" is the important bit. That said, I don't know if you can get a twofer. Let me look into it.

MV gets 6% - 8% (depending on traffic) for each purchase.

So it looks like you can simply change www.amazon.com to smile.amazon.com while leaving everything else the same, and it will apply the MV affiliate part AND the charity part (all at no additional cost to you, of course).

That said, Amazon says that the charitable part of AmazonSmile is only 0.5%, not 2%, regardless of whether there's an affiliate tag in there or not.

Is it sufficient to go to amazon home page using that link, then shop around and MV will get the credit? Reason I ask is, the tag disappears once I move on to another screen (say, click on a product or go to my cart).

Or, does this tag install a cookie that makes that machine now always credit amazon purchases to MV?

...signed,
Not a computer guy,
but,
wants MV to flourish.

Cheers,
Sully

Beautiful!! Thanks for checking. I changed my bookmark to add the tag.

Miguel

Once you follow the link, MV gets credit for anything you buy for a short amount of time. I think it's 24 hours, but I'm not positive.

EDIT: Yep. It's 24 hours.

Awesome, perfect, thanks!

And thank you for your support!

Just doing some catching up of past threads while I was gone. This one intrigued me. Didn't realize the partnership with MV and Amazon. Great way to help fund the site.

As a frequent Amazon user, I made my own app for my Apple devices. MV will get the credit for sure. Thanks Jess!

What I have noticed is upon revisit of a thread, the total number of "views" documented, does not increase, as previously.

When a particular thread is "viewed" by the same participant, the number of views remains stagnant (the same). Hence, this gives MV the actual number of "real viewers." Using the word, HENCE, again, gives MV, the actual number of real users!??

Any one else notice that?

It's a bit subtler than that, but you're right in that it is a relatively recent change in behavior.

When I switched to the new database, I was alarmed at how much write IO was happening. MV's usage model should be read-heavy, with occasional writes for the occasional post, plus statistics and such. It was the opposite -- much more write operations than read operations. Worse, the SSD storage limits how many total IOPS I can have in a given period of time, and we were definitely pushing the boundaries of usage.

Digging in to some of the neglected innards of the forum software, I first started (as one does) with low-hanging fruit. Every time someone views a thread, it was writing to the database to increment the number of views. I didn't want to give up that functionality, but with all the search engines hammering the site, that was bound to be responsible for a lot of the write operations. So I created a mechanism whereby all the topic views are tallied up in a memory-based table, and then added to the "real" on-disk table every 5 minutes. Every successive view will still be added to the table, but not in real-time.

There were other performance involving memory-based tables, too. The problem with this kind of table is that if the database CPU goes down, all the information in the table is lost. So really, you can only put information in one of those tables that you don't mind losing. Every session record is in one of those tables, which would at worst require everyone to log in if the CPU went down. Hourly statistics are in a memory table as well, as we can certainly afford to lose one hour's worth of statistics.

The rest of the optimizations (all of which happened over a three-week period) were just basic database optimizations that happened to reduce the number of write ops. And eventually, I got the write ops down to the point where I felt it was stable and acceptable.