Volunteers Needed to Beta-Test New Server
Post Reply    Forum -> Not-So-Modern 12Next
Author Message
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Tue Apr 20, 2021 2:32 pm quote
Cross-posting this here, because I know you guys like to break shit.

Volunteers Needed to Beta-Test New Server
Moderator
VNB VSC VBC VSX
Joined: 28 May 2008
Posts: 4844
Location: Hustletown, TX
Tue Apr 20, 2021 2:33 pm quote
That was not directed at Ned or Mike.
bodgemaster
63 GL, 76 Super (x 2), 74 Primavera (x 2), 06 Fly 150
Joined: 26 Sep 2013
Posts: 5809
Location: So Cal
Tue Apr 20, 2021 2:36 pm quote
Testing... 1...2...3

I gotta be meeee
I gotta be meee

Anything broken yet?
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Tue Apr 20, 2021 2:38 pm quote
SoCalGuy wrote:
Anything broken yet?
Not a single log message has appeared in my console. So, you're not trying hard enough.
Ossessionato
76 Sprint V, 63 GL, 62 VBB, 05 Stella
Joined: 16 Jun 2011
Posts: 3176
Location: Nashville
Tue Apr 20, 2021 4:27 pm quote
I'll take a look. How hard should I try to break things?

(I might be a bit of a professional in this area)
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Tue Apr 20, 2021 9:11 pm quote
chandlerman wrote:
I'll take a look. How hard should I try to break things?

(I might be a bit of a professional in this area)
Coverage is probably more important than determination. It's still a chumpchange AWS server that will max out if you push it, so DDoS is probably not necessary. What I'm really after are all the PHP exceptions that I haven't found yet.
Ossessionato
62 VBB1T Round Tail W/ leaner sidecar
Joined: 26 Jan 2019
Posts: 2238
Location: california
Tue Apr 20, 2021 10:08 pm quote
Does it need to be welded or ported?
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Tue Apr 20, 2021 10:47 pm quote
charlieman22 wrote:
Does it need to be welded or ported?
It is shiny, so one could easily make that mistake. However, I can tell you from personal experience that servers do not like anything that arcs or anything with abrasive grit.
Style Maven
74 50s x3 78 P200 x2 84 Cosa PK50XL2 58 AllState 68 Sprint 80 50special '66(?) Super125
Joined: 22 Nov 2010
Posts: 8151
Location: seattle/athens
Wed Apr 21, 2021 1:32 am quote
jess wrote:
.... I can tell you from personal experience that servers do not like anything that arcs or anything with abrasive grit.
Well, I guess that leaves me out then
Molto Verboso
2007 Stella 225
Joined: 02 Nov 2019
Posts: 1663
Location: Rochester, Minnesota
Wed Apr 21, 2021 3:41 am quote
Trying the new server, my web browser has always flagged NSM as a non secure site. Are you going to update that for the updated site?
Ossessionato
76 Sprint V, 63 GL, 62 VBB, 05 Stella
Joined: 16 Jun 2011
Posts: 3176
Location: Nashville
Wed Apr 21, 2021 3:50 am quote
jess wrote:
It is shiny, so one could easily make that mistake. However, I can tell you from personal experience that servers do not like anything that arcs or anything with abrasive grit.
Or dry chemical fire extinguishers. They really, really hate those. We had electricians working in our data center many years ago who started a small fire, looked around and grabbed the first fire extinguisher they saw. The problem was that they were working with the doors open, so the closest one was *outside* the DC, meaning it was DryChem rather than CO2.

They put out the fire, and eventually destroyed about $10 million worth of high end Sun servers (E6k's, among other things) in the process.

Anyways...I won't do more than some basic mapping, scanning, and maybe a tiny bit of scanning for known vulnerabilities and fuzzing.
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Wed Apr 21, 2021 6:09 am quote
Christopher_55934 wrote:
Trying the new server, my web browser has always flagged NSM as a non secure site. Are you going to update that for the updated site?
Yes. I've done it successfully in a test environment to prove out the process (automated with Let's Encrypt, thank you EFF). Since the server domain name has to match the certificate exactly, and since the new server is going to change to the main modernvespa.com domain, I opted to do the final signing after the switchover. That will probably involve some brief downtime for the site, but only for a bit.
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Wed Apr 21, 2021 6:12 am quote
V oodoo wrote:
Well, I guess that leaves me out then
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Wed Apr 21, 2021 6:14 am quote
chandlerman wrote:
They put out the fire, and eventually destroyed about $10 million worth of high end Sun servers (E6k's, among other things) in the process.
Oof. That's a bad day at the office.
chandlerman wrote:
Anyways...I won't do more than some basic mapping, scanning, and maybe a tiny bit of scanning for known vulnerabilities and fuzzing.
Thanks, that'd be very helpful.
Ossessionato
76 Sprint V, 63 GL, 62 VBB, 05 Stella
Joined: 16 Jun 2011
Posts: 3176
Location: Nashville
Wed Apr 21, 2021 8:01 am quote
LetsEncrypt on Linux is SUPER easy to get going, it just needs an A Record on a Public IP address that the server is running on/behind.

Since you already have that, you could set it up and test it now (use the test side to validate the config, because they heavily rate limit the prod certs and you'll get locked out for 24 hours). I use certbot on my servers and have a cron job that auto-renews everything every 30 days. It's like buttah!
Ossessionato
76 Sprint V, 63 GL, 62 VBB, 05 Stella
Joined: 16 Jun 2011
Posts: 3176
Location: Nashville
Wed Apr 21, 2021 8:37 am quote
couple things...are images supposed to be larger now?

(Just checking...I actually like it )

And I'm seeing intermittent looooooong page load times, like this form took 12 seconds to load, all waiting on posting.php. Second sub-load of it took 112 ms, though.
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Wed Apr 21, 2021 9:22 am quote
chandlerman wrote:
couple things...are images supposed to be larger now?

(Just checking...I actually like it )
They shouldn't be laid out on the page any differently, as I haven't touched any HTML or CSS. And ultimately all the uploaded images get resized after upload, so unless Imagick is broken (or missing).... hmmm. I should probably check that.
chandlerman wrote:
And I'm seeing intermittent looooooong page load times, like this form took 12 seconds to load, all waiting on posting.php. Second sub-load of it took 112 ms, though.
That's definitely not expected. I'm consistently seeing 100ms load times on my end. The next time it happens, can you check the stats at the bottom of the page? I'd be curious to know what the page render time was. That will help pin down where the delay is happening.



Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Wed Apr 21, 2021 9:39 am quote
jess wrote:
And ultimately all the uploaded images get resized after upload, so unless Imagick is broken (or missing).... hmmm. I should probably check that.
This was in fact the problem. ImageMagick was not installed on the new server (and obviously not there by default). Good catch! Thanks for pointing this out.

You bring up a valid point about size, too -- I could probably bump the pixel size that all images get shrunk to. It's currently at 800, we could probably enter the 21st century and go all the way to 1024!
Ossessionato
76 Sprint V, 63 GL, 62 VBB, 05 Stella
Joined: 16 Jun 2011
Posts: 3176
Location: Nashville
Wed Apr 21, 2021 9:52 am quote





six second query

Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Wed Apr 21, 2021 9:59 am quote
Long searches unfortunately do not surprise me. The search function on this old forum is pathetic. We will almost certainly retire the entire software installation in favor for something modern, but the migration is going to be... challenging.

So for the short term, I'm focusing on getting the old software up on modern hardware.

Thanks again for all the help.
Ossessionato
76 Sprint V, 63 GL, 62 VBB, 05 Stella
Joined: 16 Jun 2011
Posts: 3176
Location: Nashville
Wed Apr 21, 2021 10:11 am quote
You going to get redis going, or just live with a FULLTEXT index on the subject fields and body text? I'd think redis would be the easiest option to hammer-fit something in.
Molto Verboso
2007 Stella 225
Joined: 02 Nov 2019
Posts: 1663
Location: Rochester, Minnesota
Wed Apr 21, 2021 10:31 am quote
It's 3x + faster on my android tablet using chrome.
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Wed Apr 21, 2021 12:33 pm quote
chandlerman wrote:
You going to get redis going, or just live with a FULLTEXT index on the subject fields and body text? I'd think redis would be the easiest option to hammer-fit something in.
I might use redis for general cacheing of infrequently-changed data. I don't think the whole of 15 years of MV posts would fit in the amount of RAM we have in our current instance, though.

Years ago I had started a project using Sphinx as a search replacement. It showed promise, but I couldn't get it quite the way I wanted it and I think I had too many irons in the fire at the time, so ended up not finishing it.

The debate I'm having is how much effort to invest in the old phpBB2 forum software. It really is almost 20 years old now. I could probably continue to modify and extend it, but ultimately it's still built on technology mostly from the last century.

To that end, I recently spun up a Discourse install to try it out and see what it was like to administer. I like a lot of the features it brings, but the complexity level is high and the migration is going to be wholly custom. The jury is still out on that one.
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Wed Apr 21, 2021 12:35 pm quote
Christopher_55934 wrote:
It's 3x + faster on my android tablet using chrome.
Excellent, thanks!
Ossessionato
62 VBB1T Round Tail W/ leaner sidecar
Joined: 26 Jan 2019
Posts: 2238
Location: california
Wed Apr 21, 2021 1:30 pm quote
Jess - is the limitation on being able to post pictures between text related to BB code or would that be possible without creating entire science project. BBC works - if not fancy - but that would be a significant upgrade for users. My $.02
Ossessionato
76 Sprint V, 63 GL, 62 VBB, 05 Stella
Joined: 16 Jun 2011
Posts: 3176
Location: Nashville
Wed Apr 21, 2021 1:57 pm quote
jess wrote:
I might use redis for general cacheing of infrequently-changed data. I don't think the whole of 15 years of MV posts would fit in the amount of RAM we have in our current instance, though.

Years ago I had started a project using Sphinx as a search replacement. It showed promise, but I couldn't get it quite the way I wanted it and I think I had too many irons in the fire at the time, so ended up not finishing it.
Ooh...good point about memory requirements for redis. I used sphinx in one of my past rails projects and while it was kind of a pain to get going, once I had it, it was fast and rock solid. That was a multi-tenant rails app, which was the real problem with it.
jess wrote:
The debate I'm having is how much effort to invest in the old phpBB2 forum software. It really is almost 20 years old now. I could probably continue to modify and extend it, but ultimately it's still built on technology mostly from the last century.

To that end, I recently spun up a Discourse install to try it out and see what it was like to administer. I like a lot of the features it brings, but the complexity level is high and the migration is going to be wholly custom. The jury is still out on that one.
it looks like there are phpbb to Discourse migration tools, and I'd be happy to see a switch to markdown personally. I've never built or configured a discourse server, though, so I can't comment on how much of a pain it is to maintain or admin.
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Wed Apr 21, 2021 3:19 pm quote
chandlerman wrote:
it looks like there are phpbb to Discourse migration tools, and I'd be happy to see a switch to markdown personally. I've never built or configured a discourse server, though, so I can't comment on how much of a pain it is to maintain or admin.
There are, but for phpBB3. Which we are definitely not. The official advice is to upgrade from phpBB2 to phpBB3 and then migrate to Discourse, but we are so heavily customized that I'm pretty sure even the migration from phpBBJess to phpBB3 would be just as much work as writing a wholly custom migration straight to Discourse.

The complexity of Discourse is mostly one of (for me) unfamiliarity: it runs in a Docker container, it uses Postgres, it is written in RoR, and it comes with support out-of-the-box for a slew of modern features. It really does have everything we could ever ask for in a forum package, but that comes with baggage. I don't know Ruby at all, I am barely functional in Docker, and Postgres is wholly alien to me. Is that a dealbreaker? No. But I am weighing carefully the trade-off between Discourse and what is a comparatively simple LAMP install here on MV (though we use an external MySQL database and some other AWS infrastructure).

The ultimate goal is for the forum to outlive me. To do that, it might need to be less complex, not more.
Style Maven
74 50s x3 78 P200 x2 84 Cosa PK50XL2 58 AllState 68 Sprint 80 50special '66(?) Super125
Joined: 22 Nov 2010
Posts: 8151
Location: seattle/athens
Wed Apr 21, 2021 5:11 pm quote
jess wrote:
.....

The ultimate goal is for the forum to outlive me. To do that, it might need to be less complex, not more.
Fascinating stuff, but you lost me long ago. Sounds more unnecessarily complicated than a dang four stroke scooter, but necessarily modern.

I just posted to say you nailed it on for me in your lovely closing. That's just how I feel about my Vespas.
Ossessionato
76 Sprint V, 63 GL, 62 VBB, 05 Stella
Joined: 16 Jun 2011
Posts: 3176
Location: Nashville
Wed Apr 21, 2021 6:21 pm quote
jess wrote:
The complexity of Discourse is mostly one of (for me) unfamiliarity: it runs in a Docker container, it uses Postgres, it is written in RoR, and it comes with support out-of-the-box for a slew of modern features. It really does have everything we could ever ask for in a forum package, but that comes with baggage. I don't know Ruby at all, I am barely functional in Docker, and Postgres is wholly alien to me. Is that a dealbreaker? No. But I am weighing carefully the trade-off between Discourse and what is a comparatively simple LAMP install here on MV (though we use an external MySQL database and some other AWS infrastructure).

The ultimate goal is for the forum to outlive me. To do that, it might need to be less complex, not more.
That makes sense. I'm old-school with Rails and even older-school on Postgres, but my Docker-fu is pretty weak. In my spare time, I've been working on learning and building a proper CI-CD pipeline for Rails->Docker or Kubernetes with Github Actions, but I'm nobody's idea of a Site Reliability Engineer.

From an architectural perspective, containerized deployments are where the future is headed. Increasingly, anything that's not SaaS is being designed to use Kubernetes clusters as the scale-out mechanism and then just loosely coupling the subsystems from there. IBM is in the process of porting pretty much their entire Enterprise & Security software businesses that way, but using OpenShift since they own it via the Redhat acquisition.

Doing a little googling, it seems like Discourse has pretty well killed off most of the competition, other than other apps which are basically clones.

If I can help out on any of the backend pieces, let me know. Depending on how busy I am over the next few days, maybe I'll take a run at setting up a Discourse instance to see what all's involved.

Is the current backend database schema itself customized, or still vanilla phpBB2 on MySQL?
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Wed Apr 21, 2021 9:00 pm quote
chandlerman wrote:
In my spare time, I've been working on learning and building a proper CI-CD pipeline for Rails->Docker or Kubernetes with Github Actions, but I'm nobody's idea of a Site Reliability Engineer.
My day job is as a software engineer for a Very Large Corporation. Mostly device communication stuff. All this IT, web, and server stuff is just something I've picked up out of necessity for running MV. And I am a rank amateur at it, to be sure.
chandlerman wrote:
Doing a little googling, it seems like Discourse has pretty well killed off most of the competition, other than other apps which are basically clones.
Discourse brings a lot to the table. It's a modern re-think of what forums should look like, and also highly customizable through themes and plugins. There are things I *don't* like about it, but there are far more things that I really *do* like.
chandlerman wrote:
If I can help out on any of the backend pieces, let me know. Depending on how busy I am over the next few days, maybe I'll take a run at setting up a Discourse instance to see what all's involved.
Thanks! That said, don't invest too much time unless you're really curious. I managed to get it set up, it wasn't too bad. Most of the install is automated. It's just the migration that was a sticking point.
chandlerman wrote:
Is the current backend database schema itself customized, or still vanilla phpBB2 on MySQL?
It bears a vague resemblance to phpBB2 I've denormalized some parts of it for performance, and simply done away with bits that didn't serve an immediate purpose. I think the real issue is the features that have been added on. Attachments are handled much differently than stock phpBB (and stored in S3, too). Come to think of it, stock phpBB2 didn't even support attachments -- it was a mod, and some bits of it are still in place here now. It's... not a mess, exactly. But definitely a one-off software install.
Molto Verboso
2007 Stella 225
Joined: 02 Nov 2019
Posts: 1663
Location: Rochester, Minnesota
Thu Apr 22, 2021 2:29 am quote
Shows secured for me this morning, but no images in android using chrome browser.
Hooked
P Series / Li / LML / Motobi
Joined: 24 Jun 2019
Posts: 453
Location: UK - 3rd Rock From the Sun
Thu Apr 22, 2021 3:59 am quote
im on it now.... seems snappy enough

ill see if i can break anything
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Thu Apr 22, 2021 4:54 am quote
Christopher_55934 wrote:
Shows secured for me this morning, but no images in android using chrome browser.
Sigh.

This might be a dealbreaker for HTTPS and MV.
Ossessionato
76 Sprint V, 63 GL, 62 VBB, 05 Stella
Joined: 16 Jun 2011
Posts: 3176
Location: Nashville
Thu Apr 22, 2021 7:35 am quote
jess wrote:
Christopher_55934 wrote:
Shows secured for me this morning, but no images in android using chrome browser.
Sigh.

This might be a dealbreaker for HTTPS and MV.
It's working fine for me on Android, Chrome on Windows, and Chrome on Linux as of 10:35 CDT.
Veni, Vidi, Posti
300CC Italian scooter (RIP), 750CC Russian Sidecar,1650CC Bavarian ADV Tourer.
Joined: 21 May 2017
Posts: 8080
Location: The Six
Thu Apr 22, 2021 7:53 am quote
chandlerman wrote:
jess wrote:
Christopher_55934 wrote:
Shows secured for me this morning, but no images in android using chrome browser.
Sigh.

This might be a dealbreaker for HTTPS and MV.
It's working fine for me on Android, Chrome on Windows, and Chrome on Linux as of 10:35 CDT.
I'll say one thing man, you know your stuff.

FWIW images load in Chrome on iOS however the browser is running WebKit in the Apple walled garden.
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Thu Apr 22, 2021 8:06 am quote
chandlerman wrote:
It's working fine for me on Android, Chrome on Windows, and Chrome on Linux as of 10:35 CDT.
Thanks for the update. I have moved the entirety of our static content (*.modernvespa.net) to CloudFront with a new certificate. I don't have any Android devices, and Chrome on Mac isn't as strict, so had no way of knowing if it was working or not.
Molto Verboso
2007 Stella 225
Joined: 02 Nov 2019
Posts: 1663
Location: Rochester, Minnesota
Thu Apr 22, 2021 8:38 am quote
jess wrote:
chandlerman wrote:
It's working fine for me on Android, Chrome on Windows, and Chrome on Linux as of 10:35 CDT.
Thanks for the update. I have moved the entirety of our static content (*.modernvespa.net) to CloudFront with a new certificate. I don't have any Android devices, and Chrome on Mac isn't as strict, so had no way of knowing if it was working or not.
I'll try it again when I get home it'll be around 16:00.
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Thu Apr 22, 2021 9:15 am quote
Christopher_55934 wrote:
I'll try it again when I get home it'll be around 16:00.
Thanks much!
Banned
2:6
Joined: 11 Jan 2007
Posts: 7719
Location: San Francisco
Thu Apr 22, 2021 10:58 am quote
the hell is wrong with the world!

useful help coming from NSM to modernize modern vespa, FFS!
Ossessionato
76 Sprint V, 63 GL, 62 VBB, 05 Stella
Joined: 16 Jun 2011
Posts: 3176
Location: Nashville
Thu Apr 22, 2021 1:19 pm quote
jess wrote:
chandlerman wrote:
It's working fine for me on Android, Chrome on Windows, and Chrome on Linux as of 10:35 CDT.
Thanks for the update. I have moved the entirety of our static content (*.modernvespa.net) to CloudFront with a new certificate. I don't have any Android devices, and Chrome on Mac isn't as strict, so had no way of knowing if it was working or not.
Got the LetsEncrypt going, too, I see.
  DoubleGood Vespa T-Shirts  
Post Reply    Forum -> Not-So-Modern 12Next
[ Time: 0.2418s ][ Queries: 70 (0.1854s) ][ Debug on ][ Thing Two ]