!!! Major Server Problems !!!
Post Reply    Forum -> General Discussion 123Next
Author Message
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Mon May 03, 2021 12:41 pm quote
The prognosis on the database upgrade from last night is... not good. In addition to the bug that was uncovered this morning where posts canít be edited, we now appear to be experiencing table deadlock, and it is grinding the whole site to a halt. The culprit, as usual, seems to be search.

I have just upped the IO capacity of the database somewhat, as we were bumping up against some hard IO limits and then getting throttled. Iím not entirely sure this is the answer, though. I am monitoring the database to see if it starts grinding to a halt again.

If the upgraded IO capacity doesnít solve the problem, I will most likely have to revert to the database as it stood last night at 6PM PDT. That would be... a disaster, and not something I say lightly. We would lose almost 24 hours of data, including any new user registrations that happened during that time. But it might be our only option.

Fingers crossed. The next few hours are going to be tense.
Hooked
Supertech 2019 (EURO3/APAC)
Joined: 17 Nov 2020
Posts: 193
Location: Australia
Mon May 03, 2021 1:51 pm quote
Can you disabled search until you can do some more research?
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Mon May 03, 2021 1:59 pm quote
steelbytes wrote:
Can you disabled search until you can do some more research?
I did consider it! I decided instead, though, to change the two largest tables in the forum database (both of them related to search) back to the MyISAM type. I think this will alleviate the deadlock we were seeing. I am keeping a close eye on it.
Ossessionato
1979 P150X, 1983 P200E, 1988 T5, 1995 PX200E, 2011 Yamaha Fazer 600 S2
Joined: 02 Aug 2015
Posts: 2257
Location: Veria, Greece
Mon May 03, 2021 2:12 pm quote
Jess, although not related with the upgrades I have a continuous issue with Safari on my iOS devices. Every time I visit the site I'll need to login again. Sometimes I will stay logged in for days but out of nowhere the login "circle" will start again. Cookies are enabled on Safari and don't have any limitations for MV. I'd rather not use another browser. Is it a known issue??

Thanks...
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Mon May 03, 2021 2:14 pm quote
I have occasionally seen that happen, but I often go back and forth between the real site and a test version of the site, which also triggers it -- so I thought it was maybe just me I appreciate the report, though -- I will look into it.
Hooked
Supertech 2019 (EURO3/APAC)
Joined: 17 Nov 2020
Posts: 193
Location: Australia
Mon May 03, 2021 2:52 pm quote
myisam is definitely worth a shot. innodb can be a real pain
Ossessionato
VSX, Stella 177, VNX1T, V9B1T
Joined: 20 Jul 2018
Posts: 2963
Location: Staten Island, NY
Mon May 03, 2021 2:55 pm quote
postgre!
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Mon May 03, 2021 2:57 pm quote
swiss1939 wrote:
postgre!
Yes. But not until we change over to an entirely different forum package.
Ossessionato
VSX, Stella 177, VNX1T, V9B1T
Joined: 20 Jul 2018
Posts: 2963
Location: Staten Island, NY
Mon May 03, 2021 3:02 pm quote
jess wrote:
swiss1939 wrote:
postgre!
Yes. But not until we change over to an entirely different forum package.
I saw you mentioning ruby on rails and some other frameworks. I tried some of them like 10 yrs ago but the only one that I really liked was django. Tend to prefer python. But that would require lots of overhead to build in a new framework!

Haven't touched any of that stuff in years so I have no idea which is stable and or seamless anymore.

Or there is always the big one.. ASP.net MVC.
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Mon May 03, 2021 3:09 pm quote
Geek Content Ahead.

Here's a graph showing the Burst Balance for the database. Burst, in this context, relates to IO, not processing power. The database has a certain amount of credits that allows it to exceed its "nominal" IO rate, but only for a while -- when the Burst Balance goes to zero, the IO starts getting throttled. And that's exactly what happened.

You can see in the graph that it was a steady decline in Burst Balance until we hit zero.

I was able to buy my way out of this deficit by increasing the database storage capacity. It seems weird, but the IO capacity is a function of storage capacity (at least it is with Amazon AWS RDS instances). MV only uses about 4gb of our allotted 30gb database storage capacity, so I already had much more storage than I needed. But bumping the storage up to 50gb increased my IO capacity by 50%, so... now I have 4gb of actual data in a bigger partition.

You can also see in the graph the moment I upgraded the capacity, which automatically renewed my burst balance. It dipped a little bit while I was changing the search tables back to MyISAM, but it's been recovering ever since -- so things are headed in the right direction, at least.



Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Mon May 03, 2021 3:14 pm quote
swiss1939 wrote:
I saw you mentioning ruby on rails and some other frameworks. I tried some of them like 10 yrs ago but the only one that I really liked was django. Tend to prefer python. But that would require lots of overhead to build in a new framework!

Haven't touched any of that stuff in years so I have no idea which is stable and or seamless anymore.
The likeliest candidate for new forum software would be Discourse. It's relatively modern (at least compared to most PHP-based forum packages), has a lot of really awesome features, and... it's pretty freaking complicated. But keeping MV's 20-year-old infrastructure running in a modern world is getting to be a nightmare, so maybe the added complexity of Discourse is not the worst of my potential problems.
Veni, Vidi, Posti
MP3 500, GTS 250 (both 2008 MY), 2012 Honda NC700 DCT, 2014 Can Am Spyder RT
Joined: 02 Mar 2013
Posts: 6410
Location: Madison, Wisconsin
Mon May 03, 2021 3:27 pm quote
So, I have a question.
Does the amazon link to sign in to MV still work?
I imagine this is to garner some sort of financial or other support from Amazon to the site.
I just don't recall seeing it 'advertised' recently.
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Mon May 03, 2021 3:44 pm quote
Madison Sully wrote:
Does the amazon link to sign in to MV still work?
I'm not quite sure what you're referring to? I mean, my memory is not what it used to be, but amazon sign in?

We do in fact earn a small commission from amazon links posted here, and our servers do in fact run on Amazon AWS. But AFAIK, that's it.
Hooked
2020 GTS 300 HPE
Joined: 27 Sep 2019
Posts: 414
Location: San Francisco
Mon May 03, 2021 3:47 pm quote
I wonder if there is a way to share the maintenance burden, or at least have a way to eventually do that. For instance, the YouTube embedder was broken for a quite a while. I bet there are enough savvy people around here to handle problems like that, even when you want to take a break.

Thanks for all you do to keep this place going, jess!
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Mon May 03, 2021 4:12 pm quote
mayorofnow wrote:
I wonder if there is a way to share the maintenance burden, or at least have a way to eventually do that. For instance, the YouTube embedder was broken for a quite a while. I bet there are enough savvy people around here to handle problems like that, even when you want to take a break.
It's... complicated. Just getting to the point of having a functioning development environment set up with all the necessary access and tools is a herculean task. So much so that I had, until recently, allowed my own setup to fall into disrepair. It got to the point I thought that we were going to have to watch MV just spin slowly into oblivion like an unmanned and out-of-control concrete surfacer.

I just don't think bringing someone up to speed is practical. Heck, *I* barely know how some of it works. And it's a dead-end investment. MV will either transition to newer, better forum software or it will grind slowly to a halt when all y'all stop visiting forums. Honestly, it could go either way at this point. Possibly both.

But there's no future in the current setup no matter what.

One way that I have found that is a practical way to lighten the load is to have trusted moderators minding the store. Without them, MV would have been dead a long time ago.

Last edited by jess on Mon May 03, 2021 4:27 pm; edited 1 time in total
Molto Verboso
'95 Yamaha Riva 125- '05 Piaggio BV200-'05 Honda Reflex-'08 Honda Metropolitan
Joined: 22 Oct 2005
Posts: 1727
Location: Ohio
Mon May 03, 2021 4:21 pm quote
I don't know anything about what you're dealing with but for the sake of the greatest scooter community in the world I hope you're successful with working this out.

Good luck!!!

Bob
Veni, Vidi, Posti
MP3 500, GTS 250 (both 2008 MY), 2012 Honda NC700 DCT, 2014 Can Am Spyder RT
Joined: 02 Mar 2013
Posts: 6410
Location: Madison, Wisconsin
Mon May 03, 2021 4:45 pm quote
jess wrote:
Madison Sully wrote:
Does the amazon link to sign in to MV still work?
I'm not quite sure what you're referring to? I mean, my memory is not what it used to be, but amazon sign in?

We do in fact earn a small commission from amazon links posted here, and our servers do in fact run on Amazon AWS. But AFAIK, that's it.
Something about having modernvespa.com as part of the amazon sign-on.
Like they do for some charities. Here is my current link to sign onto amazon.com:

https://www.amazon.com/?tag=modevesp-20&linkCode=ur1

I had assumed there was some form of remuneration for having modern vespa in the link. And, if I recall correctly, there was an amazon ad on MV home page that had the link above.

Here I thought I was helping enrich MV and its benefactor(s)...
Ossessionato
VSX, Stella 177, VNX1T, V9B1T
Joined: 20 Jul 2018
Posts: 2963
Location: Staten Island, NY
Mon May 03, 2021 4:51 pm quote
That's just telling Amazon what website referred you to Amazon. Whoever paid for the ad that you are referencing probably profited in some small way off of any purchases using that referral campaign link meta. The extra meta probably included a code for the ad campaign they were running, and this tag for which site referred the click to the campaign. On the other end they get a report on how many clicks and views come from which referral site.

You see the same thing more commonly on YouTube links. ?v=some code equals the specific video, and more meta tags follow it with ampersands.. such as &t=1m4s would start your play of that video at 1 min 4 sec Mark.

It's one of the many ways websites can perform advanced actions from a simple link click.
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Mon May 03, 2021 5:24 pm quote
Madison Sully wrote:
https://www.amazon.com/?tag=modevesp-20&linkCode=ur1
Ah. I see what you're referring to. That is in fact our referral tag, and the one that gets applied to Amazon links here. And if you buy something at Amazon within something like 24 hours after you use that link, then we do in fact get credit for it. It's not a huge amount, but it's something.

I had forgotten that you could just go directly to Amazon with the tag, independent of a product link. So yes, you are in fact helping to offset the MV costs by using that link.

That said, I pay Amazon much more every month than they pay me.
Veni, Vidi, Posti
MP3 500, GTS 250 (both 2008 MY), 2012 Honda NC700 DCT, 2014 Can Am Spyder RT
Joined: 02 Mar 2013
Posts: 6410
Location: Madison, Wisconsin
Mon May 03, 2021 5:28 pm quote
jess wrote:
Madison Sully wrote:
https://www.amazon.com/?tag=modevesp-20&linkCode=ur1
Ah. I see what you're referring to. That is in fact our referral tag, and the one that gets applied to Amazon links here. And if you buy something at Amazon within something like 24 hours after you use that link, then we do in fact get credit for it. It's not a huge amount, but it's something.

I had forgotten that you could just go directly to Amazon with the tag, independent of a product link. So yes, you are in fact helping to offset the MV costs by using that link.

That said, I pay Amazon much more every month than they pay me.
Well I'm happy to throw whatever pennies I can in your general direction.
Since it's the link I use to open amazon every time I open amazon, well, you get that little tiny something on all of my purchases. And I'm glad to hear it.
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Mon May 03, 2021 5:51 pm quote
Madison Sully wrote:
Well I'm happy to throw whatever pennies I can in your general direction.
Since it's the link I use to open amazon every time I open amazon, well, you get that little tiny something on all of my purchases. And I'm glad to hear it.
Thank you for your support!
Veni, Vidi, Posti
300CC Italian scooter (RIP), 750CC Russian Sidecar,1650CC Bavarian ADV Tourer.
Joined: 21 May 2017
Posts: 8080
Location: The Six
Tue May 04, 2021 3:44 am quote
Wow, don't check in as often and it's amazing the things you miss.

I expect there will be some hiccups with a major upgrade and I'm going to assume this is a phpbb to discourse migration. Anyway I know for a fact the vast majority of us are going to be patient and it's been pretty damn smooth so far. At least now we don't need to worry about the server suffering a slow death as it's platform becomes more and more obsolete and no longer supported on modern OS's which translates to modern hardware/infrastructure.
Addicted
Bo - midnight blue GT200 R.I.P. Tethys - 300 GTS Titanium
Joined: 22 Jul 2008
Posts: 520
Location: Bowie, MD
Tue May 04, 2021 3:56 am quote
jess wrote:
Madison Sully wrote:
Well I'm happy to throw whatever pennies I can in your general direction.
Since it's the link I use to open amazon every time I open amazon, well, you get that little tiny something on all of my purchases. And I'm glad to hear it.
Thank you for your support!
I think we all wouldn't mind doing that. I use Smile, but I'll switch all my bookmarks over to your tag.

I know it can be more effort than it is worth to get someone up to speed on your current set up. I've got some scary servers like that at work. But, could those of us with certain tech skills help you start setting up discourse? I can donate my time. I know I come and go, but I would be a sad puppy without MV. We just need to find a way to make it more sustainable. I don't mind putting sweat equity or $$ into that.

Let us know if there is a way to help that doesn't hurt!
KimPossible.
Veni, Vidi, Posti
In garage: Yamaha Tricity 155 Urban 2019 - MV Agusta 125 RS 1956
Joined: 21 Jul 2007
Posts: 5650
Location: Latina (Italy)
Tue May 04, 2021 4:12 am quote
If there is an annual contribution to be made, I am available.
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Tue May 04, 2021 6:23 am quote
KimPossible wrote:
I think we all wouldn't mind doing that. I use Smile, but I'll switch all my bookmarks over to your tag.
Thank you!
KimPossible wrote:
But, could those of us with certain tech skills help you start setting up discourse? I can donate my time. I know I come and go, but I would be a sad puppy without MV. We just need to find a way to make it more sustainable. I don't mind putting sweat equity or $$ into that.
I very much appreciate it. And there may yet be something to do there. I have already done a trial install of Discourse -- me and the moderators took it for a joyride for a couple weeks, posting gibberish and generally trying to figure out the high points and the low points. From a user's perspective, it's got a lot to offer, though it's a bit more complex in some ways than phpBB.

From an administration perspective, it's a *lot* more complex, but there's actually a team of people developing it, rather than just a team of one (me) making a mess of things. So that's a balancing act.

The real obstacle, though, is migration. Discourse has built-in migration scripts, but they don't directly apply to MV's rather, errr, customized data set. So it will really come down to whether I can write a custom migration in a language I don't know yet (Ruby on Rails) into a database I am unfamiliar with (Postgres) using some technologies that didn't exist when MV was started (Docker). Even simple things like migrating the user accounts with passwords intact -- considering that all MV passwords are stored hashed -- will be a major challenge.

So given that daunting task, I've chosen to focus on some of the more immediate issues facing MV. We've successfully moved off of a 10-year-old AWS instance to a modern ARM-based platform with a mostly-current PHP version, so that's a big relief. We are still on an aging database version that is near EOL (MySQL 5.7) but I've managed to get most of the tables converted to InnoDB (though not without some bumps, as per this thread). The forum software appears to have problems with MySQL 8.0, and I think that's a lurking character set issue that may yet rear its ugly head in the near future. I am debating if we can ride out the remainder of MySQL 5.x's lifespan or not. And search is hopelessly broken, but I think I would rather move to a new forum package than roll a custom search solution.
KimPossible wrote:
Let us know if there is a way to help that doesn't hurt!
KimPossible.
If any of that sounds like something in your wheelhouse, let me know.
Addicted
Bo - midnight blue GT200 R.I.P. Tethys - 300 GTS Titanium
Joined: 22 Jul 2008
Posts: 520
Location: Bowie, MD
Tue May 04, 2021 7:24 am quote
jess wrote:
The real obstacle, though, is migration.
Eh, don't do it.

Let the Wayback Machine https://archive.org/web/ take care of it. Allow crawling and turn the Internet Archive loose on MV.

And if you want more control you can do a basic subscription of Archive-IT https://www.archive-it.org/blog/learn-more/ to shape what is kept and how it is presented. This kind of community knowledge is exactly what the Internet Archive was built for.

My advice would be a clean start on discourse(folks can take advantage of the opportunity to spruce up their avatars and sig files.) With a comprehensive (and user friendly!) 'classic' MV on the Wayback machine.
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Tue May 04, 2021 8:05 am quote
KimPossible wrote:
Eh, don't do it.

Let the Wayback Machine https://archive.org/web/ take care of it. Allow crawling and turn the Internet Archive loose on MV.
Iíll admit that is a tempting path. I fear, though, that without the 15+ year legacy, MV is, well... something less than whole. I donít think we have enough active posters around at present to sustain the critical mass that forums require to survive. Moving everyone to an empty Discourse install would then possibly be the beginning of the end.

Maybe.
Ossessionato
2009 GTV250 (Gone), 2003 Inder trailer, 2001 BMW R1100RT
Joined: 24 Jun 2009
Posts: 4783
Location: Santa Cruz California
Tue May 04, 2021 8:07 am quote
I don't have anything to contribute from a technology perspective since I decided decades ago to not get involved in Internet, database or search engine engineering.

But I was thinking the same thing as Kimpossible. A fresh start with someway to search the "classic" MV site but don't allow posting to it.

BTW, MV's current search engine seems to work for me but maybe you've shifted back to the old server for the moment. As an alternative search function, I also use Google, and I presume other search engines, to search MV using "<search> site:modernvespa.com" in the search filed. That seems to work quite well for me and often ties a bit better than the MV Search function.

BTW, I like some other members have been reasonably successful in life professionally and would be glad to contribute to MV's upkeep. NO reason for you to carry all the water Jess.

A BMW motorcycle site I like (bmwst.com) has a page for donations. It, like MV, is all volunteers but I send a few bucks every year to help pay for the site operating fees and licenses.

Best
Miguel
Addicted
Bo - midnight blue GT200 R.I.P. Tethys - 300 GTS Titanium
Joined: 22 Jul 2008
Posts: 520
Location: Bowie, MD
Tue May 04, 2021 8:13 am quote
Take a look at the redirect service. Discourse MV wouldn't have to be divorced at all from its historical content. I was thinking this would probably be the best option for MV anyway.

https://archive-it.org/blog/files/2021/03/Archive_It_redirection.pdf

So I'll post a link and then stop beating a dead horse.
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Tue May 04, 2021 8:37 am quote
Miguel wrote:
I don't have anything to contribute from a technology perspective since I decided decades ago to not get involved in Internet, database or search engine engineering.
I canít decide if you missed the boat or dodged a bullet.
Miguel wrote:
But I was thinking the same thing as Kimpossible. A fresh start with someway to search the "classic" MV site but don't allow posting to it.
Iím not against the idea, per se, but it wouldnít be my first option. My first option would be to get MV fully migrated to Discourse. My next option would be to keep MV running in its current state, possibly with some improvements (search, etc). Moving to Discourse without the legacy history would probably be my third option.
Miguel wrote:
BTW, MV's current search engine seems to work for me but maybe you've shifted back to the old server for the moment.
I switched the two problem tables back to their original form. Itís still all part of the new server(s), but the search tables are using an older (and apparently less problematic) database engine type (MyISAM). So itís working as well as it ever has, which is to say itís functional but not very good.
Miguel wrote:
As an alternative search function, I also use Google, and I presume other search engines, to search MV using "<search> site:modernvespa.com" in the search filed. That seems to work quite well for me and often ties a bit better than the MV Search function.
That should definitely work better than MVís built-in search function. That said, Iíd prefer to have a native search function that didnít require going out to Google (or giving Google any more data than they already have).
Miguel wrote:
BTW, I like some other members have been reasonably successful in life professionally and would be glad to contribute to MV's upkeep. NO reason for you to carry all the water Jess.
The financial burden isnít really the issue for me at the moment. Though MVís operating costs have definitely escalated over the years -- from something like $20/month on our first webhost provider in 2005, up to the $150/month or so that it was costing us up until last month. I have continued to pay the bills mostly out of my own pocket because this is what I signed up for: providing MV as a community service.

The financial forecast for MV is actually improving, too. I expect the operating costs to plummet now that Iíve switched over to modern servers and consolidated some things. Weíre only 4 days into the new month, so itís hard to make a solid estimate yet, but I will be shocked if we spend more than $100/month from here on out. In the grand scheme of things, itís not onerous. I spend more than that on my familyís phone service.
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Tue May 04, 2021 8:43 am quote
KimPossible wrote:
Take a look at the redirect service. Discourse MV wouldn't have to be divorced at all from its historical content. I was thinking this would probably be the best option for MV anyway.

https://archive-it.org/blog/files/2021/03/Archive_It_redirection.pdf

So I'll post a link and then stop beating a dead horse.
Thatís a really cool service, actually. I will seriously consider it. Itís still not my first option, but itís definitely a viable option.
Ossessionato
1991 T5 Pole Position, 2008 LXS 125, 2013 Peugeot Metropolis RS
Joined: 06 May 2011
Posts: 3486
Location: Staffordshire England
Tue May 04, 2021 8:53 am quote
I won't pretend to understand what's the problem, but I just want to tell you, good luck. We're all counting on you
Veni, Vidi, Posti
'07 LX150 (Sold), '17 GTS300, '16 BV350, '15 EN650, '09 FXDF
Joined: 26 Jul 2017
Posts: 5568
Location: Home of the Alamo
Tue May 04, 2021 8:54 am quote
jess wrote:
Miguel wrote:
I don't have anything to contribute from a technology perspective since I decided decades ago to not get involved in Internet, database or search engine engineering.
I canít decide if you missed the boat or dodged a bullet.
Miguel wrote:
But I was thinking the same thing as Kimpossible. A fresh start with someway to search the "classic" MV site but don't allow posting to it.
Iím not against the idea, per se, but it wouldnít be my first option. My first option would be to get MV fully migrated to Discourse. My next option would be to keep MV running in its current state, possibly with some improvements (search, etc). Moving to Discourse without the legacy history would probably be my third option.
Miguel wrote:
BTW, MV's current search engine seems to work for me but maybe you've shifted back to the old server for the moment.
I switched the two problem tables back to their original form. Itís still all part of the new server(s), but the search tables are using an older (and apparently less problematic) database engine type (MyISAM). So itís working as well as it ever has, which is to say itís functional but not very good.
Miguel wrote:
As an alternative search function, I also use Google, and I presume other search engines, to search MV using "<search> site:modernvespa.com" in the search filed. That seems to work quite well for me and often ties a bit better than the MV Search function.
That should definitely work better than MVís built-in search function. That said, Iíd prefer to have a native search function that didnít require going out to Google (or giving Google any more data than they already have).
Miguel wrote:
BTW, I like some other members have been reasonably successful in life professionally and would be glad to contribute to MV's upkeep. NO reason for you to carry all the water Jess.
The financial burden isnít really the issue for me at the moment. Though MVís operating costs have definitely escalated over the years -- from something like $20/month on our first webhost provider in 2005, up to the $150/month or so that it was costing us up until last month. I have continued to pay the bills mostly out of my own pocket because this is what I signed up for: providing MV as a community service.

The financial forecast for MV is actually improving, too. I expect the operating costs to plummet now that Iíve switched over to modern servers and consolidated some things. Weíre only 4 days into the new month, so itís hard to make a solid estimate yet, but I will be shocked if we spend more than $100/month from here on out. In the grand scheme of things, itís not onerous. I spend more than that on my familyís phone service.
Many thanks for your dedication to this site Jess.
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Tue May 04, 2021 9:25 am quote
Jet Peddler wrote:
Many thanks for your dedication to this site Jess.
Thanks, I appreciate it!
Ossessionato
2009 GTV250 (Gone), 2003 Inder trailer, 2001 BMW R1100RT
Joined: 24 Jun 2009
Posts: 4783
Location: Santa Cruz California
Tue May 04, 2021 10:35 am quote
jess wrote:
Miguel wrote:
I don't have anything to contribute from a technology perspective since I decided decades ago to not get involved in Internet, database or search engine engineering.
I canít decide if you missed the boat or dodged a bullet.

Definitely feel I dodged the bullet. I've focused on broadband radio, satellite and fiber communication transport systems for decades now. Lot fewer people with the skills and I love the mathematical, physics and business strategy parts of the work I do.

Cheers
Miguel
Veni, Vidi, Posti
In garage: Yamaha Tricity 155 Urban 2019 - MV Agusta 125 RS 1956
Joined: 21 Jul 2007
Posts: 5650
Location: Latina (Italy)
Tue May 04, 2021 10:41 am quote
Would you be amazed to know that I am using a 4G plus data card for my internet connection?
I pay Ä 13 per month for 100 Gb of monthly data.
Hooked
2020 GTS 300 HPE
Joined: 27 Sep 2019
Posts: 414
Location: San Francisco
Tue May 04, 2021 10:57 am quote
jess wrote:
it will really come down to whether I can write a custom migration in a language I don't know yet (Ruby on Rails) into a database I am unfamiliar with (Postgres) using some technologies that didn't exist when MV was started (Docker).
If it were me, I'd explore writing the migration in whatever language best suited my skillset and the task at hand. Discourse shouldn't care what language wrote to the database as long as it is structured in the way Discourse expects.

For places where implementation details leak (like if Discourse can't grok phpBB's hashed passwords), perhaps an intermediary layer could check your password against the old DB, and then write it to the new one if it matches. You'd have to go to that migration page once to migrate your account, and then you'd be in the Discourse promised land thereafter.
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Tue May 04, 2021 12:50 pm quote
Miguel wrote:
I've focused on broadband radio, satellite and fiber communication transport systems for decades now. Lot fewer people with the skills and I love the mathematical, physics and business strategy parts of the work I do.
I can definitely appreciate that.
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Tue May 04, 2021 12:50 pm quote
Attila wrote:
Would you be amazed to know that I am using a 4G plus data card for my internet connection?
I pay Ä 13 per month for 100 Gb of monthly data.
And on top of that, you get to live n Europe.
Petty Tyrant
GTS250 GTS300 MP3 500
Joined: 11 Oct 2005
Posts: 30075
Location: Bay Area, California
Tue May 04, 2021 12:58 pm quote
mayorofnow wrote:
If it were me, I'd explore writing the migration in whatever language best suited my skillset and the task at hand. Discourse shouldn't care what language wrote to the database as long as it is structured in the way Discourse expects.
Agreed that would be preferable. I haven't seen the Discourse schema, so it's unclear to me how much "ingestion" must be done in addition to just putting the records into place. In other words, how much additional data is cross-linked to each post?

For instance, each post at MV is stored in a row more or less as you'd expect, identifying the user, the topic, and some other stuff. But then there's a separate post hash table (to prevent duplicate postings) that also has to be kept up to date, a post cache of the rendered HTML form of each post, and then each post is also digested into the search index.

But if I can figure out how Discourse stores its data, then yes. Absolutely.
mayorofnow wrote:
For places where implementation details leak (like if Discourse can't grok phpBB's hashed passwords), perhaps an intermediary layer could check your password against the old DB, and then write it to the new one if it matches. You'd have to go to that migration page once to migrate your account, and then you'd be in the Discourse promised land thereafter.
Yep. I had something like that in mind. There are a few possibilities there, and I'm still scheming more ideas. Ultimately, someone has to implement it all, though, and that someone is most likely me.
  DoubleGood Vespa T-Shirts  
Post Reply    Forum -> General Discussion 123Next
[ Time: 0.0686s ][ Queries: 27 (0.0279s) ][ Debug on ][ Thing Two ]