jammerbirdi
Joined: 23 Sep 2004
Posts: 21046
Back to top |
 Posted: 09/23/16 1:22 am ::: |
Reply  |
|
| Bob Lamm wrote: |
| Thanks to Reb and to the moderators for all that you do. It shouldn't be taken for granted. I am very grateful. |
On behalf of the admins, thank you. But please, all praise be to Reb. Seriously.
Posted: 09/12/16 10:12 pm
| rebkell wrote: |
Well, tonight was a hell of a night. I spent probably 6 hours going absolutely nowhere on the new server. I couldn't even access the damn thing with a web browser. It was a setting on my end on my PC that I didn't set. I still don't exactly understand what the difference was, but no matter, at least I can get to it and tinker.
This software works over there, which was really a surprise to me. Funny thing though the same damn thing that was wrong with this board over the weekend was wrong over there, the actual username that the board uses to access the db wasn't able to connect. I think I now understand the problem, don't know why it would have happened, but it must have had something to do with the changeover and the password checking software. It says the password is too weak and won't accept it. Whatever, at least I have a better understanding of the problem.
Anyway, I may try to get started on installing a new board, if I've got anything left in the tank tomorrow, this was one damn frustrating stretch this evening. It helps a lot that things finally fell into place a bit, I wasn't prepared to relearn the entire damn system.
I'll try to keep you in the loop, when I get the new software installed, I'll tell you how to access it, you'll have to make a couple of tweaks to your PCs, the way I've got mine set right now, I can get to both boards, the new one is www.boards.rebkell.net and the other one is boards.rebkell.net. It won't work for you guys until you make the changes on your pc and there isn't anything to see yet anyway. |
Posted: 09/13/16 8:18 pm
| rebkell wrote: |
Didn't get much accomplished tonight, had to work late and just too worn out for a long session. I did get phpBB3 installed onto the server(it's not functional, just got all the files on the server), now I need to do a bit of reading. I have considered just starting the new board from scratch, maybe prune it and just have the last year or so of posts and leaving access to the old board, just read only. Nothing is definite, just a thought.
We may end up with a fairly plain jane style at least to start with, one thing that messed up the last installation a bit was with the style and mods that I made to the board it made it much more difficult to apply security patches and just normal upgrades. I need to make sure the styles we install are compatible with things.
Hopefully tomorrow will be a little less intense at work and I can play around some more.
Anyway, that is all for tonight. |
Posted: 09/14/16 10:19 pm
| rebkell wrote: |
Another night, learned quite a bit, also wasted several hours on some ridiculous errors. The instructions for upgrading are quite lacking in their steps. Then got some silly errors that I can't believe aren't very common, yet searches practically came up with nothing. Finally did find someone that had the error about a search being too long. The ridiculous part about the error is that the common thing you have to do is rename the tables and so the simplest thing was tacking a 3 for phpBB3 onto the name, and this made the search term too long.
Sorry I'm rambling, but finally did figure out how to start the conversion and then really ran into some stuff that shouldn't be able to happen and it concerns some of the tables on this current database. I had a page full of duplicate usernames, that shouldn't be possible, yet there was a shit pot full of them.
I do know one thing, getting the challenge on signups should help a ton. We have probably around 600,000 usernames, from spammers coming by in droves and signing up. I'm expecting the challenge will keep them from even showing up in the database. The way it is now, they end up as users, they just don't get activated, but they still clog up the works.
Anyway, I'll probably give it a fresh go tomorrow, but I'm going to try and clean out the inactive users and see how the conversion goes then.
I know most of this doesn't make a lot of sense, but just updating. |
Posted: 09/15/16 5:40 am
| rebkell wrote: |
Nah, nothing that can be done like that. That was one of the real weaknesses of this board. There are over 750,000 signups, with around 2500 active or have been active at one time. Believe me the 2500 aren't a problem. The board software has a one member at a time removal method. That would take who knows how long.
I'll write a script to get rid of them, if they were just in the one table it would be a snap, but every signup puts them in several tables. By the time the move takes place we'll probably be humming.
With all this it still isn't going to solve the problem of getting bombarded with 100's of connections at one time, from web crawlers or just assholes on the web. |
Posted: 09/15/16 9:43 pm
| rebkell wrote: |
Well, no real updates for tonight. I'm working on writing some scripts to delete the 750,000+ users. It's a bit tricky, it's amazing, but I can probably delete the whole lot in less than a second, so I gotta make sure I know what I'm doing. So mostly it's been a study of the database and trying to figure out where all the data gets put when users are created, it's really convoluted.
The newer version of the board has ways to mass delete this stuff, but not in this old version we are running. With the weekend I hope to get the new board software installed and a method for wiping out all the crap. I'll give it a trial run and then when the time comes I should be able recreate the whole thing and with a bit of luck we can switch over the board in few minutes.
Anyway, that's all I got for tonight. It's actually a good thing that things aren't going that smoothly, I learn a lot more from things going wrong than I would if it all just fell into place. False sense of security. I sent in a request to the hosting outfit to recompile a program so the captcha will work correctly.
later |
Posted: 09/17/16 1:42 pm
| rebkell wrote: |
Still alive, it's been a little tricky figuring out these tables, deleting the bad guys could potentially take us down for hours. So I've had to figure out another method. I think I have it pretty well figured out. I figured out how to spot the bad guys, but finding out to get only the good guys wasn't as straightforward. But after some studying I feel pretty comfortable with it, then there are two other tables that get modified every time there is a new signup. So I have to redo those also.
I know that sound like a bunch of gobbledygook, and for a while every thing I was trying to do was like that, but I do see a way to create new tables with only the good guys and their groups associated with them. Now I basically need to write the scripts to do this. When this is done, then I have to convert to the new board software. This is still an unknown as far as time is concerned. I suspect it may take anywhere from 30 minutes to a few hours to convert. The big deal will be moving all the posts to the new board. I haven't done it yet, but I'm pretty sure that it will have to move all the messages from one table to another one.
I don't know all the ins and outs of the conversion and may never completely understand all it does, but I suspect that it will convert all the users over and probably give them new user_id's ( not username, just the ID associated with them) and in turn the conversion will copy all the messages over and then will have to change all the user_id's associated with the posts to the new user_id it created during the conversion.
It's rather strange how tight lipped the creators of the board software are about the internal workings of the board, being as it's open sourced software. I suspect it's because there are so many people that run this software with wildly varying degrees of knowledge of the programs and database syntax used and they don't want to put out any kind of hacks other than creating programs to do things, and they didn't have a way to fix our problem, so they just say sorry.
Anyway, can't believe Houston and Fla. State FB score today, wow.
later |
Posted: 09/17/16 8:45 pm
| rebkell wrote: |
| Currently doing a trial board update, looks like it's gonna be a 2+ hour process. I did the user table changes I talked about above and they seemed to have worked just fine. So just another update, it's been running for almost an hour on the upgrade process and it's moved over around 620,000 messages out of 1,400,000+ messages total, it's pretty consistent. I'll let you know how it ends. |
Posted: 09/17/16 11:54 pm
| rebkell wrote: |
Well, it worked. Amazing everything went pretty well after the change of the tables I mentioned above, it took around 2 hours and 45 minutes to move all the posts over to the new one, that was a bit much, but with 1.4 million posts I guess it is to be expected.
I about had a cow though, the board started up, logged in, damn near forgot my password, hadn't used it in so long and it's on autofill here and it even autofilled on the version of this they moved to the new server. But finally remembered it and got in, the admin control panel has got all kinds of shit, I don't know what all it's got, but I'm sure it's loaded with a lot more controls.
but anyway, the cow part. You actually have to log in to the ACP(admin control panel) to do anything, and you have to actually log out of the admin control panel to even surf the board. so I thought I'd check out the board and see how it worked. Clicked on this and that and moved around no problem, then I clicked on a post to read and wham error, worse than that damn database error we get here, it couldn't allocate enough memory to open any damn thing. I was just about to shit, here I go scouring the web looking for the error and everything says raise the memory allocation for php, funny thing is though on the version of this software over there I can read all the posts I can do anything and everything on it, but on the new board I couldn't open one damn post without a major error.
I finally went to the main server configuration and found the memory for php and it was sitting on 16M(meg) and all the things I was reading they were recommending to raise the limit to 128M or even 256M, so I decided 16M was quite low, so I raised it to 128M and everything worked fine.
At least I know it can be done or so it seems. I'll check in tomorrow, let me know what kind of PC's you guys are using and I'll dig up the commands you need to put on them so you can get to the new board, so you poke around. I don't think there is anyway you can get there with phones or tablets, you need to change a setting in your PC to allow it to use it.
later |
Posted: 09/20/16 8:55 pm
| rebkell wrote: |
| Luuuc wrote: |
Yeah, for the most part the place runs fine, and from everything you've been saying, the problem times are nothing whatsoever to do with the board software or the specs of the server, so I guess that will still be a separate issue to tackle but I'm sure we're far from the first site to experience this issue so there must be mitigation measures available out there somewhere. I would hope that liquidweb would be willing and able to provide some assistance in that department. |
Yeah, you can tell from some of the things in the new Admin Control Panel, that they are addressing some of the issues commonly encountered. They have a whole list of Bots listed, when I changed over the software, it didn't work correctly the first time and I was looking at some of the tables it created and the first one was the users table and the first 25 or so users were bots, 1 was Anonymous, 2 was me, and the rest were bots. I was reading that when it recognizes a bot that it doesn't send out cookies and such. You can define certain connections and list them as bots, etc... not sure if it will help or not, but I gotta believe it may give us a fighting chance and a lot more information as to what is going on.
I know we get attacked sometimes too, I don't think we really have any known security holes, nothing is totally secure, but I don't think we're any more open than most sites, but being the version we are we get a lot of attempts at some of the known exploits.
I'm looking forward to getting going with the new software. |
|
|
