Main > Forum/Website Discussion

*** SERVER ISSUES *** - we are currently experiencing a heavy load on the server

<< < (6/14) > >>

Vigo:
Saint, anything we can do to help with the recent issues?

shock_:
I just saw a slowdown at about 3-4 mins past the hour, but it's come good again. It's just now a little past midnight US Eastern. In the next couple of hours if history is a guide, the site is going to slow again and then throw 500 errors.

Given the symptom (500 errors from a server running SMF) and the timing (daily, just after "midnight", in someone's US timezone at least) I'm willing to guess:


* SMF has too many scheduled tasks set to fire at 00:00 that don't need to be running at exactly that time - see http://wiki.simplemachines.org/smf/SMF2.0:Scheduled_Tasks
* Those tasks are possibly creating a queue on MySQL.
* PHP is throwing "MySQL has gone away" errors into an appropriate log once calls out to the DB exceed its timeout
* Web server is slow to respond while it still trusts MySQL might eventually have an answer for it, then starts sending quick 500s once it's sure it won't be back
Things I'd do

Check for active queries while the site is slow.
Assuming there's direct access to the database (e.g. command line or even PHPMyAdmin), send this query to the site database as the site database's user:

SHOW FULL PROCESSLIST\G

If the database user has the necessary privileges this will braindump the active db queries. Things to look for are the lines with "Time:" and "State:" -- if there are heaps of rows and they're slow, well, yeah, database bottleneck. The queries themselves might give a clue exactly WHAT it's busy with, which might lead to some ideas about shifting that activity to some other time etc.

Check for any spikes in site traffic.

Count rows per minute in the web server's access.log file, look for any spiky patterns for IP addresses or User Agents. Could be as simple as something set to crawl the site at the same time each night.

Make sure someone with SMF knowledge is involved

I'm talking very generally here, a lot of the true fixes are going to require knowledge of how to tune SMF to avoid being wasteful. Perhaps there was a spamfest a few months ago that led to hundreds of thousands of dummy registrations in the database, now slowing down daily admin functions due to sheer size. Maybe a feature was enabled that runs some daily background processing and nobody remembered to turn it off. I don't know, I'm a Drupal guy, not an SMF guy, but all commodity web apps have their quirks and all can be abused from time to time :)

RayB:
man it's SOOOO slow.
Speaking of donations, I agree with this idea. Saint, set up a Patreon. I'd pay a buck or two a month to keep this place running.

https://www.patreon.com/

Token:
I'm all for throwing money at the problem, but that does no good if saint's TOP MEN don't know what the problem is.



It took five minutes of page refreshing to post this comment.

Nephasth:

--- Quote from: Token on May 04, 2016, 09:13:38 pm ---saint's TOP MEN...


--- End quote ---

 :laugh2:

Navigation

[0] Message Index

[#] Next page

[*] Previous page