Is this really a good idea?
Normal
Small
Posted by dballing

In Marimacc's post on the old forums, she wrote:

[the change] will also allow the forums to become more stable, as they will be hosted with the rest of the Gen Con site .

Is that actually the case? In times of super-server-stress (pre-reg opening, etc.) isn't it far FAR better that the forums be as way-the-heck-far-away from the Gen Con site as possible, so that when users bring the site to its knees, the lines of communication remain open to all attendees?

By resting within "www.gencon.com", which is also where registration happens, it means that -- at minimum -- the forums are sharing the same load-balancing hardware and -- worst case scenario -- are on the same physical servers are the event registration engine, etc.

Every fiber of my "I do this for a living" being is crying out that this is a nightmare waiting to happen. Someone explain to me how I'm wrong. (And please, use long technical terms, because that's really what I'm looking for in order to feel warm and fuzzy about this).

Normal
Small
Posted by marimaccadmin

Becuase the old forums were hosted IN the office.  They went down all the time.  For power outage reasons at the office (which actually happens a lot), for issues with that local server, etc.

They are now being hosted at the same place the rest of the site is, an actual server farm with redundancy and back up power, etc.  They will therefore, in fact, be much more stable, and much less prone to being down for a day or even a weekend or more.

Normal
Small
Posted by dballing

EXCEPT in those cases when the actual server farm and network gear is being hammered during event registration, a time when we have seen portions of the server farm infrastructure fall completely over.

And in those cases where that happens, the forums have been a vital method of communication, for (a) Gen Con to users, to communicate what's happening and resolution timeframes, (b) users to Gen Con to communicate what problems are being seen, in real time, and get them due attention, and (c) user to user to share thoughts of their registration experience.

During "peak demand" periods, if that infrastructure is hammered to the point of breaking, you lose not only the application in question, but also the communications medium which has been heretofor a primary-source for information about what's going on and how to diagnose it and get it resolved.

I don't dispute that having them hosted in an actual colocation facility - generally speaking - provides them with much greater stability. Where I get off the bus is that having them still be in the same domain-space as the registration site, using the same infrastructure as the registration site, is - frankly - a rookie mistake.

Normal
Small
Posted by marimaccadmin

Thank you for your input, as always, Derek. :)

Normal
Small
Posted by jsfetzik

As long as the forums are not on the same server as any of the registration system it "should" be ok during registration. Just make sure it is not on the same network segment within the hosting facility.

Based on observerd behavior, and 20+ years of IT experience, the problem during registration is not an overload of network bandwidth to the servers, but overload of the servers themsleves. The traffic gets there, but the server cannot respond in a timely manner and it does not failover in a graceful manner.

Normal
Small
Posted by dballing jsfetzik

jsfetzik wrote:
As long as the forums are not on the same server as any of the registration system it "should" be ok during registration. Just make sure it is not on the same network segment within the hosting facility.
Based on observerd behavior, and 20+ years of IT experience, the problem during registration is not an overload of network bandwidth to the servers, but overload of the servers themsleves. The traffic gets there, but the server cannot respond in a timely manner and it does not failover in a graceful manner.

Not entirely true.

The forum is on "www.gencon.com". The reg system is on "www.gencon.com" ... that means that -- at some layer -- the same hardware is handling both forum requests and registration requests, even if its just a load-balancing layer that is slicing /forum/ and sending to "Pool F" and slicing /reg/ (or whatever URL pattern that uses) and sending that off to "Pool R".

In which case, if you end up overloading the load-balancing layer, you will have negatively impacted forum performance at the same level of impact as you have the registration performance.

That's the BEST case scenario. The worst case is substantially worse, as you might imagine.

That's based on 20+ years of building out data-center environments with literally the same sorts of problems Gen Con faces.

Normal
Small
Posted by remnant

I figure if you want to get a hold of GenCon you email or call Customer Service (even in an emergency) no?  Expecting a forum thread to be your primary immediate path to getting things resolved... meh... maybe it's very possible but probably not the fastest. 

Personally I think it's better to sit and wait for more data on how things work on these 'new forums' get some burn in time for the new forums and let it grow on me before I make judgement.  Besides, it's their Con and their forums I can make all the suggestions about how I want things done or how I would do things but until I start my own Con or things go sidesways and I'm in charge of GenCon (Something very bad probably happened if this is the case)... Good job team GenCon and thanks. 

Normal
Small
Posted by dballing remnant

remnant wrote:
I figure if you want to get a hold of GenCon you email or call Customer Service (even in an emergency) no?  Expecting a forum thread to be your primary immediate path to getting things resolved... meh... maybe it's very possible but probably not the fastest. 

Actually, during reg problems, the forums have often the fastest means for identifying problems as being spurious (1 or 2 people) or chronic (100s/1000s), and customer service e-mails would often be swamped and non-responsive (because by definition it's 1-to-1 dialog versus 1-to-many).

You may be new, or new enough that you've not been around for a truly great melt-down (it's been a couple years), but when it happens, the forums are vital.

Personally I think it's better to sit and wait for more data on how things work on these 'new forums' get some burn in time for the new forums and let it grow on me before I make judgement.  Besides, it's their Con and their forums I can make all the suggestions about how I want things done or how I would do things but until I start my own Con or things go sidesways and I'm in charge of GenCon (Something very bad probably happened if this is the case)... Good job team GenCon and thanks. 

No argument it's their forums and such. I have no problem with the forums themselves, merely the way they're currently implementing them.
 

Normal
Small
Posted by marimaccadmin remnant

remnant wrote:
I figure if you want to get a hold of GenCon you email or call Customer Service (even in an emergency) no?  Expecting a forum thread to be your primary immediate path to getting things resolved... meh... maybe it's very possible but probably not the fastest. 
Personally I think it's better to sit and wait for more data on how things work on these 'new forums' get some burn in time for the new forums and let it grow on me before I make judgement.  Besides, it's their Con and their forums I can make all the suggestions about how I want things done or how I would do things but until I start my own Con or things go sidesways and I'm in charge of GenCon (Something very bad probably happened if this is the case)... Good job team GenCon and thanks. 

Thanks remnant!  Yes, you can always email or call Customer Service, and of course, if you have Facebook, you can check there as well.  The Gen Con guys have worked really hard on these new forums, and I think you're going to like how much more stable they are, and integrated with the actual website.  Of course, please do let us know if you have any issues, down in the technical support area.  Thanks remnant!

marimacc

-forum moderator

Normal
Small
Posted by dballing

As the forums become very sluggish and almost-unresponsive during event pre-reg, I want to just say, "Yep, I told you so."

Normal
Small
Posted by braewe

Yep. Totally agree.

Normal
Small
Posted by flamepulse

seem to be responding fine to me.

Normal
Small
Posted by dballing flamepulse

flamepulse wrote:seem to be responding fine to me.

Well, the later we get the fewer people are actively beating up the systems.

It was quite heinous in the first 30-45 minutes.

Normal
Small
Posted by dangado

I mean if you think a 15 second refersh is agonizingly slow I guess it was, but so far it's been up and steady the entire time.

Normal
Small
Posted by parody

How about 503 Service Unavailable after more than 15 seconds?  I was getting those off and on for the first half-hour or so.  (A bunch early on, then less as things calmed down.)

Normal
Small
Posted by dballing

Yeah, there were a BUNCH of 503 "No Backend Server" available errors for quite a while. There were also times where it literally took two minutes to load the page.

And "no backend server" means this wasn't "oh, your local ISP is bad", that's "the load-balancing layer is falling over".

Normal
Small
Posted by kiyote marimaccadmin

marimaccadmin wrote:
Thank you for your input, as always, Derek. :)
you have the patience of a saint.

Normal
Small
Posted by watchdog

I initially got several 503 errors too.  I wasn't trying to use the forums or process a wish list, but I like to keep track of our events and see how many tickets are sold.
 

Normal
Small
Posted by garhkal

Other than i had to wait till i got home (since i was out of town without my laptop at noon), i had no issues.

New Post Sign in to write a new post.