EllisLab.com Site Improvements

12 comments

This past weekend we were offline for an hour or two just after midnight early Saturday morning to perform some site maintenance. Hopefully you have noticed a major difference in the speed of our site. These improvements are a direct result of finally having time to move to our official hosting partner, Nexcess.

Our staff has been using Nexcess for personal sites for some time, so we could experience what our customers do when they follow our recommendation to host their ExpressionEngine sites there. We’re happy to finally have the EllisLab site at Nexcess now as well.

We still have some items on our to-do list and wish-list post migration (Varnish and Redis caching for instance) which should continue to improve the site in the days ahead. In the meantime, if you encounter any problems or errors visiting our site, please .(JavaScript must be enabled to view this email address) or post a comment below and let us know.

Comments & Feedback

  1. Where were you guys previously hosted? Dedicated hosting? DB on own box?

    Picture of MediaGirl Inc.

    MediaGirl Inc.

  2. We were on Rackspace Cloud Servers (not Cloud Sites).

    Picture of Derek Jones

    Derek Jones

  3. Got it. Should Rackspace Cloud Servers be avoided for high traffic EE sites? What issues were you seeing? Any insight is very helpful for me.

    Picture of MediaGirl Inc.

    MediaGirl Inc.

  4. No, I wouldn’t go so far as to say that. Most of our issues boiled down to how many concurrent connections our setup could handle as well as member and forum table locking issues. We were definitely at an impasse with our server configuration there, and while it would have been solvable by beefing things up, we had been looking to move to Nexcess already since our hosting there can only add to their insight on hosting ExpressionEngine sites for others. And there’s something to be said for eating your own dog food.

    Rackspace is a good host, but they can’t target ExpressionEngine with the focus that Nexcess has chosen to. And it shows, we don’t have any of the typical mechanisms that a high traffic site would put into place yet to help with the load, and the site is still flying. Practical example: previously our forum homepage during a low traffic time would take about 5 seconds of processing, and 5.5M of memory. At Nexcess, it is taking less than 0.5 seconds of processing and 3.7M of memory (memory savings is primarily from switching to PHP 5.4).

    Picture of Derek Jones

    Derek Jones

  5. Derek, am I correct to think that it’s the table locking that prevented an easy spin-up of more servers to get your Rackspace Cloud performance up?

    This would be thought to be the advantage of cloud servers, so it’s pretty important to understand what the roadblock was here.

    Would the situation have been better, if the EE forum wasn’t involved? I am thinking of the ability to turn off unnecessary access tracking which can lead to table locks, as you recommend for high-load sites.

    It’s nice to know the slowness wasn’t an effect of your marketed hosting partner, as it had appeared it might be.

    Picture of narration

    narration

  6. What about the Nexcess server setup is helping with table locking? Did you start using INNODB on the problem tables or specific DB server configs?

    Picture of MediaGirl Inc.

    MediaGirl Inc.

  7. @narration - DB in a cloud with multiple servers is much trickier, and typically requires expensive solutions to distribute a MySQL server’s load (Oracle’s solution starts at $10k/yr for instance). It’s not the same animal as putting another web server up behind a load balancer, sadly. It definitely would have been a different story if it were not for our forum and member table sizes, though we would still have had the issue of needing to grow our servers’ capabilities.

    The table locks are due to writes to tables that use the MyISAM storage engine, which we were still using. When a MyISAM table gets written to, the whole table is locked, preventing other reads/writes to that table until that query is finished and releases the lock. We hit the proverbial wall where MyISAM’s read speed advantage no longer outweighed its locking behavior given our write-heavy site. InnoDB on the other hand only locks at the row level, so reads/writes to other portions of the table do not have to wait for the release of that lock.

    We could have made changes at Rackspace, no doubt, but again timing, dogfooding, and the benefits that will come to all EE sites hosted at Nexcess made moving the right choice.

    @mediagirl - We haven’t even switched to InnoDB yet, that’s what we’re doing tonight to squeeze out some more. wink

    Picture of Derek Jones

    Derek Jones

  8. Derek, thanks for the detail—that pretty much covers it, and well understood, if there are some ins and outs since you got the performance without the InnoDB so far. Probably that amounts to avoiding the cascade effects that occur with blockable resources as you near capacity limits.

    These are all interesting factors where I’m beginning some developmental tests with Amazon EC2 and RDS arrangements.

    Dogfooding definitely a good idea, again, now that we understand this is where it really begins.

    Good fortune on the rest of what you build up, and will be interesting to hear what you gain, presume here. Thanks again.

    Picture of narration

    narration

  9. I feel like home again, thank you for getting EL back to “real life ” smile (it s was a no-no when the site was so slow…i almost broke my computer while waiting pages to load…heh)... hopefully i manage to upgrade my 2.5.5 to latest in the near future with no hassle wink

    Picture of Riverboy

    Riverboy

  10. Great to see the improvements on your overall platform Derek (and team). I certainly would be interested in hearing any tweaks/improvements you’re doing at db level as I’m going through the same on a couple of high sites as well and any insight to squeeze some more performance is always welcome.

    Picture of Carl W Crawley

    Carl W Crawley

  11. Thanks Carl. If your site is write-heavy like ours (due to forums, comments, membership, and sessions) I would definitely advocate for using InnoDB for the storage engine if your host is experienced with it. We made that switch early last week and it was like Doc Brown’s yellow stage Presto log went off in the boiler. Hard to resist a BttF reference when one of our engineers owns a DeLorean.

    Nexcess runs a number of ExpressionEngine sites with higher visibility than ours so their experience with InnoDB environments is dialed in. We’ve not yet made it through our wish-list of tweaks and optimizations yet, but we’ll be happy to blog about the changes and the improvements we’ve gained as a result of each.

    Picture of Derek Jones

    Derek Jones

  12. Thanks Derek - thats definitely on my list. At the moment, I’m just working core tag fixes at the moment (advanced conditionals and embed-crazy), but then we’re going to be turning our attention to the DB.

    I look forward to reading that blog when you do it!

    Thanks,

    Carl

    Picture of Carl W Crawley

    Carl W Crawley

You must be logged in to comment on this blog post