EllisLab text mark
Advanced Search
1 of 2
1
   
Anyone trying Amazon RDS?
Posted: 27 October 2009 03:13 AM
Avatar
Joined: 2003-08-28
2315 posts

I’ve been using Amazon S3 for a couple of years to offload graphics for some EE sites. Now Amazon has RDS.

Amazon RDS gives you access to the full capabilities of a familiar MySQL database. This means the code, applications, and tools you already use today with your existing MySQL databases work seamlessly with Amazon RDS.

Looks promising. On a per-gigabyte basis, the pricing is similar to Amazon S3, which is similar to a good (not cheapie) host service (Apache, PHP, MySQL, etc.). Amazon S3 has been remarkably dependable.

 Signature 

ronnymac
———
Ron McElfresh
Honolulu, HI
———
From EE to WP | Hobby Site | Newer Hobby Site | Friend’s Site | Another Friend’s Site

 
Posted: 27 October 2009 03:41 PM   [ # 1 ]   [ Rating: 0 ]
Avatar
Joined: 2008-02-28
2330 posts

Interesting, I have heard of people using cs3 effectively - although personally I have not had to deal with mega visitors and requests. How are you using it gramps - can you share?Reseller or just a comfort of performance?
RDS looks very…very nice.
Small DB Instance: 1.7 GB memory, 1 ECU ...man this is equal to an entry level dedicated server.
Let down is:

However, Amazon SimpleDB is not a relational database, and does not offer some features needed in certain applications, e.g. complex transactions or joins.

It would be nice if the db could do some heavy lifting rather than requests.

Thanks!

 Signature 

Defeat is a state of mind; no one is ever defeated until defeat has been accepted as a reality.

 
Posted: 27 October 2009 03:49 PM   [ # 2 ]   [ Rating: 0 ]
Avatar
Joined: 2003-08-28
2315 posts
lebisol - 27 October 2009 07:41 PM

How are you using it gramps - can you share? Reseller or just a comfort of performance?

Not yet. Still trying to see if the price/performance is competitive. At this point, it looks better for those sites with LOTS of traffic and a need to have a dedicated or load-balanced server.

I’ve had good success with Amazon S3 to offload graphics from clients with shared servers (dependability is better than price). RDS features are good, though.

RDS looks very…very nice.
Small DB Instance: 1.7 GB memory, 1 ECU ...man this is equal to an entry level dedicated server.

Let down is:

However, Amazon SimpleDB is not a relational database, and does not offer some features needed in certain applications, e.g. complex transactions or joins.

It would be nice if the db could do some heavy lifting rather than requests.

That’s SimpleDB, not Amazon RDS. They’re different. RDS is MySQL 5.1.x but it’s also configured as InnoDB and not ISAM. Does that pose a problem with EE?

 Signature 

ronnymac
———
Ron McElfresh
Honolulu, HI
———
From EE to WP | Hobby Site | Newer Hobby Site | Friend’s Site | Another Friend’s Site

 
Posted: 27 October 2009 03:56 PM   [ # 3 ]   [ Rating: 0 ]
Avatar
Joined: 2008-02-28
2330 posts

Yeah I was thinking the same…why is EE not using stored procedures of MySQL 5.x? I suppose it makes the application more portable and cross-host ready…but back in day I was always advised to ‘let the db’ do as much work as possible rather than web server (pages/scripts). It has been a long since I did anything from scratch…

 Signature 

Defeat is a state of mind; no one is ever defeated until defeat has been accepted as a reality.

 
Posted: 27 October 2009 04:04 PM   [ # 4 ]   [ Rating: 0 ]
Avatar
Joined: 2004-05-15
29075 posts

Heck, we’re still supporting MySQL 3.23.32 FWIW… Well, EE 1.6.8 is, that’s going to change, I think.

 
Posted: 27 October 2009 04:11 PM   [ # 5 ]   [ Rating: 0 ]
Avatar
Joined: 2003-08-28
2315 posts
Ingmar Greil - 27 October 2009 08:04 PM

Heck, we’re still supporting MySQL 3.23.32 FWIW… Well, EE 1.6.8 is, that’s going to change, I think.

It’s obviously a pragmatic business decision for EE (and other PHP/MySQL apps) to support a wide spectrum of PHP and MySQL versions. Both are free and nearly ubiquitous on hosts throughout the world, hence easy to set up and manage, and scale to substantial traffic.

Can EE handle InnoDB tables, or is there a specific requirement for ISAM (or, is it table dependent?)?

 Signature 

ronnymac
———
Ron McElfresh
Honolulu, HI
———
From EE to WP | Hobby Site | Newer Hobby Site | Friend’s Site | Another Friend’s Site

 
Posted: 27 October 2009 04:16 PM   [ # 6 ]   [ Rating: 0 ]
Avatar
Joined: 2004-05-15
29075 posts
gramps - 27 October 2009 08:11 PM

It’s obviously a pragmatic business decision for EE (and other PHP/MySQL apps) to support a wide spectrum of PHP and MySQL versions.

Let’s put it the other way round: it makes no sense to demand the latest & greatest, there still are people out there on PHP 4. It’s a reality we’re facing.

Can EE handle InnoDB tables, or is there a specific requirement for ISAM (or, is it table dependent?)?

Yes, the storage engine should not matter.

 
Posted: 27 October 2009 04:50 PM   [ # 7 ]   [ Rating: 0 ]
Avatar
Joined: 2003-08-28
2315 posts
Ingmar Greil - 27 October 2009 08:16 PM

Yes, the storage engine should not matter.

That’s good news.

I’m not comfortable with RDS pricing. Add up all the pieces and anything but a big site with lots of traffic would do better with EngineHosting or PairLite (both of which are moderately priced hosts).

Pricing is based on a combination of necessary components, including DB instances, storage, backup, bandwidth, and I/O requests. That can add up. Amazon RDS isn’t apples to apples with Amazon S3, but pricing is similar. For example, I have a few sites hosted at PairLite (moderately priced shared host), which includes 50 gigs of bandwidth for what amounts to $8.25 a month. Amazon RDS bandwidth (data transfer) alone for the same amount would be $8.50 a month, before adding the other requirements.

I’m convinced that Amazon’s offerings are more beneficial for sites with heftier DB and bandwidth requirements.

 Signature 

ronnymac
———
Ron McElfresh
Honolulu, HI
———
From EE to WP | Hobby Site | Newer Hobby Site | Friend’s Site | Another Friend’s Site

 
Posted: 28 October 2009 04:16 AM   [ # 8 ]   [ Rating: 0 ]
Avatar
Joined: 2007-02-06
743 posts

RDS isn’t much different than running MySQL on EC2 with EBS. With RDS you sacrifice some control for some automation (automatic backups and software updates). If RDS ever offers automatic failover, that will be a game changer.

 Signature 

“I am the terror that flaps in the night”

 
Posted: 01 December 2009 11:17 PM   [ # 9 ]   [ Rating: 0 ]
Avatar
Joined: 2003-08-28
2315 posts
Rick Jolly - 28 October 2009 08:16 AM

RDS isn’t much different than running MySQL on EC2 with EBS. With RDS you sacrifice some control for some automation (automatic backups and software updates). If RDS ever offers automatic failover, that will be a game changer.

Agreed.

Any experience in setting up EE configuration with RDS or EC2/EBS?

 Signature 

ronnymac
———
Ron McElfresh
Honolulu, HI
———
From EE to WP | Hobby Site | Newer Hobby Site | Friend’s Site | Another Friend’s Site

 
Posted: 25 January 2010 10:20 PM   [ # 10 ]   [ Rating: 0 ]
Joined: 2009-12-29
12 posts

I’m interested in the whole Amazon EC2/RDS combination for EE.  Currently using Rackspace Cloud and have had a ton of issues with them the last 6-7 weeks and need an option.

Is anyone using EE with Amazon?  Easy to set up?  Tips?

Thanks!  wink

grrramps - 02 December 2009 04:17 AM
Rick Jolly - 28 October 2009 08:16 AM

RDS isn’t much different than running MySQL on EC2 with EBS. With RDS you sacrifice some control for some automation (automatic backups and software updates). If RDS ever offers automatic failover, that will be a game changer.

Agreed.

Any experience in setting up EE configuration with RDS or EC2/EBS?

 
Posted: 26 January 2010 07:57 PM   [ # 11 ]   [ Rating: 0 ]
Joined: 2009-12-29
12 posts

December and January have been bad for performance from our perspective.  Between January 2009 and early November 2009 we’d been happy with performance (and it did well on a large Halloween site (halloween.com), without hiccups that our DQC server had in 2008).  See the first two links below give a pretty good overview of the recent issues - TechCrunch was quite unhappy as are a lot of other people.

I am to the point of moving the site to AWS until Rackspace gets performance up.  My understanding of the process is this for EE:

1. Swapping RDS addresses for the current mysql addresses in the EE config files.
2. Picking an AMI to use.
3. Customize it.
4. Uploading the content to a persistent directory and then moving it to the server or doing a sym link to that directory.
5. Getting an elastic IP from Amazon.
6. Build and upload the new AMI
7. After testing, switching DNS entries to point there.

If anyone has additional tips (e.g. a good AMI to use, whether to use a persistent directory and sym link or what?) or steps that are missing etc, I would love to hear them.

I truly think that Rackspace is trying, but I think the clusters in the cloud are over-sold or over-used (or have config issues?) and while they are attempting to fix it, it isn’t as if they can just switch on 500 more machines over night to fix it.  All that is conjecture since they haven’t really given a lot of details which is frustrating.  (If you look here, you can see even more, including the spikes in page download time in January: http://www.christianriley.com/2010/01/the-cloud-is-raining-degraded-rackspace-cloud-and-no-suitable-nodes-on-rackspace-cloud/  )


Anyway, these two links give a good overview, here is what TechCrunch said: 
http://www.techcrunch.com/2009/12/18/rackspace-down/
and here:
http://strategyinprinciple.com/case-study-how-do-we-get-fanatical-support

Some other links that describe the issues:
http://www.datacenterknowledge.com/archives/2010/01/14/performance-problems-for-rackspace-cloud/
http://cloudfail.net/category/rackspace/rackspace-cloud-sites
http://www.rackspace.com/blog/?p=767

Savvyness - 26 January 2010 11:41 PM

I’m also interested in Amazon EC2/RDS combination for EE. From what I read, AMI’s are configured to run Tomcat and work with EC2Deploy.

The EC2 core framework manages EC2 instances, configures MySQL, Tomcat, Terracotta and Apache and deploys the application.

Chris, what kind of issues have you had using Rackspace Cloud? I was thinking about that as an alternative option.

 
Posted: 26 January 2010 07:58 PM   [ # 12 ]   [ Rating: 0 ]
Joined: 2009-12-29
12 posts

p.s.  I really want to like it as we did until November and I really hope they get any issues resolved because the concept is good.

 
Posted: 28 January 2010 10:43 PM   [ # 13 ]   [ Rating: 0 ]
Joined: 2009-12-29
12 posts

I did successfully move one EE site from Rackspace Cloud to AWS/EC2.  It has only been a few hours, but it already appears that AWS is doing much better performance-wise.

I ended up not using RDS yet, just mysql running on the instance.  That may change in the future, but if I can get it together, I’ll post the steps I used to make the transition. 

It certainly is possible, just required a few changes in organization and some minor customizations on the AMI.

 
Posted: 12 February 2010 03:05 PM   [ # 14 ]   [ Rating: 0 ]
Joined: 2010-02-12
2 posts
lebisol - 27 October 2009 07:56 PM

why is EE not using stored procedures of MySQL 5.x? I suppose it makes the application more portable and cross-host ready…but back in day I was always advised to ‘let the db’ do as much work as possible rather than web server (pages/scripts).

Stored Procedures aren’t very helpful in terms of performance. They also typically mean you’re hand-writing your SQL, which usually means you’re doing something wrong.
http://codebetter.com/blogs/jeremy.miller/archive/2006/05/25/145450.aspx
http://weblogs.asp.net/fbouma/archive/2003/11/18/38178.aspx
As always, Google has more.

 
Posted: 12 February 2010 03:49 PM   [ # 15 ]   [ Rating: 0 ]
Joined: 2010-02-12
2 posts
ChrisR1776 - 27 January 2010 12:57 AM

My understanding of the process is this for EE:

1. Swapping RDS addresses for the current mysql addresses in the EE config files.
2. Picking an AMI to use.
3. Customize it.
4. Uploading the content to a persistent directory and then moving it to the server or doing a sym link to that directory.
5. Getting an elastic IP from Amazon.
6. Build and upload the new AMI
7. After testing, switching DNS entries to point there.

If anyone has additional tips (e.g. a good AMI to use, whether to use a persistent directory and sym link or what?) or steps that are missing etc, I would love to hear them.

This is coming from an avid AWS user, but not Expression Engine (I run several sites on ASP.Net hosted on EC2).

First of all, Amazon has made AWS ridiculously simple. All of the steps I’m about to describe involve clicking around in the web app at http://aws.amazon.com/console/, unless otherwise specified.

Second, yes, signing up for this stuff and using it does cost money to mess with, but it’s pennies. Experimenting with it might cost you $5 at most, probably more like $2. It’s really cheap so just set it up.

There are basically 3 pre-built AMIs for Windows provided for you by Amazon, making the selection process very simple. They have obvious differences of size and cost - use the instance named “Getting Started on Microsoft Windows Server 2008” if you’re not sure - launch it on a Small Instance.

Once you’re signed in, get an Elastic IP (point and click - it will be obvious).

Launch your AMI, and assign your Elastic IP to it. Begin installing the necessary software over Remote Desktop. While it installs, create an EBS volume, then Attach it to your Instance and name it /dev/sda2. Then go back to Remote Desktop and open Disk Manager. The EBS Volume appears as an unformatted drive - format it and give it a drive letter. You want to put your content on this second drive.

At this point you might want to backup what you’ve done so far. Select your Instance in AWS and click Instance Actions > Create Image (EBS AMI). This will take your Instance offline for about 20 minutes while AWS freezes the instance and copies everything over to an AMI backup. Should your Instance ever fail in the future, you can launch it again from this AMI. Since it freezes your Instance, you want to do this infrequently - only after Configuration changes to your Windows instance.

For changes besides Windows Configuration (content, app code, etc), those should all live on your second EBS Volume you attached. To back that up you select it and click Create Snapshot. The Snapshot is created without freezing anything, almost instantly. You’ll want to do these frequently (once a day, or once an hour, depending on your backup policy).

There’s an additional detail of configuring both firewalls - the AWS firewall (“Security Group”) and the Windows Firewall to let in all the traffic you want (basically open port 80) - I can cover that later if anyone is curious.

Launching an RDS instance is less simple - it doesn’t have an AWS Console option yet, so you need to use the command-line to launch an RDS instance:
http://docs.amazonwebservices.com/AmazonRDS/latest/GettingStartedGuide/
Like:

rds-create-db-instance ee0
   
--allocated-storage 20 --db-instance-class db.m1.xlarge --engine MySQL5.1 
   
--master-username root --master-user-password root 
   
--db-name EE --headers 

Obviously use a harder to guess user/pass than root/root.

Follow the instructions in the above link to open up access to your EC2 Security Group, and probably your current IP for now, then get the name of it via the command-line - it will look something like:
ee0.crwjauxgijdf.us-east-1.rds.amazonaws.com

Then connecting to it is as simple as connecting to any MySQL server. Use whatever tools you normally use to interact with MySQL and enjoy.

If at any time you need to shut all this down and want to come back to it later, just save off an AMI of the RDS and the EC2 instance and shut them down. You’ll be charged pennies (really, a few cents a month.. it’s next to nothing) to keep storing the AMIs until you come back to them, so don’t fret the cost of storing them between when you have time to explore.

 
1 of 2
1