Grant Elastic Beanstalk Environment Access to RDS Automatically

Today, let’s take a colloquial look at automatically granting ingress access to your Elastic Beanstalk apps that need to use RDS resources.

When we first started, we brought up our RDS instance and our Elastic Beanstalk instance. The two weren’t connected. Of course, as soon as the instance tried to connect to the SQL Server, it couldn’t. Why? Because the RDS security group blocks all access by default, only allowing in what you specify.

Our naive approach was to manually go into the RDS security group (we’ll call this sg-RDS) and grant the EC2 instance security group (we’ll call this sg-EC2) access on port 1433. That works!

Until you want to rebuild your environment.

Tearing Down is a Problem

Granting sg-EC2 access to sg-RDS is a manual step. If you attempt to terminate or rebuild the environment, Elastic Beanstalk is unable to complete because sg-RDS has a reference to sg-EC2. To correctly rebuild an environment, you’d need to manually go into sg-RDS and remove the permissions granted to sg-EC2. Of course, as soon as your environment is done rebuilding, you need to set up that manual connection again.

Turns out, AWS warned me about this, I just hadn’t RTFM. They have a whole document on Using Elastic Beanstalk with RDS. In it, they warn about the dependency issue:

It is possible to connect your environment to a database by adding a rule to your database’s security group that allows ingress from the autogenerated security group that Elastic Beanstalk attaches to your environment’s Auto Scaling group. However, doing so creates a dependency between the two security groups. Subsequently, when you attempt to terminate the environment, Elastic Beanstalk will be unable to delete the environment’s security group because the database’s security group is dependent on it.

Exactly what I said.

EC2 in RDS Security Group

The solution is deceptively easy. I’ll write it down and then we’ll unpack it.

  1. Grant sg-RDS access to sg-RDS on port 1433
  2. Change EB setup to automatically put EC2 instance in sg-RDS

It’s that first step that’s a doozy. Essentially, you’re giving resources in the group access to resources in the group. Just take a sec to try to grok that. Here’s a screenshot that shows what it’s supposed to look like:

Screenshot of security group inbound settings edit showing how I'm grant the security group itself access to ... itself

Notice that the ID of the security group I’m editing is the same as the security group I’ve added to the inbound rules?

Meme image of Keanu Reeves as Ted from Bill & Ted's Excellent Adventure saying "WHOA"

In step 2, we need to setup our EB environment such that any EC2 instance spun up as part of the environment is automatically added to the sg-RDS. By default, it’s given it’s own security group with a name given by EB. That sg is what I called sg-EC2 above. The trick is to add the instances to both sg-EC2 AND sg-RDS. And it’s pretty easy. Check out the screenshot:

Screenshot of adding the RDS security group to the list of security groups for the EC2 instance

The setting “EC2 security groups” accepts a comma-delimited list of the security group names that you want the EC2 instances to be a part of. In the screenshot, I’ve got 2 listed. DIA SQL Restricted is the name of sg-RDS and awseb-e-….. is the name of sg-EC2. Upon creation of an instance in this environment, EB will place it in both sg-RDS and sg-EC2.

With step 1 and 2 combined, the environment can be terminated or rebuilt without the need for manual intervention. The bulk of the document referenced above goes over the steps in more precision.