Allow AWS Lambda to access RDS Database
Solution 1:
You can't enable this via IP. First you will need to enable VPC access for the Lambda function, during which you will assign it a Security Group. Then, within the Security Group assigned to the RDS instance you will enable access for the Security Group assigned to the Lambda function.
Solution 2:
You can configure Lambda to access your RDS instance.
You can enable this using Lambda management console. Select Lambda function which need access to RDS instance and then go to Configuration -> Advanced settings and select the VPC (which is your RDS instance is in) you need it to access.
find out more here http://docs.aws.amazon.com/lambda/latest/dg/vpc.html
Solution 3:
For anyone else searching for a concise solution, or lambda config provisioned via AWS SAM / Cloudformation, what worked for me was:
i. create a Security Group (SG) allowing outbound traffic on the desired port you'd like to connect over (eg: 5432 or 3306. Note, inbound rules have no affect on lambda I believe, currently) Apply that SG to your lambda.
ii. create an SG allowing inbound traffic on the same port (say 5432 or 3306) which references the lambda SG, so traffic is locked down to only the lambda. And outbound on the same port (5432 or 3306). Apply that SG to your RDS instance.
Further detail:
Lambda SG:
Direction Protocol Port Source
Outbound TCP 5432 ALL
RDS SG:
Direction Protocol Port Source
Inbound TCP 5432 Lambda SG
Outbound TCP 5432 ALL
SAM template.yaml to provision the main resources you'll probably require including: an RDS cluster (Aurora Postgres serverless to minimise running costs is shown in this example), a Postgres master user password stored in secrets manager, a lambda, an SG that is applied to the lambda allowing outbound traffic on port 5432, an SG that is applied to the RDS cluster referencing the lambda SG (locking down traffic to the lambda) and I have also shown optionally how you may wish to connect to the RDS from your local desktop machine using a desktop DB client (eg DBeaver) over SSH tunnel via a bastion (eg a nano EC2 instance with an EIP attached so it can be stopped and all config remain the same) to admin the RDS from your local machine.
(Please note for a production system you may wish to provision your RDS into a private subnet for security. Provisioning of subnets not covered here for brevity)
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Provisions stack with Aurora Serverless
Parameters:
AppName:
Description: "Application Name"
Type: String
Default: RDS-example-stack
DBClusterName:
Description: "Aurora RDS cluster name"
Type: String
Default: rdsexamplecluster
DatabaseName:
Description: "Aurora RDS database name"
Type: String
Default: examplerdsdbname
DBMasterUserName:
AllowedPattern: "[a-zA-Z0-9_]+"
ConstraintDescription: must be between 1 to 16 alphanumeric characters.
Description: The database admin account user name, between 1 to 16 alphanumeric characters.
MaxLength: '16'
MinLength: '1'
Type: String
Default: aurora_admin_0
Resources:
# lambdas
someLambda:
Type: AWS::Serverless::Function
Properties:
FunctionName: !Sub '${AWS::StackName}-someLambda'
# Role: !GetAtt ExecutionRole.Arn # if you require a custom execution role and permissions
VpcConfig:
SubnetIds: [subnet-90f79cd8, subnet-9743e6cd, subnet-8bf962ed]
SecurityGroupIds: [!Ref lambdaOutboundSGToRDS]
Handler: index.handler
CodeUri: ./dist/someLambda
Runtime: nodejs14.x
Timeout: 5 # ensure matches your PG/ mySQL connection pool timeout
ReservedConcurrentExecutions: 5
MemorySize: 128
Environment: # optional env vars useful for your DB connection
Variables:
pgDb: !Ref DatabaseName
# dbUser: '{{resolve:secretsmanager:some-stackName-AuroraDBCreds:SecretString:username}}'
# dbPw: '{{resolve:secretsmanager:some-stackName-AuroraDBCreds:SecretString:password}}'
# SGs
lambdaOutboundSGToRDS: # Outbound access for lambda to access Aurora Postgres DB
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: !Sub ${AWS::StackName} access to Aurora PG DB
GroupName: !Sub ${AWS::StackName} lambda to Aurora access
SecurityGroupEgress:
-
CidrIp: '0.0.0.0/0'
Description: lambda to Aurora access over 5432
FromPort: 5432
IpProtocol: TCP
ToPort: 5432
VpcId: vpc-f6c4ea91
RDSSG:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: !Sub ${AWS::StackName} RDS ingress and egress
SecurityGroupEgress:
-
CidrIp: '0.0.0.0/0'
Description: lambda RDS access over 5432
FromPort: 5432
IpProtocol: TCP
ToPort: 5432
SecurityGroupIngress:
-
SourceSecurityGroupId: !Ref lambdaOutboundSGToRDS # ingress SG for lambda to access RDS
Description: lambda to Aurora access over 5432
FromPort: 5432
IpProtocol: TCP
ToPort: 5432
- # optional
CidrIp: '172.12.34.217/32' # private IP of your EIP/ bastion instance the EIP is assigned to. /32 ie a single IP address
Description: EC2 bastion host providing access to Aurora RDS via SSH tunnel for DBeaver desktop access over 5432
FromPort: 5432
IpProtocol: TCP
ToPort: 5432
VpcId: vpc-f6c4ea91
DBSubnetGroup: # just a logical grouping of subnets that you can apply as a group to your RDS
Type: AWS::RDS::DBSubnetGroup
Properties:
DBSubnetGroupDescription: CloudFormation managed DB subnet group.
SubnetIds:
- subnet-80f79cd8
- subnet-8743e6cd
- subnet-9bf962ed
AuroraDBCreds: # provisions a password for the DB master username, which we set in Parameters
Type: AWS::SecretsManager::Secret
Properties:
Name: !Sub ${AWS::StackName}-AuroraDBCreds
Description: RDS database auto-generated user password
GenerateSecretString:
SecretStringTemplate: !Sub '{"username": "${DBMasterUserName}"}'
GenerateStringKey: "password"
PasswordLength: 30
ExcludeCharacters: '"@/\'
Tags:
-
Key: AppName
Value: !Ref AppName
RDSCluster:
Type: AWS::RDS::DBCluster
Properties:
DBClusterIdentifier: !Ref DBClusterName
MasterUsername: !Join ['', ['{{resolve:secretsmanager:', !Ref AuroraDBCreds, ':SecretString:username}}' ]]
MasterUserPassword: !Join ['', ['{{resolve:secretsmanager:', !Ref AuroraDBCreds, ':SecretString:password}}' ]]
DatabaseName: !Ref DatabaseName
Engine: aurora-postgresql
EngineMode: serverless
EngineVersion: '10' # currently provisions '10.serverless_14' 10.14
EnableHttpEndpoint: true
ScalingConfiguration:
AutoPause: true
MaxCapacity: 2
MinCapacity: 2
SecondsUntilAutoPause: 300 # 5 min
DBSubnetGroupName:
Ref: DBSubnetGroup
VpcSecurityGroupIds:
- !Ref RDSSG
# optional outputs useful for importing into another stack or viewing in the terminal on deploy
Outputs:
StackName:
Description: Aurora Stack Name
Value: !Ref AWS::StackName
Export:
Name: !Sub ${AWS::StackName}-StackName
DatabaseName:
Description: Aurora Database Name
Value: !Ref DatabaseName
Export:
Name: !Sub ${AWS::StackName}-DatabaseName
DatabaseClusterArn:
Description: Aurora Cluster ARN
Value: !Sub arn:aws:rds:${AWS::Region}:${AWS::AccountId}:cluster:${DBClusterName}
Export:
Name: !Sub ${AWS::StackName}-DatabaseClusterArn
DatabaseSecretArn:
Description: Aurora Secret ARN
Value: !Ref AuroraDBCreds
Export:
Name: !Sub ${AWS::StackName}-DatabaseSecretArn
DatabaseClusterID:
Description: Aurora Cluster ID
Value: !Ref RDSCluster
Export:
Name: !Sub ${AWS::StackName}-DatabaseClusterID
AuroraDbURL:
Description: Aurora Database URL
Value: !GetAtt RDSCluster.Endpoint.Address
Export:
Name: !Sub ${AWS::StackName}-DatabaseURL
DatabaseMasterUserName:
Description: Aurora Database User
Value: !Ref DBMasterUserName
Export:
Name: !Sub ${AWS::StackName}-DatabaseMasterUserName
Solution 4:
Here is what I did
I assigned same Subnets and VPCs to both services Lambda and RDS. Now I created a NAT Gateway choosing one of the subnet so that Lambda can use that NAT Gateway to interact with the outside world.
Last thing is to add inbound entry in the security group that is attached to RDS as well as Lambda functions. Whitelist DB port 5432 in my case for postgresql and add security group name in the source.
Security group is somehow whitelisting itself by adding an entry in inbound rules.
This worked for me pretty well.