Saving costs on RDS Audit log exports
If you’re sending RDS Audit logs to CloudWatch Logs, you might be overspending. Learn how to optimize your costs.
RDS Audit logs
RDS audit logs are a crucial component of monitoring and maintaining the security and compliance of a database. These logs capture all activities and events within the RDS instance, providing a detailed record of who accessed the database, when they accessed it, and what actions they performed.
CloudWatch Log exports
By default, RDS audit log exports are sent to CloudWatch Logs. However, for databases with high activity, this can result in significant costs reaching tens of thousands of dollars per month. As a result, it is important to explore alternative solutions to manage these costs effectively.
Solutions
There are two solutions to this:
Download logs via RDS API and store them manually
This is the cheapest method, but requires manual handling and combining of these logs, since logs are rotated by RDS quite frequently. You need to make sure you store the logs to alternative location, before they expire. Fortunately calls to RDS API are free and you will not be charged for them, but you will be charged for data transfer. Keep in mind API calls do have rate limits, so you will have to work around that.
Change CloudWatch log group to infrequent access
Since these logs are not meant to be searched all the time, changing log group to infrequent access might just do it. Its much simpler solution that only requires recreation of log group and setting it to infrequent access.
In this article, we will explore the second option as an alternative to the first, which requires coding a custom solution from scratch.
Procedure
Backup
RDS uses predefined paths for exporting logs to CloudWatch. These paths are static and cannot be changed:
/aws/rds/cluster/<cluster_name>/error
/aws/rds/cluster/<cluster_name>/slowquery
/aws/rds/cluster/<cluster_name>/audit
/aws/rds/cluster/<cluster_name>/general
Because of this reason, there is a small drawback that you will have to delete them. They cannot be renamed, so if you want to keep logs, you can export them to S3 beforehand by clicking on log group and under action you can choose:
Changing log group to Infrequent Access
Disable current CloudWatch Exports, so your current log group gets released. Unless you do this, log groups will be constantly recreated by RDS.
Delete log groups:
/aws/rds/cluster/<cluster_name>/error
/aws/rds/cluster/<cluster_name>/slowquery
/aws/rds/cluster/<cluster_name>/audit
/aws/rds/cluster/<cluster_name>/general
Recreate all log groups with exact same name, but with Infrequent Access log class
Enable log exports on your cluster again
Log groups that you created will be reused, meaning they will be using new Infrequent Access log class.
Summary
By switching log exports to infrequent access, you can reduce your storage costs by approximately 50%. This reduction also applies to API calls, which are the primary cost generator when publishing to CloudWatch logs.
Reference pricing table:
Latest prices for your region: https://aws.amazon.com/cloudwatch/pricing/
Great one. ❤️