Purpose of the article: To explain how to create a new cloud watch log group, log stream with user defined names, and pass a log message to defined log group from lambda function.
Intended Audience: Python, AWS services (IAM, Lambda, CloudWatch)
Tools and Technology: Python, AWS services (IAM, Lambda, CloudWatch)
Keywords: IAM, Lambda, CloudWatch
Introduction
In the world of AWS, effective logging and monitoring are essential for understanding the performance and behavior of your cloud-based applications.The main goal of the blog is sending logs to a defined cloudwatch log group from the lambda function.This overview discusses the process of creating customized CloudWatch log groups and streams, followed by integrating these with an AWS Lambda function using Python.
Once your log messages are flowing into CloudWatch, you gain valuable insights. You can set up alarms, and employ CloudWatch Logs Insights for in-depth log analysis. This comprehensive monitoring approach is pivotal for optimizing AWS applications, troubleshooting issues, and ensuring system reliability.
Architecture
STEP-1
Create AWS IAM Role to Execute Lambda Function
- Navigate to the IAM service in AWS, and select Roles, then create a role.
- In this section, select AWS service and lambda as use case and click on ‘Next’
- Now, add permission by searching CloudWatchFullAccess in search bar. Adding this permission gives an access to create log groups and log streams from the lambda function.
- After adding permissions, click on ‘Next’ at right bottom. Once here, enter an appropriate name and description for your service role; then ‘Create Role.’
STEP-2
- Search for lambda in AWS services list; it then shows the lambda as service, after opening it, we have an option such as ‘Create Function’, click on it.
- Now, we need to provide some information to create a new function such as function name, run time and python as run time.
- Change default execution role to use an existing role and select a role that we created in step 1, then create function by selecting ‘Create Function.’
- Now function will be ready. It shows default code in the code source. There we need to build our python code for creating cloud watch log group and log stream.
cloudwatchlogs.create_log_group(logGroupName=log_group_name)
- The above piece of code takes a name of the log group as input and creates a new log group with that name in cloud watch log groups. Otherwise, execute an except block with an error message.
response=cloudwatchlogs.create_log_stream(logGroupName=log_group_name, logStreamName=log_stream_name)
- The above piece of code takes input as log stream name and log group name in which log group we are creating that log stream.
cloudwatchlogs.put_log_events(
logGroupName=log_group_name,
logStreamName=log_stream_name,
logEvents=[
{
‘timestamp’: int(round(time.time() * 1000)),
‘message’: log_message
}
]
)
The above code loads the log message to the defined log stream.
timestamp: This field contains the timestamp of the log event.
Message: contains the actual message that we are sending to the log stream
log_group_name : It is a variable that stores the name of the log group.
log_stream_name : It is a variable that carries the name of log stream.
- After completing writing of the code in code source, deploy it and execute the lambda function.
- Open the log group; we can see the log stream with the name given in the lambda code and it contains log messages in it, which we send.
The process of customizing CloudWatch logs with AWS Lambda provides a powerful solution for effective logging and monitoring in cloud-based applications. By creating a Lambda function and utilizing a Python script, users can seamlessly integrate CloudWatch log groups and streams, allowing for the efficient organization and analysis of log data.
- User-defined names for CloudWatch log groups enhance organization and clarity.
- Log stream creation allows for fine-grained control and categorization of log data.
- Sending specific log messages ensures capturing relevant information for quick issue resolution.
References:
- https://learn.microsoft.com/en-us/azure/logic-apps/logic-apps-overview
- https://learn.microsoft.com/en-us/connectors/salesforce/
- https://salesforcefaqs.com/get-security-token-salesforce/
- https://learn.microsoft.com/en-us/azure/?product=popular
- https://learn.microsoft.com/en-us/azure/connectors/connectors-native-http?tabs=standard
Author Bio:
Mounika CHILUVURU
Associative Software Engineer - Data Engineering-Analytics
I’m Mounika CHILUVURU, and I've been with Mouri Tech for 1year as Associate Software Engineer in Data Engineering. Have a good history with AWS Services and expertise in Python and SQL coding language.
