{"id":2952,"date":"2024-09-24T20:36:40","date_gmt":"2024-09-25T01:36:40","guid":{"rendered":"https:\/\/badecho.com\/?p=2952"},"modified":"2024-09-24T20:36:47","modified_gmt":"2024-09-25T01:36:47","slug":"aws-lambda-json-logs","status":"publish","type":"post","link":"https:\/\/badecho.com\/index.php\/2024\/09\/24\/aws-lambda-json-logs\/","title":{"rendered":"Simple JSON Logging with AWS Lambda Running .NET"},"content":{"rendered":"\n
\"Simple<\/figure><\/div>\n\n\n\n

Greetings, dear readers. It’s been a minute since I last wrote, so I figured I’d break the silence with a brief article on a problem I was recently working on.<\/p>\n\n\n\n

I’ve been doing a lot of AWS cloud infrastructure design lately; in fact, I’ve had the pleasure of rearchitecting and implementing new infrastructure (this time following best practices) for an existing widely used product (whose cloud implementation was left unfinished).<\/p>\n\n\n\n

This product has several AWS Lambda functions running .NET that handle various REST API calls. One issue I had with the previous iteration of the product was its disgusting plaintext log entries being fed to CloudWatch by the Lambda functions.<\/p>\n\n\n\n

So, this article will go over a very simple and quick way to get your logs outputted in JSON, which looks much better and is easier to query than the plaintext mess you get by default.<\/p>\n\n\n\n

How a Lambda Function is Logged<\/h2>\n\n\n\n

Assuming the Lambda function’s execution role allows for it, logs are automatically captured for all requests handled by your function and sent to CloudWatch.<\/p>\n\n\n\n

Anything emitted to a standard output stream will wind up in a log stream on CloudWatch. That’s it; there’s really no magic in it.<\/p>\n\n\n\n

How do these logs look?<\/p>\n\n\n\n

\"A
A dreary, gray blob of plaintext junk.<\/figcaption><\/figure><\/div>\n\n\n\n

Eww. The above entries are pretty spartan; things begin to get painful when we need to dig through more complicated transactions.<\/p>\n\n\n\n

Never Fear! JSON Is Here<\/h2>\n\n\n\n

Thankfully, AWS gives us a gift from above: the ability to change a Lambda function’s log format.<\/p>\n\n\n\n

If using the AWS console, from the Lambda function, go to Configuration<\/strong> -> Monitoring and operations tools<\/strong> and then hit Edit<\/strong> in the Logging configuration<\/strong> box.<\/p>\n\n\n\n

\"Here
Here we can change the log format of our Lambda function.<\/figcaption><\/figure><\/div>\n\n\n\n

Or, if you’re like me and use Terraform for pretty much everything, you can add a logging_config<\/code> block to your aws_lambda_function<\/code> resource:<\/p>\n\n\n\n

resource \"aws_lambda_function\" \"foo\" {\n  function_name = \"foo\"\n  ...\n  ...\n  logging_config {\n    log_format = \"JSON\"\n  }\n}<\/code><\/pre>\n\n\n\n
Doing the above will cause all system logs to be captured in JSON structured format. System logs include messages such as the Lambda function’s container spinning up and the usage report at the end of execution.<\/p>\n\n\n\n
On the other hand, application logs (i.e., the messages generated by the function’s code) will only be captured in JSON structured format out of the box for a select few runtimes, namely Python, Node.js, and Java.<\/p>\n\n\n\n
This is accurate at the time of writing; I expect support for additional runtimes to be added as time goes on.<\/em><\/em><\/em><\/p>\n\n\n\n
So that’s a big stinker; if you’re using Amazon’s “recommended” way of logging with .NET (logging.AddLambdaLogger()<\/code>), your logs will continue to be plaintext.<\/p>\n\n\n\n
.NET Has Lambda JSON Support…Sort Of<\/h2>\n\n\n\n
The folks from Amazon recently created a PR that seems to add JSON structured support<\/a> for Lambda functions using the .NET runtime.<\/p>\n\n\n\n
And indeed, it probably does just that! However, after looking at it a bit closer, it appears that the only logger granted this support is the one provided by the ILambdaContext<\/code> passed into the Lambda function’s handler.<\/p>\n\n\n\n
This is something you’re more likely to use if your Lambda function is just a simple routine; this won’t help you if your Lambda function is hosting a lightweight ASP.NET API that relies on ILogger<\/code>, which the logger exposed by ILambdaContext<\/code> does not implement.<\/p>\n\n\n\n
What Does the Lambda ILogger Even Do?<\/h2>\n\n\n\n
Amazon provides the ILoggingBuilder.AddLambdaLogger<\/code> extension method with their Amazon.Lambda.Logging.AspNetCore<\/code> package, recommending it as the “way to lay down them logs” (my words, not theirs) when running a .NET host.<\/p>\n\n\n\n
While looking into what Amazon’s Lambda ILogger<\/code> implementation does exactly and if the changes to ILambdaContext<\/code>‘s logger affect it, I found that, in the end\u2026it really doesn’t do that much.<\/p>\n\n\n\n
I’m sure it has some fun bells and whistles that tie into Lambda functionality, but literally, here’s what logging with the Lambda logger boils down to:<\/p>\n\n\n\n
LambaLogger.cs<\/h6>\n\n\n
\npublic static class LambdaLogger\n{\n    private static Action<string> _loggingAction = new Action<string>(LambdaLogger.LogToConsole);\n\n    private static void LogToConsole(string message) => Console.WriteLine(message);\n\n    public static void Log(string message) => LambdaLogger._loggingAction(message);\n}\n<\/pre>\n\n\n
All of the Lambda ILogger<\/code> implementation’s logging methods call this static class, resulting in everything just ending up\u2026being written to the console.<\/p>\n\n\n\n
So…Just Use a JSON Console<\/h2>\n\n\n\n
Take the line of code that says logging.AddLambdaLogger()<\/code> in your function, throw it out the window, and just slap on a call to AddJsonConsole()<\/code> instead.<\/p>\n\n\n\n
That’s it. This change and the configuration change to the Lambda function’s log format will give you purty JSON logs.<\/p>\n\n\n\n
LambdaEntryPoint.cs<\/h6>\n\n\n
\nbuilder.ConfigureLogging((logging =>\n                         {\n                             \/\/ Any other logic you have here to account for\n                             \/\/ local developer sessions, etc....\n                             logging.ClearProviders();\n                             logging.AddJsonConsole();\n                         })\n<\/pre>\n\n\n
Throw in some scopes while you’re at it!<\/p>\n\n\n\n
\"Nice
Oh, to gaze upon such comely logs in CloudWatch.<\/figcaption><\/figure><\/div>\n\n\n\n
Probably Should Use Serilog<\/h2>\n\n\n\n
The stock JSON console and accompanying formatter provided by Microsoft do their job, but they are pretty limited in regards to customization, etc. If you want some more serious structured logging support, you’ll probably want to look into more powerful solutions like Serilog.<\/p>\n\n\n\n
Regardless, the quick bit of code provided above provides logs that check all my boxes, at least at the time of writing.<\/p>\n\n\n\n
Happy coding, travelers.<\/p>\n","protected":false},"excerpt":{"rendered":"
Greetings, dear readers. It’s been a minute since I last wrote, so I figured I’d break the silence with a […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[10],"tags":[41,86,42,87,88],"yoast_head":"\r\nSimple JSON Logging with AWS Lambda Running .NET - omni's hackpad<\/title>\r\n<meta name=\"description\" content=\"Demonstrates a quick and easy way to capture logs generated by AWS Lambda functions running .NET in JSON structured format.\" \/>\r\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\r\n<link rel=\"canonical\" href=\"https:\/\/badecho.com\/index.php\/2024\/09\/24\/aws-lambda-json-logs\/\" \/>\r\n<meta property=\"og:locale\" content=\"en_US\" \/>\r\n<meta property=\"og:type\" content=\"article\" \/>\r\n<meta property=\"og:title\" content=\"Simple JSON Logging with AWS Lambda Running .NET - omni's hackpad\" \/>\r\n<meta property=\"og:description\" content=\"Demonstrates a quick and easy way to capture logs generated by AWS Lambda functions running .NET in JSON structured format.\" \/>\r\n<meta property=\"og:url\" content=\"https:\/\/badecho.com\/index.php\/2024\/09\/24\/aws-lambda-json-logs\/\" \/>\r\n<meta property=\"og:site_name\" content=\"omni's hackpad\" \/>\r\n<meta property=\"article:published_time\" content=\"2024-09-25T01:36:40+00:00\" \/>\r\n<meta property=\"article:modified_time\" content=\"2024-09-25T01:36:47+00:00\" \/>\r\n<meta property=\"og:image\" content=\"https:\/\/badecho.com\/wp-content\/uploads\/2024\/09\/SimpleJsonLogging.png\" \/>\r\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\r\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebSite\",\"@id\":\"https:\/\/badecho.com\/#website\",\"url\":\"https:\/\/badecho.com\/\",\"name\":\"omni's hackpad\",\"description\":\"Game Code Disassembly. Omnified Modification. Madness.\",\"publisher\":{\"@id\":\"https:\/\/badecho.com\/#\/schema\/person\/3de67496328be7ae6e1f52faf582e9d2\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":\"https:\/\/badecho.com\/?s={search_term_string}\",\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/badecho.com\/index.php\/2024\/09\/24\/aws-lambda-json-logs\/#primaryimage\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/badecho.com\/wp-content\/uploads\/2024\/09\/SimpleJsonLogging.png\",\"width\":856,\"height\":448,\"caption\":\"Simple JSON Logging with AWS Lambda Running .NET\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/badecho.com\/index.php\/2024\/09\/24\/aws-lambda-json-logs\/#webpage\",\"url\":\"https:\/\/badecho.com\/index.php\/2024\/09\/24\/aws-lambda-json-logs\/\",\"name\":\"Simple JSON Logging with AWS Lambda Running .NET - omni's hackpad\",\"isPartOf\":{\"@id\":\"https:\/\/badecho.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/badecho.com\/index.php\/2024\/09\/24\/aws-lambda-json-logs\/#primaryimage\"},\"datePublished\":\"2024-09-25T01:36:40+00:00\",\"dateModified\":\"2024-09-25T01:36:47+00:00\",\"description\":\"Demonstrates a quick and easy way to capture logs generated by AWS Lambda functions running .NET in JSON structured format.\",\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/badecho.com\/index.php\/2024\/09\/24\/aws-lambda-json-logs\/\"]}]},{\"@type\":\"Article\",\"@id\":\"https:\/\/badecho.com\/index.php\/2024\/09\/24\/aws-lambda-json-logs\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/badecho.com\/index.php\/2024\/09\/24\/aws-lambda-json-logs\/#webpage\"},\"author\":{\"@id\":\"https:\/\/badecho.com\/#\/schema\/person\/3de67496328be7ae6e1f52faf582e9d2\"},\"headline\":\"Simple JSON Logging with AWS Lambda Running .NET\",\"datePublished\":\"2024-09-25T01:36:40+00:00\",\"dateModified\":\"2024-09-25T01:36:47+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/badecho.com\/index.php\/2024\/09\/24\/aws-lambda-json-logs\/#webpage\"},\"commentCount\":2,\"publisher\":{\"@id\":\"https:\/\/badecho.com\/#\/schema\/person\/3de67496328be7ae6e1f52faf582e9d2\"},\"image\":{\"@id\":\"https:\/\/badecho.com\/index.php\/2024\/09\/24\/aws-lambda-json-logs\/#primaryimage\"},\"keywords\":\".NET,AWS,C#,Lambda,Terraform\",\"articleSection\":\"General Dev\",\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/badecho.com\/index.php\/2024\/09\/24\/aws-lambda-json-logs\/#respond\"]}]},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\/\/badecho.com\/#\/schema\/person\/3de67496328be7ae6e1f52faf582e9d2\",\"name\":\"Matt Weber\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/badecho.com\/#personlogo\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/7e345ac2708b3a41c7bd70a4a0440d41?s=96&d=mm&r=g\",\"caption\":\"Matt Weber\"},\"logo\":{\"@id\":\"https:\/\/badecho.com\/#personlogo\"}}]}<\/script>\r\n<!-- \/ Yoast SEO plugin. -->","_links":{"self":[{"href":"https:\/\/badecho.com\/index.php\/wp-json\/wp\/v2\/posts\/2952"}],"collection":[{"href":"https:\/\/badecho.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/badecho.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/badecho.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/badecho.com\/index.php\/wp-json\/wp\/v2\/comments?post=2952"}],"version-history":[{"count":8,"href":"https:\/\/badecho.com\/index.php\/wp-json\/wp\/v2\/posts\/2952\/revisions"}],"predecessor-version":[{"id":2964,"href":"https:\/\/badecho.com\/index.php\/wp-json\/wp\/v2\/posts\/2952\/revisions\/2964"}],"wp:attachment":[{"href":"https:\/\/badecho.com\/index.php\/wp-json\/wp\/v2\/media?parent=2952"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/badecho.com\/index.php\/wp-json\/wp\/v2\/categories?post=2952"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/badecho.com\/index.php\/wp-json\/wp\/v2\/tags?post=2952"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}