Simple JSON Logging with AWS Lambda Running .NET

Greetings, dear readers. It’s been a minute since I last wrote, so I figured I’d break the silence with a brief article on a problem I was recently working on.

I’ve been doing a lot of AWS cloud infrastructure design lately; in fact, I’ve had the pleasure of rearchitecting and implementing new infrastructure (this time following best practices) for an existing widely used product (whose cloud implementation was left unfinished).

This product has several AWS Lambda functions running .NET that handle various REST API calls. One issue I had with the previous iteration of the product was its disgusting plaintext log entries being fed to CloudWatch by the Lambda functions.

So, this article will go over a very simple and quick way to get your logs outputted in JSON, which looks much better and is easier to query than the plaintext mess you get by default.

How a Lambda Function is Logged

Assuming the Lambda function’s execution role allows for it, logs are automatically captured for all requests handled by your function and sent to CloudWatch.

Anything emitted to a standard output stream will wind up in a log stream on CloudWatch. That’s it; there’s really no magic in it.

How do these logs look?

Eww. The above entries are pretty spartan; things begin to get painful when we need to dig through more complicated transactions.

Never Fear! JSON Is Here

Thankfully, AWS gives us a gift from above: the ability to change a Lambda function’s log format.

If using the AWS console, from the Lambda function, go to Configuration -> Monitoring and operations tools and then hit Edit in the Logging configuration box.

Or, if you’re like me and use Terraform for pretty much everything, you can add a logging_config block to your aws_lambda_function resource:

resource "aws_lambda_function" "foo" {
  function_name = "foo"
  ...
  ...
  logging_config {
    log_format = "JSON"
  }
}

Doing the above will cause all system logs to be captured in JSON structured format. System logs include messages such as the Lambda function’s container spinning up and the usage report at the end of execution.

On the other hand, application logs (i.e., the messages generated by the function’s code) will only be captured in JSON structured format out of the box for a select few runtimes, namely Python, Node.js, and Java.

This is accurate at the time of writing; I expect support for additional runtimes to be added as time goes on.

So that’s a big stinker; if you’re using Amazon’s “recommended” way of logging with .NET (logging.AddLambdaLogger()), your logs will continue to be plaintext.

.NET Has Lambda JSON Support…Sort Of

The folks from Amazon recently created a PR that seems to add JSON structured support for Lambda functions using the .NET runtime.

And indeed, it probably does just that! However, after looking at it a bit closer, it appears that the only logger granted this support is the one provided by the ILambdaContext passed into the Lambda function’s handler.

This is something you’re more likely to use if your Lambda function is just a simple routine; this won’t help you if your Lambda function is hosting a lightweight ASP.NET API that relies on ILogger, which the logger exposed by ILambdaContext does not implement.

What Does the Lambda ILogger Even Do?

Amazon provides the ILoggingBuilder.AddLambdaLogger extension method with their Amazon.Lambda.Logging.AspNetCore package, recommending it as the “way to lay down them logs” (my words, not theirs) when running a .NET host.

While looking into what Amazon’s Lambda ILogger implementation does exactly and if the changes to ILambdaContext‘s logger affect it, I found that, in the end…it really doesn’t do that much.

I’m sure it has some fun bells and whistles that tie into Lambda functionality, but literally, here’s what logging with the Lambda logger boils down to:

LambaLogger.cs

public static class LambdaLogger
{
    private static Action<string> _loggingAction = new Action<string>(LambdaLogger.LogToConsole);

    private static void LogToConsole(string message) => Console.WriteLine(message);

    public static void Log(string message) => LambdaLogger._loggingAction(message);
}

All of the Lambda ILogger implementation’s logging methods call this static class, resulting in everything just ending up…being written to the console.

So…Just Use a JSON Console

Take the line of code that says logging.AddLambdaLogger() in your function, throw it out the window, and just slap on a call to AddJsonConsole() instead.

That’s it. This change and the configuration change to the Lambda function’s log format will give you purty JSON logs.

LambdaEntryPoint.cs

builder.ConfigureLogging((logging =>
                         {
                             // Any other logic you have here to account for
                             // local developer sessions, etc....
                             logging.ClearProviders();
                             logging.AddJsonConsole();
                         })

Throw in some scopes while you’re at it!

Probably Should Use Serilog

The stock JSON console and accompanying formatter provided by Microsoft do their job, but they are pretty limited in regards to customization, etc. If you want some more serious structured logging support, you’ll probably want to look into more powerful solutions like Serilog.

Regardless, the quick bit of code provided above provides logs that check all my boxes, at least at the time of writing.

Happy coding, travelers.