What is structured logging and why should I use JSONL for it?

Structured logging means emitting log entries as data with named fields instead of free-form text strings. JSONL is the ideal format because each line is a self-contained JSON object that log platforms can parse without regular expressions. This gives you reliable field extraction, type preservation, and compatibility with every major log aggregation tool including ELK, Fluentd, CloudWatch, and GCP Logging.

How do I send JSONL logs to the ELK Stack?

The simplest approach is to use Filebeat with json.keys_under_root set to true. Filebeat tails your JSONL log files, parses each line as JSON, and forwards the structured fields to Elasticsearch. If you need transformation or enrichment, add Logstash between Filebeat and Elasticsearch with the json codec. Kibana then lets you search, visualize, and alert on any field.

What is the difference between Fluentd and Fluent Bit?

Fluent Bit is a lightweight log processor designed for resource-constrained environments like Kubernetes sidecars and IoT devices. It uses roughly 450KB of memory and focuses on collection and forwarding. Fluentd is the full-featured version with over 500 plugins, supporting complex routing, buffering, and transformation. Use Fluent Bit for collection and Fluentd for aggregation, or use Fluent Bit alone for simple pipelines.

Do cloud logging services automatically parse JSONL?

Yes. AWS CloudWatch, GCP Cloud Logging, and Azure Monitor all automatically parse JSON log lines into structured fields. On AWS, CloudWatch Logs Insights can query any JSON field directly. On GCP, writing JSONL to stdout on Cloud Run or GKE creates structured LogEntry objects. On Azure, Application Insights parses JSON traces and makes fields available in KQL queries.

Which logging library should I use for JSONL output?

For Python, use structlog for its processor pipeline and clean API. For Node.js, use pino for its speed and native JSONL output. For Go, use zerolog for zero-allocation performance. All three produce one JSON object per line by default and support adding custom fields, child loggers for request context, and configurable log levels.

How do I avoid performance issues with structured logging?

Use asynchronous loggers like pino or zerolog that serialize JSON off the main thread. Avoid logging large payloads at INFO level. Sample high-volume events instead of logging every occurrence. Write logs to local files and use a sidecar like Filebeat or Fluent Bit to ship them asynchronously. Set a maximum line size to prevent oversized entries from affecting the pipeline.

Structured Logging with JSONL

A comprehensive guide to structured logging using JSONL (JSON Lines) format. Learn to integrate with ELK Stack, Fluentd, CloudWatch, GCP Logging, and Azure Monitor with production-ready code examples.

Last updated: February 2026

Why Structured Logging with JSONL?

Traditional plain-text logs are human-readable but nearly impossible for machines to parse reliably. A line like '2024-01-15 ERROR User login failed for admin' buries the severity, the event type, and the username inside an unstructured string. Every log aggregation tool must resort to fragile regular expressions to extract meaning, and any change to the log format breaks the pipeline.

Structured logging solves this problem by emitting each log entry as a JSON object with well-defined fields. JSONL (JSON Lines) takes this one step further: one JSON object per line, no wrapping array, no trailing commas. This makes JSONL logs append-friendly, streamable, and trivially parseable by every major log platform. In this guide you will learn how to produce JSONL logs from Python, Node.js, and Go, and how to ship them to ELK, Fluentd, and the three major cloud providers.

Compared to other structured formats like CSV or XML, JSONL offers the best balance of flexibility, tooling support, and human readability. Fields can be nested, types are preserved, and you never need to define a fixed schema upfront. This is why JSONL has become the de facto standard for structured logging in modern infrastructure.

Logging Libraries by Language

Most modern logging libraries support structured JSON output natively or via a simple configuration change. Below are the recommended libraries for Python, Node.js, and Go, each producing JSONL-compatible output.

Python: structlog

Recommended

structlog is the leading structured logging library for Python. It wraps the standard logging module and produces JSON output by default. Its processor pipeline lets you enrich, filter, and transform log entries before serialization.

Python: structlog

import structlog

# Configure structlog for JSONL output
structlog.configure(
    processors=[
        structlog.processors.TimeStamper(fmt="iso"),
        structlog.processors.add_log_level,
        structlog.processors.StackInfoRenderer(),
        structlog.processors.JSONRenderer(),
    ],
    wrapper_class=structlog.BoundLogger,
    context_class=dict,
    logger_factory=structlog.PrintLoggerFactory(),
)

log = structlog.get_logger()

# Each call produces one JSONL line
log.info("user.login", user_id="u-42", ip="10.0.0.1")
# {"event":"user.login","user_id":"u-42","ip":"10.0.0.1","level":"info","timestamp":"2026-02-15T08:30:00Z"}

log.error("db.connection_failed", host="db-primary", retries=3)
# {"event":"db.connection_failed","host":"db-primary","retries":3,"level":"error","timestamp":"2026-02-15T08:30:01Z"}

Node.js: pino

Fastest

pino is the fastest Node.js logger, producing JSONL output by default with zero configuration. It achieves low overhead through asynchronous writing and a minimal serialization path. The pino-pretty companion package provides human-readable output for local development.

Node.js: pino

import pino from 'pino';

// pino outputs JSONL by default
const log = pino({
  level: 'info',
  timestamp: pino.stdTimeFunctions.isoTime,
});

// Each call produces one JSONL line
log.info({ userId: 'u-42', ip: '10.0.0.1' }, 'user.login');
// {"level":30,"time":"2026-02-15T08:30:00.000Z","userId":"u-42","ip":"10.0.0.1","msg":"user.login"}

log.error({ host: 'db-primary', retries: 3 }, 'db.connection_failed');
// {"level":50,"time":"2026-02-15T08:30:01.000Z","host":"db-primary","retries":3,"msg":"db.connection_failed"}

// Child loggers inherit context
const reqLog = log.child({ requestId: 'req-abc-123' });
reqLog.info({ path: '/api/users' }, 'request.start');

Go: zerolog

Zero Alloc

zerolog is a zero-allocation JSON logger for Go, designed for maximum performance in high-throughput services. It writes directly to an io.Writer without intermediate allocations, making it ideal for latency-sensitive applications. Its chained API is both ergonomic and efficient.

Go: zerolog

package main

import (
    "os"
    "github.com/rs/zerolog"
    "github.com/rs/zerolog/log"
)

func main() {
    // Configure JSONL output to stdout
    zerolog.TimeFieldFormat = zerolog.TimeFormatUnix
    log.Logger = zerolog.New(os.Stdout).With().Timestamp().Logger()

    // Each call produces one JSONL line
    log.Info().
        Str("user_id", "u-42").
        Str("ip", "10.0.0.1").
        Msg("user.login")
    // {"level":"info","user_id":"u-42","ip":"10.0.0.1","time":1739608200,"message":"user.login"}

    log.Error().
        Str("host", "db-primary").
        Int("retries", 3).
        Msg("db.connection_failed")
}

ELK Stack Integration

The ELK Stack (Elasticsearch, Logstash, Kibana) is the most widely deployed open-source log management platform. JSONL logs integrate seamlessly because Logstash and Filebeat can parse JSON natively, eliminating the need for complex grok patterns.

Filebeat is the lightweight log shipper that tails your JSONL log files and forwards them to Logstash or Elasticsearch. Setting json.keys_under_root to true promotes the JSON fields to top-level Elasticsearch fields, making them directly searchable in Kibana.

Filebeat Configuration

# filebeat.yml
filebeat.inputs:
  - type: log
    enabled: true
    paths:
      - /var/log/app/*.jsonl
    json.keys_under_root: true
    json.add_error_key: true
    json.message_key: message
    json.overwrite_keys: true

output.elasticsearch:
  hosts: ["http://elasticsearch:9200"]
  index: "app-logs-%{+yyyy.MM.dd}"

# Optional: send to Logstash instead
# output.logstash:
#   hosts: ["logstash:5044"]

If you need to transform or enrich logs before indexing, Logstash sits between Filebeat and Elasticsearch. The json codec parses each JSONL line automatically. Use the mutate filter to rename fields, add tags, or remove sensitive data before storage.

Logstash Pipeline

# logstash.conf
input {
  beats {
    port => 5044
    codec => json
  }
}

filter {
  # Parse timestamp from the JSON field
  date {
    match => ["timestamp", "ISO8601"]
    target => "@timestamp"
  }

  # Add environment tag
  mutate {
    add_field => { "environment" => "production" }
    remove_field => ["agent", "ecs", "host"]
  }

  # Route by log level
  if [level] == "error" or [level] == "fatal" {
    mutate { add_tag => ["alert"] }
  }
}

output {
  elasticsearch {
    hosts => ["http://elasticsearch:9200"]
    index => "app-logs-%{+YYYY.MM.dd}"
  }
}

Fluentd & Fluent Bit

Fluentd and its lightweight sibling Fluent Bit are CNCF-graduated projects widely used in Kubernetes environments. Both handle JSONL logs natively and can forward them to virtually any destination, from Elasticsearch to S3 to cloud-native log services.

Fluent Bit is the preferred log collector for Kubernetes and resource-constrained environments. It consumes roughly 450KB of memory and can process hundreds of thousands of records per second. The json parser handles JSONL input without any custom configuration.

Fluent Bit Configuration

[SERVICE]
    Flush        5
    Daemon       Off
    Log_Level    info
    Parsers_File parsers.conf

[INPUT]
    Name         tail
    Path         /var/log/app/*.jsonl
    Parser       json
    Tag          app.*
    Refresh_Interval 5

[FILTER]
    Name         modify
    Match        app.*
    Add          cluster production-us-east-1
    Add          service my-api

[OUTPUT]
    Name         es
    Match        app.*
    Host         elasticsearch
    Port         9200
    Index        app-logs
    Type         _doc
    Logstash_Format On

Fluentd offers a richer plugin ecosystem compared to Fluent Bit, with over 500 community plugins. Use it when you need advanced routing, buffering, or transformations. The in_tail plugin with format json reads JSONL files natively.

Fluentd Configuration

# fluentd.conf
<source>
  @type tail
  path /var/log/app/*.jsonl
  pos_file /var/log/fluentd/app.pos
  tag app.logs
  <parse>
    @type json
    time_key timestamp
    time_format %Y-%m-%dT%H:%M:%S%z
  </parse>
</source>

<filter app.logs>
  @type record_transformer
  <record>
    hostname "#{Socket.gethostname}"
    environment production
  </record>
</filter>

<match app.logs>
  @type elasticsearch
  host elasticsearch
  port 9200
  index_name app-logs
  logstash_format true
  <buffer>
    @type memory
    flush_interval 5s
    chunk_limit_size 5m
  </buffer>
</match>

Cloud Logging Platforms

All three major cloud providers accept JSONL-formatted logs and automatically parse JSON fields into searchable, filterable attributes. This eliminates the need to run your own log infrastructure while giving you the same structured querying capabilities.

AWS CloudWatch Logs

AWS

CloudWatch Logs automatically parses JSON log entries, making every field queryable with CloudWatch Logs Insights. When your application runs on ECS, Lambda, or EC2 with the CloudWatch agent, JSONL output is indexed as structured data with no extra configuration. Use Logs Insights SQL-like syntax to query across millions of log entries in seconds.

AWS CloudWatch Logs

# CloudWatch Logs Insights query examples
# Find all errors in the last hour
fields @timestamp, level, event, user_id
| filter level = "error"
| sort @timestamp desc
| limit 100

# Aggregate error counts by event type
fields event
| filter level = "error"
| stats count(*) as error_count by event
| sort error_count desc

# P99 latency by endpoint
fields path, duration_ms
| filter ispresent(duration_ms)
| stats pct(duration_ms, 99) as p99 by path
| sort p99 desc

GCP Cloud Logging

GCP

Google Cloud Logging (formerly Stackdriver) treats JSON payloads as structured log entries with first-class support. When you write JSONL to stdout on Cloud Run, GKE, or Cloud Functions, the logging agent parses each line into a structured LogEntry. The severity field maps directly to Cloud Logging severity levels, and jsonPayload fields become indexed and searchable.

GCP Cloud Logging

# Python example for Cloud Run / Cloud Functions
import json
import sys

def log(severity: str, message: str, **fields):
    """Write a JSONL log entry compatible with GCP Cloud Logging."""
    entry = {
        "severity": severity.upper(),  # Maps to Cloud Logging severity
        "message": message,
        **fields,
    }
    print(json.dumps(entry), file=sys.stdout, flush=True)

# GCP automatically parses these as structured logs
log("INFO", "user.login", user_id="u-42", ip="10.0.0.1")
log("ERROR", "db.timeout", host="db-primary", latency_ms=5200)

# Query in Cloud Logging:
# jsonPayload.user_id = "u-42"
# severity >= ERROR

Azure Monitor Logs

Azure

Azure Monitor ingests JSONL logs through the Azure Monitor Agent or direct API ingestion. Application Insights automatically parses JSON traces and custom events. KQL (Kusto Query Language) provides powerful querying across structured log data with support for aggregation, time-series analysis, and anomaly detection.

Azure Monitor Logs

// KQL queries for JSONL logs in Azure Monitor
// Find recent errors
AppTraces
| where SeverityLevel >= 3
| extend parsed = parse_json(Message)
| project TimeGenerated, parsed.event, parsed.user_id,
          parsed.error_message
| order by TimeGenerated desc
| take 100

// Error rate over time (5-minute buckets)
AppTraces
| where SeverityLevel >= 3
| summarize ErrorCount = count() by bin(TimeGenerated, 5m)
| render timechart

// Top error events
AppTraces
| where SeverityLevel >= 3
| extend parsed = parse_json(Message)
| summarize Count = count() by tostring(parsed.event)
| top 10 by Count

Best Practices for JSONL Logging

Following consistent conventions across your services makes JSONL logs easier to query, correlate, and alert on. These best practices are drawn from production systems processing billions of log entries per day.

Use Consistent Log Levels

Adopt a standard severity scale and use it consistently across all services. The most common convention uses five levels: DEBUG for development diagnostics, INFO for normal operations, WARN for recoverable issues, ERROR for failures requiring attention, and FATAL for unrecoverable crashes. Map numeric levels (like pino's 10-60 scale) to these string names at ingestion time for uniform querying.

Use Consistent Log Levels

{"level":"debug","event":"cache.miss","key":"user:42","timestamp":"2026-02-15T08:30:00Z"}
{"level":"info","event":"request.completed","method":"GET","path":"/api/users","status":200,"duration_ms":45,"timestamp":"2026-02-15T08:30:01Z"}
{"level":"warn","event":"rate_limit.approaching","client_id":"c-99","current":950,"limit":1000,"timestamp":"2026-02-15T08:30:02Z"}
{"level":"error","event":"payment.failed","order_id":"ord-123","error":"card_declined","timestamp":"2026-02-15T08:30:03Z"}
{"level":"fatal","event":"startup.failed","reason":"missing DATABASE_URL","timestamp":"2026-02-15T08:30:04Z"}

Standardize Field Names

Define a shared schema for common fields across all services. Use snake_case for field names, ISO 8601 for timestamps, and consistent naming for correlation identifiers. At minimum, every log entry should contain: timestamp, level, event (a machine-readable event name), and service. Request-scoped entries should include request_id and user_id for tracing.

Standardize Field Names

// Recommended field schema
{
  "timestamp": "2026-02-15T08:30:00.000Z",  // ISO 8601, always UTC
  "level": "info",                            // debug|info|warn|error|fatal
  "event": "order.created",                    // dot-notation event name
  "service": "order-api",                      // emitting service
  "version": "1.4.2",                          // service version
  "request_id": "req-abc-123",                 // correlation ID
  "trace_id": "4bf92f3577b34da6",              // distributed trace ID
  "user_id": "u-42",                           // authenticated user
  "duration_ms": 120,                          // numeric, not string
  "order_id": "ord-789",                       // domain-specific context
  "items_count": 3                             // domain-specific context
}

Performance Considerations

Structured logging adds serialization overhead to every log call. In high-throughput services, this can become significant. Use asynchronous logging (pino, zerolog) to move serialization off the hot path. Avoid logging large objects or request/response bodies at INFO level. Use sampling for high-volume debug logs. Write to local files and let a sidecar (Filebeat, Fluent Bit) handle shipping, rather than sending logs over the network synchronously.

Use asynchronous loggers (pino, zerolog) to avoid blocking the main thread during JSON serialization.

Never log full request or response bodies at INFO level. Use DEBUG level and enable it only when troubleshooting.

Sample high-volume events. Log 1 in 100 health-check requests instead of all of them.

Write logs to local files and ship them asynchronously with Filebeat or Fluent Bit. Avoid synchronous network calls in the log path.

Set a maximum log line size (e.g., 16KB) to prevent oversized entries from clogging the pipeline.

Rotate log files with logrotate or built-in library rotation to prevent disk exhaustion.

Validate Your JSONL Logs Online

Use our free browser-based tools to inspect, validate, and convert JSONL log files. No uploads required, all processing happens locally.

JSONL streaming

large JSONL files

JSONL validator

Inspect JSONL Logs in Your Browser

View, search, and validate JSONL log files up to 1GB. Instant loading, client-side processing, 100% private.

Structured Logging with JSONL

Why Structured Logging with JSONL?

Logging Libraries by Language

Python: structlog

Node.js: pino

Go: zerolog

ELK Stack Integration

Fluentd & Fluent Bit

Cloud Logging Platforms

AWS CloudWatch Logs

GCP Cloud Logging

Azure Monitor Logs

Best Practices for JSONL Logging

Use Consistent Log Levels

Standardize Field Names

Performance Considerations

Validate Your JSONL Logs Online

Inspect JSONL Logs in Your Browser

Frequently Asked Questions

What is structured logging and why should I use JSONL for it?

How do I send JSONL logs to the ELK Stack?

What is the difference between Fluentd and Fluent Bit?

Do cloud logging services automatically parse JSONL?

Which logging library should I use for JSONL output?

How do I avoid performance issues with structured logging?