Sunday, August 30, 2026

How to Set Up Let’s Encrypt SSL Certificate with Nginx

If we are hosting a web application on a Linux server, enabling HTTPS is one of the first things we should take care of before exposing the application to users.

One of the easiest ways to do this is by using Let’s Encrypt. It provides free SSL/TLS certificates, and with Certbot we can automate both certificate installation and renewal.

In this article, we will go step by step through the process of configuring a Let’s Encrypt certificate with Nginx. We will also look at certificate renewal and some common issues that can prevent the certificate from being generated or renewed successfully.

What We Are Going to Set Up

We will configure Nginx to serve our application over HTTPS using a certificate issued by Let’s Encrypt.

The setup will include:

  • A domain pointing to our server

  • Nginx configured as the web server or reverse proxy

  • A Let’s Encrypt SSL certificate

  • HTTPS access on port 443

  • HTTP to HTTPS redirection

  • Automatic certificate renewal

Prerequisites

For this guide, we will assume that:

  • We have a Linux server.

  • Ubuntu is being used as the operating system.

  • Nginx is installed or can be installed.

  • We have a domain name.

  • We have access to the DNS configuration for the domain.

  • We have sudo access to the server.

  • Ports 80 and 443 are accessible from the internet.

For the examples below, we will use:

Domain: example.com
Server IP: 203.0.113.10

Replace these values with the actual domain and server details.

Step 1: Point the Domain to the Server

Before requesting an SSL certificate, our domain needs to resolve to the server where Nginx is running.

In the DNS configuration, create an A record.

Type: A
Name: @
Value: 203.0.113.10

If we also want to support www.example.com, we can create another record:

Type: A
Name: www
Value: 203.0.113.10

Depending on the DNS provider, the interface will look different, but the concept remains the same.

We should verify that DNS is resolving correctly before moving forward.

From the server or our local machine, run:

nslookup example.com

or:

dig example.com

The returned IP address should match our server's public IP.

If DNS is not resolving correctly, there is no point proceeding with Certbot yet. Let's Encrypt needs to verify that we control the domain.

Step 2: Install Nginx

If Nginx isn't already installed, we can install it using:

sudo apt update
sudo apt install nginx -y

Once the installation is complete, check the service:

sudo systemctl status nginx

We should see that the service is running.

We can also verify the configuration:

sudo nginx -t

A successful configuration check should return something similar to:

syntax is ok
test is successful

Now open the domain in a browser:

http://example.com

At this stage, we should get the Nginx default page or our application, depending on how Nginx has been configured.

Step 3: Configure Nginx for the Domain

Before requesting the certificate, we should configure a server block for our domain.

Create a configuration file:

sudo nano /etc/nginx/sites-available/example.com

Add the following:

server {
    listen 80;
    listen [::]:80;

    server_name example.com www.example.com;

    location / {
        proxy_pass http://127.0.0.1:8000;

        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

The proxy_pass value should point to the application running behind Nginx.

For example, if our application is running on port 3000:

proxy_pass http://127.0.0.1:3000;

If the application is running on another server, we can use that server's address instead.

Step 4: Enable the Nginx Configuration

Create a symbolic link to enable the configuration:

sudo ln -s /etc/nginx/sites-available/example.com /etc/nginx/sites-enabled/

Then test the configuration:

sudo nginx -t

If everything looks correct, reload Nginx:

sudo systemctl reload nginx

Now verify that the domain is accessible:

http://example.com

We should make sure the application works correctly over HTTP before moving to HTTPS.

Step 5: Install Certbot

Now we can install Certbot and the Nginx plugin.

sudo apt update
sudo apt install certbot python3-certbot-nginx -y

Verify the installation:

certbot --version

We should get the installed Certbot version as the output.

Certbot will communicate with Let's Encrypt, request the certificate and configure Nginx for HTTPS.

Step 6: Request the Let's Encrypt Certificate

Now we can request the certificate.

Run:

sudo certbot --nginx -d example.com -d www.example.com

Certbot will ask for some information, including an email address and agreement to the terms of service.

It will then communicate with Let's Encrypt and perform domain validation.

If the validation succeeds, Certbot will obtain the certificate and update the Nginx configuration.

One of the advantages of using the Nginx plugin is that we don't have to manually copy certificate paths into the Nginx configuration.

Step 7: Redirect HTTP to HTTPS

During the Certbot setup, we may be asked whether HTTP traffic should be redirected to HTTPS.

Choose the redirect option if we want all HTTP traffic to automatically use HTTPS.

After enabling the redirect, users visiting:

http://example.com

will automatically be redirected to:

https://example.com

This ensures that users always access the application through an encrypted connection.

Step 8: Verify the HTTPS Configuration

Open the following URL in a browser:

https://example.com

The browser should show the secure connection indicator.

We can also check the certificate from the command line:

openssl s_client -connect example.com:443 -servername example.com

This provides information about the certificate and TLS connection.

We can also check the Nginx configuration:

sudo nginx -t

If everything is configured correctly, Nginx should report:

syntax is ok
test is successful

Step 9: Check the Certificate

Certbot provides a convenient command to check the certificates currently installed on the server.

sudo certbot certificates

Example output:

Certificate Name: example.com
Domains: example.com www.example.com
Expiry Date: ...
Certificate Path: /etc/letsencrypt/live/example.com/fullchain.pem
Private Key Path: /etc/letsencrypt/live/example.com/privkey.pem

Certbot normally stores certificates under:

/etc/letsencrypt/

We generally shouldn't manually modify files inside this directory because Certbot manages the certificate lifecycle.

Step 10: Configure Automatic Renewal

Let's Encrypt certificates are short-lived and need to be renewed regularly.

The good part is that Certbot can handle renewal automatically.

First, check whether the Certbot renewal timer is active:

sudo systemctl status certbot.timer

We can also check the timers on the system:

systemctl list-timers | grep certbot

If the timer is configured correctly, Certbot will periodically check whether the certificate needs renewal.

Step 11: Test Certificate Renewal

We shouldn't wait until the certificate is about to expire to find out that automatic renewal doesn't work.

We can perform a dry run:

sudo certbot renew --dry-run

This performs a simulated renewal process without replacing the existing certificate.

A successful test should indicate that the renewal simulation completed successfully.

This is something worth testing after the initial setup and whenever we make significant changes to the Nginx or DNS configuration.

Common Problems

The installation itself is usually straightforward. Most problems occur during domain validation, Nginx configuration or certificate renewal.

Problem 1: Domain Doesn't Point to the Server

If Certbot reports that domain validation failed, the first thing we should check is DNS.

Run:

dig example.com

Make sure the returned IP address points to the correct server.

If DNS was changed recently, we may also need to wait for the DNS changes to propagate.

Problem 2: Port 80 Is Not Accessible

Let's Encrypt commonly needs to reach the server through HTTP during certificate validation.

Make sure port 80 is open.

On Ubuntu with UFW:

sudo ufw status

If required:

sudo ufw allow 80/tcp
sudo ufw allow 443/tcp

If the server is running in a cloud environment, we also need to check the cloud firewall or security group.

Opening the port in UFW isn't enough if the cloud firewall is blocking the traffic.

Problem 3: Nginx Configuration Test Fails

If this command fails:

sudo nginx -t

we should fix the Nginx configuration before running Certbot.

The error message usually contains the configuration file and line number where the problem exists.

We can inspect the complete configuration using:

sudo nginx -T

This prints the complete Nginx configuration currently being loaded.

Problem 4: Too Many Redirects

After enabling HTTPS, we may sometimes encounter a redirect loop.

For example, the browser keeps switching between HTTP and HTTPS.

This usually happens when there is another reverse proxy or load balancer in front of Nginx and the original protocol isn't being handled correctly.

We should check:

  • Nginx redirect rules

  • Reverse proxy configuration

  • Load balancer configuration

  • X-Forwarded-Proto headers

We should avoid adding multiple layers of HTTPS redirects without understanding how traffic flows through the infrastructure.

Problem 5: Certificate Renewal Fails

If automatic renewal fails, first check the existing certificates:

sudo certbot certificates

Then test renewal manually:

sudo certbot renew --dry-run

Check the Certbot logs if the problem isn't obvious:

sudo ls /var/log/letsencrypt/

The logs usually provide enough information to identify whether the issue is DNS, Nginx, port accessibility or certificate validation.

Useful Commands

Here are some commands we can keep handy when troubleshooting a Let's Encrypt and Nginx setup.

Check Nginx status:

sudo systemctl status nginx

Test Nginx configuration:

sudo nginx -t

Reload Nginx:

sudo systemctl reload nginx

Check certificates:

sudo certbot certificates

Test renewal:

sudo certbot renew --dry-run

Check Certbot timer:

systemctl list-timers | grep certbot

Check HTTPS certificate:

openssl s_client -connect example.com:443 -servername example.com

Check DNS:

dig example.com

A Few Things We Should Keep in Mind

Getting the certificate is only one part of enabling HTTPS.

We should also make sure that:

  • HTTP redirects to HTTPS.

  • Port 443 is accessible.

  • The certificate covers all required domains.

  • Automatic renewal is working.

  • Nginx configuration remains valid.

  • Application URLs use HTTPS where required.

  • Mixed-content issues are not introduced in the application.

HTTPS should be treated as part of the application's infrastructure rather than simply a certificate that we install once and forget about.

Saturday, July 25, 2026

How We Set Up Centralized Logging in Kubernetes with Grafana Loki

As Kubernetes environments grow, troubleshooting applications becomes increasingly difficult if we rely only on kubectl logs. While the command works well for individual pods, it becomes inefficient when applications are distributed across multiple namespaces, deployments and worker nodes.

In a production environment, pods are constantly being created, restarted and terminated. Once a pod is deleted, its logs are often lost unless they have been collected and stored somewhere centrally.

This is where a centralized logging solution becomes essential.

In this article, we will set up centralized logging in Kubernetes using Grafana, Loki and Promtail. We will also look at some of the common issues that teams encounter during installation and how to resolve them.

Why Do We Need Centralized Logging?

For a small cluster with only a few applications, checking logs using kubectl logs may be sufficient.

kubectl logs <pod-name>

However, as more applications are deployed, finding logs quickly becomes difficult.

Some common challenges include:

  • Pods restart frequently.

  • Multiple replicas generate separate logs.

  • Applications run across different namespaces.

  • Logs disappear when pods are recreated.

  • Searching across multiple applications becomes time consuming.

A centralized logging solution solves these problems by collecting logs from every node and storing them in one place.

Why Grafana, Loki and Promtail?

There are several logging solutions available for Kubernetes, including the ELK Stack.

For our Kubernetes environments, we chose Grafana Loki because it is lightweight, easy to deploy and integrates seamlessly with Grafana.

The architecture is simple.

Application Pods
        │
        ▼
Promtail (DaemonSet)
        │
        ▼
      Loki
        │
        ▼
     Grafana

Each component has a specific responsibility.

  • Promtail collects logs from every Kubernetes node.

  • Loki stores the logs.

  • Grafana provides a user interface to search and visualize them.

Prerequisites

Before starting the installation, make sure the following requirements are met.

  • Kubernetes cluster is running.

  • Helm is installed.

  • StorageClass is available.

  • You have cluster administrator access.

  • A namespace exists for monitoring.

Create the namespace if it doesn't already exist.

kubectl create namespace monitoring

Step 1: Add the Grafana Helm Repository

First, add the official Helm repository.

helm repo add grafana https://grafana.github.io/helm-charts

helm repo update

This downloads the latest Helm charts for Grafana, Loki and Promtail.

Step 2: Install Loki Stack

Install the Loki Stack using Helm.

helm install loki grafana/loki-stack \
--namespace monitoring

Depending on the chart version, Helm deploys components such as:

  • Loki

  • Promtail

  • Grafana (optional)

  • Service Accounts

  • ConfigMaps

Wait for the installation to complete before moving to the next step.

Step 3: Verify the Installation

Check whether all pods are running successfully.

kubectl get pods -n monitoring

A healthy deployment should look similar to this.

NAME                                   READY   STATUS
loki-0                                 1/1     Running
promtail-xxxxx                         1/1     Running
grafana-xxxxxxxx                       1/1     Running

If any pod is stuck in Pending or CrashLoopBackOff, investigate the issue before proceeding.

Step 4: Access Grafana

If an Ingress has not been configured yet, use port forwarding.

kubectl port-forward svc/grafana 3000:80 -n monitoring

Open the browser.

http://localhost:3000

Log in using the administrator credentials.

After logging in, navigate to:

Connections → Data Sources

Verify that Loki is configured as a data source.

Step 5: Verify Loki Connectivity

Before exploring logs, make sure Grafana can communicate with Loki.

Open the Loki data source and click "Save & Test".

If everything is configured correctly, Grafana displays a success message.

If the connection fails, verify:

  • Loki service name

  • Namespace

  • Service port

  • DNS resolution

  • Network policies

Most connectivity issues are caused by an incorrect service URL.

Step 6: Explore Logs

Open the Explore page in Grafana.

Select the Loki data source.

Run a simple LogQL query.

{namespace="default"}

This displays logs for all pods in the default namespace.

To filter logs for a specific application:

{app="payment-api"}

Search for error messages.

{namespace="production"} |= "ERROR"

You can also filter by container name.

{container="nginx"}

LogQL makes searching Kubernetes logs much easier compared to manually checking individual pods.

Common Issues During Installation

Although the installation process is straightforward, there are a few issues that commonly appear in production environments.

Grafana Cannot Connect to Loki

One issue we encountered was Grafana failing to connect to Loki even though both pods were running.

The first thing to verify is the Loki service.

kubectl get svc -n monitoring

Confirm that the service name matches the URL configured in the Grafana data source.

Also verify that both services exist in the same namespace.

Most connectivity problems are caused by an incorrect service URL or namespace mismatch.

Promtail Pods Remain Pending

Promtail usually runs as a DaemonSet and schedules one pod on every worker node.

If Promtail remains in the Pending state, check the pod description.

kubectl describe pod <promtail-pod> -n monitoring

Common reasons include:

  • Insufficient memory

  • Taints

  • Node selectors

  • Missing tolerations

In our environment, Promtail couldn't be scheduled because the worker node didn't have enough available memory.

Increasing node capacity resolved the issue.

Loki Pod Keeps Restarting

If Loki repeatedly restarts, inspect the logs.

kubectl logs <loki-pod> -n monitoring

Typical causes include:

  • Storage permission issues

  • Persistent Volume problems

  • Invalid Helm configuration

  • Missing storage class

Checking the logs usually identifies the problem quickly.

No Logs Are Appearing

Sometimes every pod appears healthy, but Grafana still doesn't display any logs.

Check whether Promtail is collecting logs.

kubectl logs <promtail-pod> -n monitoring

Also verify:

  • Promtail is running on every node.

  • The correct namespace is selected.

  • Labels match the LogQL query.

  • Loki is receiving log entries.

Most "missing logs" issues are related to label mismatches rather than Loki itself.

Useful LogQL Queries

Display logs from a namespace.

{namespace="default"}

Display logs for a deployment.

{app="payment-api"}

Show only error messages.

{namespace="production"} |= "ERROR"

Show warning messages.

{namespace="production"} |= "WARN"

Search for a specific exception.

{namespace="production"} |= "NullPointerException"

These simple queries make troubleshooting much faster than manually checking logs from individual pods.

Best Practices

After deploying centralized logging, consider the following recommendations.

  • Configure persistent storage for Loki.

  • Set appropriate retention policies.

  • Configure CPU and memory requests.

  • Configure resource limits.

  • Label workloads consistently.

  • Secure Grafana with authentication.

  • Create dashboards for frequently used queries.

  • Configure alerts for critical application errors.

Following these practices keeps the logging platform reliable as the Kubernetes cluster grows.

Centralized logging is one of the first observability tools every Kubernetes environment should have. While kubectl logs is useful for quick debugging, it becomes increasingly difficult to troubleshoot distributed applications as the number of services grows.

Grafana, Loki and Promtail provide a lightweight and scalable logging solution that integrates naturally with Kubernetes. Once the platform is configured, developers can search logs across namespaces, deployments and containers from a single interface, making production troubleshooting much faster.

In our experience, most installation issues are related to configuration rather than the tools themselves. Taking a few extra minutes to verify connectivity, resource allocation and service configuration during setup saves a significant amount of troubleshooting later.

OOMKilled in Kubernetes: Understanding Exit Code 137 and How to Fix It

If you've worked with Kubernetes for some time, you've probably come across a pod that suddenly restarts with the reason OOMKilled and an exit code of 137. It usually happens without much warning. One moment the application is running normally, and the next moment Kubernetes terminates the container and starts a new one.

The first reaction is often to increase the memory limit and redeploy the application. Sometimes that works, but in many cases the same issue returns after a few hours or days because the actual problem was never investigated.

In most production environments, OOMKilled isn't the real problem. It's an indication that the application consumed more memory than it was allowed to use. Our objective shouldn't be to simply allocate more memory. Instead, we should understand why the application exceeded its limit in the first place.

In this article, we will understand what OOMKilled means, why Kubernetes reports Exit Code 137, how to investigate memory-related issues and the best practices to prevent them from happening again.

What Does OOMKilled Mean?

OOM stands for Out Of Memory.

Every container running in Kubernetes has resource limits that define the maximum amount of memory it can consume. If the application crosses that limit, the Linux kernel immediately terminates the process to protect the node from running out of memory.

Kubernetes detects that the container has stopped unexpectedly and restarts it according to the pod's restart policy.

If the application continues exceeding the memory limit after every restart, the pod may eventually enter a CrashLoopBackOff state.

Understanding this behaviour is important because Kubernetes isn't killing the application. The operating system is. Kubernetes simply reports what happened and starts a new container.

Understanding Exit Code 137

When a Linux process exits, it returns an exit code.

For an OOMKilled container, Kubernetes usually reports something similar to the following:

Last State:
  Terminated

Reason:
  OOMKilled

Exit Code:
  137

Exit Code 137 indicates that the process was terminated using the SIGKILL signal.

In Kubernetes, this almost always means one of the following:

  • The application exceeded its configured memory limit.

  • The Linux Out Of Memory Killer terminated the process.

  • Kubernetes restarted the container after it exited.

Whenever we see Exit Code 137, memory usage should be the first thing we investigate.

Confirm That the Pod Was OOMKilled

The easiest way to verify the reason is by describing the pod.

kubectl describe pod <pod-name>

Look for the Last State section.

Last State:
  Terminated

Reason:
  OOMKilled

Exit Code:
  137

If both the reason and exit code match the output above, we've confirmed that memory exhaustion caused the container to terminate.

Before making any configuration changes, we should understand why it happened.

Review the Resource Configuration

The next step is checking the memory requests and limits configured for the container.

Run:

kubectl get pod <pod-name> -o yaml

Locate the resources section.

resources:
  requests:
    memory: "512Mi"
    cpu: "250m"

  limits:
    memory: "1Gi"
    cpu: "500m"

There is often confusion between requests and limits.

A memory request determines the amount of memory Kubernetes reserves for the container during scheduling.

A memory limit defines the maximum memory the application is allowed to consume.

Once the application crosses that limit, the Linux kernel terminates the process immediately.

Setting memory limits too low is one of the most common reasons for OOMKilled pods.

Check Current Memory Usage

Before increasing memory limits, we should understand how much memory the application is actually consuming.

If Metrics Server is installed, run:

kubectl top pod

or

kubectl top pod <pod-name>

Example:

NAME              CPU(cores)   MEMORY(bytes)
payment-api       210m         985Mi

If the container has a memory limit of 1Gi and is already consuming around 985Mi, even a small increase in workload may cause the application to exceed its limit.

Monitoring memory usage gives us a much clearer picture than simply guessing.

Check the Previous Logs

When Kubernetes restarts a container, the current logs may only contain startup information.

The actual problem usually exists in the previous container.

Run:

kubectl logs <pod-name> --previous

Depending on the application, we may find messages related to:

  • Memory allocation failures

  • Large file uploads

  • Garbage collection warnings

  • Cache growth

  • Unexpected spikes in workload

Checking the previous logs has helped us identify the root cause of many production incidents.

Common Reasons for OOMKilled

Although every application behaves differently, most OOMKilled incidents fall into a few common categories.

Memory Leaks

Applications that continuously allocate memory without releasing it eventually consume all available memory.

Initially everything appears normal.

After running for several hours or days, memory usage gradually increases until the container reaches its configured limit.

Monitoring memory growth over time usually helps identify this pattern.

Processing Large Files

Applications processing PDFs, images, videos or large Excel files often require much more memory than expected.

If the entire file is loaded into memory, temporary spikes can exceed the configured limit.

Whenever possible, processing files in smaller chunks significantly reduces memory consumption.

Loading Large Datasets

Another common mistake is loading an entire dataset into memory before processing it.

Instead of loading thousands of records at once, processing them in batches or using streaming techniques keeps memory usage under control.

Traffic Spikes

Applications that perform well under normal traffic may consume considerably more memory during peak usage.

Higher request volumes often result in additional objects being created, more active database connections and larger caches.

Without sufficient memory planning, the container eventually exceeds its limit.

Incorrect Resource Configuration

Sometimes the application itself isn't the problem.

The configured memory limit simply doesn't reflect the application's actual workload.

Development and testing environments often use much smaller datasets than production, making it difficult to estimate realistic memory requirements.

Should We Simply Increase the Memory Limit?

Increasing the memory limit may stop the restarts temporarily, but it shouldn't be the first solution.

Before changing resource limits, it's worth asking a few questions.

  • Did the issue start after a recent deployment?

  • Has application traffic increased?

  • Are larger files being processed?

  • Has a new feature introduced additional memory usage?

  • Is memory usage continuously increasing over time?

Answering these questions often leads us to the actual root cause instead of masking the problem.

Review Recent Deployments

If the application was working correctly yesterday but started failing after a deployment, compare the recent changes.

Run:

kubectl rollout history deployment <deployment-name>

Recent code changes may have introduced:

  • Larger in-memory caches

  • Additional background workers

  • New libraries

  • Bigger response payloads

  • Changes in data processing

If production is impacted, rolling back to the previous deployment can restore service while the investigation continues.

Best Practices to Prevent OOMKilled

Preventing memory issues is much easier than troubleshooting them during an outage.

Some practices that have worked well across production environments include:

  • Configure realistic memory requests and limits.

  • Continuously monitor CPU and memory usage.

  • Process large files in batches.

  • Stream large datasets whenever possible.

  • Optimise application caching.

  • Load test applications before production releases.

  • Review memory consumption after every major deployment.

  • Configure Horizontal Pod Autoscaler where appropriate.

Small improvements in memory management often make a significant difference to application stability.

Common Mistakes

These are some of the mistakes we see most frequently while investigating OOMKilled incidents.

  • Increasing memory limits without identifying the root cause.

  • Ignoring historical memory usage.

  • Deploying applications without load testing.

  • Assuming Kubernetes is responsible for application crashes.

  • Forgetting to review previous container logs.

  • Using the same resource configuration for every environment.

Avoiding these mistakes usually makes troubleshooting much faster.

OOMKilled is one of the most common Kubernetes issues, but it's also one of the easiest to diagnose once we understand what Exit Code 137 represents.

Rather than treating OOMKilled as the actual problem, we should treat it as an indication that the application consumed more memory than its configured limit.

A structured troubleshooting process always produces better results than making configuration changes based on assumptions. Confirm the reason using kubectl describe, review the configured resource limits, analyse memory usage, inspect the previous logs and compare recent deployments before increasing memory.

In many cases, Kubernetes has already provided everything we need to identify the root cause. We simply need to collect the information in the right order and let the evidence guide our investigation.