How shutdown signal can be missed by the container

Missing ‘exec’ command on dockerfile / shell script may cause the containerized process to miss the SIGTERM signal sent by the container manager.

This will cause the process being stopped with the SIGKILL signal, without properly releasing the resources, potentially leaving the system in an inconsistent state.

The example below will demonstrate this. It waits for a termination signal and triggers a shutdown hook before exiting:

public static void main(String[] args) {
    Runtime.getRuntime().addShutdownHook(
            new Thread(() -> System.out.println("Clean shutdown")));

    System.out.println("PID: " + ManagementFactory.getRuntimeMXBean().getPid());

    try {
        new CountDownLatch(1).await();
    } catch (InterruptedException e) {
        System.out.println("Interrupted");
    }
}

We will run this using a shell script:

#!/bin/sh
# docker-entrypoint.sh
java -jar app.jar

…and a Dockerfile:

FROM azul/zulu-openjdk-alpine:17-jre

WORKDIR app
COPY target/app.jar .
COPY docker-entrypoint.sh .
ENTRYPOINT ["./docker-entrypoint.sh"]

Running the container shows that the PID is not 1:

$ docker build -t shutdowndemo .
...
$ docker run --name demo shutdowndemo
PID: 5

No “Clean shutdown” message when container is stopped:

$ docker stop demo
$ _

The reason for that is, instead of our program, the shell script docker-entrypoint.sh has been assigned to PID 1, which received the SIGTERM signal from the container manager.

Executing cat /proc/1/cmdline in the container confirms this (ps command is missing in the container):

$ docker exec demo cat /proc/1/cmdline
/bin/sh./docker-entrypoint.sh

After a timeout, container manager stops the container by sending a SIGKILL signal. Since the application starts successfully, the problem may go unnoticed.

Fixing with exec 

#!/bin/sh

# docker-entrypoint.sh

# ... 
 
exec java $JAVA_OPTS -jar app.jar "$@"

Updated container triggers the hook on exit:

$ docker run --rm --name sdemo shutdowndemo
PID: 1
 
Clean shutdown
$ _

Without the shell script

The exec command should be added to Dockerfile if no shell script is used:

FROM azul/zulu-openjdk-alpine:17-jre

WORKDIR app
COPY target/app.jar .
ENTRYPOINT exec java -jar app.jar

Isn’t PID 1 the init process?

Linux provides namespaces for various system resources, including PID, which is used for virtualization.

Failing fast

This Spring Boot example aborts startup if the PID is not 1 for the ‘kubernetes’ and ‘docker’ profiles.

@Component
public class PidChecker {

    private static final Logger log = LoggerFactory.getLogger(PidChecker.class);

    private final ConfigurableApplicationContext context;

    public PidChecker(ConfigurableApplicationContext context) {
        this.context = context;
    }

    @PostConstruct
    void onStart() {
        if(Stream.of(context.getEnvironment().getActiveProfiles())
                 .anyMatch(profile -> 
                       List.of("kubernetes", "docker").contains(profile))
                && ManagementFactory.getRuntimeMXBean().getPid() != 1) {
            log.error("PID is not 1 for containerized profile, shutting down");
            context.close();
        }
    }
}