Building a Logstash Offline Plugin Pack with Docker

If you run a Logstash node in an environment where it doesn't have access to the public Internet, and need to install some extra plugins, you have to build an "offline plugin pack" (a zip containing the plugins and their dependencies) on a machine that does have public Internet access. You can then copy the pack to your Logstash node, and install the plugins from it directly.

Here's a quick little script I whipped up to build the offline plugin pack using the official Logstash docker container:

#!/bin/sh -e logstash_version=7.10.0 logstash_plugins=$(echo ' logstash-codec-cloudfront logstash-input-s3-sns-sqs ' | xargs) echo " bin/logstash-plugin install $logstash_plugins bin/logstash-plugin prepare-offline-pack \ --output /srv/logstash/logstash-plugins.zip \ $logstash_plugins " | docker run -i -u $(id -u) -v $(pwd):/srv/logstash --rm \ docker.elastic.co/logstash/logstash:$logstash_version /bin/sh

Set the script's logstash_version variable to the version of Logstash you're using, and set the (whitespace-separated) list of plugins in the logstash_plugins variable to the plugins you need. Run the script, and it will output a logstash-plugins.zip into your working directory.

You can then copy the logstash-plugins.zip file to your Logstash node (for example, to the /usr/share/logstash directory of the machine), and install the contained plugins like this:

cd /usr/share/logstash sudo -u logstash bin/logstash-plugin install file://logstash-plugins.zip

Make sure you run the logstash-plugin command as the same user you use to run Logstash itself (typically the logstash user) — otherwise the plugins will be installed with the wrong filesystem permissions (and you'll see errors about it when you run the main Logstash process).