Docker Swarm logging
This guide describes how you can set up log forwarding from your Docker Swarm environments to LOGIQ. To forward logs from Docker Swarm to LOGIQ, do the following.

Deploy fluent-bit container

Create the following files:

parsers.conf

1
[PARSER]
2
Name json
3
Format json
Copied!

fluent-bit.conf

Update the LOGIQ host and Authorization Header in the file. The token can be obtained from LOGIQ UI as described here.
1
[SERVICE]
2
Flush 1
3
Parsers_File /fluent-bit/etc/parsers.conf
4
Log_Level error
5
6
# collect docker logs using fluend logging driver
7
[INPUT]
8
Name forward
9
Listen 0.0.0.0
10
port 24224
11
12
# try parsing log as json and lift its keys to the first-level
13
[FILTER]
14
Name parser
15
Match *
16
Parser json
17
Key_Name log
18
Reserve_Data On
19
20
[OUTPUT]
21
name http
22
match *
23
host <LOGIQ-endpoint>
24
port 443
25
URI /v1/json_batch
26
Format json
27
tls on
28
tls.verify off
29
net.keepalive off
30
compress gzip
31
Header Authorization Bearer <Token>
Copied!

docker-compose.fluent.yaml

Ensure that the Fluent Bit Docker Compose uses the parsers.conf and fluent-bit.conf which we prepared in the previous step.
1
version: "3.7"
2
3
services:
4
fluent-bit:
5
image: fluent/fluent-bit:latest
6
volumes:
7
- ./fluent-bit.conf:/fluent-bit/etc/fluent-bit.conf
8
- ./parsers.conf:/fluent-bit/etc/parsers.conf
9
ports:
10
- "24224:24224"
11
- "24224:24224/udp"
Copied!
Run the following command to get fluent-bit running in your Docker Swarm cluster.
1
docker stack deploy --compose-file docker-compose.fluent.yaml fluent-bit
Copied!
Ensure that the fluent-bit container is running in the cluster.

Update services to add logging driver

Add the global logging configuration to your docker-compose files as shown below
1
x-logging: &default-logging
2
driver: fluentd
Copied!
Update the services in docker-compose files to add logging configuration and environment variables. The Namespaceand AppName set in the service definition defines how the logs from the service are indexed in LOGIQ.
1
environment:
2
Namespace: production
3
AppName: nginx
4
logging:
5
driver: fluentd
6
options:
7
env: "Namespace,AppName"
Copied!

Examples

If your default Docker Compose file looks like this:
1
version: "3.4"
2
3
Services:
4
log_creator:
5
image: alpine:3.12.4
6
command: >
7
sh -c '
8
counter=0
9
while :
10
do
11
sleep 1
12
echo "{\"message\": \"log line $counter\", \"something\": \"else\"}"
13
counter=$(($counter + 1))
14
done'
15
16
Copied!
After you’ve added the logging configuration, your Docker Compose file should look like this.
1
version: "3.4"
2
3
x-logging: &default-logging
4
driver: fluentd
5
6
services:
7
log_creator:
8
image: alpine:3.12.4
9
logging: *default-logging
10
command: >
11
sh -c '
12
counter=0
13
while :
14
do
15
sleep 1
16
echo "{\"message\": \"log line $counter\", \"something\": \"else\"}"
17
counter=$(($counter + 1))
18
done'
19
environment:
20
Namespace: logiq-namespace
21
AppName: log-generator
22
logging:
23
driver: fluentd
24
options:
25
env: "Namespace,AppName"
Copied!
Run the following command to run the container.
1
docker stack deploy --compose-file logger-test.yaml logger-test
Copied!
Your Docker Swarm logs will now be ingested into LOGIQ.
Export as PDF
Copy link
Edit on GitHub