Spring boot: 실시간 로그 수집하기 (Logstash, ELK)

Logstash

Kibana를 제외하고 ELK를 셋업 할 것이다.

ElasticSearch는 어딘가 실행되고 있다고 가정하고 Logstash를 Docker에서 돌려서 Spring Boot 로그를 수집할 것이다.

 

Logstash 셋업

Logstash 도커 파일을 만든다.

version: '3.8'
services:
  logstash:
    container_name: logstash
    image: docker.elastic.co/logstash/logstash:8.10.4
    ports:
      - "5044:5044"
      - "5000:5000/tcp"
      - "5000:5000/udp"
      - "9600:9600"
    volumes:
      - ./logstash/config:/usr/share/logstash/config
      - ./logstash/pipeline:/usr/share/logstash/pipeline

 

같은 폴더에 logstash 폴더를 하나 만들어준다.

logstash/config/logstash.yml

http.host: "0.0.0.0"
xpack.monitoring.enabled: false

logstash/config/pipelines.yml

- pipeline.id: spring-boot-logs
  path.config: "/usr/share/logstash/pipeline/spring-boot-logs.conf"

logstash/pipeline/spring-boot-logs.conf

우선 logstash console에 출력하는 방식이다.

input {
    tcp {
        port => 5000
        codec => json_lines {
            target => "[document]"
        }
    }
}
output {
    stdout {}
}

 

Docker 실행

ELK.yml 으로 저장해서 아래처럼 파일을 지정해 줬다.

docker-compose -f ELK.yml up -d

 

Spring boot 셋업

dependencies 추가

dependencies {
    // https://mvnrepository.com/artifact/net.logstash.logback/logstash-logback-encoder
    implementation 'net.logstash.logback:logstash-logback-encoder:7.4'
    // https://mvnrepository.com/artifact/ch.qos.logback/logback-classic
    implementation 'ch.qos.logback:logback-classic:1.4.11'
    // https://mvnrepository.com/artifact/ch.qos.logback/logback-core
    implementation 'ch.qos.logback:logback-core:1.4.11'
    // https://mvnrepository.com/artifact/ch.qos.logback/logback-access
    implementation 'ch.qos.logback:logback-access:1.4.11'
 }

 

src/resources/logback.xml

localhost:5000 (tcp 열어둔 ip와 로컬)

<?xml version="1.0" encoding="UTF-8" ?>
<configuration scan="true" scanPeriod="30 seconds">
    <appender name="LOGSTASH" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
        <destination>localhost:5000</destination>
        <encoder class="net.logstash.logback.encoder.LogstashEncoder">

        </encoder>
    </appender>

    <root>
        <level value="INFO"/>
        <appender-ref ref="LOGSTASH"/>
    </root>
</configuration>

 

Spring boot을 실행하고 Logstash 콘솔에 가보면 아래처럼 로그가 쫙 출력되어 있다.

ElasticSearch에 보내고 싶으면 파이프라인 conf에 elasticsearch를 이용해서 보내면 된다.

output {
  opensearch {
    hosts       => "https://domain-endpoint:443"
    user        => "my-username"
    password    => "my-password"
    index       => "logstash-logs-%{+YYYY.MM.dd}"
    ecs_compatibility => disabled
    ssl_certificate_verification => false
  }
}

 

참고

아래처럼 Docker 이미지끼리 (Logstash + Spring Boot + Redis) 통신 가능하게 했다면

 

version: '3.8'
services:
  spring-boot:
    container_name: spring-boot-my
    build:
      context: ./build/libs
      dockerfile: Dockerfile.spring
    depends_on:
      redis:
        condition: service_healthy
      logstash:
        condition: service_healthy
    ports:
      - "8080:8080"
    links:
      - redis
    healthcheck:
      test: [ "CMD-SHELL", "curl -f http://localhost:8080/actuator/health || exit 1" ]
      interval: 10s
      timeout: 5s
      retries: 5

  redis:
    container_name: beauty-redis
    image: redis:latest
    volumes:
      - ./redis/redis.conf:/usr/local/etc/redis/redis.conf
    command: redis-server /usr/local/etc/redis/redis.conf
    environment:
      - REDIS_PASSWORD_FILE=/run/secrets/redis_password
    secrets:
      - redis_password
    ports:
      - "6379:6379"
    healthcheck:
      test: ["CMD-SHELL", "redis-cli -a $$REDIS_PASSWORD ping"]
      interval: 10s
      timeout: 5s
      retries: 5
    links:
      - logstash

  logstash:
    container_name: beauty-logstash
    image: opensearchproject/logstash-oss-with-opensearch-output-plugin:8.9.0
    ports:
      - "5044:5044"
      - "5000:5000/tcp"
      - "5000:5000/udp"
      - "9600:9600"
    volumes:
      - ./logstash/config:/usr/share/logstash/config
      - ./logstash/pipeline:/usr/share/logstash/pipeline
    healthcheck:
      test: [ "CMD-SHELL", "curl -f http://localhost:9600 || exit 1" ]
      interval: 10s
      timeout: 5s
      retries: 5

secrets:
  redis_password:
    file: redis.env

 

logback.xml 에서 localhost가 아닌 logstash:5000으로 해줘야한다.