로그는 어디서 수집해야 할까요?🤔 - Spring 프로젝트에서 로그의 목적에 따라 Filter, AOP, Scheduler 활용해보기

Introduction 🙋‍♀️

When working on a project, there are times when you need to collect various types of logs. This is especially true for web projects, where it’s important to determine the appropriate collection points based on the purpose of the logs—from recording API requests and responses to user statistics and error tracing. I, too, found myself in a situation where I had to grapple with this issue.

I took a brief break from my existing project and was assigned to a new one for about two months. This project was built on Spring, and it was my first Spring project. So, while working on it, I took the time to learn and document the framework’s features. Note that this project was implemented using Kotlin and Spring, so the examples in this article will primarily be in Kotlin.

I performed various tasks on this project, but one of the main challenges was setting up a dashboard using Elasticsearch and Kibana, along with the log collection required for it. Basically, I attached a logger to the classes or methods I wanted to log and managed the logs in files via the logback-spring.xml configuration.

In this post, I’d like to summarize the process of understanding how Spring works for log collection and how I considered the collection locations based on the purpose of the logs. I hope this will be helpful to readers who are considering a log collection architecture for their Spring projects.

Types of Logs to Collect 🗂️

After reviewing the requirements, I was able to categorize the logs to be collected into three main types, as follows:

Access Log
- Logs containing the time a user called the API, request duration, header information, and more
Custom Metric Log
- Logs that periodically record the number of users in the application to visualize user trends
Error Log
- Logs that record errors generated by the application’s business logic

Therefore, we were able to determine the types of logs to collect based on Access Log, Custom metric Log, and Error Log.

Spring Framework and Log Collection 🗒️

So, where in the Spring framework should we store these logs?

📍 Where should we collect Access Logs? - Filter (vs. AOP)

There was a discussion to determine the location for collecting access logs, which was our first requirement. Initially, there was a suggestion to use AOP for log collection, but upon investigation, we found that the Spring Framework offers several methods for log collection besides AOP. Therefore, it was necessary to understand Spring’s request/response processing flow and determine the appropriate collection location based on the purpose of the logs.

Spring Framework’s Request/Response Flow and Access Logs

The Spring Framework operates at the application level on top of a servlet container. The servlet primarily manages the request/response flow, while the corresponding business logic is handled internally by the Spring Framework. As shown in the diagram, the process involves receiving a request from the client, processing the logic, and sending a response.

If you add interceptors or AOP to the Spring context, the flow is extended in detail as shown below.

Focusing on the collection of access logs we discussed earlier, there were two options:

Filter
- Controls the request/response flow before it enters the Spring Context.
- Like a doorman at the front door, a Filter can control the requests and responses passing through the door.
AOP
- AOP operates regardless of the Controller, Service, or Repository layers, provided a specific event occurs.
- If the Spring Context is a house, AOP is a role that appears only in specific situations, like a plumber or a delivery person.

So, where should we collect the access logs?

Since Access Logs need to collect information on the client’s request/response flow, we determined that it is most appropriate to handle them in a Filter, which directly controls requests and responses.

Defining the Filter Class

kotlin
import jakarta.servlet.Filter
import jakarta.servlet.FilterChain
import jakarta.servlet.FilterConfig
import jakarta.servlet.ServletRequest
import jakarta.servlet.ServletResponse
import jakarta.servlet.annotation.WebFilter
import jakarta.servlet.http.HttpServletRequest
import jakarta.servlet.http.HttpServletResponse
import org.slf4j.Logger
import org.slf4j.LoggerFactory

class LoggingFilter : Filter {
    private val logger: Logger = LoggerFactory.getLogger(LoggingFilter::class.java)

    override fun doFilter(request: ServletRequest, response: ServletResponse, chain: FilterChain) {
        // doFilter 메소드를 override하면서 로그 수집이 가능하다

        val httpRequest = request as HttpServletRequest
        val httpResponse = response as HttpServletResponse

        // 요청 정보 로그 -> logback-spring.xml을 통해 파일로 저장되는 위치
        logger.info("Incoming Request: Method=${httpRequest.method}, URI=${httpRequest.requestURI}, Query=${httpRequest.queryString}")

        val startTime = System.currentTimeMillis()
        chain.doFilter(request, response) // 다음 필터 또는 서블릿 호출
        val duration = System.currentTimeMillis() - startTime

        // 응답 정보 로그 -> logback-spring.xml을 통해 파일로 저장되는 위치
        logger.info("Outgoing Response: Status=${httpResponse.status}, Duration=${duration}ms")
    }
}

Registering the Bean

kotlin

import com.mysite.filter.LoggingFilter
import org.springframework.boot.web.servlet.FilterRegistrationBean
import org.springframework.context.annotation.Baen
import org.springframework.context.annotation.Configuration

@Configuration
class LoggingConfig {
    @Bean
    fun loggingFilter(): FilterRegistrationBean<LoggingFilter> {
    	val registrationBean = FilterRegistrationBean<LoggingFilter>()
        registrationBean.filter = LoggingFilter()
        registrationBean.addurlPatterns("/*")  // 모든 url에 대해 filter가 작동되게 함
        return registrationBean
    }
}

📍 Collecting Custom Metric Logs Using Scheduling - Scheduled

To meet the requirement for tracking “user trends,” we created a schedule component using the scheduled annotation provided by Spring by default to record logs.
Here, the scheduled component is an asynchronous task that operates independently of Spring’s request flow and is used for logging periodic data.

💡 What Are Custom Metrics Logs?
Logs accumulated for the purpose of periodically collecting and tracking specific system states or metrics. Unlike general application logs or performance logs, they are used to monitor the current state of data and analyze operational data.

Scheduled Component

kotlin
import org.slf4j.Logger
import org.slf4j.LoggerFactory
import org.springframework.scheduling.annotation.Scheduled
import org.springframework.stereotype.Component

@Component
class TableCountScheduler() {

    private val logger: Logger = LoggerFactory.getLogger(TableCountScheduler::class.java)

    @Scheduled(cron = "0 0 9 * * ?") // 매일 아침 9시에 실행
    fun logTableCount() {
        val count: Int? = userRepository.count()
        
        // logback-spring.xml을 통해 파일로 저장되는 위치
        logger.info("user count : {}", count)
    }
}

📍 “We need to log the logs generated while executing business logic!” – AOP

This was an additional requirement that came up. A colleague who was already working on this had integrated a logger into the AOP, so all I had to do was add a configuration to logback-spring.xml to enable file logging via that logger. However, I determined that I needed to verify the nature of the logs being generated by that AOP component so that I could split the files for storage and add the appropriate index to Elasticsearch to send the logs.

The logger was logging errors that occurred while executing business logic at the pointcut connected to AOP. Therefore, these logs were similar to Error log.

Since AOP was implemented to categorize specific business-level errors in detail, I felt that the logger configuration—which involved more specific customization—would work well even when implemented within AOP.

AOP Component

kotlin
import org.aspectj.lang.annotation.Aspect
import org.aspectj.lang.annotation.AfterThrowing
import org.aspectj.lang.annotation.Pointcut
import org.slf4j.Logger
import org.slf4j.LoggerFactory
import org.springframework.stereotype.Component

@Aspect
@Component
class EventErrorLoggingAspect {

    private val logger: Logger = LoggerFactory.getLogger(EventErrorLoggingAspect::class.java)

    @Pointcut("execution(* com.example.listener..*(..))")
    fun eventListenerMethods() {}

    
    @AfterThrowing(pointcut = "eventListenerMethods()", throwing = "exception")
    fun logEventListenerError(exception: Throwable) {
    	// logback-spring.xml을 통해 파일로 저장되는 위치
        logger.error("Exception in Event Listener: ${exception.message}", exception)
    }
}

Conclusion 🪵

Through this project, I was able to deeply consider how to collect logs within the Spring ecosystem. While I had known that there are various types of logs, this was the first time I had actually considered where and how to collect them within a web framework.

It was also a project that made me realize that, when using the Spring Framework, the entity responsible for managing logs within the Spring ecosystem can vary depending on the type of log and its characteristics.In particular, by learning how to utilize various Spring features such as Filters, Schedulers, and AOP based on the characteristics of each log type—such as Access Logs, Custom Metric Logs, and Error Logs—I was able to deepen my understanding of the Spring Framework’s architecture.

Moving forward, I hope that by thoroughly understanding the similarities and differences between the Spring Framework and Python web frameworks, I’ll be able to contribute to the overhaul of the logging system in my existing projects.

Where should I collect logs? 🤔 - Exploring the use of Filters, AOP, and Schedulers in Spring projects based on the purpose of logging