Python 3.8 is reaching EOL... We need to upgrade now 🥹 - Upgrading the Python version in a Flask project

Introduction🙋‍♀️

In service development, the maintenance process is just as important as adding new features. Maintenance goes beyond simply maintaining the stability and quality of the code; it also involves systematically managing the versions of the programming languages and packages in use. Especially in today’s software environment, where the development ecosystem is constantly evolving, this management is a critical factor that determines a project’s stability.

This article details the process of upgrading the Python version used to implement the project. The goal was to eliminate security vulnerabilities caused by Python 3.8—which is no longer supported—thereby reducing future package compatibility issues and facilitating long-term maintenance.

As a Python backend engineer, I’ll share the decisions I made while carrying out this task and how I executed it step by step. By focusing on the practical challenges I faced during this process and the solutions I implemented, I hope to provide tangible assistance to developers planning similar tasks.

Python 3.8... End of Support...

The project we were working on internally, which uses the Flask framework, was still being developed using Python 3.8. We had upgraded from version 3.6 to 3.8 quite some time ago, and now that Python 3.13 has been released, I realized anew that we hadn’t been managing our version effectively for quite a while. (There was a business decision to build and launch the product quickly, so there was a trade-off where we couldn’t pay much attention to version management.)

However, one day in October, I checked Python’s version lifecycle and found that...

** 🚨 Python 3.8, end-of-life in October 2024 🚨 **

"Huh...? You mean Python 3.8 is reaching end-of-life right now...?"

That’s right. We realized we could no longer delay upgrading our Python version...

Why do we need to upgrade? Is it absolutely necessary?

First, from a product perspective, we had to evaluate whether upgrading Python was truly necessary. While we thought, “Isn’t it obvious that we should upgrade?”, the reason we were even considering this was tied to the characteristics of our domain and product. Looking at the product’s characteristics:

1. The current product is deployed on a closed network.
2. It is deployed in a single-tenancy structure, and we may no longer support certain deployment environments (e.g., Client A).
3. Nevertheless, product features are continuously being developed and enhanced.

Broadly speaking, I was able to summarize the situation into the three points above.

And the general reasons for upgrading Python are as follows:

1. Security vulnerabilities arise because security patches are no longer applied.
2. Support for key libraries (packages) may be discontinued.
3. We cannot take advantage of Python’s improvements and new features.
4. The likelihood of incompatibility in deployment environments, such as Docker, increases.

And so on...

We weighed the realities against the ideals, thought deeply about “why we need to upgrade the Python version,” and reached a conclusion.

• Deploying on a closed network means we can be slightly less concerned about security than with public network services. However, this does not mean security is absolutely guaranteed. (Security issues are, of course, still possible even within a closed network.)
• We may not support deployment for certain clients, which means we could end support while keeping the current version fixed. However, this applies only to a limited number of clients, and since we plan to continue expanding our client base, this does not mean we are discontinuing our support activities altogether.
• Since we “plan to expand our customer base” and, for that reason, the development and enhancement of product features will continue, the project is ongoing.

If product feature development had been discontinued, if current features were no longer being updated, or if deployment to the latest version of the product were no longer possible, we wouldn’t need to worry about the Python version. But no matter how we look at it from various perspectives, our product is still active and under active management. Given that, there was even less reason not to upgrade the version.

Upgrading the Version Gradually

Since the project is quite large, I couldn’t gauge the scope of the side effects that might occur if I simply upgraded the Python version.

“What if a key feature breaks?”

This is similar to the thoughts that arise when fixing legacy code or refactoring. Accordingly, to ensure a stable version upgrade, we established the following steps and proceeded with the work.

• Decide on the Python version to upgrade to (but now factoring in package compatibility)
• Check for changes to built-in methods
• Enhance stability through testing and scenario verification
• Review the deployment process

Deciding on the Python version to upgrade to (while also considering package compatibility)

So, which Python version should we use going forward? Two opposing opinions emerged within the team regarding this.

"Let’s use the latest Python right away vs. Let’s use a stable Python version"

The idea of using the latest Python is certainly tempting. This is because we thought, "With just one update, we might not have to worry about version upgrades for a while?"
However, the latest version isn’t always the best choice. Python 3.13 had just been released, so it was highly likely to be unstable. And, of course, we had to consider compatibility with the packages currently installed in the project and the code we’d written. So, I investigated “compatibility between Python and the packages,” which I considered the most important factor within the scope of this project. The goal here was to find the “upper bound”—that is, the highest version of Python that the packages I’m currently using support.

Generally, Python packages are distributed on a site called PyPI. This site also publishes metadata for each package. While this information can be viewed directly on the site, it can also be retrieved via the API provided by PyPI. Since I was using over 100 packages, I decided to use the API to check. (I based my investigation on the packages listed in requirements.txt.)

python
import requests

def get_python_compatibility(package_name, package_version):
    url = f"https://pypi.org/pypi/{package_name}/{package_version}/json"
    response = requests.get(url)
    if response.status_code == 200:
        data = response.json()
        # 💡 requires_python 값이 각 패키지에서 지원하는 파이썬 버전이다.
        requires_python = data["info"].get(
            "requires_python", "지원 파이썬 버전 정보 없음"
        )
        return requires_python
    else:
        return "패키지 정보를 가져오지 못함"


file_name = "requirements.txt"

with open(file_name, "r", encoding="utf-8") as file:
    for line in file:
        package_name, package_version = line.strip().split("==")
        print(package_name, package_version)
        print(get_python_compatibility(package_name, package_version))
        print()

Among the packages I investigated, only one had an upper version limit specified. Only the package scipy explicitly stated an upper limit of <3.11. Based on this, I concluded that upgrading to Python 3.11 or higher would be difficult.

💡Conclusion: Let’s use Python 3.10.x (with the highest patch version)

(Upgrading to the next Python version will likely require updating the package versions first...)

Checking Built-in Changes

Now that we’ve decided on a Python version, we checked the changes in that version. For this, we referred to the official documentation for that specific version.

The key points to check here were whether any features had been “deprecated” and whether any existing built-in methods or classes had been “removed.” Since I didn’t need to check for newly added features to verify that the existing code still worked, I didn’t need to review those changes.

In this particular case, the path to one class had changed completely. Starting with Python 3.10, the from collection import Iterable path was completely removed (prior to that, it had only triggered a deprecation warning) and changed to from collection.abc import Iterable. In such cases, the issue could be resolved by updating the path. (You can also find a note in the Python 3.9 official documentation for collections stating that it will be removed in 3.10.)

Improving Stability Through Testing and Scenario Checks

📌 Running Test Code

If your test code is well-written, it’s a good idea to run it. Since unexpected errors can occur during execution (especially when built-in methods have changed), this made it easier to take corrective action.

📌 Verifying Based on User Scenarios

Once unit testing was complete, I also had to go through a verification process based on user scenarios. Since there was no separate automated code, I cloned the front-end project into my local environment using Git, launched a container, and performed functional tests manually. Additionally, I double-checked everything after deploying to the company’s development environment.

Reviewing the Deployment Process

Once testing on the local machine was complete, I reviewed whether there would be any issues during deployment at the CI/CD level.
In my case, I had to change the Python image specified in the Dockerfile. Previously, I was using version brunneis/python:3.8.6-ubuntu, and I changed it to the python:3.10.15-slim image.

## brunneis/python:3.8.6-ubuntu 대신 공식 이미지로 변경

python:3.10.15-slim

ENV TZ=Asia/Seoul

RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime \
    && echo $TZ > /etc/timezone \
    && apt update \
    && apt install -y vim git sudo
# (이하 생략)

However, the brunneis/python:3.8.6-ubuntu image I had been using provided more apt packages than the slim image. Consequently, based on errors that occurred during the build process, I had to install the apt packages as well.

python:3.10.15-slim

ENV TZ=Asia/Seoul

RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime \
    && echo $TZ > /etc/timezone \
    && apt update \
    && apt install -y vim git sudo \
    ## 특정 패키지가 c 빌드를 하고 있어 아래의 apt 패키지들이 빌드 때 필요했음
    && apt install -y gcc make
# (이하 생략)

After modifying the Dockerfile and running build tests, I confirmed that the service was operating without major issues once deployment was complete.

This concludes the process of upgrading from Python 3.8 to 3.10.

Conclusion 🛠️

Through this project, we were once again reminded of the importance of continuous version verification and management. In particular, while investigating the Python versions supported by various packages, we realized that hitting the upper limit of a package’s supported Python versions—which prevents further version upgrades—can compromise a project’s stability due to the end of maintenance for that package. This served as a wake-up call, making us realize that this could eventually lead to greater technical debt over time.

Based on this, we concluded that we must systematically review the versions of the packages used in our project on a regular basis and update them one by one. We also became optimistic that by checking both Python and package versions together and performing updates recursively, we can always maintain an optimal state.

I also came to think of the project as a house. I felt the similarity in that it doesn’t end once it’s completed; rather, it can only be used reliably if its robustness is maintained through continuous upkeep. Moving forward, I pledge to consistently inspect and manage the project with this mindset, treating it like a “house,” and to keep it robust.