About Python Comprehensions

Enter 🙋‍♀️

In this post, I'd like to explain Python's special syntax, comprehension, and share some minor thoughts on it.

What is comprehension?

Python has a unique syntax called comprehension. Basically, we use it a lot when traversing or assigning values to elements of an iterator (keys, values, items in a list, set, dictionary, etc).

A dictionary itself is not an iterator, but methods like keys(), values(), items(), etc. on a dictionary object return an iterator.

To give you the easiest example, if you want to traverse a variable in a list structure and assign values to it, you can basically use a for statement. The same is true in Python. To simplify it, you could write something like this

python
lst = []
for i in range(10):
    lst.append(i)

If we were to write the above in the syntax of Python's comprehension, it would look like this

python
lst = [i for i in range(10)]

~~Simple, right?
Python's comprehension maximizes the conciseness of your code. It's not just for assigning list values, it's also for things like

python
# list를 dictionary로 만들 때
lst = [('김O연', '서울', 28), ('이O우', '부산', 30), ('최O주', '광주', 25)]
member_dict = {
    '이름': name,
    '지역': region,
    '나이': age
    for name, region, age in lst
}

python
# dictionary에서 사용 가능한 iterator를 comprehension 문법으로 사용할 때
animals = {'강아지': '포유류', '앵무새': '조류', '개구리': '양서류'}
lst = [
	(key, value)
    for key, value in animals.items()
]

python
# 이중 리스트 만들 때
from pprint import pprint

pprint([
    [j for j in range(i, i+10)]
    for i in range(0, 100, 10)
])

list append vs. list comprehension

Python's comprehension syntax is not only more concise, it's also inherently faster. The speed difference is especially evident when creating lists.
You can see it in the following example.

python
from datetime import datetime
COUNT = 10000000

# list appending
st = datetime.now()
lst = []
for i in range(COUNT):
    lst.append(i)
print(datetime.now() - st)

# list comprehension
st = datetime.now()
lst = [
    i for i in range(COUNT)
]
print(datetime.now() - st)

python
0:00:01.116661  # list appending
0:00:00.590065  # list comprehension

As shown above, the time difference between list appending and list comprehension can double as the value of COUNT increases, and the difference can be even greater when using double and triple for statements.

This is a big difference from how Python's list append works. This is because Python's lists have to dynamically allocate memory as they grow in length, copying existing data, and so on.
Comprehension has optimized this part, so lists can be created much faster.

Thoughts on comprehension

I consider comprehension syntax to be very pythonic and efficient, and I think it's a good syntax for readability, but some people who are not familiar with Python say that it's a bit unfamiliar and not very readable.
When you should consider using comprehension are

plain
1) 2중, 3중 리스트 등 복잡한 객체를 만들 때, 오히려 가독성이 떨어진다.
2) list comprehension으로 표현하기에는 계산이 복잡할 때, comprehension으로 표현하는 것엔 한계가 있다.

In this case, you really have to weigh the length of your data and the complexity of the logic you need to perform. If your data isn't very long to begin with (e.g., you're fetching people's information and you're pretty sure you're only fetching 10 or so pagination's), or if you're simply creating an object, then not using comprehensions may not be a performance issue per se.
list append vs. list comprehension This is because, as the example shows, in the simplest list creation logic, the difference in performance between append and comprehension is only about 0.6 seconds after 10 million or so items.

Conclusion.

Reflections on using comprehension as a backend engineer

I actually use list comprehensions a lot in my backend work. This is especially true when fetching data from the DB and creating responses. It's definitely a good syntax when used well, considering the complexity of the function and the amount of data. However, I think it should be utilized according to the situation.
It's true that 'speed' is a very big part of backend development when considering the performance of a feature, so I used to have an obsessive compulsion (?) to reduce the time by 0.001 seconds at all costs! and used it like I was risking my life (!?) for list comprehension.
However, I thought that there is no absolute right because there may be a difference in people's taste whether it is really readable or not, and 코드를 가독성 있게 짠다 is very important, especially for collaboration. For that reason, as I got older, I decided to use comprehension grammar appropriately rather than risking my life.