Difference in read(), readline() and readlines() in Python
I was looking on a web of Python the commands mentioned in title and their difference; however, I have not satisfied with a complete basic understanding of these commands. Suppose my file has only the following content.
This is the first time I am posing a question on this site, I will appreciate if someone clarifies my doubts for learning the Python. I thank the StackOverflow for this platform.
In the commands read() , readline() and readlines() , one difference is of course reading whole file, or a single line, or specified line. But I didn’t understand the use/necessity of bracket () in these commands. For example, what is the difference in readline() and readline(7) ? If the argument 7 exceeds the number of lines in the file, what will be output? On the web mentioned above, it is explained what the argument in read() does; but it is not mentioned what the argument in readline() or readlines() does?
Difference Between read(), readline() and readlines() in Python
When reading files in Python, there are a few different functions you can use to extract text from a file.
The three main functions you can use to read content from a file are read(), readline() and readlines().
read() reads the entire file and returns a string, readline() reads just one line from a file, and readlines() returns a list of strings representing the lines of the file.
In the rest of this article, we will go into the details of each function and the differences between read(), readline() and readlines()
The power of programming in Python is that there are many ways you can accomplish similar actions. With this flexibility, it can be tricky to understand the differences between certain functions.
One such situation is when you are performing file input and output and want to read or write to files.
When reading files in Python, there are a few different functions you can use to extract text from a file: read(), readline() and readlines().
Let’s talk about how you can use each of these functions in Python to read text from a file.
Is there a difference between : «file.readlines()», «list(file)» and «file.read().splitlines(True)»?
Ofc, I tried first. They produce the exact same output. That’s why I ask if there is any difference. (added a small edit for clarity).
Jul 23, 2018 at 13:21
The biggest question is why you need that list. If you’re eventually going to iterate over it once the most pythonic thing to do is never build it and iterate over the lines of the file instead.
Jul 23, 2018 at 13:40
5 Answers 5
Explicit is better than implicit, so I prefer:
with open("file.txt", "r") as f: data = f.readlines() But, when it is possible, the most pythonic is to use the file iterator directly, without loading all the content to memory, e.g.:
with open("file.txt", "r") as f: for line in f: my_function(line) answered Jul 23, 2018 at 13:27
2,049 4 4 gold badges 20 20 silver badges 30 30 bronze badges
I don’t think I can use the iterator in this case. I need to read the first line from the file, use it and do some stuff with it. Than delete the first line from the file so that the second line become the first and so on. I was thinking using one of those than using data = data[1:] and writing back inside the file.
Jul 23, 2018 at 13:37
@Bermuda: firstline = next(f) . Then do stuff with it. Then with open(«file.txt.temp», «r») as f2: f2.write(f.read()) . Then move file.txt.temp over file.txt .
Jul 23, 2018 at 13:50
This works and this is exactly what I needed ! Which is very nice. but I don’t understand how it works. According to the doc, next() retrieve the next item from the iterator. No problem. But how come when f2.write(f.read()) later, the first line has disappeared ? Does f.read() shares the same iterator with next() and therefore starts reading from that point ?
Jul 23, 2018 at 14:03
@StevenRumbalski This is a really good way to accomplish what he wanted. I just think it deviates completely the purpose of his question. He should probably ask another question so you can post your proposed method. Personally, I wouldn’t have known how to handle this. But I don’t see how future users will find this answer considering how he formulated his question and the fact that it is a comment.
Jul 23, 2018 at 14:12
@StevenRumbalski Here is an open thread for your answer stackoverflow.com/q/51481747/7692463. Don’t hesitate giving me feedback if you think I can improve wording of the question.
Jul 23, 2018 at 14:54
TL;DR;
Considering you need a list to manipulate them afterwards, your three proposed solutions are all syntactically valid. There is no better (or more pythonic) solution, especially since they all are recommended by the official Python documentation. So, choose the one you find the most readable and be consistent with it throughout your code. If performance is a deciding factor, see my timeit analysis below.
Here is the timeit (10000 loops, ~20 line in test.txt ),
import timeit def foo(): with open("test.txt", "r") as f: data = list(f) def foo1(): with open("test.txt", "r") as f: data = f.read().splitlines(True) def foo2(): with open("test.txt", "r") as f: data = f.readlines() print(timeit.timeit(stmt=foo, number=10000)) print(timeit.timeit(stmt=foo1, number=10000)) print(timeit.timeit(stmt=foo2, number=10000)) >>>> 1.6370758459997887 >>>> 1.410844805999659 >>>> 1.8176437409965729 I tried it with multiple number of loops and lines, and f.read().splitlines(True) always seems to be performing a bit better than the two others.
Now, syntactically speaking, all of your examples seems to be valid. Refer to this documentation for more informations.
According to it, if your goal is to read lines form a file,
for line in f: . where they states that it is memory efficient, fast, and leads to simple code. Which would be another good alternative in your case if you don’t need to manipulate them in a list.
EDIT
Note that you don’t need to pass your True boolean to splitlines . It has your wanted behavior by default.
My personal recommendation
I don’t want to make this answer too opinion-based, but I think it would be beneficial for you to know, that I don’t think performance should be your deciding factor until it is actually a problem for you. Especially since all syntax are allowed and recommended in the official Python doc I linked.
So, my advice is,:
First, pick the most logical one for your particular case and then choose the one you find the most readable and be consistent with it throughout your code.
answered Jul 23, 2018 at 13:32
9,587 8 8 gold badges 36 36 silver badges 68 68 bronze badges
Thank you if the only differences are stylistic, yes better perf are always nice 🙂
Jul 23, 2018 at 13:40
@Bermuda Indeed, but note that you should also try to use timeit on your specific computer to see what’s seems to be the most efficient. Just out of curiosity, try my code and get back to me on whats seems to be the best on your computer.
Jul 23, 2018 at 13:53
Relevant to your analysis, how many lines did test.txt contain? How big was the file?
Jul 23, 2018 at 16:56
@MichaelMior I edited the question by specifying the number of lines but as stated in the answer, I also tried multiple files size and number of loops. At least from what I was able to test, f.read().splitlines(True) was performing better. You can maybe confirm you have similar behavior.
Jul 23, 2018 at 17:18
@scharette Thanks for sharing. I would be hesitant to draw any conclusions from a test with only 20 lines in a file, but I agree it’s probably true that there’s not a huge difference.
Jul 24, 2018 at 18:05
All three of your options produce the same end result, but nonetheless, one of them is definitely worse than the other two: doing f.read().splitlines(True) .
The reason this is the worst option is that it requires the most memory. f.read() reads the file content into memory as a single (maybe huge) string object, then calling .splitlines(True) on that additionally creates the list of the individual lines, and then only after that does the string object containing the file’s entire content get garbage collected and its memory freed. So, at the moment of peak memory use — just before the memory for the big string is freed — this approach requires enough memory to store the entire content of the file in memory twice — once as a string, and once as an array of strings.
By contrast, doing list(f) or f.readlines() will read a line from disk, add it to the result list, then read the next line, and so on. So the whole file content is never duplicated in memory, and the peak memory use will thus be about half that of the .splitlines(True) approach. These approaches are thus superior to using .read() and .splitlines(True) .
As for list(f) vs f.readlines() , there’s no concrete advantage to either of them over the other; the choice between them is a matter of style and taste.
file.readline
При считывании символ новой строки \n присутствует в конце каждой из строк. Его может не быть лишь в последней строке — это позволяет добиться однозначности: если метод возвращает пустую строку, значит достигнут конец файла; если строка содержит лишь символ \n , значит это просто очередная строка.
with open('my_file.txt') as f:
f.readline() # 'The first line.\n'
f.readline() # '\n'
f.readline() # 'The last line.\n'
f.readline() # ''
Для упрощения можно считывать строки из файла пройдя по его объекту в цикле:
with open('my_file.txt') as f:
for line in f:
print(line)
Такой подход эффективен с точки зрения расходования памяти, быстр, и выглядит хорошо.
- Для считывания файла кусками используйте read().
- Для считывания всех строк разом используйте readlines().