I got the idea for this article when I saw in one of the blogs that this problem is proposed to be solved with the following one-line code in Perl:
echo -e '1\n2\n3' | perl -nalE 'push(@lines, $_); }{ say $lines[$_] foreach 0..$#lines -1'What about the head utility, you ask me? Yes, this is the best option. But as it turned out, in some systems, you can't use this utility with a negative value for the n parameter (if you don't know how many lines are in the file). What was said in the original article before offering this solution.
But I think there are two problems with this code snippet:
echo -e '1\n2\n3' | perl -nale 'push(@l, $_); print shift(@l) if @l> 1;'The code is quite simple, but let's look at it in more detail. First, we do not store the entire file in memory, but only the number of lines that we do not want to output - a kind of buffer. Secondly, we immediately decide whether to output a string or not. The number of lines that we don't output from the end is set as a number. In this example, this is a single line.
I tested the memory consumption measurements on a 72m file
ls -lah big.file -rw-r--r-- 1 madskill madskill 72M ноя 1 03:56 big.fileFor the first option, got the following values
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 23400 madskill 20 0 205844 188952 3960 S 0,0 4,8 0:07.44 perlAnd for my option the following
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 23413 madskill 20 0 21056 3492 3248 S 0,0 0,1 0:10.71 perlFrom the above measurements, you can see that my version uses memory more economically. But I didn't stop there and then roughly compared both variations in performance. It turned out that there is almost no difference. With repeated launches, one or the other option was faster. And the difference between them was very small.
time cat big.file | perl -nalE 'push(@lines, $_); }{ say $lines[$_] foreach 0..$#lines -1;'> /dev/null real 0m1,789s user 0m1,662s sys 0m0,188s time cat big.file | perl -nale 'push(@l, $_); print shift(@l) if @l> 1;'> /dev/null real 0m1,732s user 0m1,688s sys 0m0,116s