'ls' command very slow

I have about 17k files in a directory. When I run ls directory, I have to wait for about 15-20 seconds before the results are displayed. On the other hand, when I run ls directory | wc -l or ls directory | grep .xyz, the results are displayed immediately.

Why does this happen and is there a way to fix this?

Solution 1:

I'm going to guess that you're using Linux.

1. If your ls command is aliased such that it shows files & folders in colour, then it needs to find out each item's permissions (a stat() call) and whether it has any "file capabilities" set (a getxattr() call) in order to choose the right colour. Depending on file system, these calls can be fairly slow if the required metadata hasn't been cached in RAM yet. [Extended attributes often live in the data area, so each getxattr results in HDD seeks.]

   On the other hand, ls | when redirected to a pipe automatically disables colouring, so it no longer needs to do any extra checks – just a straightforward readdir() loop which returns the file name and type, and the kernel likely even implements read-ahead for that.

2. Normally ls columnates its output, which means it has to read the entire directory before it can output anything at all. When run through a pipe, it automatically disables the columns mode and this buffering is no longer needed. (The total run time isn't necessarily faster, but the output begins earlier, making it feel more responsive.)

Use strace or perf trace to check which system calls, if any, are taking a long time.