Expansion with *.txt in the shell doesn't work if no .txt file exists

Solution 1:

Adapting from the bash shell man page,

bash scans each word for the characters *, ?, and [. If one of these characters appears, then the word is regarded as a pattern, and replaced with an alphabetically sorted list of filenames matching the pattern. If no matching filenames are found, and the shell option nullglob is not enabled, the word is left unchanged. If the nullglob option is set, and no matches are found, the word is removed.

In this case I presume nullglob is not enabled, so the word is left unchanged - hence the output you see.

Solution 2:

I was expecting to not get any output.

If nullglob were the default, many commands would behave quite unexpectedly, because it is (perhaps unfortunately) common that commands treat the case of zero filename arguments in a qualitatively different way than the case of one or more filename arguments.

Suppose you've enabled nullglob (shopt -s nullglob) and you're in a directory where no files match *.txt. Then *.txt will indeed expand to nothing--not an empty field, but no fields at all--as you expected. But that would have these results:

• ls *.txt would list all the files in the current directory (except hidden files), because that's what ls does when you don't pass it any filename arguments.
• cat *.txt would read from standard input, because when cat has no filename arguments, it's as if you ran cat -. If running interactively, it sits around waiting for input. Many commands behave this way.
• cp *.txt dest/ would fail with the error cp: missing destination file operand after 'dest/'. This isn't a disaster, but it's confusing and quite different from the silent success that is probably desired.
• file *.txt, and various other programs with no special behavior for the case of zero filename arguments, would still fail with an error or usage message when none are passed.
• Even cases that intuitively feel like they should work often wouldn't. printf 'Got file: "%s"\n' *.txt would print Got file: "" instead of nothing.
• Accidental failure to quote occurrences of *, ?, and [ that aren't intended to be expanded by the shell would more often produce obviously wrong results, but in ways that might be difficult to figure out. For example, if no filenames in the current directory started with gedit, then apt list gedit* (where apt list 'gedit*' was intended) would become just apt list and list all available packages.

So it's good that you don't get this behavior without requesting it. Probably the most common practical situation that is actually simplified by nullglob is for f in *.txt. See also this question (which Sergiy Kolodyazhnyy's answer linked to).

The harder question to answer is why failglob--where it's an expansion error to have a glob that doesn't match any files--isn't the default in bash. I believe Sergiy Kolodyazhnyy's answer captures the reason for this even without addressing it directly. Retaining unexpanding globs without producing an expansion error is (perhaps unfortunately) the standardized behavior, and it is also traditional, and thus expected, behavior. Although bash doesn't attempt to be fully POSIX-compliant unless it is invoked with the name sh or passed the --posix option, many of its design choices even when not in POSIX mode follow POSIX directly. They had to pick some behavior, and there are downsides associated with going against users' expectations.

I think this is the least historically influential aspect of the matter so I've saved it for last... but it is worth mentioning that there's something a bit conceptually odd about nullglob behavior.

nullglob seems elegant at first because, syntactically, it treats the case of zero matching files no differently from the case of one, two, or any other number. The commands we run, which globs expand into arguments for, don't tend to treat them the same, as detailed above. But syntactically this at least feels right, which I think is the motivation for your question.

And yet, there is another, more subtle inconsistency that nullglob doesn't address--that it actually amplifies. The case of zero globbing characters ("wildcards") is treated profoundly differently from that of one, or two, or any other number. For example, with shopt -s nullglob, if ab?d?f doesn't match any files, it is removed; if ab?d doesn't match any files, it is removed; but if ab doesn't match any files (i.e., if there is no file whose name is exactly ab) it is still not removed. Of course, it would be a disaster if it were removed, because it might not be intended to refer to an existing file in the current directory at all; it might not even refer to a file. But this still eliminates any hope for total consistency.

The three behaviors bash provides--the default of treating globs that don't match any files as though they weren't globs and passing them unexpanded, the behavior you expected of treating them (if you'll pardon this odd turn of phrase) as signifying all zero of the files that do match (nullglob), and the safe behavior of considering them errors (failglob)--all represent different approaches to the ambiguity inherent in the shell not being able to know if any particular word is intended to be a filename. The shell performs its expansions without knowledge of how the particular commands you call with it will treat their arguments.

This is one of many instances of separation of concerns. In systems whose design follows the Unix philosophy, each part is intended to do one thing and do it well. The shell processes text into commands and arguments and invokes those commands, most of which are external to the shell itself. This tends to be a lot nicer and more versatile than systems where the external commands are themselves responsible for performing those transformations (as with the traditional command processors in DOS and Windows). But it has its occasional downsides.

Solution 3:

The main reason is because this is standard behavior specified by POSIX - the standard which covers shell command language and among other things pattern matching ( shells such as bash, dash shell - Ubuntu's default /bin/sh, and ksh follow this standard ). From section 2.13.3 Patterns Used for Filename Expansion:

If the pattern does not match any existing filenames or pathnames, the pattern string shall be left unchanged.

This of course has a side effect - matching filename that may be literally *.txt. The nullglob option in bash and zsh can help: if that option is enabled via shopt -s nullglob (and it is not enabled by default which applies to this question), then globstar will be expanded to empty string when no matching filenames found. ksh93 has its own advanced pattern matching mechanism which achieves the same effect ~(N)*.txt

See also Why is nullglob not default?