Why is /Application/Utilies "empty" in macOS Catalina from Terminal but not Finder? [closed]

Intent: I'm just curious as to what's going on under the hood that explains the behavior I'm seeing. (Detailed technical answers upvoted.)

I noticed that if I use Finder and open /Applications, I see a folder called Utilities (as we have for all pre-Catalina operating systems); opening that shows the usual cadre of utility applications.

However, if I open Terminal and try to list those same files, they do not appear:

$ ls -la@  /Applications/Utilities

My question: What's really going on, and how is this possible?

macOS Catalina v10.15 (build 19A602)

My own experiments before asking publicly:

My first thought was to double check the path -- and right clicking the open Utilities window's icon in Finder showed the machine's name, the hard disk's name, then Applications, and Utilities.

Typing $ ls -la@ into the command line and dragging the window's icon to Terminal injected the very same path; running the command (obviously) gave the same results.

So then I started looking to see if there was any new super-extended attributes or something? xattr showed nothing sneaky tucked away.

Now suspecting this something new with APFS, I started researching and found three surprises:

• There's now a thing called a "firm" link, that is neither hard nor soft, that end users can't create, and it seems to be some magic about user-level mount points. Currently, I haven't found much detail on what they really are, what problem they solve, and whether or not I should actually care. Although the engineer in me would love to know what's going on.
• Doing a man synthetic.conf brings up a very interesting man page, and make me want to go look at /etc/synthetic.conf, which doesn't exist.
• The discovery of a /Systems/Applications directory. That looks new.

Curiosity caused me to try the following, which got me the filename results I expected:

$ ls /System/Applications/Utilities

So, I'm left wondering:

• What was the driving need to move things under /System?
• What is this witchcraft, if not hard links, making /Application exist?
• Why is Finder showing content (apparently) from one place that's really under another?
• Is my ~/Application directory subject to such wizardy? (Is there something else I ought to be wary of?)

I'm not used to the case where a graphical shell shows files in a directory that the raw command-line talking to the file system doesn't ...the reverse, sure.

Why would I be doing this?

I was trying to automate making a list of applications I have installed, just as I used to do for prior versions of OS X (here's the old way):

$ ls ~/Applications /Applications /Applications/Utilities | fgrep .app | sed -e 's/.app//' | sort -if

And the new (ignoring special cases of certain subdirectories):

$ ls ~/Applications  /System/Applications /System/Applications/Utilities | fgrep .app | sed -e 's/.app//' | sort -if

Solution 1:

I don't know that anyone who knows the technical details of them can share them, because they are not known outside of Apple. But here's my attempt at some replies.

What was the driving need to move things under /System?

Presumably it was part of Apple's desire to put some of the core system in the "read only" partition which is also new in Catalina.

What is this witchcraft, if not hard links, making /Application exist?

I believe what you are seeing are "firm links" but you are correct that Apple has not released much information about firm links.

Why is Finder showing content (apparently) from one place that's really under another?

Presumably because Apple knows people are used to finding them there, and thought it would be better to make them appear where people are used to finding them rather than putting them somewhere else.

Is my ~/Application directory subject to such wizardy? (Is there something else I ought to be wary of?)

No. The only reason Apple is doing this is because of the read-only partition, which doesn't apply to your ~/Applications/ folder.