Is "I like dogs but cats" a valid sentence?
Not in the sense you mean.
But, is used here as a preposition meaning "except".
What Visual Studio is saying:
Close all tabs except this.
What your friend is saying:
I like dogs except cats.
That is wrong. Except only makes sense when you're talking about a subgroup of a group, not two different groups.
You could say, for instance:
I like all animals but skunks.
There are two different meanings and syntaxes (among others) for 'but'.
In
I like dogs, but ...
'but' is acting like a conjunction. What is expected in the ellipses is a full sentence. Using a simple noun there is really wrong and doesn't make sense. One expects something like "I like dogs, but cats really bother me".
In
Close all but this
'but' is acting like a preposition (heading a prepositional phrase) meaning 'except for'. Here a noun is expected. It is not as common as 'but' used as a conjunction, but is still used often enough (and is much shorter than 'except for', and so is useful in computer user interfaces).
So "I like dogs but cats" is not valid because cats is not a full sentence.