clang doesn't find c++ stdlib on my ubuntu 18.04, but it does on a brand new install. What have I broken?

I've been building a github package for a while ... a new version comes out, and I build it. It uses clang6 which is default on my ubuntu, ubuntu 18.04

now, it fails to compile, it can't find a standard include. On a stock, brand new ubuntu 18.04 it works. So I have done something which has broken. I am using a custom kernel, based on ubutntu hwe (so 5.3). I am applying a small patch to restore intel i915 powersaving.

I can simplify the problem:

$ more test.cc
#include <vector>

It works when I do this:

clang -c test.cc -v -stdlib=libc++

Also, on a stock 18.04.4 it works after I apt install clang.

But on my laptop, it fails:

clang -c test.cc -v 
clang version 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/i686-linux-gnu/8
Found candidate GCC installation: /usr/bin/../lib/gcc/i686-linux-gnu/9
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/6
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/6.5.0
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/7
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/7.4.0
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/8
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/9
Found candidate GCC installation: /usr/lib/gcc/i686-linux-gnu/8
Found candidate GCC installation: /usr/lib/gcc/i686-linux-gnu/9
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/6
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/6.5.0
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/7
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/7.4.0
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/8
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9
Selected GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/9
Candidate multilib: .;@m64
Selected multilib: .;@m64
 "/usr/lib/llvm-6.0/bin/clang" -cc1 -triple x86_64-pc-linux-gnu -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name test.cc -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -dwarf-column-info -debugger-tuning=gdb -v -coverage-notes-file /home/tim/test.gcno -resource-dir /usr/lib/llvm-6.0/lib/clang/6.0.0 -internal-isystem /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++ -internal-isystem /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/x86_64-linux-gnu -internal-isystem /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/backward -internal-isystem /usr/include/clang/6.0.0/include/ -internal-isystem /usr/local/include -internal-isystem /usr/lib/llvm-6.0/lib/clang/6.0.0/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdeprecated-macro -fdebug-compilation-dir /home/tim -ferror-limit 19 -fmessage-length 232 -fobjc-runtime=gcc -fcxx-exceptions -fexceptions -fdiagnostics-show-option -fcolor-diagnostics -o test.o -x c++ test.cc
clang -cc1 version 6.0.0 based upon LLVM 6.0.0 default target x86_64-pc-linux-gnu
ignoring nonexistent directory "/usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/x86_64-linux-gnu"
ignoring nonexistent directory "/usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/backward"
ignoring nonexistent directory "/include"
ignoring duplicate directory "/usr/include/clang/6.0.0/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++
 /usr/include/clang/6.0.0/include
 /usr/local/include
 /usr/include/x86_64-linux-gnu
 /usr/include
End of search list.
test.cc:1:10: fatal error: 'vector' file not found
#include <vector>
         ^~~~~~~~
1 error generated.

Update: I found this post: https://stackoverflow.com/a/54926827/401226 which says to look at the highest version number here: /usr/lib/ On my laptop, it is version 9. Indeed, I have version 9 gcc installed, but I don't have v9 stdlib (libstdc++-9-dev).

I did not have g++-9 installed.


Here is what I learnt:

clang by default looks for the gcc standard C++ library and it stops if this doesn't work.

It looks for the highest version. The highest version is determined by the highest version of gcc installed. So if you don't have a g++ installed which matches your highest gcc version, clang gives up when it can't find libstdc++ the first time .

So I had gcc- but I did not have g++-9.

apt install g++-9

fixed the problem as it installs the matching libstdc++-9.dev