Deco M5 mesh with ethernet backhaul

but it seems impossible to find on docs, product pages, etc, which switches support IEEE1905.1 and which ones doesn't.

The IEEE1905.1 standard says:

Modification to the underlying home networking technologies is not required by the 1905.1 layer, and hence it does not change the behavior or implementation of existing home networking technologies.

So as long as the switch does not interfere with it in an incompatible way (which your switch UI should tell you, and it should also allow to turn that off), my understanding is that all switches should work with it.

Once I connect it to the network, the entire network becomes unstable, sometimes right away

Did you try to contact TP-Link support? It's their hardware and firmware, they should know.

And even if you start debugging this (first think I'd do is take one or multiple laptops, run Wireshark and figure out what's going over the wire), you'll likely won't be able to fix it, as you don't have the firmware source.