What is this dog sniffing out here?

Sometimes my dogs go crazy and report lots of tunnels with a big icon and leave some yellow flags behind. What exactly does he find, because no tunnel shows up and it is also mostly on the inside of the prison walls, which makes no sense.

So what exactly found he there?

Dogs are not 100% reliable. They will sometimes miss tunnels and they will sometimes report tunnels even though there aren't any. When a dog reports a tunnel in a location which is very implausible (any cells would have a far shorter route to freedom), you can just ignore them.

However, in this case there might be a tunnel. Groups of tunneling prisoners first build tunnels to connect their cells to each other. This allows them to build tunnels heading for the prison perimeter far quicker. In this case the tunnel-marker is right between the toilets of the two adjacent cells, so these two cells cooperating that way is not implausible. Try deconstructing the toilets in them and you might be in for a surprise (searching the cell does not give you a 100% chance to discover the tunnel - deconstructing the toilet ist the only way to be 100% sure).

If memory serves me right you need to find either the tunnel entrance or exit to uncover the whole tunnel.

Try searching the nearby cells or performing a shakedown on the whole cell block to see if you can find the entrance before they get outside the wall.

Be aware that this is not guaranteed to uncover all the tunnels according to the Prison Architect Wiki:

As of Alpha 34, manually searching a toilet will reveal any escape tunnels stemming from that toilet. This does not include searching a cell or cell block, but rather the suspected toilet itself. This does not get rid of the random chance that tunnels will be found on a blanket search, but now if you want the check a specific toilet, you don't have to dismantle it.

Just be careful with your prisoners as they seem rather unhappy already and the search is not going to help with that.