Connect two routers with different subnet

You say these routers support port-based VLANs, so I’d do it like this:

• Set up a port-based VLAN (untagged) on both routers with the same subnet but different IPs (for example 192.168.3.1/24 on Router A and 192.168.3.2/24 on Router B), DHCP etc not required
• Connect the VLAN’s physical ports on both routers using an Ethernet cable
• Set up static routes
  • 192.168.1.0/24 via 192.168.3.1 on Router B
  • 192.168.2.0/24 via 192.168.3.2 on Router A
• If needed, configure the firewalls to allow traffic between the “normal” network and the VLAN network

Devices should then be able to talk to each other using ping and whatever. As mentioned in the comment, discovery features like Chromecast, Windows Network Neighborhood, … will not work and in most cases cannot be made to work.