Is there a special specific term for someone who travels to other multiverses willingly?
You might be looking for the word "Sliders". To be fair it was only used in (and the title of) a TV Show back in 1995 but it fit your description perfectly.
On the Sliders' Wikipedia Page the protagonists are defined as:
A group of travelers who use wormholes to "slide" between different parallel universes