Solution 1:

Here's an example: http://duncan99.wordpress.com/2011/10/08/google-maps-api-infowindows/

marker.addListener('mouseover', function() {
    infowindow.open(map, this);
});

// assuming you also want to hide the infowindow when user mouses-out
marker.addListener('mouseout', function() {
    infowindow.close();
});

Solution 2:

var icon1 = "imageA.png";
var icon2 = "imageB.png";

var marker = new google.maps.Marker({
    position: myLatLng,
    map: map,
    icon: icon1,
    title: "some marker"
});

google.maps.event.addListener(marker, 'mouseover', function() {
    marker.setIcon(icon2);
});
google.maps.event.addListener(marker, 'mouseout', function() {
    marker.setIcon(icon1);
});

Solution 3:

Thanks to duncan answer, I end up with this:

marker.addListener('mouseover', () => infoWindow.open(map, marker))
marker.addListener('mouseout', () => infoWindow.close())