Solution 1:

"rtnetlink answers device or resource busy"

I think it means exactly what it says. The device wlan0 is busy sending and recieving data to your router or access point, getting email, etc. I suggest that you tell it to stop:

sudo ifconfig wlan0 down

And try again:

sudo iwconfig wlan0 mode ad-hoc essid my-mesh-network ap 02:12:34:56:78:9A channel 1

If you have Network Manager running, you will probably need to stop it first:

sudo service network-manager stop