Can I trace a RabbitMQ consumer to a remote host?
I often find myself troubleshooting situations where, due to user error (forgetting to shut down some processes), some extra consumers are listening on a known RabbitMQ queue. When I do rabbitmqctl list_consumers | grep <queue-name>
I can see that there are too many consumers (usually twice as many as their should be), but the users involved often don't know where those are running. Is there a way to trace a consumer from the list_consumers output to the remote host where the associated process is running? I've tried all of the list_* subcommands but haven't had any luck.
Thanks for any ideas.
After getting my bearings in the management interface (thanks @jrhodin), I was able to figure out how to do this via rabbitmqctl. The key is to pass the pid and name arguments to list channels
.
First, do list_consumers and get the consumer pid:
$ sudo rabbitmqctl list_consumers | grep <queue-name>
Copy the pid (e.g., <[email protected]>)
Then do list_channels and look for the pid:
$ sudo rabbitmqctl list_channels pid connection name | grep '<your-pid-here>'
The output will show the remote and local host as part of the channel name. If you additionally want to look up the connection, you can take the connection pid (from the above command) and grep through list_connections for it.
Here's a loop that will spit out the consumer and channel names for a given queue:
for i in $(sudo rabbitmqctl list_consumers | grep <queue-name> | cut -f 2);
do echo -n "Consumer: " $i;
echo -n " Channel: ";
sudo rabbitmqctl list_channels pid name | grep $i | cut -f 2;
done
Example output:
Consumer: <[email protected]> Channel: xx.x.xx.159:8247 -> xxx.xx.xx.119:5671 (1)
Consumer: <[email protected]> Channel: xx.x.xx.159:9002 -> xxx.xx.xx.119:5671 (1)
Consumer: <[email protected]> Channel: xx.x.xx.159:7298 -> xxx.xx.xx.119:5671 (1)
Consumer: <[email protected]> Channel: xx.x.xx.159:12113 -> xxx.xx.xx.119:5671 (1)
Consumer: <[email protected]> Channel: xx.x.xx.159:14212 -> xxx.xx.xx.119:5671 (1)
You can trace the connection (in almost all cases), but I don't think it is possible via rabbitmqctl
alone.
EDIT: It is possible, it just requires multiple queries. See answer by @em-bo.
In the management interface you can click on the channel, and then in the details section see which connection it belongs to.