How does zmq poller work?

I am confused as to what poller actually does in zmq. The zguide goes into it minimally, and only describes it as a way to read from multiple sockets. This is not a satisfying answer for me because it does not explain how to have timeout sockets. I know zeromq: how to prevent infinite wait? explains for push/pull, but not req/rep patterns, which is what I want to know how to use.

What I am attempting to ask is: How does poller work, and how does its function apply to keeping track of sockets and their requests?

Solution 1:

When you need to listen on different sockets in the same thread, use a poller:

ZMQ.Socket subscriber = ctx.socket(ZMQ.SUB)
ZMQ.Socket puller = ctx.socket(ZMQ.PULL)

Register sockets with poller (POLLIN listens for incoming messages)

ZMQ.Poller poller = ZMQ.Poller(2)
poller.register(subscriber, ZMQ.Poller.POLLIN)
poller.register(puller, ZMQ.Poller.POLLIN)

When polling, use a loop:

while( notInterrupted()){
  poller.poll()

  //subscriber registered at index '0'
  if( poller.pollin(0)) 
     subscriber.recv(ZMQ.DONTWAIT)

  //puller registered at index '1'
  if( poller.pollin(1))
     puller.recv( ZMQ.DONTWAIT)
}

Choose how you want to poll...

poller.poll() blocks until there's data on either socket.
poller.poll(1000) blocks for 1s, then times out.

The poller notifies when there's data (messages) available on the sockets; it's your job to read it.

When reading, do it without blocking: socket.recv( ZMQ.DONTWAIT). Even though poller.pollin(0) checks if there's data to be read, you want to avoid any blocking calls inside the polling loop, otherwise, you could end up blocking the poller due to 'stuck' socket.

So, if two separate messages are sent to subscriber, you have to invoke subscriber.recv() twice in order to clear the poller, otherwise, if you call subscriber.recv() once, the poller will keep telling you there's another message to be read. So, in essence, the poller tracks the availability and number of messages, not the actual messages.

You should run through the polling examples and play with the code, it's the best way to learn.

Does that answer your question?

Solution 2:

In this answer i listed

Details from the documentation http://api.zeromq.org/4-1:zmq-poll

Also i added some important explanation and things that clear confusion for new commers! If you are in a hurry! You may like to start by What the poller do and What about Recieving and A note about recieving and what about one socket only sections at the end! Starting from important notes section! Where i clear things in depth! I still suggest reading well the details in the doc ref! And first section!

Doc ref and notes

Listen to multiple sockets and events

The zmq_poll() function provides a mechanism for applications to multiplex input/output events in a level-triggered fashion over a set of sockets. Each member of the array pointed to by the items argument is a zmq_pollitem_t structure. The nitems argument specifies the number of items in the items array. The zmq_pollitem_t structure is defined as follows:

typedef struct
{
    void //*socket//;
    int //fd//;
    short //events//;
    short //revents//;
} zmq_pollitem_t;

zmq Socket or standard socket through fd

For each zmq_pollitem_t item, zmq_poll() shall examine either the ØMQ socket referenced by socket or the standard socket specified by the file descriptor fd, for the event(s) specified in events. If both socket and fd are set in a single zmq_pollitem_t, the ØMQ socket referenced by socket shall take precedence and the value of fd shall be ignored.

Big note (same context):

All ØMQ sockets passed to the zmq_poll() function must share the same ØMQ context and must belong to the thread calling zmq_poll().

Revents member

For each zmq_pollitem_t item, zmq_poll() shall first clear the revents member, and then indicate any requested events that have occurred by setting the bit corresponding to the event condition in the revents member.

Upon successful completion, the zmq_poll() function shall return the number of zmq_pollitem_t structures with events signaled in revents or 0 if no events have been signaled.

Awaiting for events and blocking

If none of the requested events have occurred on any zmq_pollitem_t item, zmq_poll() shall wait timeout microseconds for an event to occur on any of the requested items. If the value of timeout is 0, zmq_poll() shall return immediately. If the value of timeout is -1, zmq_poll() shall block indefinitely until a requested event has occurred on at least one zmq_pollitem_t. The resolution of timeout is 1 millisecond.

• 0 => doesn't wait

• -1 => block

• +val => block and wait for the timeout amount

Events

The events and revents members of zmq_pollitem_t are bit masks constructed by OR'ing a combination of the following event flags:

ZMQ_POLLIN

For ØMQ sockets, at least one message may be received from the socket without blocking. For standard sockets this is equivalent to the POLLIN flag of the poll() system call and generally means that at least one byte of data may be read from fd without blocking.

ZMQ_POLLOUT

For ØMQ sockets, at least one message may be sent to the socket without blocking. For standard sockets this is equivalent to the POLLOUT flag of the poll() system call and generally means that at least one byte of data may be written to fd without blocking.

ZMQ_POLLERR

For standard sockets, this flag is passed through zmq_poll() to the underlying poll() system call and generally means that some sort of error condition is present on the socket specified by fd. For ØMQ sockets this flag has no effect if set in events, and shall never be returned in revents by zmq_poll().

Note:

The zmq_poll() function may be implemented or emulated using operating system interfaces other than poll(), and as such may be subject to the limits of those interfaces in ways not defined in this documentation.

Return value

Upon successful completion, the zmq_poll() function shall return the number of zmq_pollitem_t structures with events signaled in revents or 0 if no events have been signaled. Upon failure, zmq_poll() shall return -1 and set errno to one of the values defined below.

Example

Polling indefinitely for input events on both a 0mq socket and a standard socket.

zmq_pollitem_t items [2];
/* First item refers to ØMQ socket 'socket' */
items[0].socket = socket;
items[0].events = ZMQ_POLLIN;
/* Second item refers to standard socket 'fd' */
items[1].socket = NULL;
items[1].fd = fd;
items[1].events = ZMQ_POLLIN;
/* Poll for events indefinitely */
int rc = zmq_poll (items, 2, -1);
assert (rc >= 0); /* Returned events will be stored in items[].revents */

Important notes

What the poller do and What about Recieving

The poller only check and await for when events occure! POLLIN is for receiving! Data is there for recieving! Then we should read through recv()! We are responsible to read or do anything! The poller is just there to listen to the events and await for them! And through zmq_pollitem_t we can listen to multiple events! If any event happen! Then the poller unblock! We can check then the event in the recv! and zmq_pollitem_t! Note that the poller queue the events as they trigger! And next call will pick from the queue! The order because of that is also kept! And successive calls will return the next event and so on! As they came in!

A note about recieving and what about one socket only

For a Router! A one router can receive multiple requests even from a one client! And also from multiple clients at once! In a setup where multiple clients are of same nature! And are the ones connecting to the router! A question that can cross the mind of a new commer is! Do i need a poller for this async nature! The anwser is no! No need for a poller and listening for different sockets!

The big note is: Receiving calls (zmq_recv(), socket.recv() some lang binding)! Block! And are the way to read! When messages come! They are queued! The poller have nothing to do with this! The poller only listen to events from different sockets! And unblock if any of them happen! if the timeout is reached then no event happen! And does no more then that!

The nature of receiving is straight forward! The recieve call blocks! Till a message in the message queue comes! When multiple come they will get queued! Then on each next call for recv()! We will pull the next message! Or frame! (Depending on what recieving method we are using! And the api level! And abstraction from the binding library to the low level one!) Because we can access too the messages by frame! a frame at each call! But then here it becomes clear! Recieve calls are the things to recieve! They block till a message enter the queue! Multiple parallel messages! Will enter the queue as they come! Then for each call! Either the queue is filled or not! consume it! or wait! That's a very important thing to know! And which can confuse new commers!

The poller is only needed when there is multiple sockets! And they are always sockets that we declare on the process code in question (bind them, or to connect to something)! Because if not! How will you recieve the messages! You can't do it well! because you have to prioritize one or another! in a loop having one recv() go first ! Which will block! Which even if the other socket get a message in it's queue! The loop is blocked and can't proceed to the next recv()! hince the poller give us the beauty to be able to tackle this! And work well with multiple sockets!

while(true) {
    socket1.recv() // this will block

    socket2.recv() // this will have to wait till the first recieve! Even if messages come in in it's queue

With the poller:

While(true) {
    zmq_poll() // block till one of the socket events happen! If the event was POLLIN!
            // If any socket get a message to it's queue
            // This will unblock
    // then we check which type and which socket was
    if (condition socket 1) {
        // treat socket 1 request
    }

    if (condition socket 2) {
        // treat socket 2 request
    }
 
    // ...
}

You can see real code from the doc at this section (scroll enough to see the code blocks, you can see in all different langs too)

Know that the poller just notify that there is messages in! If it's POLLIN!

In each iteration! The poller if many events triggered already! Let's give the example of 10 messages recieved 5 in each socket! Here the poller already have the events queued! And in each next call for the 9 times! Will resolve immediately! The message in question can be mapped to what socket (by using the poller object and mean! So binding libraries make it too simple and agreable)! Then the right socket block will make the recieve call! And when it does it will consume it's next message from it's queue!

And so you keep looping and each time consuming the next message! As they came in! And the poller tracked there order of comming in! And that through the subscription and the events that were choosen to listen to! In case of receiving, it should be POLLIN!

Then each socket have it's message queue! And each recieve call! Pull from it! The poller tracked them! And so when the poller resolve! It's assured that there is message for the sockets recieve calls!

Last example: The server client pattern

Let's take the example of one server (router) and many clients (Dealers) that connect to! As by the image bellow!

Question: many connections to the same router! Comming asynchronously at once! And bla bla bla! In the server side (Router)! Do i need a poller !? A lot of new commers, may think yes or question if it's needed! Yup you guess right!

BIG NO!

Why ? Because in the Server (router) code! We have only one socket we are dealing with! That we bind! The clients then are connecting to it! In that end! There is only one socket! And all recv() calls are on that one socket! And that socket have it's message queue! The recv() consume the message one after another! Doesn't matter asynchronous and how they comes! Again the poller is only when there is multiple sockets! And so having the mixing nature of treating messages comming from multiple sockets! If not! Then one recv() of one socket need to go first then the other! And will block the other! Not a good thing (it's bad)!

Note

This answer bring a nice clearing! Plus it mades a reference to the doc with good highlighting! Also show code by the low level lib (c lang)! The answer of @rafflan show a great code with a binding lib (seems c#)! And a great explanation! If you didn't check it you must!