Python Socket Flush

I am trying to make sure that every time I call the socket.send function my buffer is sent (flushed) to my server (which is in C using unix socket).

From my understanding (and from what I see on this board) just disabling the naggle algo. should do it but my server still receive my data in chunk of 4096 bytes (default set)...

Im using the following code in Python v2.5.4:

 self.sck = socket( AF_INET, SOCK_STREAM )

 self.sck.setsockopt( IPPROTO_TCP, TCP_NODELAY, 1 ) # That doesn't seems to work...

 self.sck.connect( ( "127.0.0.1", "12345" ) )

 while( 1 ):
      self.sck.send( "test\n" )
      self.sck.send( "" ) # Still trying to flush...

Enabling/Disabling TCP_NODELAY seems that have no effect whatsoever... Is this a bug or I am missing something?

TIA

Solution 1:

TCP does not provide any kind of guaranteed "packet" sending to the other end. You are sending data as fast as you can, and TCP is helpfully batching up the data into as much as it can send at once. Your server is receiving data 4096 bytes at a time, probably because that's what it asked for (in a recv() call).

TCP is a stream protocol and therefore you will have to implement some kind of framing yourself. There are no built-in message boundaries.

Solution 2:

The only way I got a flush to work (back to a C client) was to use:

my_writer_obj = mysock.makefile(mode='w', ...)
my_writer_obj.write('my stuff')
my_writer_obj.flush()

The big advantage is that my_writer_obj defaults to text mode, so no more byte translation.

creating a reader object did not go as smoothly, but I did not need a flush on that side.

Solution 3:

There is no way to ensure the size of the data chunks that are sent. If you want to make sure that all the data that you want to send is send, you can close the connection:

self.sck.close()

Note also, that n = socket.send() returns the number of actual sent bytes. If you definitely want to send all data, you should use

self.sck.sendall()

or loop over the data sending:

while data:
    n = self.sck.send(data)    
    data = data[n:]

(But that is roughly the same as sendall() ). If you want to receive the data in bigger chunks, you can increase the size of the buffer in recv(), but this only makes the possible chunk size bigger. There is no guaranty that the data arrives in these sizes.