Java TCP socket: data transfer is slow

I set up a server with a ServerSocket, connect to it with a client machine. They're directly networked through a switch and the ping time is <1ms.

Now, I try to push a "lot" of data from the client to the server through the socket's output stream. It takes 23 minutes to transfer 0.6Gb. I can push a much larger file in seconds via scp.

Any idea what I might be doing wrong? I'm basically just looping and calling writeInt on the socket. The speed issue doesn't matter where the data is coming from, even if I'm just sending a constant integer and not reading from disk.

I tried setting the send and receive buffer on both sides to 4Mb, no dice. I use a buffered stream for the reader and writer, no dice.

Am I missing something?

EDIT: code

Here's where I make the socket

System.out.println("Connecting to " + hostname);

    serverAddr = InetAddress.getByName(hostname);

    // connect and wait for port assignment
    Socket initialSock = new Socket();
    initialSock.connect(new InetSocketAddress(serverAddr, LDAMaster.LDA_MASTER_PORT));
    int newPort = LDAHelper.readConnectionForwardPacket(new DataInputStream(initialSock.getInputStream()));
    initialSock.close();
    initialSock = null;

    System.out.println("Forwarded to " + newPort);

    // got my new port, connect to it
    sock = new Socket();
    sock.setReceiveBufferSize(RECEIVE_BUFFER_SIZE);
    sock.setSendBufferSize(SEND_BUFFER_SIZE);
    sock.connect(new InetSocketAddress(serverAddr, newPort));

    System.out.println("Connected to " + hostname + ":" + newPort + " with buffers snd=" + sock.getSendBufferSize() + " rcv=" + sock.getReceiveBufferSize());

    // get the MD5s
    try {
        byte[] dataMd5 = LDAHelper.md5File(dataFile),
               indexMd5 = LDAHelper.md5File(indexFile);

        long freeSpace = 90210; // ** TODO: actually set this **

        output = new DataOutputStream(new BufferedOutputStream(sock.getOutputStream()));
        input  = new DataInputStream(new BufferedInputStream(sock.getInputStream()));

Here's where I do the server-side connection:

    ServerSocket servSock = new ServerSocket();
    servSock.setSoTimeout(SO_TIMEOUT);
    servSock.setReuseAddress(true);
    servSock.bind(new InetSocketAddress(LDA_MASTER_PORT));

    int currPort = LDA_START_PORT;

    while (true) {
        try {
            Socket conn = servSock.accept();
            System.out.println("Got a connection.  Sending them to port " + currPort);
            clients.add(new MasterClientCommunicator(this, currPort));
            clients.get(clients.size()-1).start();

            Thread.sleep(500);

            LDAHelper.sendConnectionForwardPacket(new DataOutputStream(conn.getOutputStream()), currPort);

            currPort++;
        } catch (SocketTimeoutException e) {
            System.out.println("Done listening.  Dispatching instructions.");
            break;
        }
        catch (IOException e) {
            e.printStackTrace();
        }
        catch (Exception e) {
            e.printStackTrace();
        }
    }

Alright, here's where I'm shipping over ~0.6Gb of data.

public static void sendTermDeltaPacket(DataOutputStream out, TIntIntHashMap[] termDelta) throws IOException {
    long bytesTransferred = 0, numZeros = 0;

    long start = System.currentTimeMillis();

    out.write(PACKET_TERM_DELTA); // header     
    out.flush();
    for (int z=0; z < termDelta.length; z++) {
        out.writeInt(termDelta[z].size()); // # of elements for each term
        bytesTransferred += 4;
    }

    for (int z=0; z < termDelta.length; z++) {
        for (int i=0; i < termDelta[z].size(); i++) {
            out.writeInt(1);
            out.writeInt(1);
        }
    }

It seems pretty straightforward so far...

You do not want to write single bytes when you are transferring large amounts of data.

import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.ServerSocket;
import java.net.Socket;

public class Transfer {

    public static void main(String[] args) {
        final String largeFile = "/home/dr/test.dat"; // REPLACE
        final int BUFFER_SIZE = 65536;
        new Thread(new Runnable() {
            public void run() {
                try {
                    ServerSocket serverSocket = new ServerSocket(12345);
                    Socket clientSocket = serverSocket.accept();
                    long startTime = System.currentTimeMillis();
                    byte[] buffer = new byte[BUFFER_SIZE];
                    int read;
                    int totalRead = 0;
                    InputStream clientInputStream = clientSocket.getInputStream();
                    while ((read = clientInputStream.read(buffer)) != -1) {
                        totalRead += read;
                    }
                    long endTime = System.currentTimeMillis();
                    System.out.println(totalRead + " bytes read in " + (endTime - startTime) + " ms.");
                } catch (IOException e) {
                }
            }
        }).start();
        new Thread(new Runnable() {
            public void run() {
                try {
                    Thread.sleep(1000);
                    Socket socket = new Socket("localhost", 12345);
                    FileInputStream fileInputStream = new FileInputStream(largeFile);
                    OutputStream socketOutputStream = socket.getOutputStream();
                    long startTime = System.currentTimeMillis();
                    byte[] buffer = new byte[BUFFER_SIZE];
                    int read;
                    int readTotal = 0;
                    while ((read = fileInputStream.read(buffer)) != -1) {
                        socketOutputStream.write(buffer, 0, read);
                        readTotal += read;
                    }
                    socketOutputStream.close();
                    fileInputStream.close();
                    socket.close();
                    long endTime = System.currentTimeMillis();
                    System.out.println(readTotal + " bytes written in " + (endTime - startTime) + " ms.");
                } catch (Exception e) {
                }
            }
        }).start();
    }
}

This copies 1 GiB of data in short over 19 seconds on my machine. The key here is using the InputStream.read and OutputStream.write methods that accept a byte array as parameter. The size of the buffer is not really important, it just should be a bit larger than, say, 5. Experiment with BUFFER_SIZE above to see how it effects the speed but also keep in mind that it probably is different for every machine you are running this program on. 64 KiB seem to be a good compromise.

Hey, I figured I'd follow up for anyone that was interested.

Here's the bizarre moral of the story:

NEVER USE DataInputStream/DataOutputStream and sockets!!

If I wrap the socket in a BufferedOutputStream/BufferedInputStream, life is great. Writing to it raw is just fine.

But wrap the socket in a DataInputStream/DataOutputStream, or even have DataOutputStream(BufferedOutputStream(sock.getOutputStream())) is EXTREMELY SLOW.

An explanation for that would be really interesting to me. But after swapping everything in and out, this is what's up. Try it yourself if you don't believe me.

Thanks for all the quick help, though.

Maybe you should try sending ur data in chunks(frames) instead of writing each byte seperately. And align your frames with the TCP packet size for best performance.

Can you try doing this over loopback, it should then transfer the data in second.

If it takes minutes, there is something wrong with your application. If is only slow sending data over the internet it could be you network link which is slow.

My guess is that you have a 10 Mb/s network between your client and your server and this is why your transfer is going slowly. If this is the case, try using a DeflatoutOutputStream and an InflatorInputStream for your connection.