Cassandra EC2 slow response time

I have been developing with Cassandra locally for a while and response times on queries are in the region of 50ms.

I have set up a 5 node EC2 cluster from the cassandra community ami, everything went fine, opscenter is up and running and there are no issues.

When I perform any query from a machine outside of the aws network, whether it be to create a keyspace, or do a select * on an empty table, the response time is over 20 seconds.

When I query from a machine in the same region, the response time is between 1 and 2 seconds, still very slow.

The EC2 Machines are all m1.large eu-west-1a

Do you have any idea why this would be happening or where I should look to find the issue?

Solution 1:

The current version of DevCenter was designed to support as main scenario running (longish) CQL scripts (vs an interactive console with queries executed one after another). DevCenter is using as an underlying connector the DataStax Java driver for Cassandra.

For the above mentioned scenario, in order to ensure there are no "conflicts", a new Session is created for each execution. When a Session is initialized, the driver performs an auto-node discovery, creates connection pools, etc. Basically it does a lot of preparation work. Depending on the latency from your client machine to the EC2 nodes, the size of the cluster and also the configuration of these nodes (see the connection requirements), this initialization phase can be quite expensive.

As you can imagine the time spent preparing wouldn't represent a large percentage of running a DDL script and a decent size of inserts/updates. But for an interactive scenario, it will result in a suboptimal behavior (the one you are describing)

The next version(s) of DevCenter will address the interactive scenario and optimize for it so the user experience would be what you'd expect. And supporting this scenario is pretty high on our list of priorities.

Taken From: https://stackoverflow.com/questions/21219300/cassandra-native-transport-port-9042-slow-on-ec2-machine/21224540#21224540