How to manage python function calls between EC2 instances

Solution 1:

An alternative approach would be to 'loosely couple' the apps via an Amazon SQS queue:

  • process1 would send a message to an Amazon SQS queue
  • process2 would continually poll the queue waiting for a message (use WaitTimeSeconds = 20 to reduce the number of calls)
  • When a message is received, process2 can process the data contained in the message

The harder part is how to get the 'response' back to process1. You could use another queue, but it then becomes difficult to match the response to the original request. If process1 is waiting for the response and is not sending more messages in the meantime, then it could simply wait for a response in a second queue, then continue processing.

Otherwise, your idea of using an HTTP call to trigger process2 seems feasible, too. Just make sure the Security Group settings allow the two instances to communicate with each other. Either use two separate Security Groups, or if you are using a single Security Group then configure it to allow access from 'itself'.