Assigning TCP/IP Ports for In-House Application Use
Ports 0-1023 are the Well Known Ports and are assigned by IANA. These should only be used for the assigned protocols on public networks.
Ports 1024-65535 used to be called Registered Port Numbers (see rfc1700) but are now split into two areas (see rfc6335).
Ports 1024-49151 are the User Ports and are the ones to use for your own protocols.
Ports 49152-65535 are the Dynamic ports and should not be prescribed to a protocol.
The User Ports can be used for any protocol, but there are a finite number, so your use will clash with someone elses use on some network somewhere. IANA keep a record of registered port numbers (0-49151). If your protocol will be used on public networks then you need to look into registering it with IANA. If you are only using it within your own network then pick a port within this area (1024-49151) and check that port against the IANA register to make sure it isn't used by a protocol that could be used on your network. For private use it is probably better to pick a number that is assigned to a protocol you know won't be used than to choose one that is unassigned and so may be assigned in the future.
Don't use a port number within the Dynamic range. These ports are assigned by the operating system, dynamically and somewhat randomly. If you open a client connection (using bind() with port=0) you will be assigned an unused port from the dynamic range. There is no way to guarantee that a port in this range will always be free for your protocol.
Pick a port number from 49152 through 65535.
IANA publishes a list of currently assigned ports.
http://www.iana.org/assignments/port-numbers
The Dynamic and/or Private Ports are those from 49152 through 65535. This is the range from where you SHOULD pick a port for your in-house applications. Of course any port belonging to one of the unassigned ranges on the published list can be used. But be aware that by picking a port number from those unassigned ranges there is no guarantee whatsoever that the port you choose will not be a reserved port in the future.
UNASSIGNED PORT NUMBERS SHOULD NOT BE USED. THE IANA WILL ASSIGN THE NUMBER FOR THE PORT AFTER YOUR APPLICATION HAS BEEN APPROVED.
And make sure that the port number you pick is configurable as you stated:
That port will be embedded in the config files for the service and the clients that are consuming the service.
This will avoid headaches in case some other 3rd party you-cannot-touch software is using your port number. If that happens you just go ahead and change it on the configuration file and it just works.
Short answer: Avoid anything up to and including 1023, or over 49152, and test the chosen port against services on your network.
If you've taken the reasonable precautions that it appears you have (putting the port number in a config file), it shouldn't be an enormous disruption if you later discover a conflict.
But (so that I can add something to the other suggestions that have popped up while I've been typing) make sure that you make it easy to change! If it's in config files, make it obvious. Document it, and point it out in troubleshooting. It's the sort of thing that could go wrong, so make it easy to debug if it needs changing.
In addition to the other suggestions about picking a common application port, I'd suggest that you make the port configurable within your application. Hard-coded port numbers are a bad idea, particularly if you later find a port conflict with another application and need to change yours.