relation error between nova-compute and nova cloud controller

I have successfully setup Open Stack cloud from the documentation: Ubuntu Cloud Infrastructure.

My Juju environment file:

environments:
maas:
type: maas
maas-server: 'http://192.168.122.4:80/MAAS'
maas-oauth: 'np2beDGZye5ktAGHZC:xEN8GM5BDtWj2ChAW4:rnX9DBQs7Cv2rRZ9g9jxncpYg4uSkTbL'
admin-secret: 'nothing'
default-series: precise

All the nodes prepared by Juju for OpenStack have one LAN card.My MAAS server has one LAN card and has the DHCP Server. I am not using any DNS, As I am using the /etc/hosts file to fix it.

Here comes the problem:

When I run Juju status, I see a relation error between at cloud-compute against nova-cloud-controller. I am able to login to OpenStack dashboard and can move in all the sections except Manage Compute section, and this may be because of the error between these two servers.

When I run Juju debug-log I see following result:

2013-05-06 21:02:53,108 unit:nova-cloud-controller/1: hook.output DEBUG: Flushed values for hook 'cloud-compute-relation-joined' on 'cloud-compute:22'
Setting changed: u'ec2_host'=u'node-aabbccddeef4.localdomain' (was unset)
Setting changed: u'network_manager'=u'FlatDHCPManager' (was unset)
Setting changed: u'volume_service'=u'nova-volume' (was unset) 
2013-05-06 21:02:53,108 unit:nova-cloud-controller/1: hook.executor DEBUG: Hook complete: /var/lib/juju/units/nova-cloud-controller-1/charm/hooks/cloud-compute-relation-joined 
2013-05-06 21:02:53,115 unit:nova-cloud-controller/1: hook.scheduler DEBUG: executing hook for nova-compute/0:modified
2013-05-06 21:02:53,116 unit:nova-cloud-controller/1: unit.relation.lifecycle DEBUG: Executing hook cloud-compute-relation-changed
2013-05-06 21:02:53,116 unit:nova-cloud-controller/1: hook.executor DEBUG: Running hook: /var/lib/juju/units/nova-cloud-controller-1/charm/hooks/cloud-compute-relation-changed 
2013-05-06 21:02:53,375 unit:nova-compute/0: hook.scheduler DEBUG: settings changed: (('nova-cloud-controller/1', 1),) 
2013-05-06 21:02:53,378 unit:nova-compute/0: hook.scheduler DEBUG: executing hook for nova-cloud-controller/1:modified 
2013-05-06 21:02:53,378 unit:nova-compute/0: unit.relation.lifecycle DEBUG: Executing hook cloud-compute-relation-changed 
2013-05-06 21:02:53,379 unit:nova-compute/0: hook.executor DEBUG: Running hook: /var/lib/juju/units/nova-compute-0/charm/hooks/cloud-compute-relation-changed 
2013-05-06 21:02:54,133 unit:nova-cloud-controller/1: hook.output DEBUG: hook cloud-compute-relation-changed exited, exit code 0.
2013-05-06 21:02:54,161 unit:nova-cloud-controller/1: hook.executor DEBUG: Hook complete: /var/lib/juju/units/nova-cloud-controller-1/charm/hooks/cloud-compute-relation-changed
2013-05-06 21:02:54,555 unit:nova-compute/0: hook.output INFO: /var/lib/juju/units/nova-compute-0/charm/hooks/cloud-compute-relation-changed: configuring FlatDHCPManager network                                                    manager 
2013-05-06 21:02:55,193 unit:nova-compute/0: unit.hook.api DEBUG: Get unit setting: 'private-address'
2013-05-06 21:03:25,361 unit:nova-compute/0: hook.output ERROR: Traceback (most recent call last):
2013-05-06 21:03:25,361 unit:nova-compute/0: hook.output ERROR:   File "<string>", line 10, in <module>
2013-05-06 21:03:25,361 unit:nova-compute/0: hook.output ERROR:   File "/usr/lib/python2.7/dist-packages/dns/resolver.py", line 770, in query
2013-05-06 21:03:25,362 unit:nova-compute/0: hook.output ERROR:
2013-05-06 21:03:25,362 unit:nova-compute/0: hook.output ERROR: raise_on_no_answer)
2013-05-06 21:03:25,363 unit:nova-compute/0: hook.output ERROR:   File "/usr/lib/python2.7/dist-packages/dns/resolver.py", line 698, in query
2013-05-06 21:03:25,363 unit:nova-compute/0: hook.output ERROR:
2013-05-06 21:03:25,366 unit:nova-compute/0: hook.output ERROR: timeout = self._compute_timeout(start)
File "/usr/lib/python2.7/dist-packages/dns/resolver.py", line 568, in _compute_timeout
raise Timeout
dns.exception.Timeout
2013-05-06 21:03:25,589 unit:nova-compute/0: unit.hook.api INFO: Unable to resolve local IP address
2013-05-06 21:03:25,611 unit:nova-compute/0: hook.output DEBUG: hook cloud-compute-relation-changed exited, exit code Traceback (most recent call last):
Failure: juju.errors.CharmInvocationError: Error processing '/var/lib/juju/units/nova-compute-0/charm/hooks/cloud-compute-relation-changed': exit code 1.
.
2013-05-06 21:03:25,611 unit:nova-compute/0: hook.executor DEBUG: Hook error: /var/lib/juju/units/nova-compute-0/charm/hooks/cloud-compute-relation-changed Error processing '/var                                                   /lib/juju/units/nova-compute-0/charm/hooks/cloud-compute-relation-changed': exit code 1.
2013-05-06 21:03:25,612 unit:nova-compute/0: unit.relation.lifecycle WARNING: Error in cloud-compute-relation-changed hook: Error processing '/var/lib/juju/units/nova-compute-0/c                                                   harm/hooks/cloud-compute-relation-changed': exit code 1.
2013-05-06 21:03:25,612 unit:nova-compute/0: unit.relation.lifecycle INFO: Invoked error handler for cloud-compute-relation-changed hook
2013-05-06 21:03:25,811 unit:nova-compute/0: statemachine DEBUG: relationworkflowstate: transition error (up -> error) {'error_message': "Error processing '/var/lib/juju/units/no                                                   va-compute-0/charm/hooks/cloud-compute-relation-changed': exit code 1.", 'change_type': 'modified'}
2013-05-06 21:03:25,811 unit:nova-compute/0: statemachine DEBUG: relationworkflowstate:  execute action do_error
2013-05-06 21:03:25,812 unit:nova-compute/0: hook.scheduler DEBUG: stopping
2013-05-06 21:03:25,813 unit:nova-compute/0: unit.relation.lifecycle DEBUG: stopped relation:cloud-compute lifecycle
2013-05-06 21:03:25,944 unit:nova-compute/0: statemachine DEBUG: relationworkflowstate: transition complete error (state error) {'error_message': "Error processing '/var/lib/juju                                                   /units/nova-compute-0/charm/hooks/cloud-compute-relation-changed': exit code 1.", 'change_type': 'modified'}
2013-05-06 21:03:25,949 unit:nova-compute/0: hook.scheduler DEBUG: hook error, stopping scheduler execution
2013-05-06 21:03:25,949 unit:nova-compute/0: hook.scheduler INFO: stopped

Do I need to make some additions to the Juju environment file for OpenStack , by defining environment for OpenStack ?

In an attempt to fix above problem, I tried using a DNS server on another system instead of using /etc/hosts to resolve DNS names.

My configured domain name on this new server is "malik.net". Name resolution works fine but juju assigns a different public address against each node's installed service. e.g juju-status reports: public-address: node-aab2ccddeeff.localdomain whereas it should be public-address: node-aab2ccddeeff.malik.net.

The nodes do not recognize the public address created by juju. Then I had to make entries in /etc/hosts at each node by writing node IP's and each_node_name.localdomain. I landed in the same relation error problem.

By telling Juju to use the right domain name might fix this issue, But i do not know how to do that .


Solution 1:

The nova-compute charm depends on DNS resolution of IP addresses; several functions within OpenStack rely on IP addresses instead of hostnames and the charm includes code that directly queries DNS to resolve hostnames -> IP addresses.

Unfortunately using /etc/hosts on a stock cloud-init built instance will always result in a hostname -> IP address mapping to 127.0.1.1 - which is not much use other than for local access.

I'd encourage you to get DNS up and running; I don't think its an insane assumption that DNS is functional in the charm execution environment.