1. AMQP and Nova

The Advanced Message Queuing Protocol (AMQP) is an open standard application layer protocol for message-oriented middleware. The defining features of AMQP are message orientation, queuing, routing (including point-to-point and publish-and-subscribe), reliability and security.

AMQP is the messaging technology chosen by the OpenStack cloud. The AMQP broker, either RabbitMQ or Qpid, sits between any two Nova components and allows them to communicate in a loosely coupled fashion.

Nova uses direct, fanout, and topic-based exchanges. The architecture looks like the one depicted in the figure below:

Each Nova service (for example Compute, Volume, etc.) create two queues at the initialization time, one which accepts messages with routing keys ‘NODE-TYPE.NODE-ID’ (for example compute.hostname) and another, which accepts messages with routing keys as generic ‘NODE-TYPE’ (for example compute). The former is used specifically when Nova-API needs to redirect commands to a specific node like ‘euca-terminate instance’. In this case, only the compute node whose host’s hypervisor is running the virtual machine can kill the instance. The API acts as a consumer when RPC calls are request/response, otherwise is acts as publisher only.

2. RabbitMQ

What is RabbitMQ?

• Robust messaging for applications
• Easy to use
• Runs on all major operating systems
• Supports a huge number of developer platforms
• Open source and commercially supported

How to use?

Sending

It’s very simple, like this:

import pika
connection = pika.BlockingConnection(pika.ConnectionParameters( host='localhost'))

#establish a connection with RabbitMQ server
channel = connection.channel()

#make sure the recipient queue exists.
channel.queue_declare(queue='hello')

#send message
channel.basic_publish(exchange='', routing_key='hello', body='Hello World!')

#closing the connection.
connection.close()

2. Receiving

The first four steps are the same:

import pika

connection = pika.BlockingConnection(pika.ConnectionParameters( host='localhost'))
channel = connection.channel()
channel.queue_declare(queue='hello')

#Definition callback function to processing the received message
def callback(ch, method, properties, body):
    # tell RabbitMQ callback function should receive messages from queue
    channel.basic_consume(callback, queue='hello', no_ack=True)

# a never-ending loop that waits for data and runs callbacks whenever necessary
channel.start_consuming()

More detailed tutorial here :

http://www.rabbitmq.com/getstarted.html

3. RabbitMQ in Nova

How Nova service to create queue at the initialization time:

if 'nova-compute' == self.binary:
    self.manager.update_available_resource(ctxt)

self.conn = rpc.create_connection(new=True) 
LOG.debug(_("Creating Consumer connection for Service %s") % self.topic) 
rpc_dispatcher = self.manager.create_rpc_dispatcher() 

# Share this same connection for these Consumers 
self.conn.create_consumer(self.topic, rpc_dispatcher, fanout=False)
node_topic = '%s.%s' % (self.topic, self.host)
self.conn.create_consumer(node_topic, rpc_dispatcher, fanout=False)
self.conn.create_consumer(self.topic, rpc_dispatcher, fanout=True) 

# Consume from all consumers in a thread 
self.conn.consume_in_thread()

rpc call operation in nova

Firstly, a Topic Publisher was created for sending the message to the queuing system; Before this, a Direct Consumer was created for receiving the response message. Secondly, the message from the publisher would be dispatched by the exchange. Once the task completed, The Direct Publisher from Worker would send the response message to the queuing system. Finally, the Direct Consumer will receiving the response message.

rpc casts operation in nova

Firstly, a Topic Publisher was instantiated to send the message to the queuing system. And then, message is dispatched by the exchange, this operation not need response message.

Author: Lee Teng, Intern System Develop Engineer of Sina SWS.