[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Pulp-list] Messaging Questions

On 07/02/2010 08:10 AM, Jason Dobies wrote:
Hash: SHA1

I've been thinking about some of the edge cases we need to address in
messaging. These may be addressed already, I just wanted to how we're
handling things.

Good stuff!

- - What happens if a message is put on the queue for an agent, but the
agent never comes online to pick it up? Do they expire? We'll need some
sort of reaper to time out the request in the server database so it
doesn't perpetually look like it's in progress.

Synchronous messages will fail immediately if the agent is unavailable so let's assume for the moment, we're only talking about asynchronous messages (RMI). I believe that all asynchronous messages to the agent should be dispatched through the Tasking framework. That way *all* policy around asynchronous operations will be in one place. The lifecycle of the asynchronous message should be tied to (and implemented by) the Task. So long as the task lives, the message should also live. If the task times out, the message should be dequeued. So, the messaging framework need to support message dequeuing. It can do this by sending a cancellation message with higher priority if dequeuing not directly supported by qpid.

This leaves orphaned queues for consumer un-registration. Seems like the ConsumerApi could be responsible for this by doing something like:

> from pulp.agent import Agent
> Agent.purge('foo')

which would remove the associated queue.

- - What happens if an agent picks up a message but never replies? Same
thing here, we'll need a reaper to mark requests as timed out or in this
case, that the agent never replied.

The messaging framework (pmf) ensures that messages are processed (dispatched) before they are acknowledged (taken from the queue). This prevents against cases where the agent consumes a message then dies and thus never processes it. Due to guaranteed message delivery, the agent will always reply unless it's dead. In which case, see above.

- - Is there anything in the reply that validates that it came from the
correct agent? For instance, I send package install request 1234 to
agent A, but agent B for some reason replies that 1234 was completed.

Yes. All requests (messages) have unique serial numbers which are placed in the reply and matched by the message framework. Agent B, will never see request 1234. This behaviour is standardized and enforced by the messaging framework.

- - This one affects just about all of the previous: what happens if a
consumer re-registers? Will it reuse the same queue as it previously

Assuming that it cannot re-register with the same ID, it would be considered a new consumer. The previous registration, will orphan many resources in pulp - including the queue. Orphans need to be addressed across the board. See comment above for queue clean up.

If not, when does that queue get deleted? What happens if that
re-registration happens while the agent is doing a task before it
replies, will it confuse the server that the reply came from a
"different" consumer?

- - Are replies back to the server guaranteed delivery as well?


thinking of the situation where the server is offline when the agent
finishes doing its business.

- --
Jason Dobies
RHCE# 805008743336126
Freenode: jdob
Version: GnuPG v2.0.14 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/


Pulp-list mailing list
Pulp-list redhat com

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]