[Pulp-list] Messaging Questions

[Jeff Ortel]

On 07/02/2010 08:10 AM, Jason Dobies wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> I've been thinking about some of the edge cases we need to address in
> messaging. These may be addressed already, I just wanted to how we're
> handling things.

Good stuff!

>
>
> - - What happens if a message is put on the queue for an agent, but the
> agent never comes online to pick it up? Do they expire? We'll need some
> sort of reaper to time out the request in the server database so it
> doesn't perpetually look like it's in progress.

Synchronous messages will fail immediately if the agent is unavailable so let's assume for 
the moment, we're only talking about asynchronous messages (RMI).  I believe that all 
asynchronous messages to the agent should be dispatched through the Tasking framework. 
That way *all* policy around asynchronous operations will be in one place.  The lifecycle 
of the asynchronous message should be tied to (and implemented by) the Task.  So long as 
the task lives, the message should also live.  If the task times out, the message should 
be dequeued.  So, the messaging framework need to support message dequeuing.  It can do 
this by sending a cancellation message with higher priority if dequeuing not directly 
supported by qpid.

This leaves orphaned queues for consumer un-registration.  Seems like the ConsumerApi 
could be responsible for this by doing something like:

 >
 > from pulp.agent import Agent
 > Agent.purge('foo')
 >

which would remove the associated queue.

>
> - - What happens if an agent picks up a message but never replies? Same
> thing here, we'll need a reaper to mark requests as timed out or in this
> case, that the agent never replied.

The messaging framework (pmf) ensures that messages are processed (dispatched) before they 
are acknowledged (taken from the queue).  This prevents against cases where the agent 
consumes a message then dies and thus never processes it.  Due to guaranteed message 
delivery, the agent will always reply unless it's dead.  In which case, see above.

>
> - - Is there anything in the reply that validates that it came from the
> correct agent? For instance, I send package install request 1234 to
> agent A, but agent B for some reason replies that 1234 was completed.

Yes.  All requests (messages) have unique serial numbers which are placed in the reply and 
matched by the message framework.  Agent B, will never see request 1234.  This behaviour 
is standardized and enforced by the messaging framework.

>
> - - This one affects just about all of the previous: what happens if a
> consumer re-registers? Will it reuse the same queue as it previously
> did?

Assuming that it cannot re-register with the same ID, it would be considered a new 
consumer.   The previous registration, will orphan many resources in pulp - including the 
queue.  Orphans need to be addressed across the board.  See comment above for queue clean up.

If not, when does that queue get deleted? What happens if that
> re-registration happens while the agent is doing a task before it
> replies, will it confuse the server that the reply came from a
> "different" consumer?
>
> - - Are replies back to the server guaranteed delivery as well?

Yes.

I'm
> thinking of the situation where the server is offline when the agent
> finishes doing its business.
>
>
>
> - --
> Jason Dobies
> RHCE# 805008743336126
> Freenode: jdob
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.14 (GNU/Linux)
> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/
>
> iQEcBAEBAgAGBQJMLeUqAAoJEOMmcTqOSQHCxBwH/1WQxzUAtGJqA1IlbC26ZWaj
> zl7+lCcoVuvr3OKhWFmtZ2oFCBg76xdILgP0xPWq/whrDmNeqGUW+SXsxGheC+Zs
> eoLoWYcmkAwz3tLKuq6C9U8DT+fkpC+/9FEIJDFvNlojF0CiuBd/K2XqySEUhjK5
> Hh9Pfg8JuavbE6TNrmscHRu9qpv1VEIPq1IUsqXvfd5c0ACqNBQuCwfjGVROgKb7
> aqzo2g26+oDEV3NDwbIuZ7I9esOuMBlSCvoEJLWvQN1Mzrmeu71zRYrrIOzLKVff
> /KL0L+95hMQbi8p04ySqkmLzCNR+XqKA5VWWyKhq3TNEAZe+C6Mi4qHVyCU06yU=
> =W6kO
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Pulp-list mailing list
> Pulp-list at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5126 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20100702/983e7787/attachment.p7s>