Your IP : 172.28.240.42
Tc @ s d Z d d l Z d d l Z d d l m Z d d l m Z m Z d d l m Z d d l
m Z m Z d d l
m Z m Z d d l m Z d d l m Z m Z m Z d
e f d YZ d Z d S(
sb6 Manage outgoing and incoming messages when communicating with the server.
The protocol to communicate between the client and the server has been designed
to be very robust so that messages are not lost. In addition it is (vaguely)
symmetric, as the client and server need to send messages both ways.
Client->Server Payload
======================
All message payloads are bpickled with L{landscape.lib.bpickle.dumps}. Client
to server payloads are C{dict}s of the form::
{'server-api': SERVER_API_VERSION,
'client-api': CLIENT_API_VERSION,
'sequence': SEQUENCE_NUMBER,
'accepted-types': SERVER_ACCEPTED_TYPES_DIGEST,
'messages': MESSAGES,
'total-messages': TOTAL_COUNT_OF_PENDING_MESSAGES,
'next-expected-sequence': EXPECTED_SEQUENCE_NUMBER,
'client-accepted-types': CLIENT_ACCEPTED_TYPES (optional)}
The values have the following semantics:
- C{SERVER_API_VERSION}: The API version that is required on the server
in order to process the messages in this payload (the schema and semantics
of message types are usually different for different API versions).
- C{CLIENT_API_VERSION}: The API version of the client, hinting the server
about the schema and semantics of the messages types accepted by the client
(see below).
- C{SEQUENCE_NUMBER}: A monotonically increasing nonnegative integer. The
meaning of this is described below.
- C{SERVER_ACCEPTED_TYPES_DIGEST}: A hash of the message types that the
client thinks are currently accepted by the server. The server can use it
to know whether to send the client a new up-to-date list of accepted
message types.
- C{MESSAGES}: A python list of messages, described below.
- C{TOTAL_COUNT_OF_PENDING_MESSAGES}: The total number of messages in the
client outgoing queue. This is includes the number of messages being sent
in this payload, plus any other messages still pending and not included
here.
- C{EXPECTED_SEQUENCE_NUMBER}: The sequence number which the client expects
the next message sent from the server to have.
- C{CLIENT_ACCEPTED_TYPES}: Optionally, a list of message types that the
client accepts. The server is supposed to send the client only messages of
this type. It will be inlcuded in the payload only if the hash that the
server sends us is out-of-date. This behavior is simmetric with respect to
the C{SERVER_ACCEPTED_TYPES_DIGEST} field described above.
Server->Client Payload
======================
The payloads that the server sends to not-yet-registered clients (i.e. clients
that don't provide a secure ID associated with a computer) are C{dict}s of the
form::
{'server-uuid': SERVER_UUID,
'server-api': SERVER_API,
'messages': MESSAGES}
where:
- C{SERVER_UUID}: A string identifying the particular Landscape server the
client is talking to.
- C{SERVER_API}: The version number of the highest server API that this
particular server is able to handle. It can be used by the client to
implement backward compatibility with old servers, knowing what message
schemas the server expects (since schemas can change from version to
version).
- C{MESSAGES}: A python list of messages, described below.
Additionally, payloads to registered clients will include these fields::
{'next-expected-sequence': EXPECTED_SEQUENCE_NUMBER,
'next-expected-token': EXPECTED_EXCHANGE_TOKEN,
'client-accepted-types-hash': CLIENT_ACCEPTED_TYPES_DIGEST,
where:
- C{EXPECTED_SEQUENCE_NUMBER}: The sequence number which the server expects
the next message sent from the client to have.
- C{EXPECTED_EXCHANGE_TOKEN}: The token (UUID string) that the server expects
to receive back the next time the client performs an exchange. Since the
client receives a new token at each exchange, this can be used by the
server to detect cloned clients (either the orignal client or the cloned
client will eventually send an expired token). The token is sent by the
client as a special HTTP header (see L{landscape.broker.transport}).
- C{CLIENT_ACCEPTED_TYPES_DIGEST}: A hash of the message types that the
server thinks are currently accepted by the client. The client can use it
to know whether to send to the server an up-to-date list the message types
it now accepts (see CLIENT_ACCEPTED_TYPES in the client->server payload).
Individual Messages
===================
A message is a C{dict} with required and optional keys. Messages are packed
into Python lists and set as the value of the 'messages' key in the payload.
The C{dict} of a single message is of the form::
{'type': MESSAGE_TYPE,
...}
where:
- C{MESSAGE_TYPE}: A simple string, which lets the server decide what handler
to dispatch the message to, also considering the SERVER_API_VERSION value.
- C{...}: Other entries specific to the type of message.
This format is the same for messages sent by the server to the client and for
messages sent by the client to the server. In addition, messages sent by the
client to the server will contain also the following extra fields::
{...
'api': SERVER_API,
'timestamp': TIMESTAMP,
...}
where:
- C{SERVER_API}: The server API that the client was targeting when it
generated the message. In single exchange the client will only include
messages targeted to the same server API.
- C{TIMESTAMP}: A timestamp indicating when the message was generated.
Message Sequencing
==================
A message numbering system is built in to the protocol to ensure robustness of
client/server communication. The way this works is not totally symmetrical, as
the client must connect to the server via HTTP, but the ordering that things
happen in over the course of many connections remains the same (see also
L{landscape.broker.store} for more concrete examples):
- Receiver tells Sender which sequence number it expects the next batch of
messages to start with.
- Sender gives some messages to Receiver, specifying the sequence number of
the first message. If the expected and actual sequence numbers are out of
synch, Sender resynchronizes in a certain way.
The client and server must play the part of *both* of these roles on every
interaction, but it simplifies things to talk about them in terms of a single
role at a time.
When the client connects to the server, it does the following things acting
in the role of Sender (which is by far its more burdened role):
- Send a payload containing messages and a sequence number. The sequence
number should be the same number that the server gave as
next-expected-sequence in the prior connection, or 0 if there was no
previous connection.
- Get back a next-expected-sequence from the server. If that value is is not
len(messages) + previous-next-expected, then resynchronize.
It does the following when acting as Receiver:
- Send a payload containing a next-expected-sequence, which should be the
sequence number of the first message that the server responds with. This
value should be previous-next-expected + len(previous_messages).
- Receive some messages from the server, and process them immediately.
When the server is acting as Sender, it does the following:
- Wait for a payload with next-expected-sequence from the client.
- Perhaps resynchronize if next-expected-sequence is unexpected.
- Respond with a payload of messages to the client. No sequence identifier
is given for this payload of messages, because it would be redundant with
data that has already passed over the wire (received from the client)
during the very same TCP connection.
When the server is acting as a Receiver, it does the following:
- Wait for a payload with a sequence identifier and a load of messages.
- Respond with a next-expected-sequence.
There are two interesting exceptional cases which must be handled with
resynchronization:
1. Messages received with sequence numbers less than the next expected
sequence number should be discarded, and further messages starting at
the expected sequence numbers should be processed.
2. If the sequence number is higher than what the receiver expected, then
no messages are processed and the receiver responds with the same
{'next-expected-sequence': N}, so that the sender can resynchronize
itself.
This implies that the receiver must record the sequence number of the last
successfully processed message, in order for it to respond to the sender
with that number. In addition, the sender must save outbound messages even
after they have been delivered over the transport, until the sender receives
a next-expected-sequence higher than the outbound message. The details of
this logic are described in L{landscape.broker.store}.
Exchange Sequence
=================
Diagram::
1. BrokerService --> MessageExchange : Start
2. MessageExchange --> MessageExchange : Schedule exchange
3. [event] <-- MessageExchange : Fire "pre-exchange"
4. [optional] : Do registration
(See L{landscape.broker.registration}) : sequence
5. MessageExchange --> MessageStore : Request pending
: messages
6. MessageExchange <-- MessageStore : return( Messages )
7. MessageExchange --> HTTPTransport : Exchange
8. HTTPTransport --> {Server}LandscapeMessageSystem
: HTTP POST
9. [Scope: Server]
|
| 9.1 LandscapeMessageSystem --> ComputerMessageAPI : run
|
| 9.2 ComputerMessageAPI --> FunctionHandler : handle
|
| 9.3 FunctionHandler --> Callable : call
| ( See also server code at:
| - C{canonical.landscape.message.handlers}
| - C{canonical.message.handler.FunctionHandler} )
|
|
| 9.4 [If: the callable raises ConsistencyError]
| |
| | 9.4.1 ComputerMessageAPI --> Computer : request
| | : Resynchronize
| |
| | 9.4.2 Computer --> Computer : Create
| | : ResynchronizeRequest
| | : activity
| |
| --[End If]
|
| 9.5 ComputerMessageAPI --> Computer : get deliverable
| : activities
|
| 9.6 ComputerMessageAPI <-- Computer : return activities
|
| 9.7 [Loop over activities]
| |
| | 9.7.1 ComputerMessageAPI --> Activity : deliver
| |
| | 9.7.2 Activity --> MessageStore : add activity message
| |
| --[End Loop]
|
| 9.8 ComputerMessageAPI --> MessageStore : get pending messages
|
| 9.9 ComputerMessageAPI <-- MessageStore : return messages
|
| 9.10 LandscapeMessageSystem <-- ComputerMessageAPI : return payload
| : (See below)
|
-- [End Scope]
10. HTTPTransport <-- {Server}LandscapeMessageSystem
: HTTP response
: with payload
11. MessageExchange <-- HTTPTransport : response
12. [If: server says it expects a very old message]
|
| 12.1 [event] <-- MessageExchange : event
| (See L{landscape.broker.server}) : "resynchronize-
| : clients"
|
-- [End if]
13. [Loop: over messages in payload]
|
| 13.1 [event] <-- MessageExchange : event
| : message (message)
|
| 13.2 [Switch: on message type]
| |
| |- 13.2.1 [Case: message type is "accepted-types"]
| | |
| | | 13.2.1.1 MessageExchange -> MessageStore
| | | : set accepted types
| | |
| | | 13.2.1.2 MessageExchange -> MessageExchange
| | | : schedule urgent
| | | : exchange
| | --[End Case]
| |
| |- 13.2.2 [Case: message type is "resynchronize"]
| | |
| | | 13.2.2.1 [event] <- MessageExchange
| | | (See L{landscape.broker.server})
| | | : event
| | | : "resynchronize-
| | | : clients"
| | |
| | | 13.2.2.2 MessageExchange -> MessageStore
| | | : add "resynchronize"
| | | : message
| | |
| | | 13.2.2.3 MessageExchange -> MessageExchange
| | | : schedule urgent
| | | : exchange
| | |
| | --[End Case]
| |
| |- 13.2.3 [Case: message type is "set-intervals"]
| | |
| | | 13.2.3.1 MessageExchange -> BrokerConfiguration
| | | : set exchange
| | | : interval
| | |
| | --[End Case]
| |
| -- [End Switch]
|
-- [End Loop]
14. Schedule exchange
iN( t md5( t Deferredt succeed( t
HTTPCodeError( t got_next_expectedt ANCIENT( t is_version_highert
sort_versions( t format_delta( t DEFAULT_SERVER_APIt
SERVER_APIt
CLIENT_APIt MessageExchangec B s e Z d Z e Z d d Z d Z e d Z d Z d Z
d Z d Z d d Z d
Z d Z d Z e e d
Z d Z d Z d Z d Z d Z d Z d Z d Z d Z RS( sv Schedule and handle message exchanges with the server.
The L{MessageExchange} is the place where messages are sent to go out
to the Landscape server. It accumulates messages in its L{MessageStore}
and periodically delivers them to the server.
It is also the place where messages coming from the server are handled. For
each message type the L{MessageExchange} supports setting an handler that
will be invoked when a message of the that type is received.
An exchange is performed with an HTTP POST request, whose body contains
outgoing messages and whose response contains incoming messages.
id c C s | | _ | | _ | | _ | | _ | | _ | j | _ | j | _ | | _ d | _ d | _ t
| _ t
| _ t | _ d | _ i | _ | | _ t
| _ | j d | j | j d | j | j d | j | j d | j d S( s
@param reactor: The L{LandscapeReactor} used to fire events in response
to messages received by the server.
@param store: The L{MessageStore} used to queue outgoing messages.
@param transport: The L{HTTPTransport} used to deliver messages.
@param registration_info: The L{Identity} storing our secure ID.
@param config: The L{BrokerConfiguration} with the `exchange_interval`
and `urgent_exchange_interval` parameters, respectively holding
the time interval between subsequent exchanges of non-urgent
messages, and the time interval between subsequent exchanges
of urgent messages.
s accepted-typest
resynchronizes
set-intervalss resynchronize-clientsN( t _reactort _message_storet
_transportt _registration_infot _configt exchange_intervalt _exchange_intervalt urgent_exchange_intervalt _urgent_exchange_intervalt
_max_messagest Nonet _notification_idt _exchange_idt Falset _exchangingt _urgent_exchanget sett _client_accepted_typest _client_accepted_types_hasht _message_handlerst _exchange_storet _stoppedt register_messaget _handle_accepted_typest _handle_resynchronizet _handle_set_intervalst call_ont _resynchronize( t selft reactort storet transportt registration_infot exchange_storet configt max_messages( ( s= /usr/lib/python2.7/dist-packages/landscape/broker/exchange.pyt __init__y s* c C sp d | k r t S| d } | j j | } | d k rM t j d | t S| j j | j k } | j | S( s Returns C{True} if message is obsolete.
A message is considered obsolete if the secure ID changed since it was
received.
s operation-ids4 No message context for message with operation-id: %sN( R R" t get_message_contextR t loggingt warningR t secure_idt remove( R* t messaget operation_idt contextt result( ( s= /usr/lib/python2.7/dist-packages/landscape/broker/exchange.pyt _message_is_obsolete s
c C s | j | r- t j d | j d d Sd | k rU t | j j | d <n | j j | } | r | j
d t n | S( s Include a message to be sent in an exchange.
If urgent is True, an exchange with the server will be
scheduled urgently.
@param message: Same as in L{MessageStore.add}.
sn Response message with operation-id %s was discarded because the client's secure ID has changed in the meantimes operation-idt timestampt urgentN( R< R4 t infot getR t intR t timeR t addt schedule_exchanget True( R* R8 R> t
message_id( ( s= /usr/lib/python2.7/dist-packages/landscape/broker/exchange.pyt send s c C s | j d t d S( s9 Start scheduling exchanges. The first one will be urgent.R> N( RD RE ( R* ( ( s= /usr/lib/python2.7/dist-packages/landscape/broker/exchange.pyt start s c C si | j d k r. | j j | j d | _ n | j d k r\ | j j | j d | _ n t | _ d S( s Stop scheduling exchanges.N( R R R t cancel_callR RE R# ( R* ( ( s= /usr/lib/python2.7/dist-packages/landscape/broker/exchange.pyt stop s c C s t | j j } t | d } t | | } | j j | t j d | | j j d ry | j d t n x( | | D] } | j
j d | t q Wx( | | D] } | j
j d | t q Wd S( s
When the server updates us about the types of message it
accepts, update our message store.
If this makes existing held messages available for sending,
urgently exchange messages.
If new types are made available or old types are dropped a
C{("message-type-acceptance-changed", type, bool)} reactor
event will be fired.
t typess Accepted types changed: %si R> s message-type-acceptance-changedN(
R R t get_accepted_typest get_accepted_types_difft set_accepted_typesR4 R? t get_pending_messagesRD RE R t fireR ( R* R8 t old_typest new_typest difft type( ( s= /usr/lib/python2.7/dist-packages/landscape/broker/exchange.pyR% s c C sN | d } | j d } | j i d d 6| d 6 | j j d d | d S( Ns operation-idt scopesR
RT s resynchronize-clients( R@ RG R RP ( R* R8 t opidRU ( ( s= /usr/lib/python2.7/dist-packages/landscape/broker/exchange.pyR& s
c C s$ | j j | | j d t d S( NR> ( R t drop_session_idsRD RE ( R* RU ( ( s= /usr/lib/python2.7/dist-packages/landscape/broker/exchange.pyR) s c C s} d | k r6 | d | j _ t j d | j j n d | k rl | d | j _ t j d | j j n | j j d S( Nt exchanges$ Exchange interval set to %d seconds.s urgent-exchanges+ Urgent exchange interval set to %d seconds.( R R R4 R? R t write( R* R8 ( ( s= /usr/lib/python2.7/dist-packages/landscape/broker/exchange.pyR' s c s
j r t d St _ j j d j t j j rj t j
d j j n t j
d j j t
f d f d } f d } j j | | j j j j j j d S( s| Send pending messages to the server and process responses.
A C{pre-exchange} reactor event will be emitted just before the
actual exchange takes place.
An C{exchange-done} or C{exchange-failed} reactor event will be
emitted after a successful or failed exchange.
@return: A L{Deferred} that is fired when exchange has completed.
s pre-exchanges) Starting urgent message exchange with %s.s" Starting message exchange with %s.c sQ j d t j j d t j d t t j j d d S( Nt forces
exchange-dones! Message exchange completed in %s.(
RD RE R RP R4 R? R RB t callbackR ( ( t
start_timet deferredR* ( s= /usr/lib/python2.7/dist-packages/landscape/broker/exchange.pyt exchange_completed* s
c s t _ | rc j r1 t j d t _ n j | j j t j j
n j j d t j d d S( Ns" Switching to normal exchange mode.s exchange-faileds Message exchange failed.( R R R R4 R? t _handle_resultR t record_successRA R RB RP ( R; ( R* R^ t payload( s= /usr/lib/python2.7/dist-packages/landscape/broker/exchange.pyt
handle_result1 s
"
c s t _ t | t r] | j d k r] j j t k r] j j t j d Sn j
j d j j t
j
j t j d d S( Ni s exchange-faileds Message exchange failed.( R R t
isinstanceR t http_codeR t get_server_apiR t set_server_apiRX R RP t record_failureRA RB R4 R? ( t error_classt errort traceback( R^ R* ( s= /usr/lib/python2.7/dist-packages/landscape/broker/exchange.pyt handle_failure> s
s
server-apiN( R R R RE R RP t
_make_payloadRB R R4 R? R t get_urlR t call_in_threadRX R R6 t _get_exchange_tokenR@ ( R* Rb Rk ( ( R\ R^ Ra R] R* s= /usr/lib/python2.7/dist-packages/landscape/broker/exchange.pyRX s*
c C s | j S( sM Return a bool showing whether there is an urgent exchange scheduled.
( R ( R* ( ( s= /usr/lib/python2.7/dist-packages/landscape/broker/exchange.pyt is_urgentV s c C s | j r
d S| j r | s< | j d k s< | r | j r | rN t | _ n | j rm | j j | j n | j r | j j } n | j j
} | j d k r | j j | j n | d } | j j | | j
| _ | j j | | j | _ n d S( s_ Schedule an exchange to happen.
The exchange will occur after some time based on whether C{urgent} is
True. An C{impending-exchange} reactor event will be emitted
approximately 10 seconds before the exchange is started.
@param urgent: If true, ensure an exchange happens within the
urgent interval. This will reschedule the exchange if necessary.
If another urgent exchange is already scheduled, nothing happens.
@param force: If true, an exchange will necessarily be scheduled,
even if it was already scheduled before.
Ni
( R# R R R R RE R RI R R R R t
call_latert _notify_impending_exchangeRX ( R* R> RZ t intervalt notification_interval( ( s= /usr/lib/python2.7/dist-packages/landscape/broker/exchange.pyRD [ s&
c C s0 | j j } | j j d | j j | S( s Get the token given us by the server at the last exchange.
It will be C{None} if we are not fully registered yet or if something
bad happened during the last exchange and we could not get the token
that the server had given us.
N( R t get_exchange_tokent set_exchange_tokenR t commit( R* t exchange_token( ( s= /usr/lib/python2.7/dist-packages/landscape/broker/exchange.pyRo s
c C s | j j d d S( Ns impending-exchange( R RP ( R* ( ( s= /usr/lib/python2.7/dist-packages/landscape/broker/exchange.pyRr s c C s4 | j } | j | j } | j | j } | j } | r | d j d } x9 t | D]% \ } } | j d | k rb Pqb qb Wd } | d k r | | 3q n | j } i | d 6t
d 6| j d 6| d 6| d 6| d 6| j d 6} | j
} | j | }
|
| j k r0| | d
<n | S( s Return a dict representing the complete exchange payload.
The payload will contain all pending messages eligible for
delivery, up to a maximum of C{max_messages} as passed to
the L{__init__} method.
i t apis
server-apis
client-apit sequences accepted-typest messagess total-messagess next-expected-sequences client-accepted-typesN( R t _hash_typesRL RO R t count_pending_messagesR@ t enumerateR Re R t get_sequencet get_server_sequencet! get_client_accepted_message_typesR ( R* R, t accepted_types_digestR{ t total_messagest
server_apit iR8 Ra t accepted_client_typest accepted_client_types_hash( ( s= /usr/lib/python2.7/dist-packages/landscape/broker/exchange.pyRl s2
c C s d j | } t | j S( Nt ;( t joinR t digest( R* RK t accepted_types_str( ( s= /usr/lib/python2.7/dist-packages/landscape/broker/exchange.pyR| s c
C s! | j } | j d | _ | j d } | j } | d k re | j } | t | d 7} n t | | } | t k r t j d | j
i d d 6 | j j d n | j
| j d | j } | j d } | | k r.t j d
| | f | j j d | | | j | n | j d d
} t | | j r~t | | j g d }
| j |
n | j | j } xH | j d d D]4 } | j | | d 7} | j | | j qW| j d r t j d | | k r | j d t q n d S( s Handle a response from the server.
Called by L{exchange} after a batch of messages has been
successfully delivered to the server.
If the C{server_uuid} changed, a C{"server-uuid-changed"} event
will be fired.
Call L{handle_message} for each message in C{result}.
@param payload: The payload that was sent to the server.
@param result: The response got in reply to the C{payload}.
s client-accepted-types-hashs next-expected-sequenceR{ sI Server asked for ancient data: resynchronizing all state with the server.R
RT s resynchronize-clientss next-exchange-tokens server-uuids% Server UUID changed (old=%s, new=%s).s server-uuid-changeds
server-apis 3.2ii s0 Pending messages remain after the last exchange.R> N( ( R R@ R R R t lenR R R4 R? RG R RP Rv t get_server_uuidt set_server_uuidR Re R t _apiRf Rw R t handle_messaget set_server_sequenceRO RD RE (
R* Ra R; t
message_storet
next_expectedt old_sequencet message_store_statet old_uuidt new_uuidR t lowest_server_apiRz R8 ( ( s= /usr/lib/python2.7/dist-packages/landscape/broker/exchange.pyR_ sH
c C s0 | j j | g j | | j j | d S( s Register a handler for the given message type.
The C{handler} callable will to be executed when a message of
type C{type} has been received from the server.
Multiple handlers for the same type will be called in the
order they were registered.
N( R! t
setdefaultt appendR RC ( R* RT t handler( ( s= /usr/lib/python2.7/dist-packages/landscape/broker/exchange.pyR$ s c C s d | k r3 | j j | d | j j | d n | j j d | | d | j k r x& | j | d D] } | | qk Wn d S( s
Handle a message received from the server.
Any message handlers registered with L{register_message} will
be called.
s operation-idRT R8 N( R" t add_message_contextR R6 R RP R! ( R* R8 R ( ( s= /usr/lib/python2.7/dist-packages/landscape/broker/exchange.pyR $ s c C s | j j t | d S( N( R RC t str( R* RT ( ( s= /usr/lib/python2.7/dist-packages/landscape/broker/exchange.pyt% register_client_accepted_message_type8 s c C s
t | j S( N( t sortedR ( R* ( ( s= /usr/lib/python2.7/dist-packages/landscape/broker/exchange.pyR = s N( t __name__t
__module__t __doc__R
R R2 R< R RG RH RJ R% R& R R) R' RX Rp RD Ro Rr Rl R| R_ R$ R R R ( ( ( s= /usr/lib/python2.7/dist-packages/landscape/broker/exchange.pyR g s.
$ J ( ( R c C s t | } t | } | | } | | @} | | } g } | j g | D] } d | ^ qI | j g | D] } d | ^ qm | j g | D] } d | ^ q d j | S( Ns +%ss %ss -%st ( R t extendR ( RQ RR t added_typest stable_typest
removed_typesRS RT ( ( s= /usr/lib/python2.7/dist-packages/landscape/broker/exchange.pyRM A s
$$$( R RB R4 t landscape.lib.hashlibR t twisted.internet.deferR R t landscape.lib.fetchR t landscape.lib.messageR R t landscape.lib.versioningR R t
landscape.logR t landscapeR R
R t objectR RM ( ( ( s= /usr/lib/python2.7/dist-packages/landscape/broker/exchange.pyt <module>X s