[Rhm-users] XML Exchanges

Tue Jul 22 14:04:55 UTC 2008

Thanks. I would say just 2 or 3 dozen queues at once.  Is there a practical
limit I'm likely to hit?  Are there any XML exchange examples(java) floating
around?

Thanks,
Andrew

-----Original Message-----
From: Jonathan Robie [mailto:jonathan.robie at redhat.com] 
Sent: Monday, July 21, 2008 7:18 PM
To: Andrew M
Cc: rhemrg-users-list at redhat.com; rhm-users at redhat.com
Subject: Re: [Rhm-users] XML Exchanges

Hi Andrew,

You're right - the XML Exchange is the only exchange type that can look 
at the content of the message. The other exchange types look at message 
properties (the XML Exchange can look at both). If clients set up their 
own queues and created queries that route messages to those queues, the 
sending applications need no knowledge of the queries that exist or even 
of which parts of the message are queried.

Performance is not as good as for other exchange types, but you seem to 
be aware of that already. At this point, we have less performance data 
on the XML Exchange than for other exchange types (it's much newer!), 
but from what I can tell, this architecture could well make sense. Any 
idea how many queues might be active at the same time?

Honestly, the XML Exchange is new enough that the best way to get a feel 
for its performance with the kind of topology you suggest is probably to 
come up with a simple prototype.

Jonathan

Andrew M wrote:
> Jonathan,
> I'm checking out the FieldTable you mentioned last week. Thanks.
>
> I'd like to give you a little background on my application.  If you can
make
> any suggestions I'd love to hear them.  I'm revamping my financial order
> routing system.  I have trading applications and traders that generate
> orders something like this:
>
> <side=buy share_qty=100 symbol=IBM source=Joe exchange=NYSE>
> <side=buy share_qty=100 symbol=IBM source=Joe exchange=ArcaECN>
> <side=buy share_qty=100 symbol=MSFT source=automated_strategy_1
> exchange=ArcaECN>
>
> I have connections to multiple stock and options exchange destinations,
some
> use FIX protocol, going forward some will use their own proprietary
> protocol. To some exchanges I have multiple connections for redundancy and
> load distribution.  Currently I'm using FIX protocol for everything both
> internal and external and my goal is to use AMQP internally to carry order
> data in a protocol independent format.  AMQP msgs bound for a destination
> talking FIX would eventually hit my FIX engine for translation into
whatever
> FIX version that counterparty wants.  
>
> For routing, I would like an app/trader to just place an order on the bus
> and then based on the stock/options exchange destination specified, have
the
> order routed to the appropriate destinations.  It would need to hit a
queue
> for orders going to the exchange and also a queue for an order book app
that
> needs all outgoing orders and incoming responses.  Also copies would hit
> queues for other applications which display the order book on
workstations,
> manage risk, etc.
>
> When an app/trader places an order on the bus he specifies an exchange
> destination, say, NASDAQ.  There might be several FIX connections to
NASDAQ,
> I assume each FIX engine would draw from its own queue on the broker.  I
> would like the broker to route all orders generated by certain sources
down
> specific exchange connections. In this way, latency sensitive applications
> could have their traffic kept separate from other less sensitive stuff.
Ie.
> Joe and Bill both send orders specifying NASDAQ.  Joe's orders need to go
to
> the connection NASDAQ1 and Bill's to NASDAQ 2.  This would require routing
> based on both exchange and order source. In order for the routing to be
> based on both sources and destination exchanges I think the XQuery would
be
> required, right? 
>
> Some things I'd like to do in the future:
>
> - Orders originating from some sources would go right to the exchange
> without approval from a risk mgmt application.  Other traders' orders
would
> be required to hit a risk mgmt application which would look at orders and
> place approved ones back on the bus for delivery to the exchange.  Broker
> would need to know which sources' orders require approval and route
> accordingly.
>
> - Using multiple connections to an exchange, if the message
> translation/gateway app talking to one of those links detects the link is
> down it modifies the routing rules MRG is using so MRG routes all messages
> for that exchange to queues for links which are up. 
>
> Speed should obviously be as good as possible with latency to a minimum
but
> I'm not going to be shoveling OPRA message rates through it so I think the
> XQuery performance hit would be ok.  I would think an 8 core box could
still
> saturate my exchange links w/orders anyhow.
>
> Maybe I'm trying to do too much w/a broker?  Maybe building all this
> functionality requires an ESB?
>
> I appreciate any suggestions...
>
> Andrew
>
>
> -----Original Message-----
> From: Jonathan Robie [mailto:jonathan.robie at redhat.com] 
> Sent: Thursday, July 17, 2008 3:46 PM
> To: Andrew M
> Cc: rhemrg-users-list at redhat.com; rhm-users at redhat.com
> Subject: Re: [Rhm-users] XML Exchanges
>
> Andrew - do you have to route based on the values in these maps?
>
> If not, the XML Exchange is not the most direct way to solve your 
> problem. The XML Exchange should work fine in Java JMS, but you 
> currently have to set up your queues in either the low level Java API or 
> in C++ or Python. I haven't actually tested Java JMS programs that use 
> the XML Exchange.
>
> Jonathan
>
> Andrew M wrote:
>   
>> I need to put Map objects on the bus and pass them around for use by 
>> java, C++ and (potentially).NET applications.  Maps will definitely 
>> consist of String-to-String mappings. There may also be 
>> String-to-Float, or String-to-Long, etc.  What is the best way to do 
>> this?  It does not appear that AMQP supports some native Map type like 
>> JMS.    Is the correct plan to have C++ apps translate the Map to/from 
>> XML using this:
>>
>> http://www.codesynthesis.com/products/xsd/
>>
>>  
>>
>> and java apps do the same using this:
>>
>> http://xerces.apache.org/
>>
>>  
>>
>> Then put those bits of XML on the bus destined for an XML Exchange?
>>
>>  
>>
>> This document...
>>
>>
>>     
>
http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/pdf/Messaging_Tu
> torial/Messaging_Tutorial.pdf 
>   
>> talks about XML based routing in Python and C++ but not in Java.  Is 
>> it not possible in Java?
>>
>>  
>>
>> Thanks,
>>
>> Andrew
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Rhm-users mailing list
>> Rhm-users at redhat.com
>> https://www.redhat.com/mailman/listinfo/rhm-users
>>     
>
>
>   