Send MSMQ Messages Securely Across the Internet with HTTP and SOAP
MSMQ and .NET
Send MSMQ Messages Securely Across the Internet with HTTP and SOAP
David S. Platt
This article assumes you're familiar with .NET Framework and C#
Level of Difficulty 1 2 3
SUMMARY
When creating a distributed system you frequently need to provide for communication between two entities that are not in sync. Microsoft Message Queue Server (MSMQ) provides the kind of store-and-forward messaging in a pre-built infrastructure that can help you address these kinds of messaging needs. In the past, MSMQ was accessed using a COM wrapper. Now there's a .NET wrapper that lets you accomplish your messaging goals easily from your Framework-based code. To illustrate the use of the wrapper, the author builds a messaging application, sends MSMQ messages over the Web, and discusses messaging security.
Contents
Designers of distributed systems often need to communicate between two systems that are not running simultaneously. Store-and-forward messaging systems such as Microsoft® Message Queue Server (MSMQ) provide the prefabricated infrastructure that applications need to accomplish this task. In this article, I will discuss how to use MSMQ from the Microsoft .NET Framework, as well as the latest improvements in MSMQ 3.0, particularly its operation over HTTP networks.
Simplest Example
Programmers originally accessed MSMQ either through a native API or a COM wrapper. Microsoft has since added a .NET wrapper layer that provides easy access to .NET Framework-based applications, as shown in Figure 1.
Figure 1 MSMQ Wrappers
To get you started, I've written a very simple example that demonstrates the use of MSMQ in the .NET Framework. It is a .NET solution containing three projects: a sender, a recipient, and an administrator. You can download it and other samples from the link at the top of this article. For convenience, I've written them to work on a single box.
The .NET wrapper that provides access to all MSMQ operations lives in the system assembly named System.Messaging.dll. To access MSMQ from .NET, you must first add a reference to System.Messaging.dll to the project.
A queue is a persistent structure that exists on a particular machine. You need to create this persistent structure before you can write to it or read from it. In many sample programs, either the sender or the recipient creates the queue, but in production, an application's installation or configuration program usually does that. The sample program, SimplestQueuingAdministrator, simulates the production example. When the user clicks Create, I call the static method System.Messaging.MessageQueue.Create, which creates the persistent structure at the location specified by the textbox string. Since attempting to create a queue that already exists causes an exception, the method MessageQueue.Exists allows you to check beforehand whether the queue with the specified name exists. Figure 2 shows the code for these methods. (Note that Exists and Create don't work on remote private queues.)
Figure 2 Check for Queue and Create Queue
Queues can be either public or private. The term "private" as used in MSMQ is highly misleading. It doesn't mean that a program needs special permission to access it, but rather that the queue is not listed by name in the directory of the MSMQ domain controller. Any program that knows the name of the private queue can access it in the same manner as it would for a public queue. Unlisted would be a better name than private because a private queue behaves exactly like an unlisted phone number, no more and no less. Creating public queues requires a domain controller to maintain that public directory, so I've made the default queue names private. That way you can run the sample programs on a single machine without having a domain controller somewhere in the system. You specify a private queue by placing the string "\private$\" between the machine name and the queue name.
Figure 3 Simplest Queuing Sender
Now that I've created the queue, I can send messages to it with the sender application, called SimplestQueueingSender, shown in Figure 3. I find that the coolest part of the .NET wrapper is that it abstracts away almost all of the details of MSMQ. As Alan Cooper wrote in his book About Face 2.0: The Essentials of Interaction Design (John Wiley & Sons, 2003), "any good interface makes simple things simple and complex things possible." You don't have to think about message headers or priorities if you don't want to. Instead, you can just shove .NET objects into one end and they magically come out the other end in a reasonable default fashion. The sample program demonstrates this by sending objects of class System.Drawing.Point, as shown in the code in Figure 4.
Figure 4 Sample Sending Program Listing
The sender first has to open the queue for writing. This is done by instantiating an object of class System.Messaging.MessageQueue. The nomenclature can be misleading. Instantiating the object does not create the persistent queue structure, even if it doesn't yet exist. Only the previously discussed static method, MessageQueue.Create, does that. Instantiating the object opens the queue for reading or writing, which is conceptually similar to opening a file rather than creating one. If you think of the object you instantiate as a message queue connection (as I've done in the program comments) you'll have the right mental model. Once you've made the connection to the queue, you can create an object of any .NET class that supports XML serialization and pass it to the Send method of the queuing connection. The .NET wrapper now serializes the object into an XML stream and transmits that stream as the body of an MSMQ message. Other types of serialization are supported with a little more work, which I'll discuss later in the article, but XML is the default. The object will be automatically reconstituted on the recipient side, as you shall see later in this section.
If I don't want the default options for my message, I can exercise more control over it by instantiating an object of class System.Messaging.Message, and placing the object I want to send into the property named Body (not shown). I can then set the other properties of the message to the values that I want. MSMQ provides many powerful and flexible options. For example, the property Message.TimeToReachQueue specifies the amount of time allowed between a message being sent and the message reaching its final destination queue. If it doesn't reach the queue by that time, it is destroyed. The property Message.TimeToBeReceived specifies the amount of time allowed between a message being sent and that message being retrieved from the queue by the recipient.
The message is now in the queue and ready for reading. If the sender's and recipient's machines are not the same (as they are here for ease of use), the message would be buffered in an outgoing queue on the sender's machine and sent to the recipient when a connection between the two machines became available. You can see the message in the queue by using the MMC Computer Manager snap-in. The administrative sample program in this article demonstrates how easy such administrative programs are to write. When you click Enumerate, I open the queue and call the method MessageQueue.GetAllMessages. This returns an array of objects of class System.Messaging.Message, providing a static snapshot of all messages in the queue. Each message contains MSMQ properties such as priority and time sent. If you double-click the message in the listbox, my code pops up a form showing some interesting properties of the message including the body of the message containing the serialized Point structure (see Figure 5). The code is shown in Figure 6.
Figure 6 Enumerate Messages
Figure 5 Properties Popup
The .NET wrapper provides two ways of receiving an MSMQ message: synchronous and asynchronous. I'll describe the synchronous message reception first since it's easier to understand, although it's probably not quite as useful in production situations. The code for a synchronous receive operation is shown in Figure 7. When the user clicks the Do It button in the Synchronous Read group box, I first create a connection to the queue, as I did for sending. I now need to specify the formatter to be used for reception. When the client sent the Point object, it didn't have to specify a formatter, as the default was set to XML. However, on reception, I do need to specify a formatter because even the default XML formatter needs to know which types of object it might see in the incoming stream. For more information on XML serialization, see Chapter 8 of my book, Introducing Microsoft .NET, Third Edition (Microsoft Press®, 2003). When I create the formatter, I pass it an array containing System.Type objects describing all the types of .NET objects that the message might contain. I could also pass an array of type names, from which the wrapper could fetch the System.Type objects, if I prefer.
Figure 7 Synchronous Reception Operation
Once I've specified the formatter, I receive the message synchronously by calling the method MessageQueue.Receive. This method blocks the calling thread until a message is received or until a specified timeout expires, in which case it throws an exception. If you click the button while there's no message waiting, the calling thread will block. You can verify this by trying to drag the window around on the screen while waiting for a synchronous receive operation to complete. You'll find that you can't do it. Since I make the call to Receive from the same thread that handles the Windows® Forms user interface, the client can't process its internal message loop to respond to the mouse clicks attempting to move it. If the function does return successfully, the Body field of the message contains an object of the deserialized type, in this case a Point.
If I don't want the calling thread to block while it waits for an incoming message, I can do it asynchronously (see Figure 8). I open a connection to the queue and set its formatter as before. This time, instead of calling Receive directly, I create an event handler delegate and add it to the queue's ReceiveCompletedEventHandler property. This sets a handler function that will be called when the queue fires that event, which it does when a message arrives. I then call MessageQueue.BeginReceive. This hands off the operation to a background thread from the system thread pool and returns immediately. When a message does come in, the background thread calls my handler function. In it, I call the method MessageQueue.EndReceive, which returns the received message. I fetch the body and use it as in the synchronous case. Finally, I call BeginReceive again to replant the asynchronous read operation.
Figure 8 Asynchronous Message Reception
As you would expect, the .NET wrapper reports all MSMQ errors by means of .NET exceptions. Unfortunately, there's only one queuing-related exception—MessageQueueException. You've seen handlers for it in several of the code listings shown so far. To differentiate one type of MSMQ error from another, you have to examine the property MessageQueueErrorCode. The human-readable messages are, as usual, somewhat cryptic. For example, running the sender program without first creating the queue with the administrator program returns an error saying, "Queue is not registered in DS." The queue in question is a private queue and is therefore never registered with any DS. The error really means that MSMQ can't find that queue, but it can't quite figure out how to explain that simply and correctly to a human.
More Complex Example: Formatter Choices
As I explained in the previous section, when you send a .NET object in an MSMQ message, a piece of code called a formatter serializes that object into the message's body just before sending it. In the recipient program, a formatter deserializes the message's body into an object just before returning it to the reader. The .NET Framework provides you with a choice of three formatters: XML, binary, and ActiveX®. You can also create your own formatter by writing a class that implements the IMessageFormatter interface.
Most programmers use the XML formatter, which is the system default for sending. The sender-side object gets serialized into an XML packet, as shown in Figure 5. When the object gets deserialized, the recipient program has to provide the System.Type object describing the class to which the recipient wants the message reconstituted. This strategy is called loose coupling because the message itself does not specify the type of object to which it maps on the recipient. The recipient can deserialize the message into any class that can handle an XML packet of that layout. The sender and recipient do not have to share any code; they need only agree on the XML schema of the serialized object.
Figure 9 Serialized XML
I've written a sample program that demonstrates the formatters used for loose and tight coupling. You'll find it in the FormatterChoices subdirectory in the download. The sender program creates an object of class SharedSchemaSendersPoint and sends it to the recipient using the XML formatter. The code is similar to the simplest example, so I won't bother showing it. The serialized XML of the message body is shown in Figure 9. The recipient, however, deserializes it into an object of a different class, SharedSchemaRecipientsPoint, as shown here:
I've used the .NET attribute Xml.Serialization.XmlRoot to control the XML serialization so that this class expects to be deserialized from exactly this type of XML packet. Neither sender nor recipient knows or cares which class of code object the other is using to generate the XML packet. Each knows only to send or receive a message using a particular XML schema. You are basically sending XML packets through MSMQ, and the .NET object on either end is merely a convenient representation of the XML schema. Hence, loose coupling.
[System.Xml.Serialization.XmlRoot ("SharedSchemaSendersPoint")]
public class SharedSchemaRecipientsPoint
{
public int X ;
public int Y ;
}
But suppose this loosely coupled behavior isn't what you want. The sample program also demonstrates the use of a binary formatter. When the user clicks Send SharedCodePoint, I create an object of this class and specify the use of the BinaryFormatter for the message queue, as shown in Figure 10. This causes the .NET MSMQ wrapper class to serialize the object into a binary packet rather than an XML packet. The serialized message is shown in Figure 11. You see that the serialized packet contains the assembly name, class name, and version of the object which it contains. The recipient's binary formatter will attempt to create an object of this particular class in the recipient app. As you can see in the recipient's code in Figure 12, you don't have to specify the class as you did for the XML serializer. This information is in the incoming message. But this means that the exact assembly needed by the message has to exist on the recipient side. I call this tight coupling because the sender and recipient have to agree on code, not just on data, and make sure that the right code is in the right places.
Figure 12 Recipient Using Binary Formatter
Figure 10 Sending MSMQ Message with Binary Formatter
Figure 11 Serialized Message
The ActiveX formatter is used when sending COM objects, which is less useful when working with the .NET Framework, so I won't be covering the use of that object in this article.
MSMQ and .NET Framework Trust
The .NET Framework provides code access security that allows an administrator to specify which operations an assembly is and isn't allowed to perform. For a fuller explanation of this, see Don Box's article "
Security in .NET: The Security Infrastructure of the CLR Provides Evidence, Policy, Permissions, and Enforcement Services
" in the September 2002 issue of MSDN Magazine. This control is often quite finely grained. For example, the default settings that specify an assembly's access to system environment variables allow code that runs from your local machine to read or write any of them; code that comes from the local intranet to have read-only access to the Username variable; and code that comes from the Internet to make no use of environment variables at all. The administrator can also concoct any combination of access permissions that they desire in a custom permission set, and specify which assemblies have the rights granted in that set.
The code access security permissions for the use of MSMQ are much more coarsely grained. The immediate caller of any MSMQ method needs to have the Full Trust permission set or the .NET Framework will throw a security exception. There's no explicit "Message Queuing" permission that can be granted to one assembly over another. There's also no fine-grained control that would allow an administrator to construct a permission set that would allow, say, read/write access to queue A, read-only access to queue B, and no access to any other queues. This coarse-grainedness is the automatic behavior of any shared assembly compiled without the AllowPartiallyTrustedCallers attribute. If you use ILDASM to examine System.Messaging.dll, you'll find it absent, as opposed to, say, System.Windows.Forms.dll, on which it is present and which does not enforce such restrictions on its callers.
For logistical convenience, it's pretty common to distribute production code to users from a local intranet server or even an Internet server. The full trust requirement thus poses difficulties when you want an assembly that you obtain in this manner to use MSMQ. You probably don't want to grant full trust to any random assembly from the Internet, so you have to create a new code group to which your specific MSMQ-enabled assembly will belong, and to which you grant full trust. You will want the membership condition of this code group to be as explicit as possible—being signed with a strong name that ensures it came from a trusted company, or perhaps that it came from a specific Web site that you know your administrator has control over.
Sending MSMQ Messages over the Internet
One of the most exciting things about MSMQ 3.0 (the version included in Windows XP and Windows Server™ 2003) is that you can use HTTP and SOAP to send MSMQ messages over the Internet, instead of requiring a dedicated Microsoft network connection. The basics aren't difficult. In fact, you can configure the simplest example program, which I showed in the previous section, to do this just by changing the queue names.
I've written a sample program that demonstrates the use of MSMQ over the Internet. You'll find it in the InternetQueueing subdirectory of the code download. It uses the queue created by the simplest example program. The only difference between this sending application and the simplest sender is the name of the queue. You see in the figure that I've prepended the string "FormatName:Direct=http://localhost/msmq/" to the name of the queue, replacing the machine name. The substring "localhost" represents the name of the server, here configured to run on a single machine. That's all the sender has to do to tell MSMQ to send the message via SOAP and HTTP. (Note that you can't open a queue for receive using the HTTP format name in Windows Server 2003 or Windows XP SP2.)
Similarly, the sample recipient shown in Figure 13 executes the same code to receive MSMQ messages from the Internet, as did the simplest sample program. The only change is in the name of the queue. So how do I know that the message came over HTTP and SOAP, and what's that SOAP-looking thing in the sample program's window? The native MSMQ API supports a property called SoapEnvelope, which represents the outer SOAP packet containing the message and its routing information. That's what I show in the sample program's display window. However, the .NET MSMQ wrapper does not expose this property. To display it to you, I had to import the COM-based MSMQ wrapper which does expose it, remove the message out of the queue into a COM-based wrapper, and fetch the SoapEnvelope property from that. I don't know why this property has been omitted. In any case, I won't bother listing the code that does this as it's not an important feature for most users, but it's in the online sample if you're interested.
Figure 13 Sample Recipient
I'll quickly discuss the SOAP envelope, but only as a matter of academic interest since it's unlikely you'll need to fiddle with it. The whole point of the .NET wrapper is to hide these details from you. But it's often nice to have at least some notion of what lower layers are doing on your behalf. You can see that the SOAP envelope contains a <Header> element, which is the standard SOAP place to put infrastructure-specific information (as opposed to the <Body> element, which is used for application-specific information). This in turn contains the <path> element, the namespace of which tells you that it's part of the WS-Routing protocol. The <action>, <to>, and <id> elements are interpreted according to it. In the <to> element you see the name of the destination queue. The <properties> element belongs to the SOAP Reliable Message Protocol (SRMP). The <sentAt> and <expiresAt> elements contain information used by MSMQ to support its properties, such as SentTime, TimeToBeReceived, and timeToReachQueue message properties. Finally, the <MSMQ> element contains such MSMQ-specific data as the <Priority> element. You'll note that the message contents aren't sent in the SOAP packet. They're sent as SOAP attachments, probably in order to save space.
Security is a top priority—arguably the top priority—for any computer connected to the Internet. Before the inclusion of HTTP messaging in version 3.0, MSMQ usually ran on dedicated Microsoft-only subnets over which the administrator had a great deal of control. Both sender and recipient were usually part of the same trust domain instead of random Internet surfers-by. When you want to face your MSMQ server outwards to the anarchic Web, you need to ensure that you close every possible security hole. That's why MSMQ now supports Hardened MSMQ mode. You turn it on through the user interface shown in Figure 14, which toggles a registry setting. When you turn this mode on, MSMQ stops listening on any incoming port, thereby avoiding any kind of direct attack. (I'm reminded of the Greek philosopher Socrates, asking his friends if he could persuade them not to drag him out drinking one night. They say that he can't because they simply won't listen to him.) Instead, it accepts messages only from IIS, which accepts them over the Web and places them into the incoming queues on the server machine. All outgoing queues other than those that use HTTP are locked. Messages can be placed in them, but they won't actually be sent until the hardened mode is cancelled (which may be never, so don't plan on this).
Figure 14 Security Modes
While I'm on the topic of security, I'll discuss how the privacy and encryption requirements have changed from the previous inward-facing case to the new outward-facing global Internet case. When all the communicating parties are members of the same trust domain, you can take advantage of the MSMQ built-in authentication mechanism that generates a certificate using the sending process's logged-on identity. However, when you're on the open Internet, it's less likely that a long-term intimate relationship exists between sender and recipient, and more likely that you're talking to random strangers. The sender may well not have a login ID on the recipient's domain and it's entirely reasonable that your systems designs would now not rely on the MSMQ built-in authentication mechanism, but instead would handle authentication as part of the business process. You could also send a user ID and password as part of the message contents.
Message privacy is always a large consideration, all the more so if your messages are now carrying authentication credentials. You need to encrypt everything all the time and the easiest way to do this when using MSMQ over the Internet is to use HTTPS. Simply prefix the queue name with HTTPS rather than HTTP, and configure the server with an identity certificate from a trust provider.
While it's beyond the scope of this article, MSMQ's support of HTTP is highly flexible, so you can tweak it as much as you need to. For example, you can specify the use of proxy servers and set up automatic redirections of messages from one queue to another. For more detail on this advanced messaging topic, read the Platform SDK article "
Delivering Messages Sent over the Internet
" in the MSDN Library.
Conclusion
MSMQ is a useful tool that slots well into .NET Framework-based applications. It provides easy access for programmers who want simple default functionality. It also provides a high degree of fine-grained control for programmers who need to further tune its operation. Its operation over HTTP networks provides a greater degree of freedom than has ever before been possible for loosely coupled systems.