Businesses often find themselves stuck with delays between data transfer among several applications. For example, you may find an order processing application slow as the third-party resources involved in it can be unavailable at times. Messaging Architecture & development tool like MSMQ is ideal for these kinds of scenarios.
MSMQ(Microsoft Message Queuing) provides a distributed and decoupled way of sending and receiving messages between applications. MSMQ acts as a queue manager that easily decides when applications should be isolated and work even if other applications they interact with are down or unavailable.
In this article, you will learn how about Microsoft MSMQ in detail.
What is Microsoft MSMQ?
Microsoft Message Queuing is a messaging infrastructure and development technology that allows two or more applications to send messages to each other without having to immediately know the results. Developed for the Windows Operating System, this development tool acts as a queue manager that assists you in sending objects (serialized) to the queue where they will stay until you receive them. MSMQ allows you to interact between remote machines, even over the Internet in a decoupled way.
What is the purpose of Microsoft MSMQ?
Using MSMQ, you can allow several applications running at different times to talk with each other across a combination of heterogeneous networks(a network that uses multiple network architectures and operating systems) and systems that may be temporarily offline. MSMQ provides a messaging system that facilitates a queue to hold messages that are generated by multiple sending applications and read by multiple receiving applications.
Key Applications of Microsoft MSMQ
MSMQ ensures message delivery, effective routing, security, and priority-based messaging. Owing to MSMQ’s flexibility, you can perform both asynchronous and synchronous messaging with high performance. Check out the following scenarios where you can employ MSMQ to your advantage:
- Financial Services: Ensure fluent & secured message delivery for electronic commerce use cases.
- Portable Applications: Effective communication between embedded systems in an automatic baggage system that routes baggage through airports.
- Seamless Sales: Empower sales automation applications for traveling sales representatives with efficient messaging.
- Fluent Workflow: MSMQs provide you with the ability to build smooth workflows that update your systems. You can design an agent that interacts with each system and also minimizes the effect of any changes done in one system on others. It loosens the coupling between different systems and makes it super easy to upgrade individual systems.
Key Types of Message Queuing Applications
You can classify the Message Queuing Applications into the following 2 categories:
Receiving Applications
MSMQ’s Receiving applications can read the message from a queue using 2 different methods. It can either read the message & leave it there in the queue or extract the message, thereby taking out the message from the queue. It is to be noted that you don’t require access to the directory service as messages can be sent to the queue and read from the queue without it.
Sending Applications
The Sending applications send the messages directly to the queue and not to the receiving application. This removes any dependency on the receiving application to be available for receiving the message from the sending application.
Understanding Microsoft MSMQ Operations
To understand how MSMQ can simplify your messaging workflow, you can go through the following basic MSMQ operations:
Creating a Queue
To start with MSMQ, you can begin with creating a simple public or private queue based on the queue path name provided. This example introduces you to a basic application-defined function that returns the public or private format name of the queue created and the length of the format name including the null-terminating character.
While defining a queue, you can go through the following set of simple steps:
- Step 1: To start with MSMQ, begin with defining the MQQUEUEPROPS structure.
- Step 2: Now, configure the queue properties. For this example, the following properties are considered:
- Step 3: After setting the properties, start initializing the MQQUEUEPROPS structure.
- Step 4: Finally, call the MQCreateQueue function to create the queue.
#include "windows.h"
#include "mq.h"
#include "tchar.h"
#define BUFLEN = 256 ;
HRESULT CreateMSMQQueue(
LPWSTR wszPathName,
PSECURITY_DESCRIPTOR pSecurityDescriptor,
LPWSTR wszOutFormatName,
DWORD *pdwOutFormatNameLength
)
{
// Setting the maximum number of queue properties to 2.
const int NUMBEROFPROPERTIES = 2;
// Build the queue property structure and the structures required to initialize it.
MQQUEUEPROPS QueueProps;
MQPROPVARIANT aQueuePropVar[NUMBEROFPROPERTIES];
QUEUEPROPID aQueuePropId[NUMBEROFPROPERTIES];
HRESULT aQueueStatus[NUMBEROFPROPERTIES];
HRESULT hr = MQ_OK;
// Check the input parameters.
if (wszPathName == NULL || wszOutFormatName == NULL || pdwOutFormatNameLength == NULL)
{
return MQ_ERROR_INVALID_PARAMETER;
}
// Configure the queue properties.
DWORD cPropId = 0;
aQueuePropId[cPropId] = PROPID_Q_PATHNAME;
aQueuePropVar[cPropId].vt = VT_LPWSTR;
aQueuePropVar[cPropId].pwszVal = wszPathName;
cPropId++;
WCHAR wszLabel[MQ_MAX_Q_LABEL_LEN] = L"Test Queue";
aQueuePropId[cPropId] = PROPID_Q_LABEL;
aQueuePropVar[cPropId].vt = VT_LPWSTR;
aQueuePropVar[cPropId].pwszVal = wszLabel;
cPropId++;
// Initialize the MQQUEUEPROPS structure.
QueueProps.cProp = cPropId; // Number of properties
QueueProps.aPropID = aQueuePropId; // IDs of the queue properties
QueueProps.aPropVar = aQueuePropVar; // Values of the queue properties
QueueProps.aStatus = aQueueStatus; // Pointer to the return status
// Call MQCreateQueue to create the queue.
WCHAR wszFormatNameBuffer[BUFLEN];
DWORD dwFormatNameBufferLength = BUFLEN;
hr = MQCreateQueue(pSecurityDescriptor, // Security descriptor
&QueueProps, // Address of queue property structure
wszFormatNameBuffer, // Pointer to format name buffer
&dwFormatNameBufferLength); // Pointer to receive the queue's format name length in Unicode characters not bytes.
// Return the format name if the queue is created successfully.
if (hr == MQ_OK || hr == MQ_INFORMATION_PROPERTY)
{
if (*pdwOutFormatNameLength >= dwFormatNameBufferLength)
{
wcsncpy_s(wszOutFormatName, *pdwOutFormatNameLength - 1, wszFormatNameBuffer, _TRUNCATE);
// ************************************
// You must copy wszFormatNameBuffer into the
// wszOutFormatName buffer.
// ************************************
wszOutFormatName[*pdwOutFormatNameLength - 1] = L'';
*pdwOutFormatNameLength = dwFormatNameBufferLength;
}
else
{
wprintf(L"The queue was created, but its format name cannot be returned.n");
}
}
return hr;
}
If you try to pass a NULL pointer to a SECURITY_DESCRIPTOR structure to this function in pSecurityDescriptor, you will eventually get a queue with the default security descriptor.
Locating Queues
Using the properties of a queue, you can easily query the directory service for a set of public queues in MSMQ. For instance, you can query a directory service for all the public queues with a specific label or a specific queue identifier. You can also locate private queues in MSMQ 3.0.
Consider the following image shown below that depicts a query that retrieves 2 queues based on the service type GUID. Here, the calling application opens up the queue to send or reds a message. You can use this type of query for helping with load balancing when you need to send a message to a queue on a separate machine.
Image Source
The calling application must follow a defined process to perform a query depending on following type of calls made:
- API function calls: Fo locating a query, you can define a condition based on a valid queue property other than the queue pathname. The calling application must mention the queue property that MSMQ returns as a query result.
- COM Methods calls: For this COM, your search criteria are limited to queue labels, queue identifiers, service types, creation times, modification times, and multicast addresses. MSMQ returns an MSMQQueueInfos object that defines a query, queries a directory service, and provides a method that returns an MSMQQueueInfo object for each queue found in the query.
You can use the following API functions and COM methods to locate public queues:
Opening Queues
Another important MSMQ operation is opening a queue. It is essential for an application to first open a queue before starting sending messages to a queue or receiving messages from a queue. Once your application has opened a queue, MSMQ returns a queue handle that allows you the application to execute the following tasks:
- Send messages to one or more (in MSMQ 3.0) destination queues.
- Look at or extract messages from a single queue.
- Create cursors for navigating through the queue (Message Queuing COM components provide a single implicit cursor each time a queue is open).
- Create format names.
- Close the queue.
While making the call to open a queue, an application has to ensure to mention the following 3 details:
- The queue format name.
- The access mode.
- The share mode.
Navigating Queues
When you are trying to extract a message, the receiving application can use the cursor or the message lookup identifier (introduced in MSMQ 3.0) to navigate the messages in the queue. For cases when you use the cursor, the receiving application starts at the beginning of the queue and moves to the end of the queue in order. If you use a message lookup ID, the receiving application can start at the beginning of the queue, the message associated with a particular lookup ID, or the end of the queue. After reading the message via the message identifier, the receiving application can move to the top or bottom of the queue.
Understanding Microsoft MSMQ Functions
MSMQ provides a robust architecture that allows you to manage queues and messages for Message Queuing applications. The Message Queuing API contains functions for:
- Creating, opening, and deleting queues.
- To find existing queues and messages in the queue.
- To send and read messages in the queue.
- To set and extract properties.
You can go through the following list of functions:
MQCreateCursor
Using the MQCreateCursor function, you can create a cursor for a particular queue and returns a handle to the cursor. This cursor is enables you to maintain a certain location in a queue when reading the queue’s messages. The sample syntax for this function is as follows:
HRESULT APIENTRY MQCloseCursor(
HANDLE hCursor
);
This function has 2 parameters:
- queue: This represents a handle to the queue for which you want to create a cursor.
- phCursor: It denotes a pointer to a variable that receives the resulting cursor handle.
MQCloseCursor
This MSMQ function closes a particular cursor specified by you and frees up the resources related to that cursor.
You can use this function via the following syntax:
HRESULT APIENTRY MQCloseCursor(
HANDLE hCursor
);
This function has only 1 parameter hCursor that represents a handle to the cursor you want to close.
MQFreeMemory
You can employ this MSMQ function to free up memory allocated by Message Queuing. You execute this function by using the following syntax:
Copy
VOID MQFreeMemory(
PVOID pvMemory
);
This function has only 1 parameter pvMemory that demonstrates a pointer to the memory to be freed.
MQSetQueueProperties
Using this MSMQ function, you can configure the properties of a particular queue.
Copy
HRESULT APIENTRY MQSetQueueProperties(
LPCWSTR lpwcsFormatName,
MQQUEUEPROPS * pQueueProps
);
This function has the following 2 parameters:
- lpwcsFormatName: This acts as a pointer to the format name string of the queue whose properties you want to configure. You can use a public or private format name to specify the queue. You can only use a direct format name to configure the properties of a local private queue.
- pQueueProps: It is a pointer to the MQQUEUEPROPS structure that mentions the properties you want to configure.
For the input, the cProp member of MQQUEUEPROPS mentions the number of properties you want to configure, the aPropID array helps in mentioning their property identifiers, and the aPropVar array indicates the new values of the particular properties.
For the output, the optional aStatus array(if included in MQQUEUEPROPS
)indicates the status of the properties.
MQGetMachineProperties
This MSMQ function extracts details about a queue manager computer. You can create this function via the following syntax:
Copy
HRESULT APIENTRY MQGetMachineProperties(
LPCWSTR lpwcsMachineName,
const GUID * pguidMachineID,
MQQMPROPS * pQMProps
);
This function has the following parameters:
- lpwcsMachineName: This is the name of the queue manager computer you want to access. You have to mention the NetBIOS or DNS name of the computer. If you are parameter is used, set pguidMachineID to NULL.
- pguidMachineID: The identifier of the queue manager computer you want to access. If this parameter is used, set lpwcsMachineName to NULL.
- pQMProps: It is a pointer to a queue manager properties structure (MQQMPROPS) mentioning which properties to extract.
Apart from these 5 functions, you can go through the following table to learn about more MSMQ functions:
| Function | Description |
| MQADsPathToFormatName | Translates the ADs path of a queue, queue alias, or distribution list into a format name. |
| MQBeginTransaction | Creates an internal Message Queuing transaction object that can be used to send messages to a queue or read messages from a queue. |
| MQCloseQueue | Closes a given queue. |
| MQCreateCursor | Creates a cursor for a specific queue and returns its handle. |
| MQCreateQueue | Creates a public or private queue. |
| MQDeleteQueue | Deletes a queue from the directory service or from the local computer. |
| MQFreeSecurityContext | Frees the memory allocated by MQGetSecurityContext. |
| MQGetOverlappedResult | Retrieves the Message Queuing receives success or error code from the OVERLAPPED structure used to retrieve a message asynchronously. |
| MQGetPrivateComputerInformation | Retrieves information about the local computer without accessing the directory service. |
| MQGetQueueProperties | Retrieves the specified set of properties for a specific queue. |
| MQGetQueueSecurity | Retrieves information from the access control security descriptor of the specified queue. |
| MQGetSecurityContext | (Superseded by MQGetSecurityContextEx.) Retrieves security information needed to attach a certificate to a message for authentication. |
| MQGetSecurityContextEx | (Introduced in Windows® 2000 Service Pack 2, supersedes MQGetSecurityContext.) Retrieves the security information needed to attach a certificate to a message for authentication. |
| MQHandleToFormatName | Returns a format name for the queue based on its handle. |
| MQInstanceToFormatName | Returns a format name for the queue based on the identifier provided. |
| MQLocateBegin | Starts a query to locate a public queue, returning a query handle. |
| MQLocateEnd | Ends a query, releasing the resources associated with the query. |
| MQLocateNext | Retrieves the requested queue information from the query. |
| MQMgmtAction | (Introduced in MSMQ 3.0.) Changes the state of a computer or a queue. |
| MQMgmtGetInfo | (Introduced in MSMQ 3.0.) Retrieves information about a computer or a queue. |
| MQOpenQueue | Opens a queue for sending, peeking at, retrieving, or purging messages. |
| MQPathNameToFormatName | Returns a format name based on the Message Queuing path name provided. |
| MQPurgeQueue | (Introduced in MSMQ 3.0.) Deletes all of the messages in a queue. |
| MQReceiveMessage | Allows you to read messages in the queue. |
| MQReceiveMessageByLookupId | (Introduced in MSMQ 3.0.) Peeks at or retrieves messages in the queue based on message lookup identifiers. |
| MQSendMessage | Sends a message to the queue corresponding to the handle hDestinationQueue. |
| MQSetQueueSecurity | Sets the access control security descriptor of a queue. |
| MQMoveMessage | Moves messages between a queue and its subqueue, or between two subqueues within the same main queue. |
| MQMarkMessageRejected | Marks a message in a queue as rejected. |
Conclusion
In this article, you have learned about MSMQ and how it works. MSMQ guarantees a secured, in-time, and priority-based message delivery. Offering both asynchronous and synchronous messaging, MSMQ is free and comes with Windows, but is not installed by default. Using MSMQ functions, you can perform operations such as opening, creating, & deleting queues. You can also look at or extract messages from a single queue, create cursors for navigating through the queue, create format names and close a queue.
As you collect and manage your data across several applications and databases in your business, it is important to consolidate it for complete performance analysis of your business. However, it is a time-consuming and resource-intensive task to continuously monitor the Data Connectors. To achieve this efficiently, you need to assign a portion of your engineering bandwidth to Integrate data from all sources, Clean & Transform it, and finally, Load it to a Cloud Data Warehouse, BI Tool, or a destination of your choice for further Business Analytics. All of these challenges can be comfortably solved by a Cloud-based ETL tool such as Hevo Data.
Visit our Website to Explore Hevo
Hevo Data, a No-code Data Pipeline can transfer data in Real-Time from a vast sea of 100+ sources to a Data Warehouse, BI Tool, or a Destination of your choice. It is a reliable, completely automated, and secure service that doesn’t require you to write any code!
If you are using CRMs, Sales, HR, and Marketing applications and searching for a no-fuss alternative to Manual Data Integration, then Hevo can effortlessly automate this for you. Hevo, with its strong integration with 100+ sources and BI tools(Including 40+ Free Sources), allows you to not only export & load data but also transform & enrich your data & make it analysis-ready in a jiffy.
Share your experience of learning about MSMQ! Let us know in the comments section below!
Sanchit Agarwal is an Engineer turned Data Analyst with a passion for data, software architecture and AI. He leverages his diverse technical background and 2+ years of experience to write content. He has penned over 200 articles on data integration and infrastructures, driven by a desire to empower data practitioners with practical solutions for their everyday challenges.
