What is Microsoft MSMQ? | Comprehensive Guide for 2023

• May 9th, 2022

msmq - Featured Image

Businesses often find themselves stuck with delays between data transfer among several applications. For example, you may find an order processing application slow as the third-party resources involved in it can be unavailable at times. Messaging Architecture & development tool like MSMQ is ideal for these kinds of scenarios.

MSMQ(Microsoft Message Queuing) provides a distributed and decoupled way of sending and receiving messages between applications. MSMQ acts as a queue manager that easily decides when applications should be isolated and work even if other applications they interact with are down or unavailable.

In this article, you will learn how about Microsoft MSMQ in detail.

Table of Contents

What is Microsoft MSMQ?

MSMQ - MSMQ basic working structure
Image Source

Microsoft Message Queuing is a messaging infrastructure and development technology that allows two or more applications to send messages to each other without having to immediately know the results. Developed for the Windows Operating System, this development tool acts as a queue manager that assists you in sending objects (serialized) to the queue where they will stay until you receive them. MSMQ allows you to interact between remote machines, even over the Internet in a decoupled way. 

What is the purpose of Microsoft MSMQ?

Using MSMQ, you can allow several applications running at different times to talk with each other across a combination of heterogeneous networks(a network that uses multiple network architectures and operating systems) and systems that may be temporarily offline. MSMQ provides a messaging system that facilitates a queue to hold messages that are generated by multiple sending applications and read by multiple receiving applications.

msmq - working of msmq
Image Source

Key Applications of Microsoft MSMQ

MSMQ ensures message delivery, effective routing, security, and priority-based messaging. Owing to MSMQ’s flexibility, you can perform both asynchronous and synchronous messaging with high performance. Check out the following scenarios where you can employ MSMQ to your advantage:

  • Financial Services: Ensure fluent & secured message delivery for electronic commerce use cases.
  • Portable Applications: Effective communication between embedded systems in an automatic baggage system that routes baggage through airports.
  • Seamless Sales: Empower sales automation applications for traveling sales representatives with efficient messaging.
  • Fluent Workflow: MSMQs provide you with the ability to build smooth workflows that update your systems. You can design an agent that interacts with each system and also minimizes the effect of any changes done in one system on others. It loosens the coupling between different systems and makes it super easy to upgrade individual systems.

Load your data in Minutes Using Hevo’s No-Code Data Pipeline

Hevo Data, a Fully-managed No-Code Data Pipeline, can help you automate, simplify & enrich your data ingestion and integration process in a few clicks. With Hevo’s out-of-the-box connectors & complete support for Change Data Capture, you can load data in real-time from 100+ Data Sources(including 40+ free data sources) and load it straight into your Data Warehouse, Databases, or any destination. To further streamline and prepare your data for analysis, you can process and enrich Raw Granular Data using Hevo’s robust & built-in Transformation Layer without writing a single line of code!”

Get Started with Hevo for Free

Hevo is the fastest, easiest, and most reliable data replication platform that will save your engineering bandwidth and time multifold. Try our 14-day full access free trial today to experience an entirely automated hassle-free Data Replication!

Key Types of Message Queuing Applications

You can classify the Message Queuing Applications into the following 2 categories:

Receiving Applications

MSMQ’s Receiving applications can read the message from a queue using 2 different methods. It can either read the message & leave it there in the queue or extract the message, thereby taking out the message from the queue. It is to be noted that you don’t require access to the directory service as messages can be sent to the queue and read from the queue without it.

Sending Applications

The Sending applications send the messages directly to the queue and not to the receiving application. This removes any dependency on the receiving application to be available for receiving the message from the sending application. 

Understanding Microsoft MSMQ Operations

To understand how MSMQ can simplify your messaging workflow, you can go through the following basic MSMQ operations:

Creating a Queue

To start with MSMQ, you can begin with creating a simple public or private queue based on the queue path name provided. This example introduces you to a basic application-defined function that returns the public or private format name of the queue created and the length of the format name including the null-terminating character.

While defining a queue, you can go through the following set of simple steps:

  • Step 1:  To start with MSMQ, begin with defining the MQQUEUEPROPS structure.
  • Step 2: Now, configure the queue properties. For this example, the following properties are considered: 
  • Step 3: After setting the properties, start initializing the MQQUEUEPROPS structure.
  • Step 4: Finally, call the MQCreateQueue function to create the queue.
#include "windows.h"  
#include "mq.h"  
#include "tchar.h"  
#define BUFLEN = 256 ;  
  
HRESULT CreateMSMQQueue(  
                        LPWSTR wszPathName,   
                        PSECURITY_DESCRIPTOR pSecurityDescriptor,  
                        LPWSTR wszOutFormatName,  
                        DWORD *pdwOutFormatNameLength  
                        )  
{  
  
  // Setting the maximum number of queue properties to 2.
  const int NUMBEROFPROPERTIES = 2;  
  
  // Build the queue property structure and the structures required to initialize it.  
  MQQUEUEPROPS   QueueProps;  
  MQPROPVARIANT  aQueuePropVar[NUMBEROFPROPERTIES];  
  QUEUEPROPID    aQueuePropId[NUMBEROFPROPERTIES];  
  HRESULT        aQueueStatus[NUMBEROFPROPERTIES];  
  HRESULT        hr = MQ_OK;  
  
  // Check the input parameters.  
  if (wszPathName == NULL || wszOutFormatName == NULL || pdwOutFormatNameLength == NULL)  
  {  
    return MQ_ERROR_INVALID_PARAMETER;  
  }  
  
  // Configure the queue properties.  
  DWORD cPropId = 0;  
  aQueuePropId[cPropId] = PROPID_Q_PATHNAME;  
  aQueuePropVar[cPropId].vt = VT_LPWSTR;  
  aQueuePropVar[cPropId].pwszVal = wszPathName;  
  cPropId++;  
  
  WCHAR wszLabel[MQ_MAX_Q_LABEL_LEN] = L"Test Queue";  
  aQueuePropId[cPropId] = PROPID_Q_LABEL;  
  aQueuePropVar[cPropId].vt = VT_LPWSTR;  
  aQueuePropVar[cPropId].pwszVal = wszLabel;  
  cPropId++;  
  
  // Initialize the MQQUEUEPROPS structure.  
  QueueProps.cProp = cPropId;               // Number of properties  
  QueueProps.aPropID = aQueuePropId;        // IDs of the queue properties  
  QueueProps.aPropVar = aQueuePropVar;      // Values of the queue properties  
  QueueProps.aStatus = aQueueStatus;        // Pointer to the return status  
  
  // Call MQCreateQueue to create the queue.  
  WCHAR wszFormatNameBuffer[BUFLEN];  
  DWORD dwFormatNameBufferLength = BUFLEN;  
  hr = MQCreateQueue(pSecurityDescriptor,         // Security descriptor  
                     &QueueProps,                 // Address of queue property structure  
                     wszFormatNameBuffer,         // Pointer to format name buffer  
                     &dwFormatNameBufferLength);  // Pointer to receive the queue's format name length in Unicode characters not bytes.  
  
  // Return the format name if the queue is created successfully.  
  if (hr == MQ_OK || hr == MQ_INFORMATION_PROPERTY)  
  {  
    if (*pdwOutFormatNameLength >= dwFormatNameBufferLength)  
    {  
      wcsncpy_s(wszOutFormatName, *pdwOutFormatNameLength - 1, wszFormatNameBuffer, _TRUNCATE);  
      // ************************************  
      // You must copy wszFormatNameBuffer into the   
      // wszOutFormatName buffer.  
      // ************************************  
      wszOutFormatName[*pdwOutFormatNameLength - 1] = L'';  
      *pdwOutFormatNameLength = dwFormatNameBufferLength;  
    }  
    else  
    {  
      wprintf(L"The queue was created, but its format name cannot be returned.n");  
    }  
  }  
  return hr;  
}

If you try to pass a NULL pointer to a SECURITY_DESCRIPTOR structure to this function in pSecurityDescriptor, you will eventually get a queue with the default security descriptor

Locating Queues

Using the properties of a queue, you can easily query the directory service for a set of public queues in MSMQ. For instance, you can query a directory service for all the public queues with a specific label or a specific queue identifier. You can also locate private queues in MSMQ 3.0. 

Consider the following image shown below that depicts a query that retrieves 2 queues based on the service type GUID. Here, the calling application opens up the queue to send or reds a message. You can use this type of query for helping with load balancing when you need to send a message to a queue on a separate machine.

msmq - Locating Queues
Image Source

The calling application must follow a defined process to perform a query depending on following type of calls made:

  • API function calls: Fo locating a query, you can define a condition based on a valid queue property other than the queue pathname. The calling application must mention the queue property that MSMQ returns as a query result. 
  • COM Methods calls: For this COM, your search criteria are limited to queue labels, queue identifiers, service types, creation times, modification times, and multicast addresses. MSMQ returns an MSMQQueueInfos object that defines a query, queries a directory service, and provides a method that returns an MSMQQueueInfo object for each queue found in the query.

You can use the following API functions and COM methods to locate public queues:

Opening Queues

Another important MSMQ operation is opening a queue. It is essential for an application to first open a queue before starting sending messages to a queue or receiving messages from a queue. Once your application has opened a queue, MSMQ returns a queue handle that allows you the application to execute the following tasks:

  • Send messages to one or more (in MSMQ 3.0) destination queues.
  • Look at or extract messages from a single queue.
  • Create cursors for navigating through the queue (Message Queuing COM components provide a single implicit cursor each time a queue is open).
  • Create format names.
  • Close the queue.

While making the call to open a queue, an application has to ensure to mention the following 3 details:

  • The queue format name.
  • The access mode.
  • The share mode.

Navigating Queues

When you are trying to extract a message, the receiving application can use the cursor or the message lookup identifier (introduced in MSMQ 3.0) to navigate the messages in the queue. For cases when you use the cursor, the receiving application starts at the beginning of the queue and moves to the end of the queue in order. If you use a message lookup ID, the receiving application can start at the beginning of the queue, the message associated with a particular lookup ID, or the end of the queue. After reading the message via the message identifier, the receiving application can move to the top or bottom of the queue.

What Makes Hevo’s ETL Process Best-In-Class

Providing a high-quality ETL solution can be a difficult task if you have a large volume of data. Hevo’s automated, No-code platform empowers you with everything you need to have for a smooth data replication experience.

Check out what makes Hevo amazing:

  • Fully Managed: Hevo requires no management and maintenance as it is a fully automated platform.
  • Data Transformation: Hevo provides a simple interface to perfect, modify, and enrich the data you want to transfer.
  • Faster Insight Generation: Hevo offers near real-time data replication so you have access to real-time insight generation and faster decision making. 
  • Schema Management: Hevo can automatically detect the schema of the incoming data and map it to the destination schema.
  • Scalable Infrastructure: Hevo has in-built integrations for 100+ sources (with 40+ free sources) that can help you scale your data infrastructure as required.
  • Live Support: Hevo team is available round the clock to extend exceptional support to its customers through chat, email, and support calls.
Sign up here for a 14-Day Free Trial!

Understanding Microsoft MSMQ Functions 

MSMQ provides a robust architecture that allows you to manage queues and messages for Message Queuing applications. The Message Queuing API contains functions for:

  • Creating, opening, and deleting queues.
  • To find existing queues and messages in the queue.
  • To send and read messages in the queue. 
  • To set and extract properties.

You can go through the following list of functions:

MQCreateCursor

Using the MQCreateCursor function, you can create a cursor for a particular queue and returns a handle to the cursor. This cursor is enables you to maintain a certain location in a queue when reading the queue’s messages. The sample syntax for this function is as follows:

HRESULT APIENTRY MQCloseCursor(  
  HANDLE hCursor    
);

This function has 2 parameters:

  • queue: This represents a handle to the queue for which you want to create a cursor.
  • phCursor: It denotes a pointer to a variable that receives the resulting cursor handle.

MQCloseCursor

This MSMQ function closes a particular cursor specified by you and frees up the resources related to that cursor. 

You can use this function via the following syntax:

HRESULT APIENTRY MQCloseCursor(  
  HANDLE hCursor    
);

This function has only 1 parameter hCursor that represents a handle to the cursor you want to close.

MQFreeMemory

You can employ this MSMQ function to free up memory allocated by Message Queuing. You execute this function by using the following syntax:

Copy
VOID MQFreeMemory(  
  PVOID pvMemory    
);

This function has only 1 parameter pvMemory that demonstrates a pointer to the memory to be freed.

MQSetQueueProperties

Using this MSMQ function, you can configure the properties of a particular queue.

Copy
HRESULT APIENTRY MQSetQueueProperties(  
  LPCWSTR lpwcsFormatName,    
  MQQUEUEPROPS * pQueueProps    
);  

This function has the following 2 parameters: 

  • lpwcsFormatName: This acts as a pointer to the format name string of the queue whose properties you want to configure. You can use a public or private format name to specify the queue. You can only use a direct format name to configure the properties of a local private queue.
  • pQueueProps: It is a pointer to the MQQUEUEPROPS structure that mentions the properties you want to configure.

For the input, the cProp member of MQQUEUEPROPS mentions the number of properties you want to configure, the aPropID array helps in mentioning their property identifiers, and the aPropVar array indicates the new values of the particular properties.

For the output, the optional aStatus array(if included in MQQUEUEPROPS

)indicates the status of the properties.

MQGetMachineProperties

This MSMQ function extracts details about a queue manager computer. You can create this function via the following syntax:

Copy
HRESULT APIENTRY MQGetMachineProperties(  
  LPCWSTR lpwcsMachineName,    
  const GUID * pguidMachineID,         
  MQQMPROPS * pQMProps           
); 

This function has the following parameters:

  • lpwcsMachineName: This is the name of the queue manager computer you want to access. You have to mention the NetBIOS or DNS name of the computer. If you are parameter is used, set pguidMachineID to NULL.
  • pguidMachineID: The identifier of the queue manager computer you want to access. If this parameter is used, set lpwcsMachineName to NULL.
  • pQMProps: It is a pointer to a queue manager properties structure (MQQMPROPS) mentioning which properties to extract.

Apart from these 5 functions, you can go through the following table to learn about more MSMQ functions:

FunctionDescription
MQADsPathToFormatNameTranslates the ADs path of a queue, queue alias, or distribution list into a format name.
MQBeginTransactionCreates an internal Message Queuing transaction object that can be used to send messages to a queue or read messages from a queue.
MQCloseQueueCloses a given queue.
MQCreateCursorCreates a cursor for a specific queue and returns its handle.
MQCreateQueueCreates a public or private queue.
MQDeleteQueueDeletes a queue from the directory service or from the local computer.
MQFreeSecurityContextFrees the memory allocated by MQGetSecurityContext.
MQGetOverlappedResultRetrieves the Message Queuing receives success or error code from the OVERLAPPED structure used to retrieve a message asynchronously.
MQGetPrivateComputerInformationRetrieves information about the local computer without accessing the directory service.
MQGetQueuePropertiesRetrieves the specified set of properties for a specific queue.
MQGetQueueSecurityRetrieves information from the access control security descriptor of the specified queue.
MQGetSecurityContext(Superseded by MQGetSecurityContextEx.) Retrieves security information needed to attach a certificate to a message for authentication.
MQGetSecurityContextEx(Introduced in Windows® 2000 Service Pack 2, supersedes MQGetSecurityContext.) Retrieves the security information needed to attach a certificate to a message for authentication.
MQHandleToFormatNameReturns a format name for the queue based on its handle.
MQInstanceToFormatNameReturns a format name for the queue based on the identifier provided.
MQLocateBeginStarts a query to locate a public queue, returning a query handle.
MQLocateEndEnds a query, releasing the resources associated with the query.
MQLocateNextRetrieves the requested queue information from the query.
MQMgmtAction(Introduced in MSMQ 3.0.) Changes the state of a computer or a queue.
MQMgmtGetInfo(Introduced in MSMQ 3.0.) Retrieves information about a computer or a queue.
MQOpenQueueOpens a queue for sending, peeking at, retrieving, or purging messages.
MQPathNameToFormatNameReturns a format name based on the Message Queuing path name provided.
MQPurgeQueue(Introduced in MSMQ 3.0.) Deletes all of the messages in a queue.
MQReceiveMessageAllows you to read messages in the queue.
MQReceiveMessageByLookupId(Introduced in MSMQ 3.0.) Peeks at or retrieves messages in the queue based on message lookup identifiers.
MQSendMessageSends a message to the queue corresponding to the handle hDestinationQueue.
MQSetQueueSecuritySets the access control security descriptor of a queue.
MQMoveMessageMoves messages between a queue and its subqueue, or between two subqueues within the same main queue.
MQMarkMessageRejectedMarks a message in a queue as rejected.

Conclusion

In this article, you have learned about MSMQ and how it works. MSMQ guarantees a secured, in-time, and priority-based message delivery. Offering both asynchronous and synchronous messaging, MSMQ is free and comes with Windows, but is not installed by default. Using MSMQ functions, you can perform operations such as opening, creating, & deleting queues. You can also look at or extract messages from a single queue, create cursors for navigating through the queue, create format names and close a queue.

As you collect and manage your data across several applications and databases in your business, it is important to consolidate it for complete performance analysis of your business. However, it is a time-consuming and resource-intensive task to continuously monitor the Data Connectors. To achieve this efficiently, you need to assign a portion of your engineering bandwidth to Integrate data from all sources, Clean & Transform it, and finally, Load it to a Cloud Data Warehouse, BI Tool, or a destination of your choice for further Business Analytics. All of these challenges can be comfortably solved by a Cloud-based ETL tool such as Hevo Data.   

Visit our Website to Explore Hevo

Hevo Data, a No-code Data Pipeline can transfer data in Real-Time from a vast sea of 100+ sources to a Data Warehouse, BI Tool, or a Destination of your choice. It is a reliable, completely automated, and secure service that doesn’t require you to write any code!  

If you are using CRMs, Sales, HR, and Marketing applications and searching for a no-fuss alternative to Manual Data Integration, then Hevo can effortlessly automate this for you. Hevo, with its strong integration with 100+ sources and BI tools(Including 40+ Free Sources), allows you to not only export & load data but also transform & enrich your data & make it analysis-ready in a jiffy.

Want to take Hevo for a ride? Sign Up for a 14-day free trial and simplify your Data Integration process. Do check out the pricing details to understand which plan fulfills all your business needs.

Share your experience of learning about MSMQ! Let us know in the comments section below!

No-code Data Pipeline For Your Data Warehouse