Introduction


In case we face a challenge to build a Web Job that listens or monitors a queue in the Microsoft Azure Service Bus within a certain namespace to process each message that is sent there by a message producer this article will provide you a walk-through how to accomplish that. The Web Job in this case acts as a message consumer of the messages on the queue. Below a high level diagram of a scenario that will be explained in this article and how to face the challenge.


Picture 1. High Level Overview.

Steps to follow


The following steps demonstrate how to build, deploy and test the Web Job.
  

Step 1: Build Azure Web Job


We can build WebJobs inside Visual Studio by installing the WebJobs SDK. Once we have installed the SDK we have a template available to build a Web Job in C# or Visual Basic.



Picture 2. Visual Studio Templates.

We can select this template, specify a name for the Web Job and click Ok. We will see that a program class will be created.

namespace ServiceBusMonitorWebJob
{
    // To learn more about Microsoft Azure WebJobs SDK, please see http://go.microsoft.com/fwlink/?LinkID=320976
    class Program
    {
        // Please set the following connection strings in app.config for this WebJob to run:
        // AzureWebJobsDashboard and AzureWebJobsStorage
        static void Main()
        {
            var host = new JobHost();
            // The following code ensures that the WebJob will be running continuously
            host.RunAndBlock();
        }
    }
}
And a Functions.cs.
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Microsoft.Azure.WebJobs;
 
namespace WebJob1
{
    public class Functions
    {
        // This function will get triggered/executed when a new message is written
        // on an Azure Queue called queue.
        public static void ProcessQueueMessage([QueueTrigger("queue")] string message, TextWriter log)
        {
            log.WriteLine(message);
        }
    }
}

By default a method will be created for us to monitor or listen to an Azure Storage Queue, not Service Bus Queue! To have a method that will be triggered/executed when a message is written to an Azure Service Bus Queue we will need to have ServiceBusTrigger. This not available in the project and we will need to add the Microsoft.Azure.WebJobs.ServiceBus NuGet package.



Picture 3. Add NuGet Package Microsoft.Azure.WebJobs.ServiceBus.

Now a can change to ServiceBusTrigger in method ProcessQueueMessage and specify the queue I want to listen to.

public static void ProcessQueueMessage([ServiceBusTrigger("inboundqueue")] BrokeredMessage message, TextWriter log)

Next change is changing the type of the message from string type to BrokeredMessage type. This type is not available in our class unless we add using statement for Microsoft.ServiceBus.Messaging. The package is already in the project, because it is part of the imported NuGet package. The TextWriter object can be used to write log statements that can be viewed in the AzureWebJob Dashboard.

When a message arrives on inboundqueue it will be picked up by Web Job and enter the ProcessQueueMessage method at runtime. Here I can extract the message body and send it for instance to Redis cache as a key value pair. To send it to Redis Cache I need to import another NuGet Package i.e. StackExchange.Redis (client library). Now the complete code for the functions class looks like below:

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Microsoft.Azure.WebJobs;
using Microsoft.ServiceBus.Messaging;
using StackExchange.Redis;
using System.Configuration;
 
namespace ServiceBusMonitorWebJob
{
    public class Functions
    {
        private static Lazy<ConnectionMultiplexer> lazyConnection = new Lazy<ConnectionMultiplexer>(() =>
        {
            return ConnectionMultiplexer.Connect(ConfigurationManager.AppSettings["RedisConnectionString"]);
        });
 
        public static ConnectionMultiplexer Connection
        {
            get
            {
                return lazyConnection.Value;
            }
        }
 
        /// <summary>
        /// Method to Add Key/Value pair to Redis Cache
        /// </summary>
        /// <param name="key">Key as string</param>
        /// <param name="value">Value as string</param>
        static void AddToRedis(string key, string value)
        {
            IDatabase cache = Connection.GetDatabase();
 
            cache.StringSet(key, value);
        }
 
        // This function will get triggered/executed when a new message is written
        // on an Azure Queue called queue.
        public static void ProcessQueueMessage([ServiceBusTrigger("inboundqueue")] BrokeredMessage message, TextWriter log)
        {
            //Log
            log.WriteLine("Message picked up from inboundqueue : " + message.MessageId);
 
            //Retrieve the message body regardless of the content as a stream
            Stream stream = message.GetBody<Stream>();
            StreamReader reader = new StreamReader(stream);
            string s = reader.ReadToEnd();
 
            //Log
            log.WriteLine("Message Body : " + s);
 
            //Add to Redis Cache
            AddToRedis(message.MessageId, s);
        }
    }
}

Before the Web Job can be deployed to a WebApp a few configuration settings have to be done in the app.config. The connection strings for the AzureWebJobsDashboard and AzureWebJobsStorage need to be provided in the connectionStrings Section. These are required to view the log in Azure i.e. AzureWebJobsDashboard. The connection string that needs to be specified is the connection string to an Azure Storage account. The format is as follows:

DefaultEndpointsProtocol=https;AccountName=[Storage Account Name];AccountKey=[Access Key]

The other connection string that has to be provided is for the AzureWebJobsServiceBus. Format is:

Endpoint=sb://namespace.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=[Access Key]

Finally in the appSettings section the connection string for the Redis needs to be specified. Connection string has the format of:

namespace.redis.cache.windows.net,abortConnect=false,ssl=true,password=[password]

Once configuration is done the app.config will look like:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <connectionStrings>
    <add name="AzureWebJobsDashboard" connectionString="DefaultEndpointsProtocol=https;AccountName=mijnnuon2;AccountKey=xxxxxxxxxxxxxxxxxxxxxxxx" />
    <add name="AzureWebJobsStorage" connectionString="DefaultEndpointsProtocol=https;AccountName=mijnnuon2;AccountKey=yyyyyyyyyyyyyy" />
    <add name="AzureWebJobsServiceBus" connectionString="Endpoint=sb://dev-mijnnuon-servicebus.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=zzzzzzzzzzzzzzzzzzzzz"/>
  </connectionStrings>
    <startup>
    <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" />
    </startup>
  <runtime>
    <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
      <dependentAssembly>
        <assemblyIdentity name="Microsoft.WindowsAzure.Storage" publicKeyToken="31bf3856ad364e35" culture="neutral" />
        <bindingRedirect oldVersion="0.0.0.0-4.2.1.0" newVersion="4.2.1.0" />
      </dependentAssembly>
      <dependentAssembly>
        <assemblyIdentity name="Newtonsoft.Json" publicKeyToken="30ad4fe6b2a6aeed" culture="neutral" />
        <bindingRedirect oldVersion="0.0.0.0-6.0.0.0" newVersion="6.0.0.0" />
      </dependentAssembly>
      <dependentAssembly>
        <assemblyIdentity name="Microsoft.ServiceBus" publicKeyToken="31bf3856ad364e35" culture="neutral" />
        <bindingRedirect oldVersion="0.0.0.0-2.7.0.0" newVersion="2.7.0.0" />
      </dependentAssembly>
    </assemblyBinding>
  </runtime>
  <system.serviceModel>
    <extensions>
      <!-- In this extension section we are introducing all known service bus extensions. User can remove the ones they don't need. -->
      <behaviorExtensions>
        <add name="connectionStatusBehavior" type="Microsoft.ServiceBus.Configuration.ConnectionStatusElement, Microsoft.ServiceBus, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
        <add name="transportClientEndpointBehavior" type="Microsoft.ServiceBus.Configuration.TransportClientEndpointBehaviorElement, Microsoft.ServiceBus, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
        <add name="serviceRegistrySettings" type="Microsoft.ServiceBus.Configuration.ServiceRegistrySettingsElement, Microsoft.ServiceBus, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
      </behaviorExtensions>
      <bindingElementExtensions>
        <add name="netMessagingTransport" type="Microsoft.ServiceBus.Messaging.Configuration.NetMessagingTransportExtensionElement, Microsoft.ServiceBus,  Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
        <add name="tcpRelayTransport" type="Microsoft.ServiceBus.Configuration.TcpRelayTransportElement, Microsoft.ServiceBus, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
        <add name="httpRelayTransport" type="Microsoft.ServiceBus.Configuration.HttpRelayTransportElement, Microsoft.ServiceBus, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
        <add name="httpsRelayTransport" type="Microsoft.ServiceBus.Configuration.HttpsRelayTransportElement, Microsoft.ServiceBus, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
        <add name="onewayRelayTransport" type="Microsoft.ServiceBus.Configuration.RelayedOnewayTransportElement, Microsoft.ServiceBus, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
      </bindingElementExtensions>
      <bindingExtensions>
        <add name="basicHttpRelayBinding" type="Microsoft.ServiceBus.Configuration.BasicHttpRelayBindingCollectionElement, Microsoft.ServiceBus, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
        <add name="webHttpRelayBinding" type="Microsoft.ServiceBus.Configuration.WebHttpRelayBindingCollectionElement, Microsoft.ServiceBus, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
        <add name="ws2007HttpRelayBinding" type="Microsoft.ServiceBus.Configuration.WS2007HttpRelayBindingCollectionElement, Microsoft.ServiceBus, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
        <add name="netTcpRelayBinding" type="Microsoft.ServiceBus.Configuration.NetTcpRelayBindingCollectionElement, Microsoft.ServiceBus, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
        <add name="netOnewayRelayBinding" type="Microsoft.ServiceBus.Configuration.NetOnewayRelayBindingCollectionElement, Microsoft.ServiceBus, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
        <add name="netEventRelayBinding" type="Microsoft.ServiceBus.Configuration.NetEventRelayBindingCollectionElement, Microsoft.ServiceBus, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
        <add name="netMessagingBinding" type="Microsoft.ServiceBus.Messaging.Configuration.NetMessagingBindingCollectionElement, Microsoft.ServiceBus, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
      </bindingExtensions>
    </extensions>
  </system.serviceModel>
  <appSettings>
    <add key="RedisConnectionString" value="mijnnuon.redis.cache.windows.net,abortConnect=false,ssl=true,password=xxxxxxxxxxx"/>
  </appSettings>
</configuration>

Step 2: Deploy Azure Web Job



Before the deployment (Publish to Azure) of the Web Job can be done a configuration setting in Web App has to be done to enable AzureWebJobDashboard.


Picture 4. Web App Settings, add the AzureWebJobsDashboard connection string.

This is an important step. We have to specify the name AzureWebJobsServiceBus, the connection string in format: 

DefaultEndpointsProtocol=https;AccountName=[Storage Account Name];AccountKey=[Access Key]

and choose type custom. In case we forget this than observing the Web Job logs will result in the following error:



Picture 5. Error message when not configuring the connection string in the Web App.

Now the Web Job can be deployed via Visual Studio to a WebApp.  Right click on the project and choose Publish as Azure WebJob...



Picture 6. Publish the Web Job to Azure in Visual Studio.

We will see a Publish Web dialog and here we import the publishing setting from WebApp. These settings can be downloaded from Azure Portal.



Picture 7. Import the publishing settings of the Web App.

Next we can click OK and we will go to the next section of the dialog i.e. Connection.



Picture 8. Validate connection to Web App.

Click Validate Connection to see if connection info is correct. When valid we can click Publish. Now the Web Job will be published to Web App. In the output window of Visual Studio we will see that deployment went successful.

Step 3: Test the Azure Web Job

In the Azure Portal we can see the Web Job in the Web App.



Picture 9. Web Job inside the Web App.

When we click on the logs URL we will be redirected to the Microsoft Azure Web Jobs portal.



Picture 10. Continuous WebJob details.

Nothing much has happened so far, only that the Job has started. In case I send a message to the queue using for instance ServiceBus Explorer, will see some action. Send a message via the ServiceBus Explorer to the queue.



Picture 11. Send Message to Service Bus Queue inboundqueue.

Refresh the AzureWeb Job Portal and a new entry is available.



Picture 12. Invoke Function entry in Microsoft Azure WebJobs portal.

Once we click on the Functions.ProcessQueueMessage we examine the logs by click Toggle OutPut.



Picture 13. Web Job log entries.

To explore what is in my REDIS cache we need to navigate to the service in the Azure Portal and open a console. Enter GET and messagid.





Picture 14. REDIS Cache Output in Console.

As we can see the message is now in the Cache.

Wrap up

It took me some time to get the into ServiceBusTrigger details. After some digging around I was able to get the ServiceBusTrigger working and see its behavior through the Azure Web Job Portal. The trigger is not limited to queues as it will also work for Service Bus Topic/Subscription. The signature of the method would look like:

public static void ProcessQueueMessage([ServiceBusTrigger(Topic, Subscription)] BrokeredMessage message, TextWriter log)

See Also

Resources to explore with regards to this article are:

For a demonstration see channel 9: https://channel9.msdn.com/Blogs/Microsoft-Integration/Microsoft-Integration-WebJob-Service-Bus-Triggers