Rebuilding the SiteMonitR using Azure WebJobs SDK
Some time back, I created the SiteMonitR sample to demonstrate how SignalR could be used to tie together a Web Site and a Cloud Service. Since then Azure has evolved quite rapidly, as have the ASP.NET Web API and SignalR areas. One recent addition to the arsenal of tools available to application developers in Azure is WebJobs. Similar to the traditional Worker Role, a WebJob allows for the continuous or triggered execution of program logic. The main difference between a Worker Role and a WebJob is that the latter runs not in the context of a separate Cloud Service, but as a resource of a Web Site. WebJobs also simplify development of these routine middleware programs, too, since the only requirement on the developer is to reference the WebJobs NuGet package. Developers can write basic console applications with methods that, when decorated with properties resident in the WebJobs SDK, will execute at appropriate times or on schedules. You can learn more about the basics of WebJobs via the introductory article, the WebJobs SDK documentation, or from Hanselman’s blog post on the topic.
In this blog post, I’m going to concentrate on how I’ve used WebJobs and some of my other favorite technologies together to re-create the SiteMonitR sample. I’ve forked the original GitHub repository into my own account to provide you with access to the new SiteMonitR code. Once I wrap up this post I’ll also update the MSDN Code Sample for SiteMonitR, so if you prefer a raw download you’ll have that capability. I worked pretty closely with Pranav Rastogi, Mike Stall and the rest of the WebJobs team as I worked through this re-engineering process. They’ve also recorded an episode of Web Camps TV on the topic, so check that out if you’re interested in more details. Finally, Sayed and Mads have developed a prototype of some tooling features that could make developing and deploying WebJobs easier. Take a look at this extension and give us all feedback on it, as we’re trying to conceptualize the best way to surface WebJobs tooling and we’d love to have your input on how to make the whole process easier.
SiteMonitR is a very simple application written for a very simple situation. I wanted to know the status of my web sites during a period where my [non-cloud] previous hosting provider wasn’t doing so well with keeping my sites live. I wrote this app up and had a monitor dedicated to it, and since then the app has served as proving ground for each new wave of technology I’d like to learn. This implementation of the application obviously makes use of WebJobs to queue up various points in the work flow of site-monitoring and logging. During this workflow the code also updates a SignalR-enabled dashboard to provide constant, real-time visibility into the health of a list of sites. The workflow diagram below represents how messages move throughout SiteMonitR.
The goal for the UX was to keep it elegant and simple. Bootstrap made this easy in the last version of the application, and given the newest release of Boostrap, the templates Pranav and his team made available to us in Visual Studio 2013, it seemed like a logical choice for this new version of SiteMonitoR. I didn’t change much from the last release aside from making it even moresimple than before. I’ve been listening to the team, and the community, rave about AngularJS but I’d not
The entire solution is comprised of 4 projects. Two of these projects are basic console applications I use as program logic for the WebJobs. One is [duh] a Web Application, and the last is a Common project that has things like helper methods, constants, and configuration code. Pretty basic structure, that when augmented with a few NuGet packages and some elbow grease, makes for a relatively small set of code to have to understand.
With a quick high-level introduction to the application out of the way, I’ll introduce the WebJob methods, and walk through the code and how it works.
Code for Harnessing WebJobs
One of the goals for WebJobs was to provide ASP.NET developers a method of reaching in and doing things with Azure’s storage features without requiring developers to learn much about how Azure storage actually works. The team’s architects thought (rightfully) that via the provision of convenience attributes that referred to abstract use-cases facilitated in many common Azure scenarios, more developers would have the basic functionality they need to use storage without actually needing to learn how to use storage. Sometimes the requirement of having to learn a new API to use a feature mitigates that feature’s usefulness (I know, it sounds crazy, right?).
So, Mike and Pranav and the rest of the team came up with a series of attributes that are explained pretty effectively in their SDK documentation. I’m going to teach via demonstration here, so let’s just dive in and look at the first method. This method, CheckSitesFunction, lives in the executable code for a WebJob that will be executed on a schedule.Whenever the scheduler service wakes up this particular job, the method below will execute with two parameters being passed in. The first parameter references a table of SiteRecord objects, the second is a storage queue into which the code will send messages.
You could’ve probably guessed what I’m going to do next. Iterate over all the records in the table, grab the URL of the site that needs to be pinged, then ping it and send the results to a storage queue. The outparameter in this method is actually a queue itself. So the variable resultList below is literally going to represent the list of messages I’m planning on sending into that storage queue.
Now, if you’re obsessive like me you’ll probably have that extra monitor set up just to keep tabs on all your sites, but that’s not the point of this WebJob. As the code executes, I’m also going to call out to the Web API controller in the web site via the UpdateDashboard method. I’ll cover that in more detail later, but that’s mainly to provide the user with real-time visibility into the health of the sites being checked. Realistically all that really matters is the log data for the site health, which is why I’m sending it to a queue to be processed. I don’t want to slow down the iterative processing by needing to wait for the whole process so I queue it up and let some other process handle it.
In addition to a scheduled WebJob, there’s another one that will run on events. Specifically, these WebJobs will wake up whenever messages land into specific queues being observed by this WebJob. The method signatures with appropriately-decorated attributes specifying which queues to watch and tables to process, are shown in the code below.
One method in particular, AddSite,runs whenever a user event sends a message into a queue used to receive requests to add sites to the list of sites being watched. The user facilitates this use-case via the SiteMonitR dashboard, a message is sent to a queue containing a URL, and then, this method just wakes up and executes. Whenever a user sends a message containing a string URL value for the site they’d like to monitor, the message is then saved to the storage table provided in the second parameter. As you can see from the method below there’s no code that makes explicit use of the storage API or SDK, but rather, it’s just an instance of an IDictionary implementer to which I’m just adding items.
The SaveSiteLogEntry method below is similar to the AddSite method. It has a pair of parameters. One of these parameters represents the incoming queue watched by this method, the second represents a table into which data will be stored. In this example, however, the first parameter isn’t a primitive type, but rather a custom type I wrote within the SiteMonitR code. This variation shows the richness of the WebJob API; when methods land on this queue they can be deserialized into object instances of type SiteResult that are then handled by this method. This is a lot easier than needing to write my own polling mechanism to sit between my code and the storage queue. The WebJob service takes care of that for me, and all I need to worry about is how I handle incoming messages. That removes a good bit of the ceremony of working with the storage SDK; of course, the trade-off is that I have little no no control over the inner workings of the storage functionality.
That’s the beauty of it. In a lot of application code, the plumbing doesn’t really matter in the beginning. All that matters is that all the pieces work together.
Finally, there’s one more function that deletes sites. This function, like the others, takes a first parameter decorated by the QueueInput attribute to represent a queue that’s being watched by the program. The final parameters in the method represent two different tables from which data will be deleted. First, the site record is deleted, then the logs for that site that’ve been stored up are deleted.
The SiteMonitR Dashboard
The UX of SiteMonitR is built using Web API, SignalR, and AngularJS. This section will walk through this part of the code and provide some visibility into how the real-time updates work, as well as how the CRUD functionality is exposed via a Web API Controller. This controller’s add, delete, and list methods are shown below. Note, in this part of the code I’ll actually be using the Storage SDK via a utility class resident in the Web Project.
Remember the scheduled WebJob from the discussion earlier? During that section I mentioned the Web API side of the equation and how it would be used with SignalR to provide real-time updates to the dashboard fromthe WebJob code running in a process external to the web site itself. In essence, the WebJob programs simply make HTTP GET/POST calls over to the Web API side to the methods below. Both of these methods are pretty simple; they just hand off the information they obtained during the HTTP call from the WebJob up to the UX via bubbling up events through SignalR.
The SignalR Hub being called actually has no code in it. It’s just a Hub the Web API uses to bubble events up to the UX in the browser.
The code the WebJobs use to call the Web API make use of a client I’ll be investigating in more detail in the next few weeks. I’ve been so busy in my own project work I’ve had little time to keep up with some of the awesome updates coming from the Web API team recently. So I was excited to have time to tinker with the Web API Client NuGet’s latest release, which makes it dirt simple to call out to a Web API from client code. In this case, my client code is running in the scheduled WebJob. The utility code the WebJob calls that then calls to the Web API Controller is below.
As I mentioned, I’m using AngularJS in the dashboard’s HTML code. I love how similar AngularJS’s templating is to Handlebars. I didn’t have to learn a whole lot here, aside from how to use the ng-class attribute with potential multiple values. The data-binding logic on that line defines the color of the box indicating the site’s status. I love this syntax and how easy it is to update UX elements when models change using AngularJS. I don’t think I’ve ever had it so easy, and have really enjoyed the logical nature of AngularJS and how everything just seemed to work.
Speaking of my AngularJS controller, here it is. This first code is how the Angular controller makes HTTP calls to the Web API running on the server side. Pretty simple.
Here’s how the Angular controller handles the events bubbled up from the Angular service that abstracts the SignalR connection back to the server. Whenever events fire from within that service they’re handled by the controller methods, which re-bind the HTML UX and keep the user up-to-date in real-time on what’s happening with their sites.
With that terse examination of the various moving parts of the SiteMonitR application code out of the way the time has come to learning how it can be deployed.
There are three steps in the deployment process for getting the site and the WebJobs running in Azure, and the whole set of code can be configured in one final step. The first step should be self-explanatory; the site needs to be published to Azure Web Sites. I won’t go through the details of publishing a web site in this post. There are literally hundreds of resources out there on the topic, from Web Camps TV episodes to Sayed’s blog, as well as demo videos on AzureConf, //build/ keynotes, and so on. Save it to say that publishing and remote debugging ASP.NET sites in Azure Web Sites is pretty simple. Just right-click, select Publish, and follow the steps.
Once the code is published the two WebJob executables need to be zipped up and deployed. First, I’ll pop out of Visual Studio to the bin/Release folder of my event-driven WebJob. Then I’ll select all the required files, right-click them, and zip them up into a single zip file.
Then in the portal’s widget for the SiteMonitR web site I’ll click the WebJobs tab in the navigation bar to get started creating a new WebJob.
I’ll give the WebJob a name, then select the zip file and specify how the WebJob should run. I’ll select Run Continuously for the event-driven WebJob. It has code in it that will halt the process in wait for incoming queue messages, so this selection will be adequate.
Next I’ll zip up the output of the scheduled WebJob’s code.
This time, when I’m uploading the zip file containing the WebJob, I’ll select the Run on a Schedule option from the How to Run drop-down.
Then I’ll set up the schedule for my WebJob. In this example I’m going to be a little obsessive and run the process to check my sites every fifteen minutes. So, every fifteen minutes the scheduled task, which is responsible for checking the sites, will wake up, check all the sites, and enqueue the results of each check so that the results can be logged. If a user were sitting on the dashboard they’d observe this check happen every 15 minutes for each site.
The two WebJobs are listed in the portal once they’re created. Controls for executing manual or scheduled WebJobs as well as stopping those currently or continuously running appear at the bottom of the portal.
The final step in deployment (like always) is configuration. SiteMonitR was built for Azure, so it can be entirely configured from within the portal. The first step in configuring SiteMonitR is to create a Storage Account for it to use for storing data in tables and for the queue being used to send and receive messages. An existing storage account could be used, and each object created in the storage account is prefixed with the sitemonitr phrase. That said, it could be better for some to have an isolated storage account. If so, create a storage account in the portal.
Once the storage account has been created, copy the name and either primary or secondary keys so that you can build a storage account connection string. That connection string needs to be used, as does the URL of my site (in this case, sitemonitr.azurewebsites.net, which is in fact a live instance of this code) as connection strings and an appSetting (respectively). See below on how to configure these items right from within the Azure portal.
Once these values are configured, the site should be ready to run.
To test it out, open the site in another browser instance, then go to the WebJobs tab of the site and select the scheduled task. Then, run it from within the portal and you should see the HTML UX reacting as the sites are checked and their status sent back to the dashboard from the WebJob code.
Speaking of dashboards, the WebJobs feature itself has a nice dashboard I can use to check on the status of my WebJobs and their history. I’ll click the Logs link in the WebJobs tab to get to the dashboard.
The WebJobs Dashboard shows all of my jobs running, or the history for those that’ve already executed.
I’m really enjoying my time spent with WebJobs up to this point. At the original time of this post, WebJobs was in the first Alpha release stage. so they’re still pretty preview. I’m seeing huge potential for WebJobs, where customers who have a small middle-tier or scheduled thingthat needs to happen. In those cases a Worker Role could be quite overkill, so WebJobs is a great middle-of-the-road approach to giving web application owners and developers these sorts of scheduled or event-driven programs that enable a second or multiple tiers running. I’ve really enjoyed playing with WebJobs, learning the SDK, and look forward to more interesting things coming out in this space.