Microsoft's platform strategy fails

Hybrid connectivity: Connect Azure websites to line-of-business applications using PortBridge

  • 17 minutes to read

This article has been machine translated.

Azure websites

Tejaswi Redkar

Download the code samples

Contrary to popular belief, not everything is going to the cloud, at least not yet. Just as all cars are not yet hybrid and most devices are not yet energy efficient, a good amount of software will still be running locally for years. Deep investments are hard to leave, and most organizations need significant business justification to modernize an existing application. Do you have solar panels in your home? Despite long-term energy savings and the tax benefits, people are reluctant to invest in them due to upfront costs. It's no different for organizations when it comes to moving to the cloud.

Fortunately, however, you can use hybrid connectivity to modernize your applications by tapping into the power of the cloud while still plugging into existing software services that run local in your data center. In this article, I'm going to show you how to create a website that uses the Azure Service Bus to connect to software services running in non-Azure data centers. I use a popular utility called PortBridge instead of using the Service Bus API directly to connect Azure websites to on-premises services.

Hybrid connectivity in Azure

Azure includes a range of services for building hybrid applications. Hybrid connectivity are the two most commonly used Azure Virtual Network and Azure Service Bus. Azure Virtual Network allows you to expand your local network in Azure, allowing it to be a private hybrid network between Azure and the data center. The expansion between Azure and your data center takes place in the network layer instead of the application layer in Service Bus. Since this article is about application connectivity with Service Bus, I won't cover Azure Virtual Network here. For more information, visit bit.ly/QAODgX.

The Service Bus Relay service allows you to connect two applications behind a firewall. The applications can be in Azure or your daughters-in-law data centers. Service Bus Relay gives you the ability to register your Windows Communication Foundation (WCF) application endpoints in the Service Bus registry in Azure Datacenter. Method calls can be securely exchanged between the client and the server within the service bus infrastructure and then transmitted to the respective applications in your data center. This is ideal, for example, in a scenario where a utility company needs access to HVAC or power meters send data collection and important events to these devices, performed from a central server in Azure. illustration 1 As this example illustrates, the utility company has a head-end in Azure and devices running in buildings.


Figure 1 Service Bus Relay scenario to a power company

The control element gateway is a device running in buildings that is responsible for controlling electrical devices. Each control gateway has a WCF endpoint registered with the Service Bus registration with a globally unique identifier. When the head-end service, running on one of the Azure Compute Services, wants to communicate with a specific control gateway, it opens a connection to the globally unique endpoint in Service Bus and starts sending or getting messages from the control gateway. Typically the control gateway is the building's firewall. But what if you have a non-WCF service (such as a SQL Server database) running on-premises that you want to connect from an Azure website?

Introduction of PortBridge

PortBridge is a utility that builds on the Service Bus API, which enables communication between any endpoint of the TCP-based service with local and on Azure. The core concept of the PortBridge and the prototype were developed by Clemens Vasters (bit.ly/SI93GM). I've modified it a little for this article and compiled it with the latest version of the Service Bus SDK. When building applications for Service Bus Relays, you are forced to create a WCF interface for all relevant endpoints that you want to make available in the cloud. This can get cumbersome and tedious if you have a large number of endpoints or generic platform end devices, such as databases and search engines. Adding another WCF abstraction to these services doesn't make sense. Instead, you can use PortBridge to expose non-WCF-TCP endpoints for Service Bus Relay connectivity. Front-end applications such as Azure websites can then connect to any TCP data source residing in your data center behind a firewall. In principle PortBridge creates a generic interface for WCF, as shown in Figure 2.

Figure 2 the generic PortBridge Windows Foundation communication interface

With only three generic methods PortBridge acts as a proxy between the client and server endpoints and routes all TCP calls to the designated server via a service bus relay. The biggest advantage of using PortBridge is that you don't have to build a WCF interface for every local endpoint that you want to enable for the hybrid connection.

PortBridge consists of two components:

  1. An agent that runs closer to the client application or in a virtual machine (VM) in the Azure infrastructure as a service (IaaS).
  2. A Windows service (or console application) that runs local and acts as a proxy for service endpoints running in the same environment.

Some common use cases for PortBridge are:

  • Connection to all TCP-based data storage from local
  • Connect to third-party web services that can be migrated to Azure
  • Remote desktop connections

This article shows you how to deploy and configure PortBridge to enable communication between Azure websites and a SQL Server database running on the local computer.

A hybrid solution

Sky blue websites usually form the web front end or web services of their application layer. But in many situations, you cannot migrate some of the data sources the site makes to Azure. In such cases, PortBridge is an ideal solution for connecting Azure websites to running local databases. Himmelblau Websites communicates with the PortBridge agent, which in turn communicates with the PortBridge service via the service bus. The PortBridge service then routes the call to the target database. Figure 3 illustrates the architecture of the Hybrid Website, which I will create in this article and shows a typical message exchange in a PortBridge (or Service-Bus-Relay) environment:


Figure 3 PortBridge example architecture

  1. The local PortBridge console application (or Windows service) registers the TCP endpoint database (1433 for SQL Server) within the naming service bus registry with a unique URI. Service Bus Relay opens an outbound bidirectional connection with the console application.
  2. When an HTTP or HTTPS request comes to Azure websites, the website opens the database connection as usual, but with the IP address of the PortBridge Agent VM instead.
  3. The PortBridge Agent VM serves as a database proxy in the cloud and has two communication interfaces - one for receiving TCP requests and one for connecting to the PortBridge console service bus relay endpoint.
  4. The PortBridge agent forwards the database query to the Service Bus Relay service. The beauty of the solution is that for the Azure web sites they do business as usual. The API for the database has not changed, only the IP address has.
  5. Service Bus Relay routes the call to the PortBridge application running on your localhost (or datacenter).
  6. Eventually the call reaches the database and the data is retrieved using the same connection.

Note that there are a number of components involved that communicate over PortBridge, and you need an additional VM that might not be needed if using Service Bus directly. As mentioned earlier, the benefit here is a generalization of exposing all TCP endpoints - you can reuse the same PortBridge Agent VM connecting to multiple data sources running in different locations. Now that I've shown you the solution's system architecture, I'm going to start constructing it.

Create RESTful web service

To keep this article simple, I'm going to design a relaxing ASP.NET Web API service called Countries that pulls a list of all the countries from a SQL Server database table. In the development process, I always recommend creating and testing locally before deploying to the cloud. Figure 4 shows the code for the country web service.

Figure 4 country web service

The web service has only two methods - and received (String Id). The Get method gets all countries from the database table and the Get (id) method gets a Country object under the country code. Figure 5 shows the country database table in the local SQL Server database.


Figure 5 country database table

Figure 6 shows the CountryService class, which gets the data from the database and returns country objects.

After the database and web service are ready, you can quickly test the web service locally by pressing F5, run it from Visual Studio. If all goes well, you have a web service that works locally. Now you want to move to the cloud, but you cannot migrate the database to the cloud as it is shared with other applications that are still running on-premises. PortBridge fits perfectly in this scenario with little or no code changes to the web service.

Figure 6 the CountryService class

Set up PortBridge

Before moving the web service to the cloud, you need to configure and test PortBridge. Since there are several layers in the architecture, it is important to get all of the components configured correctly at the beginning. I usually create a table of input and end points for each component (see Figure 7).

Figure 7 Endpoints and Locations

Solution componentEntrance end pointOutput end pointOutput end point
Country web service80Azure website
PortBridge Agent14331433Azure VM
PortBridge14331433Azure VM
SQL Server1433Azure VM

This table might seem trivial in this case, but for applications with hundreds of endpoints, a table like this will help you configure services correctly. As mentioned earlier, the PortBridge agent resides in a VM and has two endpoint interfaces - input from the application and output to a Service Bus Relay endpoint. Correspondingly, the PortBridge service on the local computer has two endpoints - input from the Service Bus relay and output from the SQL Server database. Once you have the configuration parameters noted down, you can start the deployment process.

Create a service bus namespace

Since the communication backbone of this solution is Service Bus Relay, I first need to create a Service Bus namespace from the Azure Portal (manage.windowsazure.com) as shown in Figure 8.


Figure 8 Creating the Service Bus Namespace

The country namespace in Figure 8 is unique in the world and is used to create a globally unique endpoint for the services. When you have created the namespace, click the Connection Information button at the bottom of the page to see the Namespace Access Infor Mation. You need standard issuer and the standard key to access the namespace shown Figure 9so write this down.


Figure 9 Bus Namespace Service Credentials

The PortBridge agent and PortBridge application require these credentials to connect to the namespace.

Configure the PortBridge application

The PortBridge application has a custom configuration section:

The issuer namespace and secret are required to establish a previously created service bus relay namespace. The target service hosts are listed in the host mapping. In my example, it's the local computer and port 1433. You can add CSV ports to externalize multiple endpoints on the same host. You can also add multiple target hosts as long as they are reachable by the PortBridge application machine. When the configuration is complete, start the PortBridge application on the local computer. If everything goes as expected you should see a screen similar to this Figure 10.


Figure 10 PortBridge application is running

The PortBridge application creates an outbound connection to the Service Bus Relay service on the country namespace and creates a unique name in the format PortBridge / [target host], where [target host] is the TargetHost parameter in the PortBridge application configuration file , like in Figure 11.


Figure 11 Service Bus Relay Connection

The PortBridge part of the name is hard-coded, while the destination host changes depending on the number of hosts you've created. It is important to note that to keep the name unique in the namespace, each target host only has one entry in the HostMappings section of the configuration file. Note that if you don't configure the TargetHost parameter correctly, the endpoint created will not match the service you want to represent, and communication will also fail.

Information on required Service Bus Relay bindings outbound ports, bit.ly/1l8lncx.

Configure the PortBridge Agent

Like the PortBridge application, the PortBridge agent also has a custom configuration section:

The PortBridge agent needs the port for the Service Bus relay endpoint (RemoteTcpPort) to have a port mapping section where you map each input endpoint port (LocalTcpPort). The destination host value must match the destination host value you previously created in the PortBridge application configuration. This value is used to connect to the appropriate Service Bus namespace endpoint. If these two values ​​do not match, the application will not work.

In the Firewall Rules section you can explicitly list the IP addresses of computers from which the PortBridge agent will accept requests. Since my PortBridge agent receives requests from an Azure website, I need the firewall rules to add the IP address ranges of the Azure data centers. You can get the Azure data center IP address ranges from bit.ly/1l8yDxV.

To deploy PortBridge Agents in Azure, navigate to the Azure Portal, create a new Windows VM, copy the PortBridge Agent files and run PortBridgeAgent.exe. As you create the VM, be sure to open the 1433 endpoint for external access so that the Azure website can access this port on the PortBridge Agent VM.

Another option for deploying the PortBridge Agent is through Apps for Azure, an easy way to deploy Windows Store apps. Apps for Azure is a free app with pre-built VMs readily available for deployment, and it has a pre-built PortBridge VM, as in Figure 12. You can install Apps for Azure from appsforazure.com.


Figure 12 Apps for Azure for deploying the PortBridge agent

The pre-built PortBridge Agent VM by default opens the 1433 and 80 endpoint ports for communication. If you want to custom configure the PortBridge Agent, you will need to modify the PortBridgeAgent.exe.config file in the C: \ ddapplications folder. After configuring the file, you need to restart the dynamicDeployInitService Windows service.

Testing and deployment of the country web service

Once the PortBridge components are installed and running, you need to change the web service database connection string on the PortBridge Agent VM to add:

Note that besides the host name, all other parameters remain unchanged. With this configuration change, you'll be pointing directly to the PortBridge Agent VM instead of the database. Next, publish the Country Web API Service to Azure websites and test it to navigate to the following URIs:

  1. http: // [Azure Web Site Host] / api / countries, for all countries.
  2. http: // [Azure Web Site Host] / code api / countries / [country code], for a specific country.

If the end-to-end communication works, both the URIs will return JSON objects for the method calls. You can now call the country web service from any device and the call will be traversed from the Azure Web Site to the PortBridge agent and to reach the SQL Server database via the PortBridge application. The data is executed from the database that is retrieved on the local computer (or data center).

Performance considerations

Since PortBridge is a level of dereferencing, it causes latency compared to architectures involving direct database communication or hybrid virtual network connections. PortBridge is only recommended in situations where a virtual network and direct communication are not possible. At the time of this writing, Azure websites do not support virtual networks, so Service Bus is the only way to create hybrid connectivity. During my tests, I saw wait times of 50ms to 300ms for the country web service. For some scenarios I recommend caching the data in the cloud and the services running in remote data centers only reach them at certain intervals.

Security considerations

PortBridge relies on the security functions of Service Bus Relays. By default, PortBridge uses connection security to connect to the Service Bus namespace, but it does not employ any transport or message encryption functions. The PortBridge agent offers a custom IP address filtering mechanism, but this shouldn't be as robust as the built-in security mechanisms of Azure Service Bus. I recommend doing extensive thread modeling before deploying the solution to production.

Scaling PortBridge

From the architecture in Figure 3, it looks like the PortBridge agent has a single point of failure. But you can scale out the PortBridge Agent by adding a load balancing endpoint for the VM and creating multiple instances of the PortBridge Agent VM. The calls coming from the Azure website will then be load balanced across multiple PortBridge agents. Although the PortBridge Agent can scale, the final destination endpoint (the database) should be designed to handle this scaling. She doesn't want to scale on the web and the middle tier, but not on the data plane. Your design should be completely elastic.

PortBridge in the real world

Over the past couple of years, based on multiple customer requests, our team built a messaging hub in the cloud for integrating third-party applications with Azure Services. The aim was to create a lightweight plug and play model for collecting data from disparate data sources and then surface the aggregated data for end users on devices of different sizes. The team chose to use PortBridge as the backbone for this plug-and-play model, as it provided the abstraction necessary to connect non-WCF services to applications in Azure. We're changing the PortBridge code for better diagnostics, performance, and caching. Some of the scenarios we successfully implemented:

  1. Create a unified messaging service that integrates the 19 different data sources of a Fortune 50 company and offers an in-service context data mashup to the customer's field employees. Quick access to the required information in the field was the main goal of this solution.
  2. Retrieve data from thousands of application instances used for commercial vehicle inventory management and store the data in a central repository in the cloud. The data collected was used for preventive maintenance and rehearsing buyer behavior. Without the service bus (and PortBridge), such an application would have been 10 times the cost and effort.
  3. Providing case management information and timesheet to thousands of attorneys on their mobile devices anywhere. Prior to this application, the attorneys had to go back to the office after each court case to record their time. With the mobile app you can register your entries directly from the courthouse.

Summary

Himmelblau websites and Service Bus complement each other in the building of the hybrid web applications. PortBridge is only an abstraction layer to make conveniently available in the cloud via service bus relays for non-WCF endpoints. I have covered several scenarios of the required building aggregation mashup web sites in which data is obtained from multiple web services running in different locations. Using PortBridge, these applications could be designed, tested, and built without significant changes to the original application code. As soon as the first PortBridge configuration is tested, the connectivity works consistently as long as the service bus and web services are available. Although PortBridge allows you to quickly build hybrid applications, keep in mind it introduces latency on your service calls. If your application's response time is different, PortBridge may not be the right choice and you should evaluate Azure Virtual Network or on-premises services to migrate to the cloud. After a walk through the solution described in this article, you should comfortably externalize any local endpoint in Azure and then call it up from an Azure website. Source code for PortBridge and its associated country web service is available on GitHub published at github.com/dynamicdeploy/portbridge.

Tejaswi Redkaris an author and software developer. He currently works as the Director of Platform Strategy and Communities Application for Microsoft. His book, "Windows Azure Web Sites: Building Web Applications at a Rapid Pace" (Dynamic Deploy LLC, 2013), is the most comprehensive and best-selling book on the subject. Redkar is also the creator of appsforazure.com and dynamicdeploy.com, where he has first hand experience of running production applications in the cloud. You can reach him at [email protected] and follow him on Twitter at twitter.com/tejaswiredkar.

Thanks to the following technical experts for reviewing this article: Microsoft Azure the Product Management Team