Introduction to Hybrid Integration

For many years enterprises have spent huge amount to build custom applications and infrastructure running on-prem. Most of these custom build application holds a major portion of business critical data. With the shift towards global markets, business are required to share their data in one or another form to the customers or the partners in a secure manner. The business owners are also making huge investment in cloud offerings simply because of multiple benefits like auto scaling, pay as you go, multiple SaaS products etc.

The challenges these Organization are facing is to bridge on premise resources sitting behind the corporate firewall to the Cloud. To solve these critical issues, the term Hybrid Integration has gained the popularity. This article will not go in details about various types of hybrid Integration but it will show how an implementation of Hybrid Integration scenario with Microsoft BizTalk 2016 and Logic Apps in synchronous manner.


We will be developing or demo based on below use case :

An organisation has its customer’s details record sitting at on-premise SQL Server which is in DMZ environment and cannot be accessed outside corporate firewall.  To integrate On-premise Application, the organization will be using Microsoft Biz Talk Server as its primary integration product. Recently the Web Application is being moved to Azure and as a Integrator we need to provide a secure Integration layer between cloud and services running on On-premise (like SQL ,SAP ,Oracle Etc) . The current requirement to to fetch customer record from on-Premise and update the record based on user choice


In the demo we will be using Logic Apps, On-Premise Data Gateway and BizTalk Server 2016 to make Synchronous real time call to the On premise SQL server and get the customer information with supplied Customer ID  (another Logic App for update).

Design

Step by Step to Setup the BizTalk Environment to use On-Premise Data Gateway and BizTalk Solution

  • Download on the On-Premise Data Gateway from On-Premise Data Gateway  on Biz Talk Server.
  • Follow the installation and configuration steps as described at Install the on-premises data gateway for Logic Apps
  • Note: If you are using your personal MSDN subscription associated with live,Hotmail or outlook then create a tenant user in AD and use the tenant user to install and configure your On-Premise Data GatewayCreating a Work or School identity in Azure Active Directory to use with Windows VMs
  • Install the Logic App adapter available with BizTalk CTP version on windows server 2012 or 2016
  • Setup the IIS to configure LogicApp Management Endpoint, Follow the link to configure and test Logic Apps endpoint  .Create separate Application poll in IIS to run LogicApp Management Endpoint and the Logic App Application.
  • Create application within IIS for GetCustomerDetails and set the application poll same as of LogicApp Management Endpoint. Set the Application path to use C:\Program Files (x86)\Microsoft BizTalk Server 2016\LogicApp Adapter\BizTalkWcfService

  • Create a GetCustomerDetails orchestration in BizTalk visual studio solution. The Orchestration will work with request response port and take request from Logic App adapter . After receiving the incoming request  the Orchestration send port will call SQL Server stored procedure  by passing customer Id  as parameter.  The SQL stored procedure will return  customer details from the Customerdetails table like phone number ,AddressLine1 ,Address Line2 etc .These customer specific customer details will be used in Web App . Customer Logged to the Web App can modify his/her  personal information and click on submit button. 
  • This will trigger another workflow updateCustomer logic app endpoint configured in  Web App a new Logic App  (one way logic App  using Logic App adapter ) and update Customer record based on customer Id .
  • Below is the snapshot of GetCustomerDetails  Orchestration which does message transformation and then sends the GetCustomerDetails response message to request response port which is using Logic App Adapter .


  • To develop this solution we have used  simple SQL stored procedure "GetCustomerDetails"  which is responsible for fetching customer details data from SQL based on the customer Id
Stored Procedure
CREATE procedure [dbo].[GetCustomerDetails]
 
 @CustomerId varchar(50)
 
As begin
 
select Cust_AddressLine1,Cust_AddressLine2,City ,PhoneNo from dbo.CustomerDetailTable where Customerid=@CustomerId
 
end
  • Once you have built the Biz Talk solution  successfully  deploy it in BizTalk administration console , bind the solution to appropriate  request-response(Logic App) and send ports(sql send-receive). The configuration of Request-Response port using Logic App adapter is shown below 


GetCustomersDetailsReciveLocation

Check the Status of On-premise Data Gate way windows service running on Biz Talk Environment. To check go to run command and type services.msc and check the status of On-premise data gateway  If you are working on corporate firewall then you need to open appropriate ports . Follow the instruction as being mentioned in the link Install the on-premises data gateway for Logic App . In my case, I have used the VPN Gateway to connect to Azure resources. 
 

Step by Step Process to Create Logic App in Azure to Call Biz Talk On-Premise

  • Login to Azure Portal with your subscription Id.
  • If you have created tenant user (you are using Hotmail ,outlook or live account ) then use the tenant user to Login to Azure Portal.
  • Create a new instance of On-premise Data Gateway in Azure and follow the instruction  on-premises data gateway for Logic Apps .
  • Create New Logic App “GetCustomerDetails” within the same region as of your On-premise Data Gateway with Request Trigger and add associated actions within the trigger , In the request, trigger specifies a JSON schema (this can be generated using jsonschema.net) . My sample JSON schema looks like below.
{
  "type": "object",
  "properties": {
    "ns0:CustomerRecord": {
      "type": "object",
      "properties": {
        "-xmlns:ns0": {
          "type": "string"
        },
        "Id": {
          "type": "string"
        }
      },
      "required": [
        "-xmlns:ns0",
        "Id"
      ]
    }
  },
  "required": [
    "ns0:CustomerRecord"
  ]
}

  • Within Logic Apps use On-premise Data Gateway to make a connection with BizTalk. Enter the correct credentials and Logic Apps Management URL (MachineName need to be changed to BizTalk system name).
  • Once successfully connected,  you will be able to see the deployed schema(prepare message from JSON action) and receive location under send message to BizTalk Action. Select the appropriate receive location within Send message to BizTalk action.
  • Save the Logic App this will generate an HTTP endpoint of the trigger . Copy the HTTP endpoint to test the developed Logic Apps .The HTTP endpoint generated contains a SaaS key token in the last which will be used for the authentication purpose . You can configure a API management  URL endpoint to do more fine-grained authentication and authorization on the generated URL.

Testing

Testing the HTTP endpoint is simple and you can use multiple tools like .Net client , Postman ,SOAPUI ,Mozilla Client or Fiddler . In this case, we have used Postman to test the hybrid solution involving Biztalk, Logic App and On-Premise Data Gateway . We have used below JSON message to test our solution

    {
  "ns0:CustomerRecord": {
    "Id": "SN001"
  }
}

 The output will be shown in the postman windows . It is good to set the application content-type or else you need to do content-type conversion within Logic Apps. 

 
Through postman, we have submitted the valid customer Id and got a response from the on-premise BizTalk instance, which is in Sync with SQL Server within corporate firewall zone.           

This approach is alternative of using HTTP Relay to get a request response(half-duplex) pattern working with an on-premise service .The reader can use this pattern in a wide variety of use cases with minimum development effort and reduce the amount of time to manage an internal integration endpoint .

See Also

An important place to find a huge amount of BizTalk related articles is the TechNet Wiki itself. The best entry point is BizTalk Server Resources on the TechNet Wiki. Another important place to find Logic App related articles is the TechNet Wiki itself. The best entry point is Logic App Resources on the TechNet Wiki.