Wagner Silveira explains how to unlock BizTalk endpoints to the outside world using Azure Services, with a focus on HTTP endpoints. The options he demonstrates in the session include Azure Relays, Logic Apps, Azure Function Proxies and Azure API Management.
Integrate 2018, June 4-6, etc.venues, London
I’m Sriram Hariharan. I’m from the Document360 team and I’m…it’s my privilege to introduce and welcome Wagner Silveira. Wagner works as the Principal Integration Architect at Theta in New Zealand. He hails from Brazil and he’s been working in the integration space since 2002. Wagner’s focus areas is on integration and service-oriented architecture and he’s been working on different areas across industries including healthcare, education, transport, utilities, and retail. Wagner has been awarded the Microsoft Most Valuable Professional Award since 2017 for his contribution to Azure and the integration space and the community activities as well. So ladies and gentlemen please welcome Wagner Silveira.
Wagner: Wow. No pressure, right? After Richard, Michael and Johan. But let’s see how we can do that. My name is Wagner Silveira. I’m the Principal Integration Architect at Theta, and I’m going to be talking about how we can expose BizTalk to the world via Azure, Right?
So in the agenda what we have is I’m going to be talking about why and how we should…and how we can expose BizTalk, right, to take other…to combine that with Azure to create our hybrid solutions. Then we talk about, like, creating those endpoints through some of the steps that Azure has available. We’re doing demos, so I can start you guys now to the sacrifice USB or hard drive, anything. So demo got to actually be in our favor. And we got some takeaways. Okay? Nice and easy.
Let’s talk about exposing BizTalk to the world. Why we want to do that, right? Why we want to actually get BizTalk that is running on premise and push it up to the cloud? One of the reasons is I might want to consume on premises…the resources, but most of my workflow [inaudible 00:02:14] workload is putting on the cloud, right? I want to combine the best of both worlds, and BizTalk is awesome for that. I think you guys saw in Steven’s presentation yesterday that BizTalk would be your foundation for that kind of hybrid solution. Then if you do that, you have a single gateway. Instead of like poking openings and firewalls and installing gateways and hybrid connections everywhere, you do it in one place that is BizTalk, and BizTalk distributed a talk to the systems that he already knows how to do. Right?
And the other thing you can do is that because of that, you can start [SP] extending cloud workflows. So I have stuff that is running on the cloud, but all of a sudden I need to push it up into…on premises. I can do that through BizTalk. I can start leveraging from a whole lot of connectors that BizTalk has that we don’t have easily accessible on the cloud. Okay?
How we can do that? We can do that through message exchange using queue topics, files, and mail like the…everyone that worked with EDI and things like that knows that we should…not going to be too far away for files even if we hate it and…or we may use this stuff like that. Right?
But in this talk, I want to actually talk a little bit about something else like expose BizTalk as HTTP endpoints. So try to compose an API that you can start to use. Right?
So the focus of the talk is going to be how I can expose BizTalk for a set of HTTP endpoints, and what is the tools that we have in Azure to do that. Right?
So, basically, what we’re doing is, like, unlocking those BizTalk endpoints, opening the door in a controlled way to the nice world outside. There is an Azure service for that. Actually, there is a lot of Azure services for that, right? You can use Azure Relay. Like, the oldest kid in the block that is still quite relevant today. We can use Logic Apps. We can use Azure Functions. So through the Azure Functions proxy, for example, we can use API Management. That’s, like, the premier tool for API and API composition and gateway.
So let’s start with the Azure Relay Services. What is that service? So it’s a way securely expose on-premises services, right? So with that, you’re going to have an endpoint on Azure that relates directly to an endpoint on-prem. Doing that we don’t have any infrastructure though, or in this case firewall changes because it’s doing bidirectional SocketChannel. So it opens a connection from on-prem to Azure, and after that, it leaves that connection open and you can go through this channel to do stuff.
And it comes in two flavors, right? You have the hybrid connections, and you have the WCF relays. What BizTalk uses for that? BizTalk goes and works with WCF relays out of the box. Right? So you have a way to create an WCF relay by just setting up a receive location. It uses the WCF HTTP BasicRelay binding for that. One of the things that we have to be aware is when you’re doing that it’s actually using XML format because it’s still based on WCF with the XML serialization. Right? We try to patch change them through that. That comes as, “What is that?” It’s like, “Hey, here is a binary for you.” It doesn’t really understand what it is.
From a security point of view, you have like a kind of two things to take in consideration. At the configuration time, you need to be able to create that relay. So you need to have access to an Azure service with SAS tokens or ACS tokens for the right level of permission. Right? And when clients are trying to use that endpoint you have two options. You can have either set to anonymous. That is not exactly that good. Or you can have…using the relay access tokens, right? But in some cases it’s going to be really hard for the…if you think about a SAS application trying to connect to you, it might not be able to create that SAS token, that relay access token. So one thing that we find out through clients, and I think it was an old blog from Richard Soroto that gave us the idea, is we can actually extend and create your own API key like security using the pipeline…a pipeline component at the beginning.
So I’ve got some scenarios and considerations around that. So if you want to expose an XML based endpoint, and you want to do that as a really fast and reliably service, Azure Relays is your man. Right? You’re going to see on the demo, like, how fast it is to do it, and the only thing you need to do is I have a relay header, and I have access. So I don’t have to go to the portal to do anything. We can do everything from BizTalk and it’s just there.
You need to consider the security implications, right, because you definitely don’t want to be doing anonymous. Everyone agree? But in some cases, you won’t be able to use the relay either, right? So a shared secret, custom shared secret using a pipeline component at the very first step of your pipeline, might be a way to work on this problem. We’re going to be doing some demos about that later on, right? But before we go and jump into the demos, let’s talk about the other ones, and then we can do the demos altogether.
So the next guy is Logic Apps. And do I need to say anything about Logic Apps? Don’t think so, right? So let’s go to that kind of first. Here is a mouthful for us is the integration platform as a service solution for Microsoft or the iPaaS. Basically, in this scenario what we’re trying to do is we already have, in Logic Apps, like, enough flow, and now we want to extend that to access on-premises solutions. And BizTalk together with Logic Apps works really well for that. If we want to create an endpoint from BizTalk that uses Logic Apps you’re going to be using the HTTP trigger. Right?
And we can have on-premises access using the on-premises without a gateway. So that means that, again, for Logic Apps we don’t have to create any kind of firewall changes. Right? So relays, we don’t need to do firewall changes, which makes it, like, a really compelling cases when your system administration says, “Over my dead body.” Right? The Logic Apps also makes it compelling because it’s going to be easy and fast and secure to do that instead of punching a hole in firewall.
So how BizTalk works with Logic Apps? BizTalk works with Logic Apps through the out of the box adapter for Logic Apps. If you’re going from BizTalk to Logic Apps, you have the Logic Apps BizTalk connectors if you’re going from Logic Apps to BizTalk. But you can also create a custom connector exposing your WCF receive location if you want to. Okay?
It allow you to do SOAP and REST. And from a security point of view, from Logic Apps to BizTalk. So when you’re doing configuration time, you can do the Windows authentication. So you need to have credentials to be able to connect to that endpoint, that is basically an IS endpoint. Or you’re going to be doing anonymous. That, again, is not fun. Okay?
And the user to Logic Apps. You have the SAS keys. So there’s the, basically, how the Logic Apps secures their endpoint. Scenarios and considerations. You can extend the cloud workflows. Right? You can leverage on-premises connectors. So if I have something that the Logic Apps doesn’t have that BizTalk gave to me, combine those two and then you’re in game.
And you can use XML and JSON formats. That give you, like, the next step. If I cannot use Azure Relays because I need JSON, you might be using Logic Apps because it allows you to do this.
Think about the security implications again, right? You might need the SAS thing. It might not be the best way to expose the Logic Apps, so you might need to put some other layer on top of that. The good thing about Logic Apps is, again, leverage the on-premise data gateway. So, again, the firewall security is fine.
Azure Function Proxies. How many people used that feature so far? Anyone? Apart from two? So that’s, basically, an API composition tool, right, that comes with Azure functions in this quite basic functionality. So it allows SOAP and REST because it’s, basically, a pass through. So it opens a channel, allow you to go. And you can have on-premise access with hybrid connections. So, again, that’s a good thing, right? You can…you don’t need to do much change on the firewall.
Between BizTalk and the Function Proxies, we have, like, no out of the box configuration. So we actually have to create a receive locations first, get that endpoint, and then configuring the proxy.
The configuration…you’re doing that configuration via the Function App UI, right? So you have to go jump into the portal and do stuff there. And the consumption plan that is, basically, usually when…how you use the functions don’t have access to hybrid connections. So if you want to use that, you have to go and jump into one of the paid…sorry. One of the service plan types. That is not consumption.
The scenarios is you want to expose the endpoints as a single API, and it’s okay to do just a pass through. Don’t need to do anything on the message before or after. Right? You okay with not using the consumption plan, which is kind of one of the big attractives of the functions apps anyway. If you want to use the consumption plan, the option would be open your firewall, and then you would have a set of IPs that is the application to whitelist.
From an API management point of view, right, the API management is your turnkey solution for API composition, API gateway, developer portal, is the tool for API management from the Azure portal. It gives you, like, a set of really comprehensive functionality. There is a lot of stuff there that you wouldn’t have on function apps, for example. Think caching, throttling, message changes, head augmentation, head removal, all of that that we saw before on [inaudible 00:14:34] presentations.
And you can do SOAP, REST, and REST to SOAP that, too. Okay. So it requires VNET integration if you want to go on prems or changing the…or firewall changes. From the BizTalk point of view, you have a wizard-based configuration, and we can have that using feature pack two straight from BizTalk, or feature pack one importing from API management. When you publish it, only allow you to publish Basic HTTP endpoints, right? The WCF Basic HTTP bindings. So if you have like a web HTTP binding, or WCF HTTP binding, it’s not going to work. And allow you to do SOAP and REST to SOAP. Finally, it requires VNET integration, or firewall changes if you want to talk to BizTalk, right. And my example here, I opened the firewall, you know, that…to…for us to talk to that.
So the scenarios. Again, you want to expose the endpoints as a single API, but you want to modernize a solution. You want to do, like, SOAP to REST, for example. You want to include caching, you want to include throttling, you want to make sure that that API is…supports a lot of hit [SP].
And you need to remember that VNET integration are going to require premium from an API management point of view, right? So if you want to use VNET integration, you need to go to premium. Otherwise, you can do like I did, do a firewall change. And the advantage of API management, in this case, is there’s only one port to whitelist…only one IP to whitelist. And is static. It’s going to continue there.
So let’s have some hands-on time.
So here is my BizTalk server, right? And the first thing I want to show you guys is the WCF relay. So how easy it is to create a WCF relay endpoint. Right?
So we’re going to be using the WCF basic HTTP relay. And in order to configure that, you just need to know where is the endpoint, right? So what is the…your namespace? In my case, W Silveira relay. And say where you want to send that. So, in this case, I will just change that, and you always should be changing demos on the fly, right?
So that means that now I’m going to be accessing that into integration demo instead of the other one that I have just to prove that it works. And if it doesn’t work, then I’ll have some trouble. From a security point of view, you can define if you want to do relay access token or no security. Right? And from a Service Bus connection formation that’s like the design time. In order to be able to create that thing, you need to use either SAS or ACS, and, basically, just need to say what is your key, and the value for that key.
So what we’re doing here, on this one, is I am creating some kind of items to come all the way to my SQL server, right? It’s a really complex application. Port A to Port B and boom.
So let’s see if that’s working.
WCF relay. And who remember what I called that?
Man: I think you canceled out.
Man 2: Yeah, I think you canceled it later.
Wagner: I did cancel?
Man 3: Yeah.
Wagner: Okay. So that means that that one is going to work. Right? Or not? Cool. Let’s see. Living dangerously. So I’m sending out requests where it’s like…let’s say please, please demo gods. And the status is, like, quite a…quite [inaudible 00:19:21] pending. So otherwise, it wouldn’t work.
It accepted. At least. That means that to relay seems to be working fine. Let’s see if it goes all the way to BizTalk. Yeah. It came. One demo complete. Hopefully, the other one is going like that. Just to show you quickly that that thing is actually under Azure relays. Here are our WCF relays. Right? And you see I have integrated 2018 service request that has one listener and that is exactly BizTalk sitting on the other side. That has no changes on firewall or anything like that. So it’s really, really fast to start using that and leverage an endpoint on the cloud. Right?
So next on the list I’m going to skip Logic Apps and leave it to the last because we’ve seen a lot of Logic Apps those days. Right? So let’s see some of the other options. So let’s go to functions proxy. As I said, for functions I actually cannot do anything on BizTalk, but if I had an endpoint there that is ready for me to use, then I can like…a proxy here. Right? And that’s exactly the case on this one. What I have is that one here, right. Can you guys see? So let me…there we go.
So when you’re creating a proxy you, basically, just say what is going to be the route. So it’s going to be using like the default address of the function app, right, and then put that route there. And it’s going to be using that. And you define your backend URL. And in my case, see that my backend URL is like my machine 8080, blah, blah, blah. That machine…see, there’s no, like, DNS name and no public IP on that. Why? Because that function app is actually higher up to that machine through hybrid connection. Right? And then because of that, I can actually do the connective…and those two guys can talk to each other.
So it’s that simple to create, but you can see that it’s not much else that I can do here, right? I can say, “What is the methods that I want to use?” I can, like, override my method. I can add a query parameter or add a header parameter, but that’s it. Compared to API management where I can do, like, heaps and heaps of putting caching, throttling, doing message transformation, all that kind of stuff. That one is almost nothing. But it’s really good for that fast kind of integration. Okay?
So, again, fingers crossed. Let’s see if that works.
So there we go. W Silveira. Function dedicated Azure website. So I’m pointing to the function website on that endpoint, right, and here is the message. I would say, “The demo gods are with us.” Resolution is “yes” since that is completed. I bet it saves, and then we’re in game. Otherwise, we’re having other fry to fish. Fish to fry.
There we go. It went to the proxy, went out to BizTalk, and came back. Right? Let’s see.
Refresh that. Yep. So quite easy to connect those two. Right? The last one that I want to show you guys is the API management. And from the API management, you can see that I can get one endpoint, like, any one of those endpoints as long as the endpoint is a basic HTTP. So if I click on that one, publish to API is disabled because these are the WCF basic relay.
If I click on that one it is also disabled because these are the web HTTP [SP], right? But any of those three I can go and publish to API management. Right? And in this case, I’m doing a SOAP pass through sending to this address. I update it f I want. And some of the gotchas that I found here is how many people here has a Microsoft account and a work account like the Office 365? Lots, right? On that case here, it only works with the work account. Right? The same thing that we had before with on-premise data gateway if you guys tried to implement that. So that’s why I’m using that admin stuff.
Also, I have that. Then you select the subscription. You select what is the resource group, your API management application is. Select the API management and that’s it. Right? Put the things, select what products you want that API associated, and it pushes it all up to you. Right? I will show the one that I already created because I also want to show a couple of things on API management. And I think we’re close to run out of time.
So not temp the gods too much on that.
If you go back to my API management, here are my APIs. Here is the Integrate 2018 SOAP that we created. And it has, like, the get requests. Right? It has the…if you look at inbound policies…let me make that a little bit smaller. Otherwise, I can’t see as well. I’ll bring it back in a minute. There is nothing because I just did a pass through, right? I select SOAP [SP] pass through. So there is no changes there.
But if you look at the settings, it’s actually pointing to my…the STEM-IP of my machine on the port that I opened, with the endpoint that is there on, IS for me that connects to that receive location. Right? And if I test that…and say, “Give me anything with status completed.” And send. Fingers crossed. Come on. Okay. Why? Oh, why? It’s not applied. We were so close, man. Let’s try that again. Let’s see if the one that I created is the…Is that the right one?
And let’s see if anything here that I actually did it wrong.
Might be that.
You guys didn’t sacrifice enough for the…enough USB keys, or I didn’t do that. But let me get back to this one here and we can do one more thing. One thing that I wanted to show you guys is this. My schema for this one, if we can see that, was not exactly a nice schema. And that’s probably the second gotcha that might be worth for you guys to know before we finish up. Okay? You see that…how well I did my schema. I just say, “Element blah with no type or anything like that.” Right? This throws off the API management pass big time. But I’m pretty sure that you have, like, lots of schemas like this lying around. All right?
So just to make sure that you are aware of that. If you try to create the…you try to create like a SOAP to REST with that, it’s just don’t do any mapping because it doesn’t know the types to do the maps. Right? If you get a message like this, proper schema like that one where I have like the types, then what you are going to find when you do the SOAP to REST…if I find my…service type test. And I do look at my “get comments”. And look at my code. There is my view code. Sorry.
It’s actually going to be doing all the translations from our REST message to the SOAP message that we need to push. Okay? But I tried that yesterday with the other schema, and all of that is blank because it doesn’t know how to actually do that transformation if it doesn’t know what types are supposed to apply. Okay? Two to one of the demos. Not too bad. I would prefer better, but that’s fine.
So some takeaways to finish her up. How are we going with time? About time, right? Mm-hmm, cool. So as anything in Azure, there’s like choices, choices, choices, and sometimes we’re going to be overwhelmed with those choices. Right? So if you’re anyone like me, you’re going to be feeling like a kid in a candy store and try everything. What is the things that you might think when you need to identify your needs in that particular case connecting BizTalk to the cloud via some APIs? Think about the use cases.
Do I need to use SOAP? Do I need to use REST? We can see that’s like from those…probably Logic Apps and functions might help you with REST. The other ones might not be that too helpful. So that might throw you in the right direction. And just to be on the same place, that doesn’t mean that the API management wouldn’t do REST for you in that case, but you would have to hire…wire everything yourself instead of using the tools that are there. Okay?
So for the same reason, you need to think like, “Do I need to use XML? Do I need to use JSON?” Right? So I might have, like, a resting point lying around that I need to use that still uses XML. Function proxy would give that to me, would allow me to pass through. But WCF relay wouldn’t. Oh, WCF relay would because it’s XML. If it was REST, then I would probably would go with function proxy. And things like that.
Think about, like, the firewall and the relay based status [SP], the WCF relay, the on-premise gateway, the hybrid connection. Like, how easy to you is going to be discuss opening BizTalk through firewall in some specific cases, versus just setting up a hybrid connection, and setting up a hybrid connection just on the base…on premise gateway just on the BizTalk box that is going to fly much better than installing that in a whole lot of different places. Right?
Where and how you’re going to be security? Like, I could go and secure before it even hits me, just before I start to process, but you need to secure, right? And, finally, the budget. Like, what is going to be from those things the one that fits you better on your current budget? Are [SP] just trying now, there’s an enterprise application, it’s something that is mission critical? So though things comes with different budgets because of that, it give you, like, different options.
So, basically, what I could say on that is, like, it feels like [inaudible 00:34:36] light, and I’m proud enough just with that. Find your thing [SP], right, find your balance. Look at those questions, answer them, and find the best option for you. And the last thing is BizTalk works really well with Azure, but like I said last year, BizTalk loves Azure. Right? If you need to contact me, I’m going to be around the rest of the day. But also you can get more information around those ones.
Fill the form below to get all the presentations delivered as a single zip file in your mailbox.
byJon Fancey & Matt Farmer
byMicrosoft Integration Team