http

Using NATS instead of HTTP for inter service communication

In the new micro service architecture the traditional monolithic application is broken down into multiple small scoped domains. That brings several benefits both from a development perspective since micro services can be iterated individually and also performance since they can be scaled separately.

Now with the move to a micro service architecture one common problem is how to make all the services communicate with one another?
Typically an application exposes a REST API interface to the client so naturally the first option is to start using HTTP.
The question then becomes, is HTTP the best messaging protocol for inter service communication?

Looking at the HTTP's definition from the official RFC

The Hypertext Transfer Protocol (HTTP) is an application-level  
protocol for distributed, collaborative, hypermedia information  
systems. It is a generic, stateless, protocol which can be used for  
many tasks beyond its use for hypertext, such as name servers and  
distributed object management systems, through extension of its  
request methods, error codes and headers.  

It is clear that HTTP wasn't built to provide a communication layer between micro services.

On the contrary, looking at the NATS protocol definition

Open Source. Performant. Simple. Scalable.  
A central nervous system for modern, reliable,  
and scalable cloud and distributed systems.  

It becomes clear that it was built with new modern systems in mind.

One thing to keep in mind is that a
RESTful API could be built on top of NATS across all micro services. The key point is that HTTP is not the best protocol for inter service communication.

Benchmark

In theory it all sounds good, but in practice how much faster is NATS?
To test both approaches a simple benchmark was put together trying to mimic a real world example.

The idea is that a HTTP proxy server will accept client connections and then route the request to the appropriated internal micro service. In one scenario HTTP was used to proxy the messages and on the other NATS.

For the HTTP proxy a node-js express server was used and for the microservice a golang app using the httprouter and the official NATS go-client

To generate the client traffic that would be hitting the proxy the bench-rest tool was used.

Sequential requests with no payload

Overall NATS performed 3x faster than HTTP.
To process 50,000 requests using HTTP as the messaging protocol between the proxy and the micro service it took ~12 mins while using NATS it only took 5.


Sequential requests with 1K payload

When adding some data to the requests the behavior was similar and NATS still processed messages ~3x faster than HTTP.

Parallel requests

When a factor of concurrency was added to the tests that's when NATS performed the best.
To process 50,000 requests with a batch of 2,500 concurrent requests at a time NATS took 1.6 mins while HTTP took 18 mins.

Overall the results shown that NATS beats HTTP both in speed and throughput.

The source code used for the benchmarks can be found here

Enabling CORS on a node.js server, Same Origin Policy issue

Recently we faced the famous “XMLHttprequest doesn’t allow Cross-Origin Resource Sharing” error.

To overcome the problem a very simple solution was needed.

Below I’ll try to give a quick overview of what is CORS and how we managed to work around the issue.

Cross-Origin Resource Sharing – CORS

In a nutshell CORS is the mechanism that allows a domain to request resources from another domain, for example, if domain http://websiteAAA tries to request resources from http://websiteBBB the browser won’t allow it due to Same Origin Policy restrictions.

The reason for having Same Origin Policy rules applied on the browser is to prevent unauthorized websites accessing content they don’t have permissions for.

I found a great example that emphasizes the need to have Same Origin Policies enforced by the browser: Say you log in to a service, like Google for example, then while logged in you go and visit a shady website that’s running some malware on it. Without Same Origin Policy rules, the shady website would be able to query Google with the authentication cookies saved in the browser from your session, which of course is a huge vulnerability.

Since HTTP is a stateless protocol, the Same Origin Policy rules allow the browser to establish a connection using session cookies and still keep each cookie private to the domain that made the request, encapsulating the privileges of each “service” running in the browser.

With that being said, imagine a situation where you, as a developer, need to communicate with an API sitting on a different domain. In this scenario you don’t want to hit the Same Origin Policy restrictions.

Workaround 1 – Request resources from a server

The most common way to get around this problem is to make the API request from your own server, where Same Origin Policy rules are not applied, and then provide the data back to the browser. However, this can be exploited:

Last semester I created an example of how an attacker would be able to spoof whole websites and apply a phishing attack circumventing Same Origin Policy restrictions.
The attack structure was very similar of how ARP poisoning is done.

A very brief overview of the attack:

  1. The user would land on a infected page
  2. The page would load a legitimate website by making a request from the attacker’s server where Same Origin Policies are not applied.
  3. The attacker would inject some code in the response to monitor the vicitms activity
  4. After the victm’s credentials were stolen he would stop the attack and redirect the user to the original requested page.

By spoofing the victim’s DNS it would make even harder to detect the attack, but even without DNS spoofing this approach would still catch some careless users.

All the code for the example is available on github
The attack was built on top of a nodeJS server and socketIO
The presentation slides for the attack can also be found here

Workaround 2 – JSONP

Another way to circumvent the problem is by using JSONP (JSON Padding). The wikipedia articles summarizes in a clear and simple way how JSONP works.

The magic of JSONP is to use a script tag to load a json file and provide a callback to run when the file finishes loading.

An example of using JSONP with jquery:

[sourcecode language=”javascript”]
$.ajax({
url: "http://website.com/file.json
dataType: ‘jsonp’,
success: function (data) {
// Manipulate the response here
}
});
[/sourcecode]

Even though making requests from your server or using JSONP can get around the Same Origin Policy restrictions it is not the best solution, which is why CORS started being implemented by the browser vendors.

With CORS a server can set the HTTP headers of the response with the information indicating if the resources can or can’t be loaded from a different origin.

If you are curious and want to snoop around looking into the HTTP response headers of a page, one way to do that is to use the developers tools that come with WebKit.
Below is a screenshot of all the resources loaded by the stack overflow home page.
Screen Shot 2013-02-14 at 6.34.24 PM

As you can see in the screenshot, the script loaded from careers.stackoverflow.com/gethired/js had the following HTTP header options appended to its response:

  • Access-Control-Allow-Headers
  • Access-Control-Allow-Methods
  • Access-Control-Allow-Origin

That means that if you want to make an ajax call to carrers.stackoverflow.com/gethired/js from your own page, the browser will not apply Same Origin Policy restrictions since the careers.stackoverflow server has indicated that the script is allowed to be loaded from different domains.
*An important note to make is that only the http://careers.stackoverflow.com/gethired/js has the Same Origin Rules turned off, however, the careersstackoverflow.com domain still has them enabled on other pages.

This means you can enable the header options on a response level, making sure a few API calls are open to the public without putting your whole server in danger of being exploited.

This lead us to our problem.

The Problem

In the set up we currently have, one computer plays the role of the API server and we were trying to query that API asynchronously from the browser with a page being served from a different domain.

The results, as expected, were that the call was blocked by the browser.

Solution

Instead of hacking around and trying to make the requests from a different server or using JSONP techniques we simply added the proper header options to the responses of the API server.

We are building our API on a nodeJS server, and to add extra headers options to the response could not have been easier:

First we added the response headers to one of the API calls and it worked perfectly, however we wanted to add the option to all our API calls, the solution: use a middleware.

We created a middleware which sets the response header options and pass the execution to the next registered function, the code looks like this:

[sourcecode language=”javascript”]
//CORS middleware
var allowCrossDomain = function(req, res, next) {
res.header("Access-Control-Allow-Origin", "*");
res.header("Access-Control-Allow-Headers", "X-Requested-With");
next();
}

app.configure(function () {
app.set(‘port’, config.interfaceServerPort);
app.set(‘views’, dirname + ‘/views’);
app.set(‘view engine’, ‘jade’);
app.use(express.favicon());
app.use(express.logger(‘dev’));
app.use(express.bodyParser());
app.use(express.methodOverride());
app.use(allowCrossDomain);
app.use(app.router);
app.use(express.static(path.join(
dirname, ‘public’)));
});

app.configure(‘development’, function(){
app.use(express.errorHandler());
});

// API routes
app.get(‘/list/vms’, routes.listGroup);
app.get(‘/list/vms/:ip’, routes.listSingle);
app.get(‘/list/daemons’, routes.listDaemons);
[/sourcecode]

That’s it for CORS, later we’ll cover another cool header option, the X-Frame-Options

If you are interested in finding more about Same Origin Policy or CORS check out this links:
http://en.wikipedia.org/wiki/JSONP
http://geekswithblogs.net/codesailor/archive/2012/11/02/151160.aspx
https://blog.mozilla.org/services/2013/02/04/implementing-cross-origin-resource-sharing-cors-for-cornice/
https://developers.google.com/storage/docs/cross-origin
http://www.tsheffler.com/blog/?p=428
http://techblog.hybris.com/2012/05/22/cors-cross-origin-resource-sharing/
http://security.stackexchange.com/questions/8264/why-is-the-same-origin-policy-so-important
http://www.w3.org/TR/cors/
https://developer.mozilla.org/en-US/docs/HTTP/AccesscontrolCORS
https://developer.mozilla.org/en-US/docs/Server-SideAccessControl
http://www.bennadel.com/blog/2327-Cross-Origin-Resource-Sharing-CORS-AJAX-Requests-Between-jQuery-And-Node-js.htm