Making waves: breaking down barriers with ultrafast networks
Fri, 15 May 2015 05:00:00 GMT
In this article, Mike O’Hara and Adam Cox of The Realization Group investigate the development of new, ultrafast wireless communication networks for trading financial markets. Mike speaks with Stéphane Tyč, CEO of network provider McKay Brothers, while Adam talks with Tariq Rashid, a long-time data centre executive, and with Alexandre Laumonier, who has published extensive analysis of the state of play in the microwave industry. Together, they describe a fast-changing, competitive technological scene that is revolutionising the world of trading.
Microwave transmission of data has been used in the telecommunications industry for decades. But it is only during the past half dozen years – when the technology has offered the tantalising prospect of both lower data transit costs and greater speed for trading firms – that financial market participants have begun to take microwave seriously. When prices for the fastest available fibre-optic networks shot up and began eating into trading firms’ margins, forward-thinking technology providers and trading groups began hunting for new solutions. Microwave technology, it turned out, fit the bill. The technology has its drawbacks, most notably a reliance on line of sight when setting up point-to-point communication links. But the advantages more than outweigh the disadvantages. Now, microwave and millimetre networks are being built all the time, both by a new breed of network providers and by some of the trading companies themselves. What is more, some people are even starting to think of ways the technology might be employed over previously unthinkable distances, such as across the Atlantic Ocean. It seems that the race to zero has somehow just got a lot faster.
The need for speed
Stéphane Tyč of McKay Brothers has been at the forefront of the microwave network business since it began around 2010. His firm built a pathway from Aurora to New Jersey that today is considered the fastest available. McKay has also built a number of popular microwave links in Europe, connecting different trading ecosystems in London with each other and with Frankfurt. The firm is looking at doing much more.
So what is fuelling all the demand? The simple answer is that speed has always ruled in the markets.
“I’m not sure anything has changed. Fundamentally, you need to be faster than someone else and the amount of time that you need to win by is dictated by the random jitter inherent to the order sending and data publishing on exchanges,” Tyč said.
Different venues have different protocols and different infrastructures, so the degree to which a speed advantage makes a difference often depends on where the trading is taking place.
“So, if when you’re faster by 10 µs, you always win, then, being faster by 10 µs really counts,” Tyč said. “There are exchanges where, if you’re faster by one µs, you’ll always win and others, if you’re faster by 100 µs, you might not win, because of the randomness of the propagation and different orders to the matching engine.”
“You need to be faster than someone else and the amount of time that you need to win by is dictated by the random jitter inherent to the order sending and data publishing on exchange”
Stéphane Tyč, CEO, McKay Brothers
But while the importance of 10 or 100 microseconds can vary from venue to venue, the importance of speed in general is not in question.
In the early days, microwave was pretty much strictly the province of high frequency trading (HFT) firms. Their trading models depend on speed more than anything else.
Tariq Rashid, a veteran UK colocation industry insider, said that after the initial success with the Chicago area-to-New Jersey routes, trading firms immediately became interested in Europe.
“It went from zero to a lot of interest very quickly because firms were seeing the advantages it had gained in the US,” Rashid said. “The interest ramped up very quickly because people saw the inherent arbitrage advantages that microwave gave them.”
Rashid’s data centre decided early on that it would have to facilitate trading firms and network providers with microwave links. “If we hadn't offered it into our facility, they would have built a tower alongside the facility and then come in for the last quarter of a mile or so using fibre, thereby creating a level playing field for participants,” he said.
Meanwhile, after an initial period of spectacular growth, HFT firms began to face a daunting array of obstacles, from increased competition to higher costs. Overall revenues and margins in the past five years are thought to have fallen, while the macroeconomic environment – with rock-bottom interest rates and record low volatility levels – has encouraged an extended period of orderly trading that has provided fewer opportunities for HFT firms to make money. Add to that intense regulatory scrutiny in the wake of the Flash Crash, high-profile mishaps, lawsuits and loud complaints from the buy side. One result of all this has been consolidation in the HFT sector.
“A lot of the bigger HFT players have swallowed up some of the smaller players, so there are fewer HFT players out there,” Rashid said.
And yet, demand for microwave networks is only growing. One reason is that it’s not just HFT firms that are seeing advantages from them.
“It’s just progressively growing and it’s spreading beyond the classical HFTs – especially on the market data side. Everyone’s kind of trying it and it’s very interesting because the price point is attractive. The number of market data clients is steadily increasing. Every month more firms are using our data service,” Tyč said.
By market data clients, he means firms that consume the data, though Tyč notes that the ones interested in microwave are only those which co-locate at an exchange. “There are a few hundred firms in the world that are our natural clients today, though that number continues to expand. More firms - whose strategies are not latency dependent - are taking advantage of being milliseconds more responsive to changing markets.”
“A lot of the bigger HFT players have swallowed up some of the smaller players, so there are fewer HFT players out there”
Typically these would be banks and smaller market makers, which, as the McKay Brothers executive notes, are not like HFTs. “The banks, for instance, have tons of extra layers of security. They cannot just shave the last hundred nanoseconds off their connection. But, still, they need to have good data to execute orders for their clients,” he said. These firms need the most up-to-date data so that their orders and trading models are more deterministic and so they know what’s traded and when. “Microwave latency is the low-hanging fruit of competitive improvement for those firms,” notes Tyč.
Artists, cartographers and technologists
Microwave networks rely on tall towers (the taller the better because of the line-of-sight factor) which network builders use as relay sites. Since regenerating a signal requires time, the fewer the relay sites, and the straighter the line on a map, the lower the latency.
Just as is the case with fibre optic, considerable effort goes into finding ways to make as straight a line from point A to point B as possible, and that means identifying locations for repeater towers that will process signals. In other words, building a microwave or millimetre wave network is as much an exercise in artistry and cartography as it is in technology. After the initial proof of concept in terms of microwaves applicability to financial trading, a large number of networks across Europe have sprung up.
Alexandre Laumonier, an anthropologist who has written extensively about microwave networks, said the main competition now is between the independent providers such as McKay Brothers and others and proprietary trading firms such as Optiver, Jump Trading and Vigilant Global.
“Independent providers need money to build their networks, before selling them to various customers, but trading firms spend their own capital to compete with the providers,” Laumonier said.
He doubts there will be many new entrants into the field given some of the competitive and infrastructural obstacles now in place in Europe. “We could say there is no space for competition anymore – the competition is how you can manage and improve a network by trying to have better routes,” he said.
Better, in other words, means faster.
McKay has current connections that link the LD4 data centre in Slough (one of the world’s largest trading ecosystems) with Frankfurt and with the LHC data centre in the Docklands area of London. The company basically is creating a mesh to link all the major sources of liquidity in London with each other and with Frankfurt.
To give an idea of the speed involved, the roundtrip journey from LD4 in Slough to Frankfurt and back again takes just 4.64 milliseconds on the McKay Brothers network. Among other things, this means that synchronisation and timestamps must be extremely precise or a firm may not know whether an executed order took place first in one location or in the other.
Further afield, various firms are looking at Zurich, Milan and Stockholm. As Laumonier noted in a popular blog that delved deep into the microwave business: “The U.S. Army, NATO and the RAF erected a lot of towers during the second half of the 20th century, and these towers are now invaded by dishes owned by various trading firms/microwave providers.”
In North America, efforts are underway to build connections between Toronto and trading centres in the United States.
But perhaps the most interesting activity is taking place far from any major trading centre.
The Hibernia project will create a new, faster fibre-optic connection under the Atlantic Ocean, with a handoff at LD4. Rather than reaching the United Kingdom at Land’s End, where an array of microwave networks currently extend, the new trans-Atlantic network will meet land at Brean, not far from Bristol.
While there has been some uncertainty as to whether there will be an opportunity for microwave networks to connect with Hibernia at or near Brean, the expectation is that that the networks extending to Land’s End will soon become redundant. It’s an example of how quickly the map can change when it comes to finding the optimum routes from one centre to another.
Cloak and dagger
As can be expected, various firms have been building pathways to Brean and buying up frequencies in the UK. In fact, frequency and tower “squatting” has become a common issue, according to Laumonier. He said some firms ask for licences for a tower they know they won't use so that competitors can't use the tower anymore; others install dishes on towers without legal authorisation, or before they received authorisation; and others are authorised to install equipment of one type or another, such as a certain-sized dish, but end up installing different-sized dishes.
“The fact is, there are some discrepancies between the public data available (the legal authorisations) and what you can see on some towers,” Laumonier said. “Some regulators seem to be more vigilant than others. It's clear that the firms who really respect the law ask for more regulation and control in order to prevent ‘unfair’ competitors from having illegal advantages.”
“The fact is, there are some discrepancies between the legal authorisations and what you can see on some towers. Some regulators seem to be more vigilant than others”
In the UK, regulators make data about who owns what network and what frequencies public. But Britain is an outlier in this respect. Tyč said France publishes some information, but Belgium and Germany do not provide any public data. “I really hope there’s a European directive that makes it all available, that would be very good,” he said.
Questions about transparency extend not just to who owns which networks, but also to how fast they really are. Laumonier said: “Those who publish latencies obviously do it to show that they are faster than competitors. But what do those latencies exactly mean? Are they the latencies between the dishes of two exchanges? Do they include the few microseconds needed by the data to travel between a dish at the top of a data centre and the trading firms’ co-located servers that process the data in the heart of the data centre?”
For example, McKay Brothers notes that between data centres in Aurora and nearby Cermak, where futures are traded, the latency is 0.184 milliseconds if a trading firm server is installed on the 2nd floor. But it is 0.183 millisecond if the server is on the 9th floor. In other words, what Laumonier calls “McLatencies” can vary by a millionth of a second from one rack-to-rack route and another.
“I'm not sure the other public latencies published by microwave competitors are so precise. That said, each microsecond counts now,” he said.
Ultimately, only the trading firms themselves really know what latencies they are dealing with actually are, whether that stems from the route provided or factors such as what floor their servers are on.
There are a variety of factors that can affect latency, but generally speaking the best route will be based on how close a microwave network can get to providing a straight line between two points. Laumonier said McKay data shows that its Illinois-New Jersey route is only three kilometres more than the 1,180 kilometre length of a straight line between the two points.
“This is impressive as McKay Brothers was not the first firm to build such a network. Probably three or four trading firms had already built their own proprietary networks before McKay, but it seems that they didn't really take as much care with the design of the paths. They only wanted to be faster than fibre-optic, and they were so. But McKay decided to design the shortest, and fastest, network,” he said.
Laumonier noted that the US McKay microwave network only needed 22 segments to join the two areas, about two-thirds the number of other competitors. That saved money because fewer dishes and cables were needed. “Achieving the shortest microwave network between two areas is a form of art, and an economical challenge too,” Laumonier said.
The only way is up
The dizzying array of networks criss-crossing Europe that Laumonier has identified is testimony to how popular microwave and millimetre wave have become. Linking cities and venues across hundreds of kilometres via this technology has become relatively straight-forward. But what about trying to go much, much further? Could wireless communication, for instance, ever take place across the Atlantic Ocean?
Even if the technology could be harnessed to allow a signal to travel such vast distances, an immediate problem comes up: the curvature of the earth. After a certain point, the shape of the planet eliminates line of sight. At the same time, building towers across the ocean is not thought to be realistic.
There has been talk of specially designed balloons that could relay signals. Google, with its Loon project, is currently exploring the use of balloons for making the internet accessible in all parts of the globe.
Another intriguing technological answer may lie in what is called Tropospheric scatter, or troposcatter. This technology uses the troposphere, the lowest level of the earth’s atmosphere, like a kind of backboard. A signal is sent up into the air at an angle; it hits a device in the troposphere and bounces back down to a dish as much as a thousand kilometres away
For shorter distances, Laumonier does not expect a better technological solution than microwave, even considering new methods such as laser technology (which is hampered by its vulnerability to fog).
“The next challenge seems to be wireless networks between continents, i.e. between London and New Jersey for instance,” Laumonier said. He knows of one firm that is currently developing technology inspired by the old troposcatter method.
What he finds interesting is how the most modern of firms are attempting to harness technology that is actually quite old. “It is fascinating to see how the trading world – HFT or not HFT – Is trying to encircle the globe with all these microwave technologies,” Laumonier said. “They are old technologies now used in a very modern way,”
Rashid is in no way surprised by this turn of events.
“Throughout history, people have been using technology to gain competitive advantage, lower latency tools to help transport information from one destination to another in order to make better trading decisions. Whether microwave or millimetre wave is the last, I suspect not,” Rashid said. “I'm sure that there will be other technologies that will be coming along in the future.”
In addition to microwave technology, there are two other forms of wireless communication that are being used to trim milliseconds: millimetre wave and laser technology. All three go at the speed of light. But there are different technical factors that affect each. What are some of the pros and cons?
Microwave: This occupies the radio spectrum between 1 and 30 GHz. It can handle less data than fibre-optic links but it is far faster. When light travels through glass, as it does in a fibre-optic cable, it is estimated to be 33% slower than when traveling through air. This is because glass has a higher refractive index. But signals need to be repeated when traveling via microwave. Tyč says microwave can go over 100 kilometres.
Millimetre wave: This is the same speed as microwave but more bandwidth. It is between 30 and 300 GHz. One drawback is the distance allowed between repeated signals is much smaller –Tyč says about 10 kilometres. For the network providers catering to this demanding set of clients, both microwave and millimetre wave are important. Heavy rain or flying birds are said to be able to disrupt signals for both communication methods.
Laser: This technology is currently the highest bandwidth. It is just being introduced for trading networks. It was developed in the 1990s for the purpose of gathering images from outer space and was adapted for use in military jets for communication. Fog will completely stop laser communication and rain will attenuate it also. In all cases of wireless communication, network providers build in fall-backs that allow switching to other methods in the event of disruption.
Images from Laumonier (click on images to see higher resolution):
An illustration from Laumonier of the impact Hibernia will have for trading networks. The top image shows the south of England and the numerous networks stretching to Land’s End to link up with trans-Atlantic fibre optic networks; the bottom image shows how many of those microwave connections will be redundant once Hibernia is operational. The question is whether Hibernia will allow microwave links at Brean.
For more information, visit: