Are PCs to blame for the financial mess we're in?
8th Oct 2011 | 09:00
How algorithmic trading has reshaped the stock market
Are PCs to blame for the financial mess we're in?
During 6 May 2010, the Dow Jones Industrial Average - the second oldest US market index and one of the most commonly used indices to reflect the state of the market as a whole - saw its biggest and fastest decline ever.
By 2:42pm that day, the Dow had dropped 300 points since the start of trading that morning. By 2:47pm, a mere five minutes later, it had dropped by a further 600 points. Suddenly, hundreds of stocks had had their prices slashed to almost zero. The crash instantly wiped out almost $1trillion in stock value.
Even major blue chip companies weren't safe from its effects: Accenture shares fell more than 90 per cent from $40 to $0.01. Even more amazingly, just 20 minutes later the market had bounced back up to almost the same level it was at previously.
This major wobble in global economic markets sent shockwaves throughout the financial world, mainly because it wasn't at all clear what had caused it. A year on, and despite some lengthy investigations by the US Securities and Exchange Commission (SEC) and the Commodity Futures Trading Commission (CFTC), it still isn't clear, but one likely causative factor is high frequency automatic algorithmic trading.
That's the buying and selling of shares not by market traders, but by financial software that automatically analyses the financial market and buy or sells shares based on that information at lightning speeds. The idea of a broker on the phone shouting, 'Buy, buy, sell, sell!' is far removed from how a lot of people are now trading on the stock market.
In some markets, more than half of the trading is done by computer programs, which use algorithms to decide whether to buy or sell stock. Generally, these are simple systems that are able to generate multiple small profits by systematically trading on electronic markets in the blink of an eye. The algorithms are usually written by proprietary trading firms and sold on to other traders.
Within the algorithm will be certain values that determine exactly when and where to trade, including the price of a security, the size of the fund available, timing considerations like how quickly an order can be executed, the best time to place an order to make sure it will be executed, how likely an order is to be filled and the overall cost of each transaction. Put simply, if the algorithm's conditions are met by a particular market scenario, the shares are traded automatically in a matter of milliseconds (and this speed is dropping all the time).
Remember that this process is entirely automatic, with no human involvement apart from what was laid down in the algorithm by the software's programmer. Some people are even buying these programs without knowing what the algorithms contain, merely trusting that they will do what they say they will do.
This is known as black box trading, which refers to the fact that you can't look into the black box to see how it works. Other firms create their own personalised algorithms, and these are evolving into much more complex programs over time as more and more information and safeguards are incorporated into their development.
The average lifetime of these trading algorithms tends to be extremely short - sometimes only a couple of days. So useful are these algorithms that they are becoming a very valuable commodity in themselves - in fact, there have already been two convictions in the US courts against people who have stolen proprietary code from these types of programmes worth millions of dollars.
The impact that these trading algorithms could have on economies was made worryingly clear by Assistant United States Attorney Joseph Facciponti when describing the case against former Goldman Sachs employee Sergey Aleynikov, who was convicted of such a theft earlier this year.
"The bank has raised the possibility that there is a danger that somebody who knew how to use this program could use it to manipulate markets in unfair ways," Facciponti said.
The dawn of automation
Clearly then, these algorithms have the power to affect the financial markets, but should we be worried about them? Don't they just provide another level of automation to make the traders lives easier? Surely they aren't actually causing any problems to the financial markets by themselves?
Andy Haldane, the Bank of England's Executive Director in charge of Financial Stability, isn't so sure. As he said in a speech on June 2011 in Beijing,
"Driven by a potent cocktail of technology and regulation, trading in financial markets has evolved dramatically during the course of this century. Platforms for trading equities have proliferated and fragmented. And the speed limit for trading has gone through the roof. This rapidly changing topology of trading raises some big questions for risk management. There are good reasons, theoretically and empirically, to believe that while this evolution in trading may have brought benefits such as a reduction in transaction costs, it may also have increased abnormalities in the distribution of risk and return in the financial system. Such abnormalities hallmarked the Flash Crash."
So what effect does he feel that high frequency algorithmic trading has had?
"High frequency trading (HFT) has had three key effects on markets," explains Haldane. "First, it has meant ever-larger volumes of trading have been compressed into ever-smaller chunks of time. Second, it has meant strategic behaviour among traders is occurring at ever-higher frequencies. Third, it's not just that the speed of strategic interaction has changed but also its nature. Yesterday, interaction was human-to-human. Today, it's machine-to-machine, algorithm-to-algorithm. For algorithms with the lifespan of a ladybird, this makes for rapid evolutionary adaptation."
When Haldane talks about a massive change in the way financial markets are run, he's not joking. There really has been a change caused by the increase in the speed of what's possible - and it's come around fast. Certainly too fast for all the implications to be fully realised, as Haldane explains:
"The average speed of order execution on the US New York Stock Exchange has fallen from around 20 seconds a decade ago to around one second today." And that's just the average. As the use of high frequency trading increases, that average is going to decrease considerably.
As Haldane explains, electronic trading itself is fast approaching light speed - the speed limit of the universe.
"A decade ago, execution times on some electronic trading platforms dipped decisively below the one-second barrier. As recently as a few years ago, trade execution times reached 'blink speed' - as fast as the blink of an eye. At the time, that seemed eye-watering at around 300-400 milliseconds, or less than a third of a second, but more recently the speed limit has shifted from milliseconds to microseconds - millionths of a second. Several trading platforms now offer trade execution measured in microseconds.
"As of today, the lower limit for trade execution appears to be around 10 microseconds. This means it would, in principle, be possible to execute around 40,000 back-to-back trades in the blink of an eye. If supermarkets ran high frequency trading programmes, the average household could complete its shopping for a lifetime in under a second. Imagine that."
The rise of HFT
Of course, the fact that HFT allows you to do a lot of trades quickly isn't necessarily a problem, but we do need to consider how much trading is being done automatically, without human intervention. The fear here is that soon the basis of the financial markets may change.
If investors start to believe that stocks aren't valued on a company's expected future earnings, but actually get their value from computers trading against other computers for speed and advantage, then the financial markets will seem like too much of a gamble to invest in. So how much trading nowadays is being done solely by computers?
Unsurprisingly, with the edge it offers over other traders in terms of speed and even knowledge, it is steadily increasing. For example, as Haldane explains, "In Europe, since 2005, HFT has risen from a tiny share to represent over 35 per cent of the equity market. In Asia and in emerging markets, it's growing fast from a lower base. What's true across countries is also true across markets.
"HFT is assuming an ever-increasing role in debt and foreign exchange markets. In some futures markets, it already accounts for almost half of turnover. In the space of a few years, HFT has risen from relative obscurity to absolute hegemony, at least in some markets."
Changing the game plan
That's not all algorithmic trading has done - it's also changed the way the financial game is played. In the past - a mere decade ago, before computer automation - traders and big financial institutions would make money by dealing in reasonably simple instruments: bonds, equities and the foreign exchange.
As automation kicked in, the profit margins on these trades were subsequently driven down. This had the knock-on effect of encouraging these firms to reduce their own costs and find competitive advantage by automating as many of their simple trades as possible.
Whereas in the past there would be hundreds of traders buying or selling small amounts of shares in an effort not to tip off the competition as to what they were doing, now it could be handled by computers, gradually buying or selling small numbers of shares over controlled time periods.
The massive increase in speed provided by these automatic processes also allowed the big financial houses to see other places where algorithms could be employed to generate wealth, places where the split second timing could allow inefficiencies in financial reporting to be exploited.
One example is arbitrage opportunities. Here, assets that are basically identical are bought and sold at the same time to make a profit from a price difference between the two assets. So if the same stock is listed at £20 on the New York Stock Exchange and £19.95 on the Philadelphia Stock Exchange, you can guarantee yourself a profit by buying stock on the Philadelphia stock exchange and then selling it straight away on the New York Stock Exchange (provided the difference offsets any dealing costs).
With algorithmic trading, even if those differences only appear for a split second, the financial houses can act on them. It's this form of algorithmic trading that has some regulators worried - they think that when searching for volatile markets to exploit, these forms of trades could be adding to the overall volatility of the market themselves.In actively seeking out these imperfections, they exacerbate their effect on the markets.
This is the news
Another useful trick that algorithmic trading can employ is to automatically scan incoming financial news feeds for buzz words or phrases like 'interest rate hike', for example. When the computers detect these they can instantly trigger the same dealing strategies that were employed when a similar situation occurred in the past.
This instant reaction allows firms to get the drop on their competitors and can prove very profitable. This is becoming very popular with the larger firms, and news feed providers have even changed the structure of their feeds so they can be more easily read by computers.
The problem here though is that if you have a lot of computer programs reacting in a very similar way as soon as news is produced, it can cause a bad reaction on the markets - especially if this then triggers further rounds of automatic trading based on the previous automatic trading. This is a major shift in the way the financial world works.
Suddenly, it isn't humans absorbing the news and acting on it based on experience. Instead, it's just a computer going through a preset process automatically. Clearly, if this form of algorithmic trading becomes endemic, the future of some stocks and shares will rely not on how the companies are performing, but how that performance is reported in the news.
Also, imagine the problems that could occur if the news writer got it wrong. In the fast-paced world of financial news reporting, it isn't unimaginable that a mistake could be made, if such a mistake could now automatically create a massive financial black hole, it isn't surprising that there are calls for regulation on how algorithmic trading can be used.
Running to stand still
Automatic trading has had another effect on the market - it's increased its load (the number of deals that are being done daily). The average trade size on equity markets in 1996 was over 1,600 shares, in the past decade it has decreased to a mere 400 shares per trade. That means there are four times as many trades needed to deliver the same volume as 10 years ago.
This fourfold increase in trade data means that when something goes wrong it's a lot harder to untangle what caused what when, especially as these trades are very hard to track back to particular companies. This is why it's so hard to lay the blame for the 2010 Flash Crash at any particular door - there just isn't the paper trail needed to work out what happened.
It's likely that some regulation may be introduced, allowing an audit trail to be seen clearly, but despite encouraging noises from regulators this doesn't seem to be happening soon.
This extra load has another very important consequence - the diminishment of risk management. Nowadays risk management is taking a back seat as it is almost impossible to consolidate the amount of data being generated from high-frequency trading. Number crunching can't keep up with the uncontrolled automation, which means that it is increasingly difficult to judge the risk of an action on the markets.
It's not just confusion over risk either - the fact that most people can't see what factors the algorithms they use for trading contain has, some argue, led to a lack of transparency in the markets.
Obviously, the people who create the algorithms need to protect their intellectual property, and so provide closed systems to the program buyers, but this means some smaller traders using HFT aren't even totally sure what the algorithms they are using for trading contain. This also means that government regulators can't see what is happening inside these black boxes either.
Some say this allows traders to hide abusive activity and they should have to show what they are doing to regulators. Traders point out that these algorithms are increasingly becoming their life-blood, and sharing the secrets of their own proprietary code would give others an unfair advantage.
Of course, this means that this type of black box trading impinges on one of the biggest facets regulators are keen to have in the marketplace - visibility. This is compounded by the fact that those high-frequency traders with the better algorithms have access to better information than other traders, and they can make the deal microseconds before anyone else. Where do these massive advantages offered by the latest algorithms fit in if markets are meant to be level playing fields?
One of the biggest questions about algorithmic trading is just how profitable it is. There are definitely big bucks being generated by established firms, but what about the newcomers who are trying to take advantage of this new type of trading?
Some industry insiders are already speculating that the profitable days of algorithmic trading are already in the past (although this means profits of merely millions as opposed to billions). It seems that increasing competition in the high frequency trading world means it's becoming harder and harder to find those little inconsistencies that provided so much profit at the introduction of this new way of trading.
Trading volumes have since slumped, as has volatility, meaning algorithmic trading has nothing to take advantage of. Of course speed is a very useful factor offered by automatic trading, but now it seems, you need to have algorithmic trading just to keep pace with the rest of the field.
How dangerous is algorithmic trading?
There is little doubt that algorithmic trading isn't for the faint of heart - if your algorithm isn't up to date, or you have applied it the wrong way or to the wrong market, it's possible to lose a lot of money very quickly. Human error can quickly become very costly in the world of automated high frequency trading.
As the Bank of England's Andy Haldane pointed out: "In 2003, a US trading firm became insolvent in 16 seconds when an employee inadvertently turned an algorithm on. It took the company 47 minutes to realise it had gone bust."
This, of course, is an extreme example, but it does highlight that by putting themselves at one remove from the actual trading, dealers may quickly lose the understanding of what is going on in the markets, and even feel disconnected from the sums of money they are playing with.
Despite all this doom and gloom surrounding high frequency algorithmic trading, many people are embracing this type of new technology with open arms. After all, electronic trading isn't a new thing in the financial markets - the first electronic exchange (NASDAQ) is over 40 years old.
One such proponent of algorithmic trading is Brian R Brown, author of Chasing the Same Signals: How Black-box Trading Influences Stock Markets from Wall Street to Shanghai. Although he agrees that caution should always be used, he thinks the new technology should be viewed as a positive thing.
"Certainly there are mixed opinions on the financial crisis," he says, "but the overwhelming evidence of cause was bad lending practices and aggressive risk taking, which computerised trading cannot be blamed for.
"Having said that, the speed and pace of today's market, as a consequence of black box strategies, definitely have eroded the common person's perspective of the economy, and that needs to be addressed by regulators and the industry. In light of the Flash Crash, our industry needs a more granular view of market activity to better audit and regulate systematic risk; that is, we have historically audited large shareholders, but now we need to audit the high-frequency participants, who may not be in the league tables of shareholders, but certainly have an influence on the stability of share prices."
But should we be laying the blame for the Flash Crash on computers? Could this form of trading ever have been possible before their existence?
"The usage of mathematical models for stock price forecasting has a history as long as the financial markets," explains Brown, "but the game changed with the advancements of technology that allowed strategies to be fully automated. The industry is very different than it was just five years ago, let alone a decade ago."
So does he see algorithmic trading as bad for the market? "There are mixed views on the impact of black-box strategies to the markets. Many strategies are widely viewed to have a stabilising effect on the market by buying out of favour stocks, or selling over-valued stocks, effectively dampening volatility. Having said that, evidence suggests that the 'tail' events are greater than ever, which is a consequence of today's complexity of our markets."
So would he put recent volatility in the market, such as the flash crash, at the door of computer-led trading? "Certainly the instances of rapid share price movements have been increasing in most global markets," explains Brown. "However, the post-mortem of the flash-crash indicated that human error was the root catalyst for the rapid impact in the S&P futures. What's most concerning is that an event in one market, ripples through to global markets, raising volatility to historical levels. We certainly have new types of 'systematic' risks that weren't in the markets a decade ago."
It's clear that Brown is aware of the risks, but he is still optimistic about the future: "Despite differing views, the recent advancements in trading technology have overwhelmingly improved market efficiency, reduced costs to investors, opened up market access and improved execution quality for investment managers. There is a teething process at times, but active investors should recognise the benefits. A retail investor in Chichester can now trade the Brazilian market just as easily as a professional institution."
Of course, computers play a much bigger role in the financial world than just high frequency trading. One person who is keen to lose the automation and bring people back into the finance markets while still using computers to do so is Jonathan Assia, CEO of Etoro.
Etoro is the world's largest investment network, with over 1.5 million users in over 130 countries.
Etoro has been championing social trading, which lets every investor see, follow and copy the actions of other investors in real time. It is Etoro's mission to open the financial markets to every individual and make them accessible through a simple, transparent and more enjoyable way to trade currencies, commodities and indices online.
We contacted Assia to ask him whether social trading using computers has opened up the financial markets to people in a way that didn't exist before.
"At the heart of social trading is a large community of investors from all walks of life and from every region of the world. The rich diversity of people who invest their money creates the largest knowledge base of free investment advice that every one can instantly tap into. It lets people drill down into each traders profile and see their performance record, risk appetite and investment strategy.
"Furthermore, people can now apply social indicators to determine a trader's performance, which is reflected by the number of followers and the number of people actively copying the investment decision and actions of any given trader. Another indicator is the rate of responsiveness and the level of a trader's/investor's interaction with the community, and the circle of direct followers that provide further information about a trader's personality, performance and investment strategy. It's this collective wisdom, often described as the wisdom of crowds, that provides novices, trader and pros with valuable information to form a conscious investment decision."
Transparency for all
Assia is certain that this form of sharing trading expertise is the future, but he wonders how quickly this transparency will spread.
"The question is whether traditional financial institutions are ready to open their doors and grant users full transparency into their financial structure," he says. "Surely they would find themselves in a very challenging position as they are not ready yet for this type of information revolution that could disclose performance inconsistencies, resulting in unwanted questions. It would put them in competition with every consumer turned trader in a world where performance is the defining factor for consumers choosing their (indirect) money manager.
"Many financial markets are regulated to provide consumers with a sense of security to invest their funds. However, many of these regulations were enforced because financial institutions and financiers were finding ways to use information, not available to the general public, to generate tremendous profits for themselves and their organisations.
"But what if 'inside' information becomes available to everyone for free? Is it then still considered to be inside information? Social investment networks like Etoro provide free information and full transparency on every executed trade in their network. This valuable information is available to everyone instantly and easily.
"Furthermore, the ground-breaking difference in most cases is that consumers copying a specific trader know upfront that the trader is investing his or her own funds. Therefore, if they do well, every CopyTrader will do well. This interlinked trader-investor relationship is one of the most important trust building blocks that is currently not available in tradition financial channels - the majority of consumer funds are invested by fund managers and traders who are not known to the investor, and who themselves have often not invested with their own money, which constitutes a much looser trader-investor relationship."
Man vs machine
So what does Assia think about automatic trading compared to social trading? "Whenever we leave all decisions to automated computer systems then we shouldn't be surprised if at some point an unexpected event topples everything we have learned so far or were able to anticipate," he says.
"Recent events have proven again and again that financial stability is at risk when confronted with unexpected events. Technology, just like in every other industry, was invented to provide competitive advantages. Algorithmic trading is already very entrenched in the financial investment community, it surely has its advantages in certain (predictable) situations - faster execution, no emotions, macro trend identification, a high amount of data processing and more. However, the human factor has some additional features to offer that a computer can't match - sensibility, intuition, experience, courage, creativity and most of all the ability to assess complex situations by filtering out the noise and to focus on the bare essentials. All of these human attributes are inherently connected to social networks, and coupled with the 'wisdom of crowds', they provide a powerful information network to allow clever investment decisions.
"Furthermore," Assia adds, "algorithms are also a reflection of a programmer or inventor's investment personality, and as such they are imperfect by nature."
Risks of algorithms
Does Assia feel that algorithmic trading could be damaging to the financial markets? "Algorithmic trading can influence the markets if the financial institutions using them are carrying with every trade decision large amounts of funds.
"Many of the algorithms use similar base functions to execute upon current market conditions and influencing events, which could trigger an avalanche of trading activity that might lead to an unfavourable outcome.
"Leaving investment decisions in the hands of computer generated programs can be dangerous if the markets become unpredictable. In these situations human intelligence would supersede any artificial intelligence as it incorporates a lifetime of experiences and a macro-economic view of global events to help form decisions."
What about algorithmic trading that responds automatically to buzz words that appear in news feeds? "There is a risk that many of the algorithms would generate similar investment decisions as they all incorporate the same news data set," says Assia.
"Possibly the best solution would be to use a hybrid (human/computer) approach to make use of each entity's best traits. However, we should always consider human judgement superior because it incorporates intangible attributes that no computer program can duplicate."
Despite these worries about automatic trading, Assia still sees some crossover between his company's social trading and algorithmic trading:
"Social investment networks apply a people-based trading algorithm. Investors can create a portfolio consisting of traders with different asset classes, risk appetite and investment strategies. They can then decide to automatically copy every trade, but with the advantage of ultimately diversifying their investments by incorporating every human and technical parameter with every indirectly executed trade.
"It's a similar yet very different approach to algorithmic trading. However, at some point someone will come along and try to create an algorithm that effectively picks the right people instead of financial instruments to create the winning investment strategy."
Computers in charge
If automated financial technologies can be trusted, does Assia see a future where computers are in charge of entire economies?
"Orwell would say 'I told you so'. Leaving the economy at the mercy of computer programs is just irresponsible and dangerous. No computer can incorporate the human elements that have built economies. Intuition, experience, compassion, courage and humanity cannot be coded into a program and all of these attributes are at the heart of every functioning economy.
"How many times have decisions been made in favour of the people that a computergenerated decision would have ignored? Macro economy is on the opposite side of what the computer is all about. The idiom clearly states (to miss the forest for the trees) so what if you could crunch all the available market data but still miss the really important things that define us as human beings?"
What if we were to do away with computers completely, then? If we didn't have computers we wouldn't be able to deal in such mind-bendingly large numbers, preventing us from the boom and bust economies we so often see? Does Assia agree that computers have allowed us to hide from the size of the figures involved?
"Possibly not. The minute we have detached ourselves from the physical currency and moved to a virtual state of money exchange, the human touch is lost. This is possibly the most dangerous aspect because it triggers riskier investment decisions - often the traded volumes are much more than the regular consumer could comprehend, and it is in our human nature to lose the conscious connection to the tools of our daily trade. How do investment managers cope with the fact that they are risking billions of dollars? They simply rationalise it to stay sane. Using computers to make investment decisions further releases the acting trader from their responsibilities to apply their best human judgment to every investment decision they make.
"Without computers people would have to communicate more and the impact of their decisions would be felt. It would place them at the centre of every decision and it would be their name signing off on every executed trade. I'm convinced that if we would have to hand-carry suitcases filled with money or valuables to every trade we would make more careful investment decision and we would reduce the amount we are willing to put down for a specific trade."
Does this mean that computers have made us poor, or is it just that another bubble has burst? "The financial market might be the greatest bubble of all time, but instead of accepting the fact that there is something wrong at its base, we find ways to create more virtual liquidity in order to compensate for the underlying problem. The question is, when will virtual wealth outweigh the actual wealth of goods and what will trigger [a meltdown]?
"History is full of events that have wiped out virtual wealth and have returned humans to the real world, where goods are the actual currency," Assia explains. "An apple becomes the most expensive good when you are hungry and even a million dollars in virtual currency cannot pay for it. Humans have experienced these conditions in the aftermath of wars, where a collapsed economy regenerated itself based on real goods and a more reasonable approach to measure the value of knowledge, skill and physical labour.
"Computers have brought about an incomprehensible improvement in our productivity, and in the ability to let our imagination run wild and create tremendous achievements in every field. In most fields, computers have helped us to improve human value - whether in medicine, science or education - but when it comes to finance, computers have detached us from the reality of how much a buck is actually worth.
Computers are responsible for our debt, and for allowing us to take uncalculated risks with our hard earned income. It's time to let the human touch back into the financial world."
Liked this? Then check out The 10 fastest computers in the world
Sign up for TechRadar's free Week in Tech newsletter
Get the best tech stories of the week, plus the most popular news and reviews delivered straight to your inbox. Sign up at http://www.techradar.com/register