SEC Gets Real-Time Market Data

I’ve written here before about the enormous growth in high-frequency equity trading that has taken place in the last few years, and about some of the side effects of that growth, such as the “Flash Crash” in May, 2010, or the trading disruptions on August 1 of this year.  After these events, it is customary for government regulators to issue reports on what went wrong; the Securities and Exchange Commission (SEC) issued a report on the Flash Crash in October, 2010.

You might visualize these regulatory agencies working in a way similar to air traffic control, monitoring the activity and health of the financial markets continuously throughout the day.  That picture is plausible enough, but it does not reflect reality, at least up to the present.

Earlier this week, The Washington Post published a report that the SEC was about to “go high tech” by installing a system, called MIDAS,  that would, for the first time, provide the regulator with real-time market data. To date, the agency’s information systems have been left far behind by developments in the markets it is supposed to monitor.

As computing power and big data have revolutionized stock trading in recent years, one market player has lagged far behind: the Securities and Exchange Commission, whose job policing the markets has been hampered by a serious technology gap.

Although the amount of data to be handled has increased significantly in the recent past, the technology of digital, nearly real-time financial market data is not new.   When I first began working in the industry, in the mid-1970s, market participants had this kind of data available, although often it was in the form of a video feed (essentially, a TV picture of a data display).  Even before that technology, market data was distributed by electro-mechanical stock “tickers”.   By the mid-1980s, there was a substantial movement toward digital distribution of market data; that change meant that the data could not only be looked at by traders, but also fed into spreadsheets and other applications.  (I did some work on digital data distribution in the late 1980s and early 1990s.)

Today, of course, the high-frequency trading that has become so significant is entirely based on the rapid processing of real-time data; speed is of the essence, as I noted back in 2010:

The time frames used in these strategies are in some  cases so short (measured in milliseconds) that firms aggressively bid for computer locations physically close to the exchange’s data center: the network propagation delay (at the speed of light!) has to be taken into account.

Given how important this type of trading has become, it is somewhat surprising that the SEC has not had the capability to monitor it.  Although it’s been a few years since I was actively involved in the industry, I was taken aback by the article, and apparently I am not alone:

“I scratch my head and say, ‘How could the SEC not have had this in place already?’ ” said Joseph C. Saluzzi, co-head of a brokerage firm called Themis Trading. “Why are they still playing catchup?”

Whatever the reasons for the delay, it is encouraging that the SEC is getting this facility in place; however, as the article points out, the system is just a starting point.  There is still more work to be done to create a comprehensive market surveillance system.

But experts who track the agency say more needs to be done. They are eager for the launch of the “consolidated audit trail,” a system that would require broker-dealers to report all their activity to a central repository and track the identities of those dealers and their clients.

Most critically, the SEC needs to be able to recruit and retain staff members that can use the data effectively, and to cultivate an organizational culture that is less legally-focused, and more data-driven.

Growth in High-Frequency Trading

In trying to examine the background and underlying causes of financial market events such as the disruption in equity trading on August 1, or the “flash crash” of May 6, 2010, the question of whether we are looking at some fundamental change in the way the markets work is frequently raised.  Trading volumes have certainly increased over the past few years, but firms engaged in high-frequency trading tend to be rather reticent about the exact nature and extent of their operations.

In one of my posts about the 2010 flash crash, I mentioned an analysis carried out by Nanex, a firm that specializes in systems and software to distribute and process market data, giving them a particularly good view of the overall market.

Our business is supplying a real-time data feed comprising trade and quote data for all US equity, option, and futures exchanges.

I have just come across a rather striking demonstration, put together by Nanex, of the growth of high-frequency trading (HFT) activity in US markets over the period January 2007 through January 2012.    The top graph on the page is an animated GIF graphic, which shows daily HFT activity for the various US exchanges (denoted by different colors).  At the beginning of the period, in 2007, the lines are relatively flat.  As time progresses, one begins to see more activity, first at the beginning and end of the trading day, then spreading throughout the day. Assuming Nanex’s data are correct (and I have no reason to doubt that), there has been a very substantial increase in HFT activity over the last few years.

The graphs on the lower part of the page show a more interesting aspect of this growth.  The graph on the left shows the growth, over time, of quote activity (that is, the posting of bid and offer prices for securities).  The graph on the right shows actual trades over the same period,  It is clear that most of the activity growth is in quotes, rather than trades.   There was some suspicion, in the 2010 flash crash, that some market participants might have engaged in “quote stuffing”, the generation of spurious quotes that would effectively clog up competitors’ systems.   There is certainly nothing in this evidence to disprove that suspicion.

Nanex also points out that spurious quote messages, like spam, are close to free for the sender, but not for other market participants.  As noted above, the large increase in quote activity does not correspond to an increase in actual trades, meaning that the cost per trade of processing quote messages has gone up substantially.

We think that a 10-fold increase in costs without any benefits would be considered “detrimental” by most business people. We think the regulators would agree with us as well.

This analysis, together with other evidence, suggests that HFT activity is not as benign as some proponents would have us believe.   As Prof. Ed Felten at Princeton observed back in 2010, it can be difficult to disentangle just what happened after the fact, but I’ll try to fit some of the pieces together in future posts.