Remember me

Register  |   Lost password?

The Trading Mesh

DarkStar's Blog

Big Data Isnt About the Data

January 6, 2012 Comments (0)           

 
I’LL HAVE THE DATA SOUP PLEASE
We had data yesterday, we have data today, and we’ll have data tomorrow.  In fact, we’ll have a lot more data tomorrow.  We’re adding data to our lives at what seems to be an exponential rate.
I belong to the Low Latency, Big Data group on Linkedin, here’s the link – Low Latency, Big Data.  There are some heavy hitters out there and we’ve been working together on a definition for big data.
WHAT...

What Exactly is Complex Event Processing Today?

January 6, 2012 Comments (0)           

As much as I disagree with much of what Curt Monash writes, he did actually ask a good question recently in his post, “Renaming CEP… or not” Without getting into a rehash of the hash over there, let’s look at things a bit differently. Let’s talk about what CEP is not. I left trading to join a firm called NEON. I was an early investor in this company, my mentor had started the firm, and to me it looked like the cat’s meow. It was a great time, a lot...

Its Time to Kill the Elephant

January 6, 2012 Comments (0)           

Google started using MapReduce about 10 years ago. Somewhere between there and now, Doug Cutting decided that he could copy it while at Yahoo and Hadoop was born. Doug now works at a company named Cloudera who bills themselves as providing the “only solution that manages Apache Hadoop across the enterprise.” Hadoop has been around for so long that even leading analyst firms are covering it, claiming that if your organization is an early adopter, you need to be looking at Hado...

Cassandras Data Model

January 6, 2012 Comments (0)           

As we prepare to implement our Market Data repository to facilitate algo development and back-testing, you should have downloaded Cassandra and installed it by now. What, you haven’t? Well, click here, get it done and then come back for some fun. To get things up and running once you’ve downloaded Cassandra, click here for some guidance (this assumes you’re running Linux but should point you in the right direction if you’re running Windoze). CONFUSION Most of...

Building a Back Testing Platform for Algorithmic Trading

January 6, 2012 Comments (0)           

On this continuing series, I am examining thoughts and specific implementation details around building a back-testing platform for algo trading. Eventually, we’ll see where complex event processing plays and how to implement it. Appendix to Part One – The Data Format Rather than looking at various database solutions first and then trying to define the problem in terms of those solutions, let’s first examine what market data looks like. In its most simple form, market da...

Building a Back Testing Platform for Algorithmic Trading

January 6, 2012 Comments (0)           

In this series, I’m going to outline in general, how to build a back-testing platform for the creation, tweaking, and subsequent execution of algorithms used in electronic trading. Part One The Data I recently made some comments on Vertica’s blog in regards to what I considered to be a fairly bold claim. They said that Vertica was the only real column store. But even if they are, so what? In my comments, I alluded to my belief that we optimize problems to solutions we tr...

Predictions for 2011

January 6, 2012 Comments (0)           

Some predictions for 2011. In no particular order or importance. 1. CEP – The Feature There’s a couple of things going on here. The most important being that Mark Palmer is writing blog posts about Richard Tibbetts writing blog posts on the Tabb Group’s site about writing better software on Wall Street (because software startups write better code and deal with bigger problems than firms on Wall Street, especially exchanges, do, right?…). No customer win....

What's Wrong With Complex Event Processing?

January 6, 2012 Comments (0)           

I spend a significant amount of my time keeping up with advances in processing high velocity big data. Over the last year, I’ve watched the NoSQL camp grow a lot. And now, some folks are even forecasting a market approaching $2 Billion USD by 2015. The last time I saw that kind of trajectory for a new software category was for Complex Event Processing. So without casting any undue aspersion on the NoSQL camp, let’s talk about why CEP has so dramatically failed to generate...

Flash Crash HFT Not To Blame so What Next?

January 6, 2012 Comments (0)           

Much to the SEC’s consternation, the recent report detailing the causes of the May 6th, 2010 Flash Crash has failed to indict High Frequency Trading as the cause. BUT WE WANT THE MONEY How does all of this tie in with the SEC’s bid to build a huge consolidated audit trail? (CAT) – well, after a lot of thought, I don’t really know. And I haven’t read a lot from anyone that purports to know either. All I know is the SEC wants a lot of money to build it because,...

Hope, FPGAs, High Frequency Trading and the New Market Access Rules

January 6, 2012 Comments (3)           

 
I recently became aware of an emerging practice most likely being implemented by clearing companies at the low end of the capitalization sprectrum offering a unique solution to the recent Market Access Rules.
NO UN-FILTERED DIRECT ACCESS
What the SEC is trying to do is remove, or reduce the opportunity for either crooks, idiots, or algo’s gone wild from doing bad things to the market. Under the new rules, order flow needs to be monitored. This is not something that the HFT crowd...