Looking at the web interface, it is claiming a 92% success rate on trades. That clearly indicates to me that those results are "in-sample". In other words, the model was trained on some data, and then backtested on that same data. In-sample results are essentially worthless. I used to work for a quant hedge fund, and at least on daily trading, you could have great results with 60% correct. There's no way you can have 92%.
Finding a useful financial signal is not primarily a search problem through a giant space of potential indicators. It is all about controlling for overfitting, and ensuring that the signal continues into the future. Also, I saw no mention of transaction costs for the trading strategy, which can often turn a great strategy into a money losing one.
Yep, it's the same thing as "testing on the training set". With memorization, one can achieve almost 100% on the training set. I'm not saying that EDW is doing this, but I'd be very surprised if they could get 92% success rate on real, unseen, test data. Heck, most Wall St companies would _kill_ for anything in approaching 60%.
Finding a useful financial signal is not primarily a search problem through a giant space of potential indicators. It is all about controlling for overfitting, and ensuring that the signal continues into the future. Also, I saw no mention of transaction costs for the trading strategy, which can often turn a great strategy into a money losing one.