• M T4 = Empty4, because it is rubbish.
• Crapergy Tester = Stratergy Tester, because it is utter bollocks arising out of rubbish.
• Craptesting = backtesting, because of all the above.

• the candle Open price.
• the candle High price.
• the candle Low price.
• the candle Close price.
• the candle open time and date.

• the spread. Think about this for a few seconds, folks. By default, Empty4's Crapergy Tester uses the current spread. Ouch. If you are 'paper testing' by manually going back over the charts, you probably won't even bother to include any kind of spread calculation. Backtesting in crapergy tester ignores the spread at the time the trade would have been taken because it has no record of it. Consider this:
  • if the trade was taken at the time a major news event was released and the spread jumped by a factor of 10, this will not show up in your backtested trade result.
  • spreads widen at the weekend, sometimes by a factor of 20 or more. Imagine what that does to EA craptesting results.
  • EA's craptesting their results use the current spread. This changes all the time even without what I previously described. A craptesting run at different times will give different results.
• swap costs. Crapergy Tester does not factor this in at all when craptesting an EA. For sure, it would take a really dedicated paper trader to factor them in, even if he/she knows what they were. Just imagine the difference that this makes to the headline profit/loss.
• commission charged by 'real' as opposed to market-makers and bucket-shop brokers. This is ignored by Crapergy Tester. Imagine the difference that makes to profitability.

SteveHopwood » Tue Dec 30, 2014 4:17 am wrote:Note. In this document:

• Empty4 = Empty4, because it is rubbish.
• Crapergy Tester = Stratergy Tester, because it is utter bollocks arising out of rubbish.
• Craptesting = backtesting, because of all the above.

The purpose of this document is to explain why craptesting in crapergy tester does not help us.

One of the hardest things that new traders have to grasp about the charts they are viewing is this: the only information each candle holds in its database is:

• the candle Open price.
• the candle High price.
• the candle Low price.
• the candle Close price.
• the candle open time and date.

That's it folks. Now think of the things that are vital to a viewing the likely outcome of trading that is not held in each candle's database:

• the spread. Think about this for a few seconds, folks. By default, Empty4's Crapergy Tester uses the current spread. Ouch. If you are 'paper testing' by manually going back over the charts, you probably won't even bother to include any kind of spread calculation. Backtesting in crapergy tester ignores the spread at the time the trade would have been taken because it has no record of it. Consider this:
  • if the trade was taken at the time a major news event was released and the spread jumped by a factor of 10, this will not show up in your backtested trade result.
  • spreads widen at the weekend, sometimes by a factor of 20 or more. Imagine what that does to EA craptesting results.
  • EA's craptesting their results use the current spread. This changes all the time even without what I previously described. A craptesting run at different times will give different results.
• swap costs. I am not sure that Crapergy Tester factors this in at all when craptesting an EA. For sure, it would take a really dedicated paper trader to factor them in, even if he/she knows what they were. Just imagine the difference that this makes to the headline profit/loss.

Have a quick think about the implications of a candle only holding the OHLC in its database, on looking back to see what indicators are showing you. They are working with each candle's close price. Ok, so they might be working with their open price, or some combination of HL, but the big thing to remember is this; what an indicator shows you on a chart is a fixed point in time, which is the candle close time.

Take an indi that we are all familiar with - Stochastic. The damn thing might have zoomed up and down like an elevator on drugs during the course of a candle and played merry Hell with your trading. This will not show up on your chart because Stoch can only work the the candle close price. Imagine what this does to an EA that depends on Stoch to trigger trade opening and closing.

So, if you run your Crapergy Tester in visual every tick mode, where do all those ticks come from? Suppose you are craptesting an EA on the H1. You will see the "Using M1", "Using M5" etc messages. Crapergy Tester is taking the OHLC of each candle it has in its database and constructing a grid of ticks based on the direction these candles took. It is guesswork.

Fine/ish if you are testing a long term strategy that only makes trading decisions at the close of each candle. Bit of a sod if you are craptesting an M5 scalping system. Absolute sod if your long term trading includes a tick-by-tick trailing stop.

And then we get to the issue of missing candles. Empty4 ships with incomplete candle OHLC data, let alone missing tick-by-tick data. Noobs do not tend to know this.

So guys, this is why back testing is informally banned at SHF and why trying to discuss it will get you into trouble an any of my threads, and most that I merely contribute to.

I use it to test that EA code is working. I would not even bother to have nightmares about using it to test an EA's potential profitability - that is a waste of time.



Subsequent note: this thread is about why those of us who have been around this stuff for years know that craptesting does not work. Posts by dimwits trying to insist it does will be deleted - guess what prompted this edit. Posts adding to the knowledge offered up here will be added to this post.

Chorlton » Sat Jul 25, 2015 4:51 am wrote:Hi Steve,

What method would you recommend to test a strategy? Is there any form of backtesting (ignoring strategy tester) that would be worthwhile?

I've asked this question on many forums and normally receive the reply "run it on a demo account". Although I can understand why this would be preferred to relying on the Empty4 Strat Tester, the issue for me is that any results are provided in real-time so if I want to get a good sample size I may need to run the EA for many months.

Just interested in how you guys validate any new strategy.

Cheers,

Chorlton

There is no quick and easy way that I know, to evaluate a strategy using Empty4. The only way to properly evaluate a strategy is to trade it on a live, real money account.

Even demo trading has its limitations. There is no slippage. Order sending/closing usually works perfectly even in the sort of storm that freezes live cash accounts with the less honest 'brokers'. The one thing that demo trading proves is that a strategy does not work. If it is not profitable on demo, it will get murdered live. If your 'broker' is in fact a criminal, then you strategy could well run perfectly on demo and then get murdered when you go live.

Chorlton » Sun Jul 26, 2015 4:05 am wrote:
Thanks Steve,

Appreciate the reply.

Given your opinion of Empty4, which platform do you use?