Backtesting Parameter Selection

afeudale · Post by **afeudale** » Thu Dec 28, 2017 2:30 am

Hope everyone is enjoying the holiday season!

Maybe more of a philosophical question, but I'm curious for thoughts/feedback from the community on Empty4 Backtesting/Optimization/Parameter selection.

As an example, my latest EA can give me a profit factor of 4+ with certain parameters from 2003-2017. However, it will only take 100 trades during this 14-year period.

Conversely, I can get a PF of 1.15 with less strict parameters on the same time period, but it will take 10,000+ trades and thus give a higher overall profit than the strict parameters.

Parameters between these two extremes will result in PF's and trades in almost linear, inverse proportion to each other. Which is the better strategy?

While I understand that a higher PF and lower drawdown is in general better (you can always just trade larger lot sizes to compensate for the fewer trades), this is countered by the fact that no system is 100% and thus you may wait all year for that one "amazing" trade only to lose out.

If you take the lower PF parameters, you will have more opportunities for trading and come closer to the backtested PF.

Post by **Shoddy** » Thu Dec 28, 2017 5:19 pm

http://www.stevehopwoodforex.com/phpBB3 ... =28&t=4020
No use backtesting, it won't give you any meaningful answers, trust us on this.

afeudale · Post by **afeudale** » Thu Dec 28, 2017 5:37 pm

Thanks Shoddy. You should see my post at the end of that thread you referenced - maybe it changes things (EDIT: looks like that post is gone...either I posted it incorrectly or it was maybe a locked thread ooops)

At the end of the day, backtesting cannot be written off entirely (even if Empty4 is not useful for this purpose). If it is useless, then you must also write off all forms of technical analysis/technical trading (which most of SHF is about). I am not about to write off all of that.

Technical analysis is ALL about the past repeating itself. For example, when this combination of indicators did abc in the past, then a certain trade was successful. So when those indicators do abc again, there is a likelihood greater than 50% it will be successful again. If this is not true, all indicators are useless, along with support/resistance, etc.

Backtesting is a way to quantify this repetitive nature.

So the better question is probably "what to use to backtest" (maybe not Empty4) OR "how to make Empty4 backtesting meaningful", rather than "to backtest or not".

Just my 2 cents

Post by **Shoddy** » Thu Dec 28, 2017 6:04 pm

ok, yes, then to be more precise: don't use Empty4 for backtesting, but if you do, then use it for functional testing, as in does it take the trade or not, does it close at the correct place or not, I'm sorry but having coded and tested hundreds of EA's and seeing how their live performance differs from backtesting, it's just not worth it, the results will be very different. Maybe there are tools that are good enough to use for backtesting, do tell us in that case, with proof please.

afeudale · Post by **afeudale** » Thu Dec 28, 2017 6:46 pm

Yes, you are right about the limitations of Empty4 backtesting.

Curious your opinion on this: If a backtester were comprehensive (i.e. contained all the required tick data, spreads, etc.) then it should produce the exact same results in a backtest as it would in a forward/live test, correct?

If it doesn't produce the same results, that infers that the past is not repeatable and therefore all indicators/predictors are useless, no?

Maybe my colleagues who trade on fundamentals were right all along...hahaha.

Thanks.

Post by **Shoddy** » Thu Dec 28, 2017 7:52 pm

It would be closer but you would still lack info on latency from your terminal to broker and back, the real order depth at that moment, slippage and whatever else I can think of that will stop you from getting the exact same results. As I said we can get close but not exact. Just deal with it, live trading is different from backtesting and demo.

Post by **SteveHopwood** » Fri Dec 29, 2017 9:50 pm

Shoddy, thanks for trying to show the OP the error of his ways. He was never going to take any notice and is on a path to SHF self-destruction.

afeudale, it was not I who deleted your post from the end of the "Why Empty4 backtesting does not work" thread - but probably only because somebody else with brains got there first. I do not care what you think. Those of us who have been around this stuff for a long time know why backtesting in Empty4 does not work; we have tried to explain it to the likes of you. It is entirely up to you whether you accept the benefit of our expertise or not.

Failing to accept it merely makes you an idiot. That is fine by me. You can be as thick as you want so long as it does not disturb me.

Disturbing me again about this will be a different matter. Attempt to post about this again and I will ban you from SHF.

This thread is locked.