What other backtesting Tools you use?

[NeoTrader]

Hi all,

I want to ask all coders here what other tools you use and/or you would recommend regarding backtesting.

I am especially interested in backtesting multi-currency and/or multi-time frame strategies.

I am really getting sick with the Empty4 Strategy Tester and it bothers me that I can't really test what I want to test. I wouldn't bother learning a new programming language if I knew that I could do what I want and not what the crap Platform let me do.

It also would be nice if the testing functionality would support actual computer technologies like multicore and multithreading to speed up the testing process.

I know Gary is using Tradestation and AnotherBrain is using AmiBroker...

As of now I think that AmiBroker could be my Platform of choice since it seems to have all requirements I need and I also can connect it later to IB and there are also some interface possibilities to Empty4.

Before I put time and effort in this Platform I wanted to ask our great community what other tools they are using, why they are using them, what the caveats are and if they would recommend it or not. And if not, why?

Thanks in advance,

NeoTrader

P.S.: I don't want to start here a flamewar against Empty4 or backtesting... just a serious discussion about what Tools would do the backtesting job best.

-

[garyfritz]

Correct, I use TS. It has some real strengths -- very flexible & capable language, can express very complex trading ideas very succinctly and quickly. Most importantly it WORKS. With few exceptions, a backtest will produce basically identical results to the same strategy running in realtime, which is of course what you want. When you apply a strategy to a chart in TS, it automatically runs a backtest on the history in the chart, and then switches to realtime monitoring and execution.

Execution speed is very good. E.g. I'm running a strategy on a chart with over 80,000 bars -- 30min chart going back 6+ years, about 350 trades -- and my strategy executes in about 3-4 seconds. So I can test 15-20 parameter sets in about a minute.

TS will chart arbitrary-minute charts (equivalent to M1, M3, M7, M43, whatever) and you can also use "tick" charts -- each bar contains 10 ticks, or 34 ticks, whatever. However you can't chart "N seconds per bar." TS provides all data very smoothly (none of the crapola games we have to deal with in Empty4, though FX data doesn't work as quickly/smoothly as stock/futures/etc data due to the conversion issues) and the TS brokerage (your only option) is very good. Data is free, TS is free for FX, very good fills and no commissions. (Futures and stock data are NOT free. You can get the platform for free with futures/stocks with a modest amount of trading.)

Disadvantages: biggest problem is the lack of multi-symbol capability. In TS a strategy works on only one symbol -- the main symbol in the chart, which you set up manually, no programmatic control. You can use data from other symbols in your decision process, if you set up those symbols in the chart ahead of time, but you can only trade one symbol. None of the flexible "grab whatever symbols you want, trade whatever symbols you want" of Empty4. TS has recently added portfolio capabilities and walk-forward analysis, but the WFA (from my brief look at it) is weird and not very useable, and the portfolio tool looks kind of clunky too but I haven't studied it closely.

Another major problem is that TS's automated execution is... a bit flaky. Network glitches can cause all charts to reload their data, which causes strategies to re-execute, which may result in multiple execution of orders or similar nasties. You have to be aware of this and code around it. I don't yet use TS for automated execution.

MultiCharts is another tool that was originally based loosely on TS. It accepts basic TradeStation EasyLanguage, so you have many of the same backtesting capabilities, but it also handles multi-symbol / portfolio testing. (I think you are still limited to a single symbol per chart like TS, i.e. a single strategy can't trade multiple pairs, but MC can combine those tests for portfolio statistics.) MC is NOT free, unless you open an account with FXCM and trade a minimum of 80 mini lots per month.

I'd like to hear Brian's opinion of AmiBroker. I've heard pretty good things about it.

[NeoTrader]

hi Gary,

thank you for your description of Tradestations Features and flaws...it is really good to know from an experienced actual user where the strengths and weaknesses of a platform are. Much appreciated.

MultiCharts is another tool that was originally based loosely on TS. It accepts basic TradeStation EasyLanguage, so you have many of the same backtesting capabilities, but it also handles multi-symbol / portfolio testing. (I think you are still limited to a single symbol per chart like TS, i.e. a single strategy can't trade multiple pairs, but MC can combine those tests for portfolio statistics.) MC is NOT free, unless you open an account with FXCM and trade a minimum of 80 mini lots per month.

I know also MultiCharts and had different Views of it...but it seems that it has highly improved the last couple of Years. And they also want to release a .NET version in the near future...sounds interesting... must dig a little deeper in this... but also has a bigger price tag than AmiBroker.
Do you or did you use(d) MultiCharts?

I'd like to hear Brian's opinion of AmiBroker. I've heard pretty good things about it.

yep...this would be great... I'm also looking forward to hear from him and what his reasons where to go with AmiBroker...

Thanks again for your detailed Post.

happy trading,

NeoTrader

-

[garyfritz]

No, I have not used MC. Khalid uses MC, though since he is not a strong programmer and not a fan of backtesting, he hasn't done a lot with the backtesting features. He could probably tell you a lot about the usability &etc.

[Jeuro]

I am using Jforex to backtest multiple-pairs strategies.
After many year of manual trading I started to look for automation some 3 years ego.
I tried the "indicators" route for about 2 year and came to the conclusion that is absolutely pointless to use them for automatic systems. Some 10 month ago I started to work price equations between certain pairs and had to decide what platform to use.

I am happy with Jforex. I like that fact that Dukascopy has very good price history.
I use a generic EA (specially coded) that to trigger up to 8 pairs by any amount of pips, lot, tp, sl, etc I want.

I am an strategist and not a programer but I am getting good help form a member of this forum. Most of all, we are quite excited that we finished testing one basket of 3 pairs with excellent results for the past 3 year. ... and now working in a different basket. I am already trading 2 real account with the system.

At the beginning was a pain having to get a new demo from Dukas every 14 days... But I opened an a real account + they game me permanent demo to continue the work. The most I like about Dukascopy is to be able to trade 1000 minimum lot size with a regular account.. and no maximum. ..
and that is great for multiple pairs. Most of the Empty4 brokers have minimum 10000 units for regular accounts.

J.

[NeoTrader]

hi jeuro,

thanks for your comment on Jforex. I am also interested to open an account with Ducascopy in the near Future. But I think I will use another platform to connect to DC. Nothing against Jforex but it is as far as I know only available on DC. Please correct me if I'm wrong.

When I put so much energy, time and money into development and testing I want to be able to take my knowledge to another criminal if I am not happy anymore...or want to divide my risk....and experience shows that this time could come...

Therefore I would prefer a Platform that I can use with different Datafeeds and/or Crims.
I plan to work with IB and/or Ducascopy in the future and maybe some Empty4 brokers..so the optimum would be if the platform had direct or indirect Interfaces that I could use with them.

This is also one Problem I have with AmiBroker...It has a couple of Datafeed Providers but can only use IB for mechanical trading...but it seems also to have some interface to Empty4 that I must check first...

MultiCharts on the other hand has over 20 Broker at hand and also seems to be very good at backtesting...in combination with a easyLanguage like Programming Language where you also have a great community of developers and available sources... this could also be a good choice ...

If someone here has more experience with MultiCharts and could explain a little more about it's quirks and goodies.....especially the things that are not written from a marketing guy...


Thanks in advance,

NeoTrader

-

[Jeuro]

Jforex is available At Alpari US and FXDD. I think that is all.
The choices to be able to backtest multi-pairs strategies is very limited. Any of the others platform also have limited brokers. So, it becomes a matter of choice of what coding language you feel more comfortable with.

After many months on a crossroad to choose one platform and to go thru the learning curve, I choose jforex because Dukascopy is the Broker that most traders with heavy volume end up anyway. I mean they are set up for bigger player. Including that from 250k we can have our deposits in our own Bank and operate via letter of credit. To me, that is the ultimate segregation of funds. Not only is not mixed with their money nor other clients money, but the profits are credited each night in my bank account, so there is no issues about withdrawals.

In any case, with all the resources and coders available for Empty4, it is quite easy to code any multi-pair strategy that show good results in Backtests in other platforms. So, even if the purpose is only backtest that is ok. With Dukascopy Europe it is possible to open account with only $100 so it it very cheap to get all their resources.

J.
.

[NeoTrader]

hi jeuro,

and thanks again for your comment...

I didn't know that other brokers also used jforex...thanks for that info...but as you described ducascopy is the way to go if you have a larger account...

The choices to be able to backtest multi-pairs strategies is very limited. Any of the others platform also have limited criminals. So, it becomes a matter of choice of what coding language you feel more comfortable with.

Yes, this is true... MultiCharts also supports Ducascopy and IB (but seems to have some problems with the Forex feed on IB) .... but I could not find anything about a reliable Empty4 connection.
AmiBroker seems only to support IB for autotrading and there are some hacks to connect it wit Empty4 Platform...but no Ducascopy connectivity...

From Backtesting point of view this two seem to be the best choices as of now....

After many months on a crossroad to choose one platform and to go thru the learning curve, I choose jforex because Dukascopy is the criminal that most traders with heavy volume end up anyway. I mean they are set up for bigger player. Including that from 250k we can have our deposits in our own Bank and operate via letter of credit. To me, that is the ultimate segregation of funds. Not only is not mixed with their money nor other clients money, but the profits are credited each night in my bank account, so there is no issues about withdrawals.

Yep, I know this. Thats why I want to start with them in the first place...and I also know java as a programming language so the learning curve would not be that great as with a complete new language...but the programing language is not the problem in the learning curve...most time in the learning curve goes to learning the quirks of the platform and how to work around them... .
I just want to have my strategies in place and tested before I start with the big bucks..

In any case, with all the resources and coders available for Empty4, it is quite easy to code any multi-pair strategy that show good results in Backtests in other platforms. So, even if the purpose is only backtest that is ok. With Dukascopy Europe it is possible to open account with only $100 so it it very cheap to get all their resources.

Yes you can code multipair and multitimeframe strategies in Empty4 but you don't have any reliable way of testing them. this is my major problem...

Gary made a good proposal with a testing library/framework that would also be capable of testing MTF and MP Strategies. But it is still in the Brainstorming Phase...
I hope that he can put his 2 Neurons together soon so that we can get started...this would be a major breakthrough for our testing needs.

Thanks again for all the Infos.

happy trading,

NeoTrader

-

[Jeuro]

....but the programing language is not the problem in the learning curve...most time in the learning curve goes to learning the quirks of the platform and how to work around them... .
I just want to have my strategies in place and tested before I start with the big bucks..

I am not a tech wiz and in about a week I got the backtesting quirks pad down.

Yes you can code multipair and multitimeframe strategies in Empty4 but you don't have any reliable way of testing them. this is my major problem...

Yes, that was the major issue for me. I figured a nice multi-pairs strategy and was able to trade it in Empty4... all forwards demos did great and the theoretical math worked fine... but 2 or 3 months forward demos usually do not tell the whole history and is a very slow process to advance. ... when finally I decided to get the whole history I paid a Russian programmer to code the tool EA and ready to proof the theory for all markets ... now looking back I regret not doing it sooner. I wasted a lot of time at turtle speed.


Gary made a good proposal with a testing library/framework that would also be capable of testing MTF and MP Strategies. But it is still in the Brainstorming Phase...
I hope that he can put his 2 Neurons together soon so that we can get started...this would be a major breakthrough for our testing needs.

I posted a thread proposing a a multi-pair testing lab long time ago... but nobody was interested... so, if we believe in our own work, we just have to keep moving and do what we have to.

J.

Thanks again for all the Infos.

happy trading,

NeoTrader

-

[NeoTrader]

hi again Jeuro,

Yes, that was the major issue for me. I figured a nice multi-pairs strategy and was able to trade it in Empty4... all forwards demos did great and the theoretical math worked fine... but 2 or 3 months forward demos usually do not tell the whole history and is a very slow process to advance. ... when finally I decided to get the whole history I paid a Russian programmer to code the tool EA and ready to proof the theory for all markets ... now looking back I regret not doing it

What did you mean with "to code the tool EA" ? Did he coded an EA in MQ4 or java? I think I'm lost now...

I posted a thread proposing a a multi-pair testing lab long time ago... but nobody was interested... so, if we believe in our own work, we just have to keep moving and do what we have to.

yeah I remember your thread...but I think it was to specific formulated for you needs...Gary's approach is a more general one that can be used in many different EA's. And this would be what we need...with his approach it would be possible to test an EA like it is running in real time against the history data of the charts (including MTF & MP) but it only makes paper trades and then if the results are positive we can just switch it to real trading. This would be a great improvement for backtesting.

happy trading,

NeoTrader

-