stevehopwoodforex.com
https://www.stevehopwoodforex.com/phpBB3/
Print view

R/Empty4 indicator
https://www.stevehopwoodforex.com/phpBB3/viewtopic.php?t=2069
Page 3 of 5
Author:  FreeFox [ Mon Nov 11, 2013 10:20 pm ]
Post subject:  Re: R/Empty4 indicator

Really interested in this thread.

I signed up for the Standford ML course a few weeks back and working through that. I've been developing EAs for a few years now and always looking for ideas.

Question for people. I have the historical trades for a good performing EA and I'd like to gain some insights in to what triggers the trades. Will the ML course skils be able to help categorise some common attributes of the trades to help me develop something similar?

Any thoughts?
Author:  6y588 [ Tue Nov 12, 2013 2:35 am ]
Post subject:  Re: R/Empty4 indicator

FreeFox wrote:Really interested in this thread.

I signed up for the Standford ML course a few weeks back and working through that. I've been developing EAs for a few years now and always looking for ideas.

Question for people. I have the historical trades for a good performing EA and I'd like to gain some insights in to what triggers the trades. Will the ML course skils be able to help categorise some common attributes of the trades to help me develop something similar?

Any thoughts?
Welcome to the thread FreeFox, as I mentioned before even if only a few participated, I will try to keep this thread alive.

The proof-of-concept below shows that the lm() function from R could be replaced by Theil–Sen estimator. The top indicator is written in R/MQL, while the bottom indicator is written completely in MQL (see second pic). Google the terms "theil linear regression" and "mql".

@FreeFox
Btw, from what I recall, the course teaches you about supervised (and unsupervised) machine learning. Unless they changed the topics for the course, I do not think that you will gain much insights in to how to reverse engineer trades from history.

@eigenvector
That is what I am trying to achieve, an automatic pair selection based on rate of change (momentum). You could even combine with a news filter to trade on high impact news. This news trader idea came only after I posted my trading system with six components concept, hence the news trader EA will be a different system.

For example, today a short AUDUSD trade was opened by the EA, which selected the best pair automatically based on rates of change (momentum) and news (high impact). See first pic.

The Theil-Sen estimator returns the slope of the EIGHT currencies, which are then sorted from highest to lowest. Then, the currency with the high(est) positive slope was used for the long symbol, while the currency with the low(est) negative (or most negative) slope was used for the short symbol. Then the news filter checks for a high impact news for either symbol and trades accordingly. You have to be careful, sometimes the pair returned may not exist, e.g UsdAud - long Usd and short Aud - so you would have to reverse the symbols, i.e. AudUsd, and open a short to trade this pair. You can use the MarketInfo() function to determine the correct order.

I hope that this helps you to gain some clarity in the concepts used for trading.
Author:  FreeFox [ Fri Nov 15, 2013 8:45 am ]
Post subject:  Re: R/Empty4 indicator

Thanks for the reply to my question 6y588.

My idea for attempting to gain insights in to how an EA trades based upon analysing the historical trades were along the following lines:

For each trade determine various attributes about it, the more the better. For example, we could categorize them as follows:
- Was is above/below 14 day EMA
- Has prior bar touched a bollinger band?
- Is RSI indicator overbought/oversold?
- Have we had x 5 minute down/up bars in a row?
- etc, etc.

(These are just ideas for how we can assess what conditions existed for when a trade was entered.) Based on these or many other factors that could be thought of you’d be able to categorize each trade and come up with some probability. For example, for 95% of the historical trades all of them met certain conditions that were categorized. So I wasn’t looking at necessarily “reverse engineering” an EA, as that would be far too complex but hopefully come up with some known facts that when taken together meant we were in an area where a trade was highly likely. This would allow me to build a trading system based on that analsysis.

That was my thought process anyway. I was thinking maybe the Standford Machine Learning (ML) course would be useful to do that sort of exercise.

Is that a possibility or I am way off in my thinking or what ML would enable me to do?
Author:  eigenvector [ Sat Nov 16, 2013 4:23 am ]
Post subject:  Re: R/Empty4 indicator

Your descriptions do help clarify some of the many ideas I've been mulling over for the past weeks, 6y588. I had thought to use session times as a filter for pair selection, but it seems using the actual news reports takes the idea one step further and makes a lot of sense conceptually. One of the difficulties comes in knowing which currency momentum discrepancies are significant and which are not. Having definite gate times/events might help to reduce the whipsaws in switching back and forth between two strong horses that both end up in basically the same place, but at different times.

I hadn't looked into Theil-Sen estimation but can appreciate how it might be a more appropriate tool for the given heteroscedastic series. The code doesn't appear too overly complex to convert to C# where I plan to implement this strategy.

From a practical implementation standpoint, this is a large project. This is due to both the number of time series involved and the innate difficulty in backtesting such a system with many inputs. I've taken quite a bit of time thinking about the various aspects from a system conceptual standpoint (making observations mainly based on absolute (rather than relative) currency movements.

I like the idea of being able to identify opportunities as they develop on pairs that I may not otherwise look at on a regular basis. Understanding trade direction is at least half the equation and this framework provides both high momentum pair and trade direction. It just seems to lack a method for exit.

In theory, when the currency slope starts to reverse or flatten, then positions could also be flattened. I'm not sure that this strategy would work as a purely reversal system because pairs might change over time, leaving many hanging pairs waiting for a signal that might not come.
Author:  instreak [ Sun Nov 17, 2013 4:41 pm ]
Post subject:  Re: R/Empty4 indicator

Great Thread!

I am very much interested in connecting R with Empty4.


from your example, I am not able to compile the GradientVolatility4.mq4. I have error stating "cannot opent the program file".

please help! what is causing this error?


thanks.
Author:  eigenvector [ Sun Nov 17, 2013 11:32 pm ]
Post subject:  Re: R/Empty4 indicator

instreak wrote:Great Thread!

I am very much interested in connecting R with Empty4.


from your example, I am not able to compile the GradientVolatility4.mq4. I have error stating "cannot opent the program file".

please help! what is causing this error?


thanks.
Try copying your entire Empty4 directory out from program files into a directory that you own, and running Empty4 from there.
Author:  6y588 [ Mon Nov 18, 2013 8:36 am ]
Post subject:  Re: R/Empty4 indicator

I have created an R/Empty4 indicator that performs a prediction of the next n-bars of RSI values based on the forecast package in R. (Note: I will be away for a week, so I may only respond after that).

I do have some to write on the architectural considerations used in a neural net design, e.g. activation function, hidden layers, etc. However, I am running short of time but instead I will post some links to neural nets in R.

(g) The package forecast by Professor Rob Hyndman. URL: http://robjhyndman.com/hyndsight/forecast4/
(h) The package nnet (Feed-Forward Neural Networks). URL: http://cran.r-project.org/web/packages/nnet/index.html
(i) The package neural net (Training by Neural Networks). URL: http://cran.r-project.org/web/packages/ ... index.html
(j) Predictive models in R. URL: http://rsnippets.blogspot.sg/2012/12/pr ... ok-in.html
(k) Time Series Analysis and Order Prediction with R. URL: http://www.inside-r.org/howto/time-seri ... ediction-r
(l) Neural Networks with R – A Simple Example. URL: http://gekkoquant.com/2012/05/26/neural ... e-example/
(m) Visualizing neural networks from the nnet package. URL: http://beckmw.wordpress.com/2013/03/04/ ... t-package/

Basically, the inputs for the RSI_NNet indicator are as follows: (a) RsiPeriod=14 - parameter for RSI period; (b) NNetLookBack=100 - the number of bars to fit the neural net; (c) NNetForecast=10 - the number of bars to forecast; (d) Rpath="C:/Program Files/R/R-2.15.3/bin/i386/Rterm.exe" - the full directory path to the Rterm 32-bit exe file. See attached picture.
1064 RSI_NNet inputs.png
The top indicator is the RSI_NNet indicator, while the bottom indicator is the original RSI indicator.
1063 RSI_NNet.gif
If anyone is interested, I have attached the RSI_NNet.mq4 indicator file below.

For convenience I am including all the steps for R/Empty4 installation again:

1) Install R project (I used version 2.15.3): http://tinyurl.com/cdpxela
- if you use a different version, make sure 32-bit exe is used and Rpath is correct.

2) Install 7bit's mt4R library (latest version): http://tinyurl.com/d9p7czb
- copy "mt4r.mqh" and "mt4r.dll" into "include" and "libraries" folders of your Empty4 installation directory.

3) Open R console application, and type in the prompt exactly below

Code: Select all

install.packages("zoo")
install.packages("forecast") 
- this will ask you for a mirror site (just select the site nearest to you).
- you must perform this in R console, and not RStudio (as RStudio keeps a separate install folder).

4) Compile the "RSI_NNet.mq4" indicator in your Empty4 client/editor and attach it to a chart.

5) If you are getting an error message on the line of code:

R = Rinit(Rterm,2);

"Rinit" function is not defined

Then, change the "Rinit" to "RInit".
Author:  dawveed [ Sat Nov 23, 2013 6:16 pm ]
Post subject:  Re: R/Empty4 indicator

Hey 6y588,

Thank you for this great thread. I join you.

Can you please share the indicators in the screen shot, so the interested could test the system and provide feedback and possible upgrades?
1058 RGradientCss v TheilCss.gif
Author:  6y588 [ Mon Nov 25, 2013 8:46 am ]
Post subject:  Re: R/Empty4 indicator

dawveed wrote:Hey 6y588,

Thank you for this great thread. I join you.

Can you please share the indicators in the screen shot, so the interested could test the system and provide feedback and possible upgrades?
Welcome to the thread dawveed, as mentioned I will try to keep this thread alive but I may not be checking the forum as often due to my starting a new job and I have several trading projects that I am working on.

I would like to integrate the Thiel estimator with the Currency Slope Strength (CSS) indicator as the TheilCss used iCustom(), which is inefficient and slow, to call the CSS indicator. If anyone would like to help integrate and make this indicator work faster, then I would be willing to explain via PM.

6y588
Author:  eigenvector [ Wed Jan 08, 2014 7:09 am ]
Post subject:  Re: R/Empty4 indicator

6y588,

I finally got around to putting together some of the preliminary parts of the project based on some of the ideas presented here as a proof of concept. Kudos to you 6y588 and hope your new job is going well!

As there were subtle bugs in all the currency strength indicators that I studied, I decided to roll my own currency strength indicator. I have simplified the code, and in the process got all the bugs worked out (hopefully), and have also added in TheilSen. Even with TheilSen included inside the same indicator it is quite slow in running. The median calculation in Empty4 is a bear but even using the mean with TheilSen is slow in Empty4. As a result of this slow performance, I took some sample code from 2012 and after studying some documentation, made a DLL in C# to handle the TheilSen calculation. The currency strength indicator runs much faster now.

I've been experimenting with various TheilSen calculations, and in the process have now made a version with the median (the standard algorithm), as well as a version that uses the mean. I also made some weighted versions that tend to trigger slightly faster but may be slightly noisier which negates the benefit of faster triggering. After a cursory review, the median seems more stable (robust) as the theory states than the mean version.

Shown below the chart in the subplots are the median, median weighted and absolute strengths of four currencies in subplots 1,2 and 3 respectively shown on a very noisy GBPJPY 1 minute chart.
TheilSen_Strength.png
All times are UTC Page 3 of 5