R/Empty4 indicator

Place your new trading idea here to see if someone can automate it.
adamfx
Posts: 4
Joined: Thu May 23, 2013 11:59 pm

Re: R/Empty4 indicator

Post by adamfx »

Empty4 Template for indicator using an R function: Facilitating the development of Empty4/R indicators

I think creating a Empty4 Template using an R function to make an indicator would be a good idea. Then new Empty4 indicators using R code could easily be made by changing the R function rather than any of the Empty4 code. Thank you for the great forum.


Also does the function have to be within the script? Is it possible to run the function in R and still have it recognized by the Empty4 script?
Last edited by adamfx on Mon May 27, 2013 9:11 pm, edited 1 time in total.
eigenvector
Trader
Posts: 20
Joined: Sat Jul 14, 2012 3:14 pm

Re: R/Empty4 indicator

Post by eigenvector »

I haven't had much time to post because I have been taking the Stanford Machine Learning course at Coursera. I just finished this week. I worked all the homework and review questions. ML is a popular course so it is likely that future sessions will run. Toward the end of the class I noticed that I didn't need to watch the videos to answer the review questions and even found this to be true for the last programming exercise. I'm not sure if the programming exercises got easier or my knowledge of Octave / Matlab increased to the point that they seemed much easier.

Takeaways from the class in terms of applicability to trading system development:

I think week 6 is really all about designing good tests and that applies equally to trading systems. For instance, the third video of week 6, "Model Selection and Train/Validation/Test sets" seems to shine a new light on commonly discussed ideas of in/out of sample testing. I've tried to apply this 60/20/20 split to the data I'm working with. The procedure is basically to do all the optimization in the 60% train period. I drop out the losing strategies. Then I run those systems again and evaluate performance during the validation 20% for model selection, mixture. Then the result is once again validate on the final out of sample period, the test 20%.

I think this is a good way to do things from the standpoint of not evaluating or selecting the strategies on the same data where you perform the optimization. If any of you have read some of the other trading system development books, they typically suggest optimizing on 70-80% of the data and then running a final test out of sample on the remaining 20-30% of the data. However there may be built-in selection bias because the initial period is used for both optimization and selection, which may ensure worse out of sample performance than using the 60/20/20 method described in the previous paragraph. Does anyone have any thoughts on this that they would like to share?

By the way, I also think that there could be a great deal of value in generating learning curves (also described in week 6). If one knows that there is a high bias problem with the data, then the approach should be different than if the problem is high variance. But unless this is performed I'm not sure how to know if a system is under-fit or over-fit.

Would anyone care to discuss their insights from the ML class as they relate to trading system development?
eigenvector
Trader
Posts: 20
Joined: Sat Jul 14, 2012 3:14 pm

Re: R/Empty4 indicator

Post by eigenvector »

6y588 wrote:Hi All

To kick off discussion, I have created an R/Empty4 indicator that performs a gradient calculation of the last n bars. Basically, in a rising market, the gradient (or slope) of the line, which is a regression line calculated using lm() function in R and using the coeff[2], is positive. Conversely, in a falling market, the gradient is negative. Also, a steeper line will result in a higher slope.

If anyone is interested, I have attached the GradientVolatility.mq4 indicator file here.

Regards

Image
It took me a while to get my head around the basic concept driving the GradientVolatility.mq4 indicator. I kept wondering how it was possible to create a bounded oscillator from regressing a gradually increasing sequence

Code: Select all

seq_along(x)
onto a stochastic price series. But the answer, it seems is in the nature of the return value from the regression function. Setting the intercept aside for a moment, the coefficients (beta) are really a representation of the slope, or first derivative. I suppose this is where the term "gradient" comes in. Since a rolling period is used, the slope adapts as the price series adapts. Shorter lookback periods produce a more chaotic and faster moving oscillator. Longer lookbacks produce a slower moving oscillator.
Raj4x

Re: R/Empty4 indicator

Post by Raj4x »

Very interesting thread and initiative. Definitely going to stay on top of this and pitch in where I can.

Thanks for the links and the courses, subscribed to a few.

Just getting my feet wet in Time Series Analysis, I believe this is the way to go when trying to design a solid trading model.

Thanks :)
6y588

Re: R/Empty4 indicator

Post by 6y588 »

Raj4x wrote:Very interesting thread and initiative. Definitely going to stay on top of this and pitch in where I can. Thanks for the links and the courses, subscribed to a few. Just getting my feet wet in Time Series Analysis, I believe this is the way to go when trying to design a solid trading model. Thanks :)


Welcome to the thread Raj4x, as I mentioned before even if only a few participated, I will try to keep the thread alive.

The main issue for me when using neural net for forecasting is that the result is prone to curve-fitting. So additional tools to determine whether results are curve fitted should be developed together with the R library. The ML course had some useful video lectures that would help in developing these tools, in particular test for high variances or high biases.
eigenvector
Trader
Posts: 20
Joined: Sat Jul 14, 2012 3:14 pm

Re: R/Empty4 indicator

Post by eigenvector »

6y588 wrote:
Raj4x wrote:Very interesting thread and initiative. Definitely going to stay on top of this and pitch in where I can. Thanks for the links and the courses, subscribed to a few. Just getting my feet wet in Time Series Analysis, I believe this is the way to go when trying to design a solid trading model. Thanks :)


Welcome to the thread Raj4x, as I mentioned before even if only a few participated, I will try to keep the thread alive.

The main issue for me when using neural net for forecasting is that the result is prone to curve-fitting. So additional tools to determine whether results are curve fitted should be developed together with the R library. The ML course had some useful video lectures that would help in developing these tools, in particular test for high variances or high biases.
After studying the ml class and then attempting to use what I learned (from the bias / variance section) on system development and the financial time series, I came to a conclusion.

The nature of the data seems to be firmly in the high variance realm. In particular, with regards to system development where optimization is used, the results are invariably high variance (overfit). High bias would be poor results both in and out of sample which are usually filtered out by the base optimization method. So the higher performing systems (those that tend to get selected) resulting from an optimization seem to exhibit tendencies consistent with high variance (poor out-of-sample performance, very high in-sample performance, smooth in sample performance). I find that this tendency to overfit is most apparent when optimizing for highest performance with metrics such as maximizing avg returns or maximizing returns/drawdown.

Based on this insight, I adjusted my optimization strategies to account for high variance based on the advice from the class and have been able to get some better OOS results as a result.
User avatar
SpiderX
Trader
Posts: 554
Joined: Thu Aug 22, 2013 4:50 pm

Re: R/Empty4 indicator

Post by SpiderX »

Hi 6y588,

Would like to point your attention to the following:
http://www.stevehopwoodforex.com/phpBB3 ... =27&t=3210

In my opinion, ML techniques could be useful for solving this as compared to what they are currently using.

cheers
"Love is patient, love is kind. It does not envy, it does not boast, it is not proud. It does not dishonor others, it is not self-seeking, it is not easily angered, it keeps no record of wrongs.Love does not delight in evil but rejoices with the truth. It always protects, always trusts, always hopes, always perseveres."
-Corinthians 13:4-8
6y588

Re: R/Empty4 indicator

Post by 6y588 »

I have been busy working on my own trading ideas. Below is an example of a component that I am working on, which is part of a bigger picture to develop my own set of tools and indicators, for a complete trading system.

The thread you mentioned has some interesting ideas, and I will follow the discussion, but not quite sure how to contribute yet. May wait until after completing my own projects and giving time for them to crystalize their ideas first.

My system has six key components: (i) multi-pair; (ii) market strength; (iii) trend; (iv) momentum; (v) price action; and (vi) parameter-less. Each individual component is not new, however this system will be unique and different from others as it uses my own integration of custom indicators. I will not further elaborate as I want to keep this thread focused on R/Empty4 discussion, and also I have my own trading blog to do that.

As mentioned above, one of my own trading idea was to integrate the GradientVolatility indicator with the CurrencySlopeStrength (CSS) indicator. See the attached picture.

The CSS measures the strength of the currencies, however it does not measure the rate of change (or slope) of the strength. For example, in the picture, the CSS showed that AUD is negative and decelerating, but it does not show the rate of deceleration. The rate of change is captured using the GradientCSS.

There is a lot of potential for this indicator, as shown yesterday, when I gained a total of +73 pips from two long EurNzd trades based on their rates of change. See the attached history (ignore the other two non-EurNzd trades).

The best part for trading on this indicator is that there won't be any optimization, as there are very few parameters. Hence, I do not have to worry about a high variance (over-fitting).

Note: There is a problem when calling the GradientCSS indicator with the iCustom() function, as it does not seem to like R very much.

(Edit: Just an update, I have completely rewritten the GradientCSS indicator from the ground up to use MQL code only, by replacing the lm() function from R. Then the iCustom() function should be able to work with this indicator.)
You do not have the required permissions to view the files attached to this post.
Last edited by 6y588 on Mon Nov 11, 2013 2:32 am, edited 9 times in total.
User avatar
SpiderX
Trader
Posts: 554
Joined: Thu Aug 22, 2013 4:50 pm

Re: R/Empty4 indicator

Post by SpiderX »

Hi 6y588,

My thoughts on that discussion and on integrating ML are as follows:
Since they were able to predict next candles based on just looking at up and down candle.
It should be possible to use an appropriate ML technique (neural network) and feed it the sequence of up/down candles.. 1,0,1,1,0 ..etc, output will be the next candle direction.
Probably the neural network will need to be continuously updated to adapt to market conditions.
Input: 0,1,0,1,1,1,1 ==> Output : 1


It should be possible for ML technique to do this prediction since there is already example of someone being able to do it.
Conceptually, it looks straightforward, since there are no doubles involved and output is so straightforward too.
Just wondering, how implementation would go.

cheers
"Love is patient, love is kind. It does not envy, it does not boast, it is not proud. It does not dishonor others, it is not self-seeking, it is not easily angered, it keeps no record of wrongs.Love does not delight in evil but rejoices with the truth. It always protects, always trusts, always hopes, always perseveres."
-Corinthians 13:4-8
eigenvector
Trader
Posts: 20
Joined: Sat Jul 14, 2012 3:14 pm

Re: R/Empty4 indicator

Post by eigenvector »

6y588,

Interesting that you are calculating the slope (GradientVolatility) on the underlying currencies. I was planning on doing the same thing given that the currencies seem relatively better behaved compared to the pairs. This is probably what you refer to regarding market strength. With this setup I can envision automatic pair selection based on relative (momentum) and absolute (trend) currency strength. Appreciate your post as it may help to crystallize the concepts for me a bit.

As an aside, the slope was useful on individual currency pairs as well, but mostly in the longer term (low frequency).
Post Reply

Return to “Ideas for Possible Automation”