Neural network system

[Gamma_gallus]

Hi everyone,

For some time now, I’m really interested to the neural network (NN) systems, because i really think this could give us a real edge for our bots. Not being a coder myself I can’t do much on this, and i also believe it could work if there’s a team working and brainstorming on it.
(Documentation at the end of the post)

So, what it is all about ?

To make it simple, we give to the EA the ability to become smart enough to analyze himself the situation (what he sees on the chart) and assess whether or not he should take any action (long/short) based on what he learnt from the past. We feed him with some data & logic, then he learns himself for a while, to finally being operationnal.

In other words, we are the teacher, the bot is the student. At school he learns the methodologies, and once adult he is able to think by himself and be « independant » (of course if the teacher is bad, i let you imagine the outcome of its adult life..)
That’s exactly the same concept than : you’ve leanrt at school that 3 x 5 = 15 so you’re able to deduct now that 30 x 50 = 1500

All current EAs do strictly follow the rules we’ve implemented/hardcoded into the code, so rather inflexible because they’re not able to adapt to changing market conditions. A NN system has the possibility to perform better and adapt itself (of course if we did code well it ..)

The good thing is that it exists a free open source Fast Artificial Neural Network Library (FANN) in mq4.
The bad thing, is that it looks rather complex to implement into a code and make it work.. (requires deep coding, math & architecture skills)

We could have simple trading concepts to bloody complex ones using this method .. here are few ideas :

- A News system : we feed the bot with the news relases from the past (high ones) and price action that occured that at that time (implies we store the past news in a text file) and we let him analyze the different scenarios that have happened.

- Enhance the CBI module (central bank intervention), when that happened in the past (JPY ; CHF) maybe there were some similar conditions just before it occurs !?

- Adapt Nanningbob v10 : bot defines if market is trending, ranging and adapts it’s trading style accordingly

- We feed the bot with few indicators (e.g. : trend following ; profit taking ; overbought/oversold and a volume one) then he’ll need to learn the different combinations that happened in the past and the corresponding price actions, so that when similar market conditions happen again he could try to anticipate.

What do you think ?

**** Documentation ****
FANN Empty4 library (open source)
http://fann4mt.thetradingtheory.com/

2 Empty4 test examples
http://articles.mql4.com/868
http://articles.mql4.com/777

A looong & complex article on this topic
http://articles.mql4.com/717

Few posts on a stock market attempt (not based on Empty4 tough)
http://leenissen.dk/fann/forum/viewtopic.php?f=1&t=152


Thanks
Gamma

[Gamma_gallus]

We could even maybe use a NN to build a statistical model as shown in the attached pdf !?
Okay, to be honest, I don't understand a word from this pdf, this is way too complex for my last living brain cell
So please be fair with me, I have no clue if this is applicable to trading .. just want to share it with you.

Gamma

[magft]

Hi everyone,

For some time now, I’m really interested to the neural network (NN) systems, because i really think this could give us a real edge for our bots. Not being a coder myself I can’t do much on this, and i also believe it could work if there’s a team working and brainstorming on it.
(Documentation at the end of the post)

So, what it is all about ?

To make it simple, we give to the EA the ability to become smart enough to analyze himself the situation (what he sees on the chart) and assess whether or not he should take any action (long/short) based on what he learnt from the past. We feed him with some data & logic, then he learns himself for a while, to finally being operationnal.

In other words, we are the teacher, the bot is the student. At school he learns the methodologies, and once adult he is able to think by himself and be « independant » (of course if the teacher is bad, i let you imagine the outcome of its adult life..)
That’s exactly the same concept than : you’ve leanrt at school that 3 x 5 = 15 so you’re able to deduct now that 30 x 50 = 1500

All current EAs do strictly follow the rules we’ve implemented/hardcoded into the code, so rather inflexible because they’re not able to adapt to changing market conditions. A NN system has the possibility to perform better and adapt itself (of course if we did code well it ..)

The good thing is that it exists a free open source Fast Artificial Neural Network Library (FANN) in mq4.
The bad thing, is that it looks rather complex to implement into a code and make it work.. (requires deep coding, math & architecture skills)

We could have simple trading concepts to bloody complex ones using this method .. here are few ideas :

- A News system : we feed the bot with the news relases from the past (high ones) and price action that occured that at that time (implies we store the past news in a text file) and we let him analyze the different scenarios that have happened.

- Enhance the CBI module (central bank intervention), when that happened in the past (JPY ; CHF) maybe there were some similar conditions just before it occurs !?

- Adapt Nanningbob v10 : bot defines if market is trending, ranging and adapts it’s trading style accordingly

- We feed the bot with few indicators (e.g. : trend following ; profit taking ; overbought/oversold and a volume one) then he’ll need to learn the different combinations that happened in the past and the corresponding price actions, so that when similar market conditions happen again he could try to anticipate.

What do you think ?

**** Documentation ****
FANN Empty4 library (open source)
http://fann4mt.thetradingtheory.com/

2 Empty4 test examples
http://articles.mql4.com/868
http://articles.mql4.com/777

A looong & complex article on this topic
http://articles.mql4.com/717

Few posts on a stock market attempt (not based on Empty4 tough)
http://leenissen.dk/fann/forum/viewtopic.php?f=1&t=152


Thanks
Gamma

I have had a play around with FANN in the past and it does look interesting. With NNs the key is to get the configuration correct with the correctly formatted data. Its no good asking it to compare apples with oranges unless we tell it how to compare them

The idea about using events and CBI is interesting but not easy to quantify, you would need to create data that shows what happens ie cause and effect and train NN on this. I am very interested in this as i did my PhD in Neural Networks using Matlab and i have thought about using them for forex for a while. There are lots of articles around on ways to use them as predictors though all the ones i tested didnt seem to work well. Maybe between the multitude of minds here we can come up with something good. I'm up for coding something if Steve isnt interested in this one.

Let the fun commence.

Mike

[Gamma_gallus]

Hi Mike,

Should there be enough people interested to a NN system, I think we should start with a very simple model to see how it goes. Then if that has some potential, let's try to go further and make it more complex by adding more data to study.

- Regarding the news events, according to me , it's a good example (see below), but since the big moves happen within a minute, it seems difficult to automate it (the time the EA retrieves the actual data from a website and place an order, it might be too late)

A good example because : let's take the GBP retail sales from below link.
We have the history, the values (actual, forecast .etc..) and the spikes. So the EA could analyze all those data, massage them (make ratios: % actual vs forecast > spike) and on the next release if the same data present an already seen situation, then it can jump on it with a pre-defined BE, TP (based on the similar historical scheme)
But once again the time we retrieve the data from the web it might be too late ... but I still give this example because it's a nice visual one.

http://www.forexpeacearmy.com/forex_new ... p?id=16740

Gamma

[hiredwhip]

Hi everyone,

For some time now, I’m really interested to the neural network (NN) systems, because i really think this could give us a real edge for our bots. Not being a coder myself I can’t do much on this, and i also believe it could work if there’s a team working and brainstorming on it.
(Documentation at the end of the post)

So, what it is all about ?

To make it simple, we give to the EA the ability to become smart enough to analyze himself the situation (what he sees on the chart) and assess whether or not he should take any action (long/short) based on what he learnt from the past. We feed him with some data & logic, then he learns himself for a while, to finally being operationnal.

In other words, we are the teacher, the bot is the student. At school he learns the methodologies, and once adult he is able to think by himself and be « independant » (of course if the teacher is bad, i let you imagine the outcome of its adult life..)
That’s exactly the same concept than : you’ve leanrt at school that 3 x 5 = 15 so you’re able to deduct now that 30 x 50 = 1500

All current EAs do strictly follow the rules we’ve implemented/hardcoded into the code, so rather inflexible because they’re not able to adapt to changing market conditions. A NN system has the possibility to perform better and adapt itself (of course if we did code well it ..)

The good thing is that it exists a free open source Fast Artificial Neural Network Library (FANN) in mq4.
The bad thing, is that it looks rather complex to implement into a code and make it work.. (requires deep coding, math & architecture skills)

We could have simple trading concepts to bloody complex ones using this method .. here are few ideas :

- A News system : we feed the bot with the news relases from the past (high ones) and price action that occured that at that time (implies we store the past news in a text file) and we let him analyze the different scenarios that have happened.

- Enhance the CBI module (central bank intervention), when that happened in the past (JPY ; CHF) maybe there were some similar conditions just before it occurs !?

- Adapt Nanningbob v10 : bot defines if market is trending, ranging and adapts it’s trading style accordingly

- We feed the bot with few indicators (e.g. : trend following ; profit taking ; overbought/oversold and a volume one) then he’ll need to learn the different combinations that happened in the past and the corresponding price actions, so that when similar market conditions happen again he could try to anticipate.

What do you think ?

**** Documentation ****
FANN Empty4 library (open source)
http://fann4mt.thetradingtheory.com/

2 Empty4 test examples
http://articles.mql4.com/868
http://articles.mql4.com/777

A looong & complex article on this topic
http://articles.mql4.com/717

Few posts on a stock market attempt (not based on Empty4 tough)
http://leenissen.dk/fann/forum/viewtopic.php?f=1&t=152


Thanks
Gamma

I have had a play around with FANN in the past and it does look interesting. With NNs the key is to get the configuration correct with the correctly formatted data. Its no good asking it to compare apples with oranges unless we tell it how to compare them

The idea about using events and CBI is interesting but not easy to quantify, you would need to create data that shows what happens ie cause and effect and train NN on this. I am very interested in this as i did my PhD in Neural Networks using Matlab and i have thought about using them for forex for a while. There are lots of articles around on ways to use them as predictors though all the ones i tested didnt seem to work well. Maybe between the multitude of minds here we can come up with something good. I'm up for coding something if Steve isnt interested in this one.

Let the fun commence.

Mike

Hi Mike,
I contributed on this thread for a while into the summer and ran out of time.....The FF Mafia shut this guy down for some reason but I went back and reread the thread, just to finish it.......It's an interesting read with alot of theory but no source code.......
I'll put a link to it here........For anything you may gain from it....

http://www.forexfactory.com/showthread.php?t=240358

have a great day

whip

[magft]

Hi Mike,
I contributed on this thread for a while into the summer and ran out of time.....The FF Mafia shut this guy down for some reason but I went back and reread the thread, just to finish it.......It's an interesting read with alot of theory but no source code.......
I'll put a link to it here........For anything you may gain from it....

http://www.forexfactory.com/showthread.php?t=240358

have a great day

whip

I do remember reading the thread, just had a scan through my PCs and i have NextClose v13.mq4, i do have various papers and indicators on the subject. I don't think any really do that well. I do remember reading an article about a trader that uses all factors that effect a currency (eg EURUSD, OIL price, FTSE100, S&P500, etc) to train NN on a daily basis. I also remember in RonaldRayguns "Old Dog New Tricks" he talked about retraining/self optimising EAs and RFKing2 said he had used the idea but switched to using NN.

I have used the FANN2MQL4 libraries and so could put ideas into practice, obviously starting off simply and increasing complexity.

Mike

[hiredwhip]

Hi Mike,
I contributed on this thread for a while into the summer and ran out of time.....The FF Mafia shut this guy down for some reason but I went back and reread the thread, just to finish it.......It's an interesting read with alot of theory but no source code.......
I'll put a link to it here........For anything you may gain from it....

http://www.forexfactory.com/showthread.php?t=240358

have a great day

whip

I do remember reading the thread, just had a scan through my PCs and i have NextClose v13.mq4, i do have various papers and indicators on the subject. I don't think any really do that well. I do remember reading an article about a trader that uses all factors that effect a currency (eg EURUSD, OIL price, FTSE100, S&P500, etc) to train NN on a daily basis. I also remember in RonaldRayguns "Old Dog New Tricks" he talked about retraining/self optimising EAs and RFKing2 said he had used the idea but switched to using NN.

I have used the FANN2MQL4 libraries and so could put ideas into practice, obviously starting off simply and increasing complexity.

Mike

If Steve has time to do it or you want to do, I'll be happy to jump in and help.....it could get really interesting...RR's bot had alot of good points also and he was off and on that thread for awhile...

whip

[Cubic]

Hi,

Heres another one

In Russian: http://codebase.mql4.com/ru/6288

In English, sorta
http://translate.google.com/translate?h ... md%3Dimvns

However, note the paragraph entitled "Notes on the forward test"

[DING]

it could get really interesting..I would join and give some help, my master degree thesis focus on NN and Data Mining.
go go Neural Network !

[Gamma_gallus]

Let's try to suggest simple systems to start with.
Below are 2 of them, that i think are easy to code (hopefully they're really easy .. )

1. Tweak an EA from Steve : Reversi (TF M15 > H1?)
First, We identify the trend (using Nanningbob v10? i like his system, or any other)
Of course if we are in an Up trend we only buy etc..
In a Up Trend, the bot will calculate the stats : after a serie of X bearish candles there's a reversal to the main trend (x% of chance) : we enter at this point.
The NN will be in charge to find the optimum entry point based on what he learned from the history (e.g. after 4 bearish candles in 80% of the cases the price retraces back to the main up trend direction).
In case of ranging market, maybe we can allow buy and sell positions ??
SL could be also defined by the NN based on a multiplier of the ATR value (with a maximum limit we define ourselves)
TP is a user input (TL, JS, fixed pip value), or ATR value, unless we let the bot automatically calculate it ?

2. Fib or Pivot points and round numbers (H1, H4)
How usually price behaves when reaching the different levels.
SL also based on a multiplier of the ATR value (max limit too) or simply set up on a different pivot/Fib level.
TP same as point 1.

Anyone having suggestions on simple systems, please feel free to intervene.

Gamma