real time data

[hedgeitall]

I am just interested if anyone uses any fundamental data or real time data for indicators or trading.
There are a few stories of trading using twitter feeds, etc. Of course there is FFCAL that gives news updates.
If you could find successful traders who post their trading results you could use that data.
You could use the data of people posting signals,etc.
Are there any sources of data worth considering?

In most cases it is easy enough to get the data into Empty4 or whichever platform you use.

Here's an example I chucked together using Python to extract the Buy/Sell % of myfxbook live accounts
the Python code is set to run in a loop updating every 60 seconds and write to a text file
It gives market sentiment based on live data.

Here is an Empty4 Function to read the file created

Code: Select all

int readfromfile(int pr){
   double values[];
   int h = FileOpen("output.txt", FILE_CSV || FILE_READ, ";");
   int i = 0;
   while (!FileIsEnding(h)){
      string s = StringTrimRight(FileReadString(h));
      if (s == "")
         {
         continue;   
         }
      double value = StrToDouble(s);
      ArrayResize(values, i + 1);
      values[i] = value;
      i++;
   }
   FileClose(h);
   return(values[pr]);
}

To get the results

Code: Select all

int buypercent;
int pai=0;
if(Symbol()=="EURUSD")pai=0;
if(Symbol()=="GBPUSD")pai=1;
if(Symbol()=="USDJPY")pai=2;
if(Symbol()=="GBPJPY")pai=3;
if(Symbol()=="USDCAD")pai=4;
if(Symbol()=="EURJPY")pai=5;
if(Symbol()=="AUDUSD")pai=6;
if(Symbol()=="EURCHF")pai=7;
if(Symbol()=="EURGBP")pai=8;
if(Symbol()=="USDCHF")pai=9;
if(Volume[0]<5)buypercent=readfromfile(pai);

[hedgeitall]

Here's an update on how a test MyFxBook-Market-Sentiment EA is performing. It is only a 30 day demo.
The EA simply buys if over 50% of myfxbook accounts are long. Sells if under 50% of myfxbook accounts are short. It uses an opposite close function so there is always an open trade for a particular pair.

[garyfritz]

Very cool! I never would have guessed the aggregate myfxbook accounts would perform that well. If anything I would have guessed they would be a contrary indicator.

I wonder if it would be possible to spider through myfxbook and glean the results of individual account holders, and only follow the positions of the best performers?

[blahn]

Interesting idea. I would assume it should be possible to aggregate the other stat sites like fxstat, mt4stats, mt4live, etc with myfxbook. The larger the aggregated pool, the stronger your sentiment signal should be. And as Gary said, you could give a higher weight ratio to those who are more successful from each site. heck, if you wanted to add a few hundred thousand more trades to your pool you could add all the signal providers that have demo accts too like zulutrade, rentasignal, etc

[hedgeitall]

MyFxbook say that 49% of accounts are profitable. So it is surprising to see it work so well.
There are probably better ways to use the data? This was just thrown together quickly.

Yes you could take data from an individual account or accounts,zulutrade, rentasignal,etc. You could set it up to search every myfxbook,zulutrade,rentasignal account and only look for profitable accounts.

You can take data from just about anywhere.

You could set up more advanced news filters.
You could look for keywords in press releases?
Scan twitter?
get data from other charts outside of Empty4?

It opens up a whole new world of data. Really every indicator used in EA's comes from the 1 source. It makes sense to look to other sources for indicators of market movement.

The issue is - Which sources are of any use? which are just garbage?

For example you could combine.
The number of cows sold to McDonalds in North America every day,
Coal exports in Canada for the month
& monthly tourism numbers in Australia
to make an indicator.

But will it give you anything of value? Probably not for forex.

[hedgeitall]

this is what we need.
http://www.youtube.com/watch?v=IBov98-L ... ure=relmfu

[blahn]

MyFxbook say that 49% of accounts are profitable. So it is surprising to see it work so well.
There are probably better ways to use the data? This was just thrown together quickly.

Yes you could take data from an individual account or accounts,zulutrade, rentasignal,etc. You could set it up to search every myfxbook,zulutrade,rentasignal account and only look for profitable accounts.

You can take data from just about anywhere.

You could set up more advanced news filters.
You could look for keywords in press releases?
Scan twitter?
get data from other charts outside of Empty4?

It opens up a whole new world of data. Really every indicator used in EA's comes from the 1 source. It makes sense to look to other sources for indicators of market movement.

The issue is - Which sources are of any use? which are just garbage?

we would probably have to look at many of these. The idea is very similar to concepts in the video you mentioned, and in what we do every day already with Gallup Polls, misc surveys, etc. They all need an appropriately high sampling of specific target data to make a good prediction. Just look how accurate gallup polls can be just using phone surveys. The trick will be finding the truly relevant sources that have enough weight to accurately model a prediction.

[garyfritz]

The fact you got results like that, with 49% winning accounts, demonstrates how powerful this is. It has tremendous potential. You could scan databases of Martian sandstorms and Facebook posting frequency and hunt all over for possible data sources, but why? Most of those will have very little correlation to FX. Scanning FX accounts makes a lot more sense IMHO.

I think you could supercharge it by selecting individual accounts to follow. Filter out those 51% losers, and even most of the marginal winners, and you will be left with much more powerful signals. I would scan through the individual accounts and weight them according to recent success rate -- % gain over the last X periods, or Kelly value or Sharpe or something like that. Filter out any accounts that fall below some cutoff. Then build your trading signal by adding the weighted buy/sell position of the remaining accounts. I bet that'll work a LOT better than the equally-weighted aggregate performance of all accounts. (I trade a system that uses weighted positions just like this, but weighting individual pairs. It makes an ENORMOUS difference in performance.)

I would suggest somebody start scraping this data immediately, and stash it away in some kind of database. Then you could test out and refine your algorithms on the historic data.

[hedgeitall]

This might super charge the data gathering.

http://rapid-i.com/content/view/181/190/

Takes 5 minutes to do, what took me a couple of days.
(well it would have taken 5 mins if I knew what i was doing)

There are heaps of tutorials. & no real coding to get results.
http://www.youtube.com/watch?v=zMyrw0Hs ... el&list=UL

[NeoTrader]

hi hedgeitall,

nice find...looks like a very promising tool...

will dug into it when i have some time...


Thanks again & happy trading,

NeoTrader

-