Script to (ATTEMPT) Force Download of historical data

Script to (ATTEMPT) Force Download of historical data

Postby mbkennel » Sun Jan 15, 2012 6:54 pm

Here's a script I cooked up to ATTEMPT to download historical data on all found symbols.

Sometimes iBars() forces a read, sometimes it doesn't work. Why? Empty4.

It Print()s messages to Experts, so read it to see what didn't work. Try it again, or go do those pairs manually by opening a chart and changing to every time frame and trying PageUp.

UPDATE: new improved version uses error checking as described below, puts up Alert() at end for failures.
You do not have the required permissions to view the files attached to this post.
"It is a capital mistake to theorize before one has data." Sir Arthur Conan Doyle.
"If your experiment needs statistics, you ought to have done a better experiment" Lord Rutherford.
mbkennel
 
Posts: 27
Joined: Tue Dec 20, 2011 5:40 am

Re: Script to (ATTEMPT) Force Download of historical data

Postby gaheitman » Mon Jan 16, 2012 7:32 am

mbkennel wrote:Here's a script I cooked up to ATTEMPT to download historical data on all found symbols.

Sometimes iBars() forces a read, sometimes it doesn't work. Why? Empty4.

It Print()s messages to Experts, so read it to see what didn't work. Try it again, or go do those pairs manually by opening a chart and changing to every time frame and trying PageUp.


Nice work. I was looking into this as well and stumbled across this: http://forum.mql4.com/45525#567226

Basically, it says that ArrayCopyRates() will also grab data, and it seemed they were satisfied that it worked for recent data. It may have the same results as iBars or your other methods, but I thought it worth mentioning.

Also, the call to iBars triggers a download, but doesn't wait. So for the first call, it will return zero bars, but it will have queued a request to download the information.

I also ran across mention of error 4066. From the documentation:
Code: [Select all]
 Please Login or Register to see this content
<

Apparently you should get this error when the data isn't available. Perhaps repeatedly calling iBars() until you no longer get error 4066 is the way to go?

George
User avatar
gaheitman
 
Posts: 644
Joined: Tue Nov 15, 2011 10:55 pm
Location: Richmond, VA, US

Re: Script to (ATTEMPT) Force Download of historical data

Postby mbkennel » Mon Jan 16, 2012 8:10 am

Thanks. I updated top-post with an updated version with that error check & minor tweaks (like I forgot about PERIOD_W1)

Let me know how it goes everyone. This is very crim-dependent.
"It is a capital mistake to theorize before one has data." Sir Arthur Conan Doyle.
"If your experiment needs statistics, you ought to have done a better experiment" Lord Rutherford.
mbkennel
 
Posts: 27
Joined: Tue Dec 20, 2011 5:40 am

Re: Script to (ATTEMPT) Force Download of historical data

Postby SteveHopwood » Tue Jan 17, 2012 11:34 pm

Fantastic again Matt. Many thanks.

I am busy catching up following my internet downtime and have not looked at the code. Is it easily added to our shell code and other EA's such as Graeme, or does it need a lot of adapting?

Cheers Matt. I believe I have mentioned occasionally that you are a star?

:D
Read the effing manual, ok?

Global Prime is the official SHF broker. Click here to sign up for a live account with Global Prime and join the several thousand SHF members who wisely choose GP as their broker of choice.

I still suffer from OCCD. Good thing, really.

Anyone here feeling generous? My paypal account is always in the market for a tiny donation. [email protected] is the account.

To see The Weekly Roundup of stuff you guys might have missed Click here

My special thanks to Tommaso (milanese) for all the incredible work he does here.
User avatar
SteveHopwood
 
Posts: 9140
Joined: Tue Nov 15, 2011 8:43 am
Location: Misterton - an insignificant village in England. Very pleasant to live in.

Re: Script to (ATTEMPT) Force Download of historical data

Postby MichaelM » Tue Jan 17, 2012 11:43 pm

Matt you're awesomeness :D
MichaelM
 
Posts: 112
Joined: Sun Dec 04, 2011 11:04 pm

Re: Script to (ATTEMPT) Force Download of historical data

Postby blazko » Tue Jan 17, 2012 11:56 pm

great great! Thank you.
cheers
User avatar
blazko
 
Posts: 84
Joined: Thu Dec 01, 2011 10:57 am

Re: Script to (ATTEMPT) Force Download of historical data

Postby hanover » Wed Jan 18, 2012 12:39 am

Great work, Matt.

Your script is very useful for loading OHLC data across all pairs/TFs for subsequent use in the Recent Strength indy that I developed, which is the prerequisite data source for Steve's 'hanover' EA (here: viewtopic.php?f=5&t=31)

Previously it was necessary to manually visit each TF for every pair, and then hold down the PageUp key to load the data. Or alternatively, wait for the RS indy to load the data, which (if using all 9 Empty4 TFs) for some crims would take anything up to 20 minutes.

I just ran your script across AxiTrader's Empty4 and then when I attached the RS indy – presto! – the plots took shape immediately.

Many thanks for your efforts.

David
User avatar
hanover
 
Posts: 9
Joined: Wed Nov 16, 2011 9:23 am
Location: Hamilton, New Zealand

Re: Script to (ATTEMPT) Force Download of historical data

Postby mbkennel » Wed Jan 18, 2012 1:21 am

SteveHopwood wrote:Fantastic again Matt. Many thanks.

I am busy catching up following my internet downtime and have not looked at the code. Is it easily added to our shell code and other EA's such as Graeme, or does it need a lot of adapting?

Cheers Matt. I believe I have mentioned occasionally that you are a star?

:D


It should be easy to add (it is mostly modularized in subroutines), but it can take quite a while to execute.

It tries to download all timeframes. If the EA doesn't use all timeframes, it would take too much time.
And if it is not multipair then it doesn't need to test the other pairs.

And it can fail, or appear to fail, and the next time you run it succeeds as Empty4 and your crims internet catches up. Or doesn't.

We could design a 'library' (I know you hate using real libraries vs cut and paste)

bool attemptDownloadSymbol(string symbol, int time_frame)
if symbol is StringLength()==0, do all, if time_frame is 0, do all. returns true if successful, false otherwise. This could be put into various EA's init().
"It is a capital mistake to theorize before one has data." Sir Arthur Conan Doyle.
"If your experiment needs statistics, you ought to have done a better experiment" Lord Rutherford.
mbkennel
 
Posts: 27
Joined: Tue Dec 20, 2011 5:40 am

Re: Script to (ATTEMPT) Force Download of historical data

Postby SteveHopwood » Wed Jan 18, 2012 2:16 am

Much food for thought here guys. Thanks.

:D
Read the effing manual, ok?

Global Prime is the official SHF broker. Click here to sign up for a live account with Global Prime and join the several thousand SHF members who wisely choose GP as their broker of choice.

I still suffer from OCCD. Good thing, really.

Anyone here feeling generous? My paypal account is always in the market for a tiny donation. [email protected] is the account.

To see The Weekly Roundup of stuff you guys might have missed Click here

My special thanks to Tommaso (milanese) for all the incredible work he does here.
User avatar
SteveHopwood
 
Posts: 9140
Joined: Tue Nov 15, 2011 8:43 am
Location: Misterton - an insignificant village in England. Very pleasant to live in.

Re: Script to (ATTEMPT) Force Download of historical data

Postby NicoScholtz » Wed Jan 18, 2012 6:34 am

Thank you very much! This is most useful! 8-)
Success is a marathon, not a sprint
NicoScholtz
 
Posts: 4
Joined: Thu Nov 17, 2011 5:44 am

Next

Return to Coders Hangout

Who is online

Users browsing this forum: No registered users and 1 guest