Script to (ATTEMPT) Force Download of historical data

mbkennel
Trader
Posts: 29
Joined: Tue Dec 20, 2011 5:40 am

Script to (ATTEMPT) Force Download of historical data

Post by mbkennel »

Here's a script I cooked up to ATTEMPT to download historical data on all found symbols.

Sometimes iBars() forces a read, sometimes it doesn't work. Why? Empty4.

It Print()s messages to Experts, so read it to see what didn't work. Try it again, or go do those pairs manually by opening a chart and changing to every time frame and trying PageUp.

UPDATE: new improved version uses error checking as described below, puts up Alert() at end for failures.
You do not have the required permissions to view the files attached to this post.
Last edited by mbkennel on Mon Jan 16, 2012 8:09 am, edited 1 time in total.
"It is a capital mistake to theorize before one has data." Sir Arthur Conan Doyle.
"If your experiment needs statistics, you ought to have done a better experiment" Lord Rutherford.
User avatar
gaheitman
Trader
Posts: 655
Joined: Tue Nov 15, 2011 10:55 pm
Location: Richmond, VA, US

Re: Script to (ATTEMPT) Force Download of historical data

Post by gaheitman »

mbkennel wrote:Here's a script I cooked up to ATTEMPT to download historical data on all found symbols.

Sometimes iBars() forces a read, sometimes it doesn't work. Why? Empty4.

It Print()s messages to Experts, so read it to see what didn't work. Try it again, or go do those pairs manually by opening a chart and changing to every time frame and trying PageUp.
Nice work. I was looking into this as well and stumbled across this: http://forum.mql4.com/45525#567226

Basically, it says that ArrayCopyRates() will also grab data, and it seemed they were satisfied that it worked for recent data. It may have the same results as iBars or your other methods, but I thought it worth mentioning.

Also, the call to iBars triggers a download, but doesn't wait. So for the first call, it will return zero bars, but it will have queued a request to download the information.

I also ran across mention of error 4066. From the documentation:

Code: Select all

ERR_HISTORY_WILL_UPDATED	4066	Requested history data in updating state.
Apparently you should get this error when the data isn't available. Perhaps repeatedly calling iBars() until you no longer get error 4066 is the way to go?

George
mbkennel
Trader
Posts: 29
Joined: Tue Dec 20, 2011 5:40 am

Re: Script to (ATTEMPT) Force Download of historical data

Post by mbkennel »

Thanks. I updated top-post with an updated version with that error check & minor tweaks (like I forgot about PERIOD_W1)

Let me know how it goes everyone. This is very crim-dependent.
"It is a capital mistake to theorize before one has data." Sir Arthur Conan Doyle.
"If your experiment needs statistics, you ought to have done a better experiment" Lord Rutherford.
User avatar
SteveHopwood
Owner
Posts: 9754
Joined: Tue Nov 15, 2011 8:43 am
Location: Misterton - an insignificant village in England. Very pleasant to live in.

Re: Script to (ATTEMPT) Force Download of historical data

Post by SteveHopwood »

Fantastic again Matt. Many thanks.

I am busy catching up following my internet downtime and have not looked at the code. Is it easily added to our shell code and other EA's such as Graeme, or does it need a lot of adapting?

Cheers Matt. I believe I have mentioned occasionally that you are a star?

:D
Read the effing manual, ok?

Afterprime is the official SHF broker. Read about them at https://www.stevehopwoodforex.com/phpBB3/viewtopic.php?p=175790#p175790.

I still suffer from OCCD. Good thing, really.

Anyone here feeling generous? My paypal account is always in the market for a tiny donation. [email protected] is the account.

To see The Weekly Roundup of stuff you guys might have missed Click here

My special thanks to Thomas (tomele) for all the incredible work he does here.
MichaelM
Trader
Posts: 117
Joined: Sun Dec 04, 2011 11:04 pm

Re: Script to (ATTEMPT) Force Download of historical data

Post by MichaelM »

Matt you're awesomeness :D
User avatar
blazko
Trader
Posts: 86
Joined: Thu Dec 01, 2011 10:57 am

Re: Script to (ATTEMPT) Force Download of historical data

Post by blazko »

great great! Thank you.
cheers
MiTija

Re: Script to (ATTEMPT) Force Download of historical data

Post by MiTija »

Well Done!!!

Thank you,

MiTija
User avatar
hanover
Posts: 8
Joined: Wed Nov 16, 2011 9:23 am
Location: Hamilton, New Zealand

Re: Script to (ATTEMPT) Force Download of historical data

Post by hanover »

Great work, Matt.

Your script is very useful for loading OHLC data across all pairs/TFs for subsequent use in the Recent Strength indy that I developed, which is the prerequisite data source for Steve's 'hanover' EA (here: http://www.stevehopwoodforex.com/phpBB3 ... p?f=5&t=31)

Previously it was necessary to manually visit each TF for every pair, and then hold down the PageUp key to load the data. Or alternatively, wait for the RS indy to load the data, which (if using all 9 Empty4 TFs) for some crims would take anything up to 20 minutes.

I just ran your script across AxiTrader's Empty4 and then when I attached the RS indy – presto! – the plots took shape immediately.

Many thanks for your efforts.

David
mbkennel
Trader
Posts: 29
Joined: Tue Dec 20, 2011 5:40 am

Re: Script to (ATTEMPT) Force Download of historical data

Post by mbkennel »

SteveHopwood wrote:Fantastic again Matt. Many thanks.

I am busy catching up following my internet downtime and have not looked at the code. Is it easily added to our shell code and other EA's such as Graeme, or does it need a lot of adapting?

Cheers Matt. I believe I have mentioned occasionally that you are a star?

:D
It should be easy to add (it is mostly modularized in subroutines), but it can take quite a while to execute.

It tries to download all timeframes. If the EA doesn't use all timeframes, it would take too much time.
And if it is not multipair then it doesn't need to test the other pairs.

And it can fail, or appear to fail, and the next time you run it succeeds as Empty4 and your crims internet catches up. Or doesn't.

We could design a 'library' (I know you hate using real libraries vs cut and paste)

bool attemptDownloadSymbol(string symbol, int time_frame)
if symbol is StringLength()==0, do all, if time_frame is 0, do all. returns true if successful, false otherwise. This could be put into various EA's init().
"It is a capital mistake to theorize before one has data." Sir Arthur Conan Doyle.
"If your experiment needs statistics, you ought to have done a better experiment" Lord Rutherford.
User avatar
SteveHopwood
Owner
Posts: 9754
Joined: Tue Nov 15, 2011 8:43 am
Location: Misterton - an insignificant village in England. Very pleasant to live in.

Re: Script to (ATTEMPT) Force Download of historical data

Post by SteveHopwood »

Much food for thought here guys. Thanks.

:D
Read the effing manual, ok?

Afterprime is the official SHF broker. Read about them at https://www.stevehopwoodforex.com/phpBB3/viewtopic.php?p=175790#p175790.

I still suffer from OCCD. Good thing, really.

Anyone here feeling generous? My paypal account is always in the market for a tiny donation. [email protected] is the account.

To see The Weekly Roundup of stuff you guys might have missed Click here

My special thanks to Thomas (tomele) for all the incredible work he does here.
Post Reply

Return to “Coders Hangout”