Proposal: Library organization to simplify EA code

[garyfritz]

I don't really enjoy slogging through thousands of lines of code, much of which is of no interest to me and may not even be called in the EA, to figure out what an EA is actually doing. We really don't need most of that code in the same file with the logic specific to a particular EA.

I don't know enough about MQL's particular variant of C to know how practical this is, but: would it be possible to move all of the standard modules into libraries or included files, so we could just have the code specific to that system in the EA file? It would certainly make it a lot easier to read and understand the EA code, and I suspect it would make it a lot easier for the EA coders too.

Assuming MQL supports it, I would put everything associated with a particular module into its module file, including externs, comments, etc. You'd probably want comments in the main file calling out the params of particular interest for that EA. E.g. if your EA uses the Hanover module, you'd want to explain how to set the Hanover params. But if you're not using it, you don't need any mention of it in your EA. People could refer to the shellEA PDF or go open the whatever.mql file to find out specifics about a particular module.

Then we could just "#include <shellEA.ex4>" or whatever to pick up the whole beastie, or maybe "#include <hanover.ex4>" to pick and choose individual modules. (Not sure if you actually import ex4's or use the library mechanism.)

We'd have to incorporate version numbers in the libraries' names, so an old EA wouldn't break when you made changes to an underlying library. You'd just release the new library with a new version name, and only newer EAs that knew how to use it would link in that newer library.

Is that do-able in MQL? I think it would make life easier for everyone, once we made the transition.
Gary

[magft]

I don't really enjoy slogging through thousands of lines of code, much of which is of no interest to me and may not even be called in the EA, to figure out what an EA is actually doing. We really don't need most of that code in the same file with the logic specific to a particular EA.

I don't know enough about MQL's particular variant of C to know how practical this is, but: would it be possible to move all of the standard modules into libraries or included files, so we could just have the code specific to that system in the EA file? It would certainly make it a lot easier to read and understand the EA code, and I suspect it would make it a lot easier for the EA coders too.

Assuming MQL supports it, I would put everything associated with a particular module into its module file, including externs, comments, etc. You'd probably want comments in the main file calling out the params of particular interest for that EA. E.g. if your EA uses the Hanover module, you'd want to explain how to set the Hanover params. But if you're not using it, you don't need any mention of it in your EA. People could refer to the shellEA PDF or go open the whatever.mql file to find out specifics about a particular module.

Then we could just "#include <shellEA.ex4>" or whatever to pick up the whole beastie, or maybe "#include <hanover.ex4>" to pick and choose individual modules. (Not sure if you actually import ex4's or use the library mechanism.)

We'd have to incorporate version numbers in the libraries' names, so an old EA wouldn't break when you made changes to an underlying library. You'd just release the new library with a new version name, and only newer EAs that knew how to use it would link in that newer library.

Is that do-able in MQL? I think it would make life easier for everyone, once we made the transition.
Gary

Gary, it is possible you create mqh files and then have an #include <hanover.mqh> at the top of the EA. I have mentioned this to Steve a while ago but he said he had used this setup in the past and had issues with Empty4 not running the EA correctly so that is why he put everything in in one! From a coders point of view if you write all the code you know what it does and what is need for a give EA. From an external point of view it makes it more difficult for take an Ea and add to it unless you are familiar with the code.

The LibOrderReliable is set up like this so you can call it from any EA, same for FANN2MQL and MT4R as well. So i think the issues "may" have be resolved in newer versions of Empty4.

Mike

[gaheitman]

I think we are talking about using the #include feature and not the #import feature. The #include feature really just alerts the compiler to replace the #include line with the contents of the file specified. You could arbitrarily cut out lines 100-200 of your code and save it to a file named 100-200.mqh. Then add "#include <100-200.mqh>" at the new line 100 in your code and it would compile just fine.

The good news, then, is that using include files should work the same as not using them.

I think separating the code into one or several include files is a good idea if we are willing to adhere to the following principles:

1. Library code does not rely on (and may not use) any global variables (meaning globals defined in the main code) other than externs
2. Variables global to the library would be named after the library (TMGT_StopLoss) to prevent name collisions (since there is no library-only global variables)
3. Library functions and procedures are classified as public (intended to be called by code outside the library) or private (intended to be called from within the library)
   1. Interfaces to public procedures are well-defined, completely parameterized and set in stone
   2. Interfaces to private procedures may change and so should not be called by code not in the library
4. No side-effects - Library code can't change the selected order (well, it has to change it back when it is done). Actually, I'd like to remove reliance on an order being selected from all of the code.

I'm sure there are more restrictions we'd want to place on the library code, but the point of doing these things is so we don't have to worry about versioning the include files. When a new version comes out, it can be dropped in place and all EAs can be recompiled and will use the new file without modification.

If we go down this road, one of the hardest things to do will be to decide on the interfaces. For example, if we consider the TradeMgmt module, it would likely handle the BreakEvenSL, TrailingSL, JumpingSL and CandleStickSL. Our first decision is whether the interface to the library is to each individual style of trailing stop or to a general TradeManagement() procedure. If we go with the latter, than the expectation is that we won't ever have to call the individual ones directly.

Now we have to decide if we want to call TradeManagement() on a single ticket at a time, or have it manage all open tickets. We also have to allow for managing multi-currency and multi-MagicNumber bots. Perhaps we'll decide to have a TradeManagementBasket() procedure that we call to manage tickets as a set, and the TradeManagement() procedure that manages each individual trade.

Setting aside the basket question, we could have any one of the following for trade management:

void TradeManagement(int Ticket); //manage individual ticket
void TradeManagement(string Symbol); //manage all trades for symbol
void TradeManagement(string MagicNumber); //manage all trades matching MagicNumber
void TradeManagement(); //manage all trades for symbol/magicnumber combo -- but caller doesn't get to choose
void TradeManagement(string Symbol; int MagicNumber); //manage all trades for symbol/magicnumber combo

If we go with the last one, we'd have to specify whether a NULL symbol means all matching symbols or current chart Symbol() and what magic number to use to select all magic numbers (-1?). If we decide 6 months down the road that there is a better way to do TradeManagement, we would have to decide to break backward compatibility (boo, hiss) or create a TradeManagement2.0() procedure that only new code knows about.

(As an aside, Empty4 allows you to have default values for parameters in procedures and to use the default you can just exclude the parameter for the call. So we could do a TradeManagement(string Symbol, int MagicNumber, bool ver2.0=false, int newparam1=0;int newparam2=0) that would not break compatibility, but would be awkward for the new code to use)

The fun part is that there are no "right" answers, but whatever the answers are they are far reaching in their impact!

Of course, none of this is an improvement unless it makes life better for Steve. I am sure he doesn't have any problems navigating the code because it is so familiar to him. From his perspective the true benefit will likely be the ease of getting a fix out to all of his creations at once.

The downside of taking on this project is that we wouldn't be doing anything that makes us money in Forex since there would certainly be a slow down in automation of other ideas.

George

[garyfritz]

Great suggestions, George! Though if you can do this without any versioning I will be impressed...

Setting aside the basket question, we could have any one of the following for trade management:

Why just one? You could offer all of them with different wrappers: TradeManagementTicket(), TradeManagementSymbol(), etc.

[gaheitman]

Great suggestions, George! Though if you can do this without any versioning I will be impressed...

Setting aside the basket question, we could have any one of the following for trade management:

Why just one? You could offer all of them with different wrappers: TradeManagementTicket(), TradeManagementSymbol(), etc.

Mostly because I'm lazy. You make a very good point, of course. No doubt they'd all call TradeManagementTicket() in the end, so there wouldn't really be differences in the core code being executed, just the looping and selection.

[SteveHopwood]

I approach this from a purely pragmatic point of view.

I know my code - well, mostly anyhow. When I code an EA, all I do is strip out the stuff I do not need and add in the stuff I do need. Stripping out the old stuff takes a couple of minutes and causes no problems. Adding in the new stuff introduces the bugs and causes me the headaches.

I once tried running a thread where running the EA depended on the user being able to add Matt Kennel's OR library to their platform. The result was painful enough for me to vow never to do so again.

My code works as is for me. Mostly. I offer it to others to use as they see fit. I do not care whether they have trouble reading it or not. As with version numbers et al, I am uninterested in the problems my approach causes others. I am interested in the way my approach solves them for me.

Once you have actually run a thread of your own where everything is not idiot-proof, you will understand my approach.

[AussieSteve]

G'day - been following the threads on here for a while, thought I'd chime in. I've taken Steve H's "Shell Code" and abstracted into two parts. 1 is the main program, with the init(), parameters, and unique code which runs the EA I'm coding. The second is the called library, which contains all the general functions which may or may not be used in each EA. The intent is to;

• 1. make my core code light and easy to read
  1. enable me to quickly update my EA to use the latest (i.e. least buggy) set of functions, without having to re-port my code into the latest drop of the shell library
  1. allow me to create alternate code in my main program (i.e. I use a very different DisplayUserFeedback() function to SH, so I call my DisplayUserFeedback_Custom()) which won't be overwritten when I merge with the newest Shell code if/when I forget .

It's been a side project of mine which I've never quite managed to get together quickly enough to drop back in on this site to share; every time I get it sorted Steve H has magic'ed something else into the shell code, or fixed a major bug. If there is some interest (and a green light from SH) I'll look to rolling it again with the current shell code and publish for others to investigate and use.

[markft]

G'day - been following the threads on here for a while, thought I'd chime in. I've taken Steve H's "Shell Code" and abstracted into two parts. 1 is the main program, with the init(), parameters, and unique code which runs the EA I'm coding. The second is the called library, which contains all the general functions which may or may not be used in each EA. The intent is to;

• 1. make my core code light and easy to read
  1. enable me to quickly update my EA to use the latest (i.e. least buggy) set of functions, without having to re-port my code into the latest drop of the shell library
  1. allow me to create alternate code in my main program (i.e. I use a very different DisplayUserFeedback() function to SH, so I call my DisplayUserFeedback_Custom()) which won't be overwritten when I merge with the newest Shell code if/when I forget .

It's been a side project of mine which I've never quite managed to get together quickly enough to drop back in on this site to share; every time I get it sorted Steve H has magic'ed something else into the shell code, or fixed a major bug. If there is some interest (and a green light from SH) I'll look to rolling it again with the current shell code and publish for others to investigate and use.

Hi Steve,

I have also undertaken a similar project. Posted info on it here: http://www.stevehopwoodforex.com/phpBB3 ... 7762#p7762 - please have a look if you are interested and perhaps we can both find it useful.