Zigzag Bollinger Band

[snailbeard]

During September many things changed while I did some deep re-factoring of the code. All part of getting transparency so that the new log files are giving me deep insights into how Zzbb gets to a final decision: whether or not to drop a trade and whether or not it should be a quick grab or a long suck. An equally important objective is to assist with manual entries by listing the pro's and con's at each potential entry.

As always I managed to mess things up with lots of new bugs part way through the month, much to the annoyance of Mrs Snailbeard who was tempted to swing a large heavy frying pan in my direction. At least I didn't have to add a new set of pairs for the Scottish Groat

However, despite the bugs, ZzBb still managed to make +12% growth in September. I was a bit mystified by some of the entries as well. By the end of the month I had addressed a whole raft of issues and was starting to see mainly positive results from the back tests.

The real-time winning pairs were GBPUSD, USDJPY and USDCAD
while EURUSD AND USDCHF had net losses.

A total of 33 trades, approximately a third were in GBPUSD. Good old Blighty!

However, this morning, out of four trades I was surprised to see two secondary entries at obvious wrong places. Looking at the difference in the log files for the live trades and a later back-test is the next task.
In the attached chart the first entry grabs a quick profit while the second entry occurs in the middle of a pull-back and goes to the stop-loss:

[snailbeard]

There is also a problem with the extra-profit-lock (EPL). According to the log file there is a switch to this method of trailing the trade, but it has no effect on the final result. The real-time trade was successful only because the FxOpen spread was less than 1 pip and the trade reached the target, while for back-testing I generally set the spread to 2 pips. Therefore, the back-test trade started later, switched to EPL later but didn't close the trade when price reversed too far.

This M1 chart of USDJPY shows the trade progressing for real-time (annotated in yellow) and afterwards a back-test (annotated in blue).

[snailbeard]

I have reimplemented some old code for tracking profit. It was originally implemented to help debug some code for trailing multiple trades per pair and it got neglected after I abandoned that method of trailing.
It was not designed to be safe for real time forward testing. However, the new version is designed for both forward and back-testing, but it cannot be used while trading more than one trade per pair until I start using classes and objects as it would be too cumbersome to do it in simple script.

It should now be possible to cross compare the the real-time results against the back-test and also find out exactly how the jumping stop losses and profit lock exits can be improved.

With reference to the earlier post for USDJPY, this is how the profit tracking looks shortly after the entry:

Code: Select all

04:30 USDJPY: :  1  $MxPf:10.71  $RPf: 7.33  Pips:4.60  $DdBp:-2.55  $DdAp:0.00  GapTp:23.10

where
$MxPf: Maximum money profit so far
$RPf: Snapshot of current money profit
Pips: Snapshot of profit in pips
$DdBp: How much the trade reversed before getting into profit
$DdAp: How much the trade reversed after it went into profit
GapTp: Snapshot of current number of pips away from the take profit price

so this extract shows the trade getting to within one pip of the take profit price:

Code: Select all

424: 05:16 USDJPY: :  1  $MxPf:53.81  $RPf: 53.81  Pips:21.10  $DdBp:-2.55  $DdAp:0.00  GapTp:6.60
425: 05:17 USDJPY: :  1  $MxPf:53.81  $RPf: 48.47  Pips:19.20  $DdBp:-2.55  $DdAp:0.00  GapTp:8.50
426: 05:18 USDJPY: :  1  $MxPf:75.18  $RPf: 75.18  Pips:28.70  $DdBp:-2.55  $DdAp:0.00  GapTp:1.00
427: 05:19 USDJPY: :  1  $MxPf:75.18  $RPf: 61.13  Pips:23.70  $DdBp:-2.55  $DdAp:0.00  GapTp:4.00
428: 05:20 USDJPY: :  1  $MxPf:75.18  $RPf: 56.06  Pips:21.90  $DdBp:-2.55  $DdAp:0.00  GapTp:5.80

That would have been a good time to extract profit or trigger an exit at some threshold.

Later in the day there was a chance to breakeven:

Code: Select all

1235: 15:20 USDJPY: :  1  $MxPf:75.18  $RPf: 14.94  Pips:7.30  $DdBp:-2.55  $DdAp:-30.53  GapTp:20.40
1236: 15:21 USDJPY: :  1  $MxPf:75.18  $RPf: -0.57  Pips:1.80  $DdBp:-2.55  $DdAp:-30.53  GapTp:25.90
1237: 15:22 USDJPY: :  1  $MxPf:75.18  $RPf: -18.65  Pips:-4.60  $DdBp:-2.55  $DdAp:-30.53  GapTp:32.30
1238: 15:23 USDJPY: :  1  $MxPf:75.18  $RPf: -15.26  Pips:-3.40  $DdBp:-2.55  $DdAp:-30.53  GapTp:31.10
1239: 15:24 USDJPY: :  1  $MxPf:75.18  $RPf: -29.11  Pips:-8.30  $DdBp:-2.55  $DdAp:-30.53  GapTp:36.00

and finally after 16:30, there was no hope:

Code: Select all

1355: 16:30 USDJPY: :  1  $MxPf:75.18  $RPf: -41.56  Pips:-12.70  $DdBp:-2.55  $DdAp:-68.76  GapTp:40.40
1371: 16:36 USDJPY: :  1  $MxPf:75.18  $RPf: -49.49  Pips:-15.50  $DdBp:-2.55  $DdAp:-68.76  GapTp:43.20
1372: 16:37 USDJPY: :  1  $MxPf:75.18  $RPf: -53.17  Pips:-16.80  $DdBp:-2.55  $DdAp:-68.76  GapTp:44.50
1373: 16:38 USDJPY: :  1  $MxPf:75.18  $RPf: -55.72  Pips:-17.70  $DdBp:-2.55  $DdAp:-68.76  GapTp:45.40
1374: 16:39 USDJPY: :  1  $MxPf:75.18  $RPf: -54.59  Pips:-17.30  $DdBp:-2.55  $DdAp:-68.76  GapTp:45.00
1375: 16:40 USDJPY: :  1  $MxPf:75.18  $RPf: -72.73  Pips:-23.70  $DdBp:-2.55  $DdAp:-72.73  GapTp:51.40
1376: 16:41 USDJPY: :  1  $MxPf:75.18  $RPf: -78.97  Pips:-25.90  $DdBp:-2.55  $DdAp:-78.97  GapTp:53.60
1377: 16:42 USDJPY: :  1  $MxPf:75.18  $RPf: -70.46  Pips:-22.90  $DdBp:-2.55  $DdAp:-78.97  GapTp:50.60
1378: 16:43 USDJPY: :  1  $MxPf:75.18  $RPf: -76.98  Pips:-25.20  $DdBp:-2.55  $DdAp:-78.97  GapTp:52.90
1379: 16:44 USDJPY: :  1  $MxPf:75.18  $RPf: -69.04  Pips:-22.40  $DdBp:-2.55  $DdAp:-78.97  GapTp:50.10
1380: 16:45 USDJPY: :  1  $MxPf:75.18  $RPf: -76.13  Pips:-24.90  $DdBp:-2.55  $DdAp:-78.97  GapTp:52.60

It is only carelessness that allows a near win to drift all the way to the original stoploss and a few tweaks should prevent this from happening in future.

[snailbeard]

Following on from previous post, I have extended the tracking to include the minimum gap to the target which can then be used to help make decisions about closing the trade

Code: Select all

05:16 USDJPY: :  1  $MxPf:53.81  $RPf: 53.81  Pips:21.10  $DdBp:-2.55  $DdAp:0.00  GapTp:6.60  MnGpTp:6.60
05:17 USDJPY: :  1  $MxPf:53.81  $RPf: 48.47  Pips:19.20  $DdBp:-2.55  $DdAp:0.00  GapTp:8.50  MnGpTp:6.60
05:18 USDJPY: :  1  $MxPf:75.18  $RPf: 75.18  Pips:28.70  $DdBp:-2.55  $DdAp:0.00  GapTp:1.00  MnGpTp:1.00
05:18 USDJPY: :  Tried OrderCloseReliable(): FAILED
05:18 USDJPY: :   ExitByExtraProfitLock

Unfortunately, the close operation failed. Hopefully that can be resolved.

Edit:
After finding and fixing a bug in the exit code the profit is retained:

Code: Select all

1	2014.10.01 04:23	buy	1	0.31	109.818	0.000	0.000	0.00	10000.00
2	2014.10.01 04:23	modify	1	0.31	109.818	109.492	110.095	0.00	10000.00
3	2014.10.01 05:18	close	1	0.31	110.085	109.492	110.095	75.18	10075.18

[snailbeard]

Thanks to the detailed logging, it was possible to fix some obvious wrong trades and tune some over zealous filters. This robot really needs a new name because the zigzags and Bollinger-bands are no-longer major actors in the logic. Perhaps it is more like the India dish Khichdi, which is a combination of several ingredients but served as a single dish. Perhaps less than 10% of the effort has been in the named parts. More effort has gone into understanding trend over different timescales and compensating for sluggishness of MA & even EMA.

PBRE (pullback-reentry) is supposed to be the main method of timing entries but requires more price movement, whereas PACL is a price movement across a threshold (such as a pivot level) in the current trend for a shorter range of pips.

Mini-PBRE is a hybrid of the PBRE entry methods but uses the PACL short range target and stop-loss.

Last week PACL by itself produced a net return of just over 12%. After sorting out a few quirks that improved to about 13%. After enabling Mini-PBRE over the same pairs and duration the win to loss ratio decreased but the net profit increased to 17%.

I now consider ZzBb3 to be stable enough to run on live accounts, although it will continue to require adjustments as the market changes.

In particular, the EURUSD fails to please any-which-way and requires it's own specific entry-exit method.
So I'll be reflecting on that in due course.

[snailbeard]

I've been testing ZzBb3 on mainly thin-spread brokers like GlobalPrime, FxOpen and Armada, but I have a long standing lot-size issue with fat-spread-Alpari. So its time to see if I can work out why this version of Empty4 gives an undersized lot-size.

Blurry eyed and before I'd finished my first coffee I ran a back test:

ZzBb-old509build05Sept14-GBPUSD.png

After the initial shock and panic, and squinting at the back-test settings, I realized that I had run the test with the old 509 build. After rerunning with the latest version the results were on par with the other brokers, although the profit was less due to the lot-size issue:

ZzBb3-latest-GBPUSD-20141010-AlpariUK.png

The log file shows the trade set-up but not how the lot-size was calculated:

Code: Select all

11:30 GBPUSD: M1 PACL: Setup VPO by  PALC Dp method  Dir: -1 At: 1.65646
11:30 GBPUSD: :  setVpoForGenericEntry():  PALC Dp method  Dir: -1 At: 1.65646
11:30 GBPUSD: :  VPO: Pair: GBPUSD, bActive: 0, Sell, Lots: 0.330, Price: 1.65626, SL: 1.65926, TP: 1.65353, MTPT: 0, TS: 2014.09.02 11:30, TE: 2014.09.02 11:45

So more detective work required...

[snailbeard]

While the Yen & the Loony generated nice profits on Monday the Euro continues to challenge. However, I feel I should trade it as much as possible while it still exists. Experts have been predicting the collapse of the Euro for years, and if the Italians and French drop out what will be left? It could still be years away - but who knows?
Anyway, I was surprised that ZzBb3 got away with a pointless trade which brought in a few pips today.
With hindsight there was a good entry at around 09:45 CET. However, it materialized on the wrong side of support so better to wait for a better set-up:

crazy-eurusd-scalp-14Oct2014.png

I didn't pay much attention to the Euro as I was tinkering with an indicator to display risk, money, lot-size: for a given risk (% of balance) and stop distance (in pips). It is almost working but it needs to also convert from the account currency e.g GBP to trade currency e.g. USD.

[snailbeard]

Although I have reverted to MetaCrap build 670 after lots of problems with Empty4 build 717, there are still regular phantom shutdowns. On restarting the platform all open positions are closed. These closes could be due to a loss of run-time data when the EA restarts. Some EAs make use of MetaCrap's persistent variables to manage live-data but I had problems with the reliability of these in the past.

One solution would be to save a row of data in a spreadsheet format and read it back after restarting, although this data could be incomplete after a several-hour shutdown.

Another option would be to scan the M1 price history and recreate any dynamic monitoring data
and finally I could add a flag so that certain functions know that the platform has been restarted.

And the simplest solution would be to avoid any dependence on tracked values by regenerating essential information when it is required.

On the other hand we could revisit the idea of moving to a more reliable platform. I see that our old friend Slipshod has moved on and is now using cTrader. I have recently opened an account with FxPro and it is a bonus that they are offering cTrader: any new projects will be created using c# & .Net

[snailbeard]

It would great if some wise sage could explain to me the conservation of ancient support and resistance and its on-going influence in the present. I would not be surprised if quite a few traders have blown their accounts or got severely burned this month.

AUDUSD-W1-death-star-DeathRays-Oct14.png

There is a very interesting and rare candle pattern (rare for AUDUSD on the weekly time-frame). September was the perfect month to sell, sell and sell, until price reached near 0.866 when a ?trader? looking at 12 months of prices noticed that AUDUSD lost a whole 12 months of gain in just three to four weeks. Someone somewhere with a lot of USD then started buying, but the bears beat the price back down. However, the contrarians were undeterred and continued to buy and the average of the high-low price for the last 3 weeks has stayed about the same. On the other hand the average of the W1 Open-close price has gone up. So the trend of the last 3 weeks has been up in terms of the open-close price.

This is quit easy to see at a glance, but ZzBb doesn't currently understood this obvious rising average and has done very badly in the last few days after relaxing some of the entry filters.

The last 3 W1 bars are also 'inside' bars which means a falling range, both body length are below average and the extremes are contracting. I can image my old teacher hitting me on the head and shouting: 'DON'T TRADE WHEN THE BARS ARE CONTRACTING!'

However, it would be more useful to be a bit more analytical since it is easy to see it historically, so I might write something about 'sticks and stickiness', which is an idea about measuring and quantifying non-trendingness in a faster way than moving averages.

However, this post is more about deep history than recent history. the next chart shows how important key price levels are to the market-movers-and-shakers and we need to know what they are thinking so that we are NOT running with the moving average days and weeks after opposite forces are battling for price control.

AUDUSD-weekly-superSR-at-0.866-Oct2014.png

ZzBb3 does not look back at what SR existed 12 months ago but perhaps the market has a 7 year itch:

AUDUSD-2007-old-swinger-at-0.88.png

[snailbeard]

Is anyone actually still trading real money with MetaCrap?
Is Build 670 really less crap than build 717? That is not my experience.
I frequently see market prices frozen for significant duration.

The freezes are also apparent in ZzBb3 log files:

Code: Select all

04:30 USDCAD: : PBRE:   dD1Stickness: 38.14 iD1CountedSticks: 3 dD1Shrinkage: 0.741
04:30 USDCAD: M1 PACL: bPACLDropBuyFlag  1
04:53 USDCAD: :  PBRE iDailyDirection:  1
04:53 USDCAD: :  PACL-A: Potential Buy at WP: bWpM15H1RisingBreak

There should have been a log update at 04:45.
Imagine what can happen to price in that 23 minute gap.
A delayed entry or exit can be a disaster!
These delays are affecting the the trailing stop and profit lock resulting in losses instead of profits.

Are MetaQuotes using customers live trades to debug their software?
Can we sue them for losses due this negligence?

Is Empty4 5 more robust, reliable and less buggy than Empty4 4?

I have just downloaded Empty4 build 735 directly from MetaQuotes web site.
Lets see how this runs for a week...

An open trade was closed on restarting the platform. Hopefully the improving logging has caught the reason for this.

Update:
I have now found the cause of close-trade on platform-restart, the fix is quite simple but only testable in real-time.