developing multi-time frame trading rules with a trend
TRANSCRIPT
Developing Multi-Time Frame Trading Rules with a Trend
Following Strategy, using GP
Jaime Morais Monteiro de Jacinto Machado
Thesis to obtain the Master of Science Degree in
Electrical and Computer Engineering
Supervisors: Prof. Rui Fuentecilla Maia Ferreira Neves
Prof. Nuno Cavaco Gomes Horta
Examination Committee
Chairperson: Prof. Horácio Cláudio de Campos Neto
Supervisor: Prof. Rui Fuentecilla Maia Ferreira Neves
Members of the Committee: Prof. Aleksandar Ilić
November 2015
i
Resumo A otimização de regras de negociação que se adaptam às mudanças nos mercados tem
merecido crescente atenção por parte do mundo financeiro e por parte dos académicos. Este
trabalho propõe uma metodologia para criar regras de negociação com a característica de
seguir tendências e que combina várias escalas temporais. As regras de negociação são
baseadas em indicadores técnicos tais como o RVI e o MACD e são individualmente otimizadas
para 19 índices bolsistas e 11 mercadorias no período entre 2006 e 2014. São usadas
estruturas em árvore para representar as regras que estas são gradualmente melhoradas
através de algoritmos evolucionários. As melhores soluções para negociar estes ativos são
depois simuladas individualmente e em Portfolios.
A utilização de duas escalas temporais permite a redução do risco visto que são combinados
dois perfis de negócios diferentes, naquilo que poderá ser designado por diversificação de
sistemas de negociação. Um inovador algoritmo para eliminar soluções semelhantes é
também apresentado, usando o Erro Quadrático Médio dos sinais gerados.
As regras de negociação geradas por este método obtêm lucros em tendências ascendentes e
descendentes reagindo rapidamente a quedas fortes. Nas quedas dos mercados em 2008 as
regras otimizadas não só impediram quebras fortes no capital como conseguiram lucrar com
as quedas.
Palavras-chave: Algoritmos Evolutivos, Programação Genética, Otimização, Mercados
Financeiros, Bolsa de Valores, Análise Técnica, Indicadores Técnicos, Regras de Negociação,
Seguidor de Tendências, Portfolio.
ii
iii
Abstract The optimization of trading rules that adapt to changing market conditions has been subject
to increasing attention from the financial world and from academics. This work presents a
methodology to develop trading rules with a trend-following approach combining several time
frames. Rules are based on technical indicators like the RVI and MACD and are individually
optimized for 19 stock indices and 11 commodities in the period from 2006 to 2014. Tree
structures are used to represent the trading rules which are gradually evolved through
evolutionary algorithms. The best solutions to trade these assets are then simulated
individually and in portfolios.
The use of two time frames allows a reduction in risk since two different profiles of trades are
combined in what can be described as trading system diversification. An innovative algorithm
to delete similar rules is also presented, based on the Mean Squared Error of the generated
trading signals.
The trading rules obtained by this method are able to profit from upward and downward
trends and react fast to sharp falls. In the bear market of 2008 the optimized rules not only
prevented sharp drops in capital but managed to profit from the declining prices.
Keywords: Evolutionary Algorithms, Genetic Programming, Optimization, Financial Markets,
Stock Markets, Technical Analysis, Technical Indicators, Trading Rules, Trend-following,
Portfolio.
iv
v
Acknowledgements I thank my supervisor Rui Neves for his guidance, support and feedback. I am grateful for his
openness and flexibility, allowing me a great deal of freedom in my work. I also thank my co-
supervisor, Nuno Horta, for his input and feedback.
I thank my family for her support, especially my mother for her continued support in all
domains and my young son, Miguel.
vi
vii
Dedication I dedicate this work to my father, Fernando Machado.
viii
ix
Table of Contents RESUMO............................................................................................................................................................ I
ABSTRACT ....................................................................................................................................................... III
ACKNOWLEDGEMENTS .................................................................................................................................... V
DEDICATION ................................................................................................................................................... VII
TABLE OF CONTENTS ....................................................................................................................................... IX
LIST OF TABLES .............................................................................................................................................. XIII
LIST OF FIGURES: ............................................................................................................................................ XV
LIST OF ACRONYMS: ..................................................................................................................................... XVII
1 INTRODUCTION ....................................................................................................................................... 1
1.1 WORKS PURPOSE ....................................................................................................................................... 2
1.2 RELATED PUBLICATIONS ............................................................................................................................... 2
1.3 DOCUMENT STRUCTURE .............................................................................................................................. 2
2 RELATED WORK ....................................................................................................................................... 5
2.1 MARKET ANALYSIS ...................................................................................................................................... 5
2.1.1 Efficient Markets and the Random Walk Theory ............................................................................... 5
2.1.2 Fundamental Analysis ....................................................................................................................... 6
2.1.3 Technical Analysis.............................................................................................................................. 6 2.1.3.1 Nomenclature and definitions ................................................................................................................ 10
2.1.4 Technical or Fundamental Analysis? ............................................................................................... 10
2.1.5 Trend-Following............................................................................................................................... 10
2.2 PORTFOLIO THEORY .................................................................................................................................. 12
2.2.1 Passive management ...................................................................................................................... 12
2.2.2 Active management ........................................................................................................................ 13
2.3 EVOLUTIONARY ALGORITHMS ..................................................................................................................... 13
2.3.1 Genetic Algorithms .......................................................................................................................... 13
2.3.2 Genetic Programming ..................................................................................................................... 14
2.4 EXISTING SOLUTIONS ................................................................................................................................. 16
2.4.1 Single asset optimization................................................................................................................. 16 2.4.1.1 Solution using Genetic Algorithms ......................................................................................................... 17 2.4.1.2 Solutions using Genetic Programming.................................................................................................... 18 2.4.1.3 Conclusions ............................................................................................................................................. 20
2.4.2 Portfolio Optimization ..................................................................................................................... 20 2.4.2.1 Solutions using the Classic Portfolio Theory ........................................................................................... 20 2.4.2.2 Solutions using TA and FA ...................................................................................................................... 22 2.4.2.3 Conclusions ............................................................................................................................................. 24
2.5 CHAPTER CONCLUSIONS ............................................................................................................................. 24
3 SOLUTION’S ARCHITECTURE .................................................................................................................. 27
3.1 OPTIMIZATION MODULE ............................................................................................................................ 28
3.1.1 Optimization Windows .................................................................................................................... 28
3.1.2 Optimization divided in generations ............................................................................................... 29
3.1.3 Elimination of similar solutions ....................................................................................................... 30
3.1.4 Only Trend Following Solutions ....................................................................................................... 31
3.1.5 Combining several time frames ....................................................................................................... 31
x
3.1.6 Genetic Programming ..................................................................................................................... 33 3.1.6.1 Representation ....................................................................................................................................... 33 3.1.6.2 Genes ...................................................................................................................................................... 35 3.1.6.3 Reproduction Operators ......................................................................................................................... 36
3.1.7 Data organized in vectors ................................................................................................................ 37
3.1.8 Interpretation of Genomes .............................................................................................................. 38
3.1.9 String representation of Genomes .................................................................................................. 39
3.1.10 Enforcing Trend-following rules .................................................................................................. 41
3.1.11 Evaluating the performance ....................................................................................................... 42 3.1.11.1 Sharpe Ratio and Sortino Ratio .............................................................................................................. 42 3.1.11.2 Risk Return Ratio .................................................................................................................................... 43 3.1.11.3 Modified Risk Return Ratio ..................................................................................................................... 44 3.1.11.4 Lake Ratio ............................................................................................................................................... 44
3.1.12 Fitness Function .......................................................................................................................... 45
3.2 PORTFOLIO MODULE ................................................................................................................................. 45
3.3 TECHNICAL INDICATORS AND RULES .............................................................................................................. 46
3.3.1 Moving Averages ............................................................................................................................. 46
3.3.2 Price Channels ................................................................................................................................. 47
3.3.3 MACD .............................................................................................................................................. 48
3.3.4 Relative Volatility Index ................................................................................................................... 50
3.3.5 Stop and Reverse ............................................................................................................................. 51
3.3.6 Composite indicators ....................................................................................................................... 52
3.4 CHAPTER CONCLUSIONS ............................................................................................................................. 53
4 IMPLEMENTATION ................................................................................................................................. 55
4.1 ASSETS ................................................................................................................................................... 55
4.2 CACHE .................................................................................................................................................... 56
4.3 OPTIMIZATION PARAMETERS ....................................................................................................................... 56
4.3.1 Window sizes ................................................................................................................................... 56
4.3.2 Size of the population of solutions .................................................................................................. 57
4.3.3 Method of chromosome creation .................................................................................................... 57
4.3.4 Stopping criteria .............................................................................................................................. 57
4.3.5 Penalizing bigger solutions .............................................................................................................. 58
4.4 INDIVIDUAL SIMULATIONS (OPTIMIZATION) .................................................................................................... 58
4.4.1 Fixed capital .................................................................................................................................... 58
4.4.2 Maximum Drawdown ...................................................................................................................... 59
4.4.3 Position sizing .................................................................................................................................. 59
4.4.4 Commissions and Slippage paid ...................................................................................................... 59
4.4.5 Position size recalculation ............................................................................................................... 60
4.5 PORTFOLIO SIMULATIONS .......................................................................................................................... 60
4.5.1 Time synchronization of asset data ................................................................................................. 60
4.5.2 Equal weight portfolio ..................................................................................................................... 60
4.6 CHAPTER CONCLUSIONS ............................................................................................................................. 61
5 RESULTS ................................................................................................................................................. 63
5.1 FORMAT OF THE RESULTS ........................................................................................................................... 63
5.2 SETUP WITH A LARGE SEARCH SPACE ............................................................................................................. 64
5.3 SETUP USING ONLY BOOLEAN GENES (3 LEVELS) ............................................................................................. 65
5.4 SETUPS WITH RESTRICTED BOOLEAN GENES.................................................................................................... 65
5.5 SETUP 10 – BEST GENES WITH 1 LEVEL DEPTH ............................................................................................... 67
xi
5.5.1 Example Genome from S&P500 ...................................................................................................... 71
5.6 EQUAL WEIGHT PORTFOLIO (REFERENCE) ....................................................................................................... 72
5.7 SETUP 10 PORTFOLIO ................................................................................................................................ 73
5.8 SETUP 10 PORTFOLIO - LONG AND MEDIUM TERM COMPONENTS ....................................................................... 74
5.9 CHAPTER CONCLUSIONS ............................................................................................................................. 76
6 CONCLUSIONS AND FUTURE WORK ....................................................................................................... 77
6.1 CONCLUSIONS .......................................................................................................................................... 77
6.2 CURRENT LIMITATIONS .............................................................................................................................. 78
6.3 FUTURE WORK ........................................................................................................................................ 78
REFERENCES ................................................................................................................................................... 81
APPENDIX A - USED ASSETS ............................................................................................................................ 85
APPENDIX B – GROUPS OF GENES .................................................................................................................. 87
APPENDIX C – TECHNICAL PRIMITIVE GENES .................................................................................................. 89
APPENDIX D – CACHE OF VECTORS ................................................................................................................. 91
APPENDIX E – PERFORMANCE METRICS ......................................................................................................... 93
xii
xiii
List of tables
Table 1 – Array to be optimized (element of the population). ................................................ 17
Table 2 – Element from the population representing a possible portfolio. ............................ 20
Table 3 – Overview of different approaches for single asset optimization. ............................ 21
Table 4 – Overview of different approaches on portfolio optimization. ................................. 25
Table 5 – Meaning of outputs from a chromosome that produces long and short signals .... 38
Table 6 – Combination of outputs in sub-systems with two chromosomes ........................... 38
Table 7 – Logical table of using Majority Vote on 3 single indicator rules .............................. 52
Table 8 – Results from optimization with all operators (3 levels) ........................................... 64
Table 9 – Results from optimization with Boolean operators (3 levels) .................................. 65
Table 10 – Average results for setups with depth 2 and Boolean genes. ................................ 66
Table 11 – Average results for setups with depth 1 and Boolean genes. ................................ 67
Table 12 – Results from optimization with setup 10 – best genes with tree depth 1 ............. 69
Table 13 – Results summary of an equal weight portfolio ...................................................... 72
Table 14 – Results summary of portfolio with setup 10 optimizations ................................... 74
Table 15 – Annualized percentage returns of portfolio with setup 10 optimizations ............. 74
Table 16 – Results summary comparing portfolios with different time frames ...................... 75
Table 17 – Annualized percentage returns from portfolios with different time frames ......... 75
Table 18 – List of stock indices used ........................................................................................ 85
Table 19 – List of commodities used ........................................................................................ 85
Table 20 – Gene groups and characteristics ............................................................................ 87
Table 21 – Description of LONG Technical Primitive genes ..................................................... 90
Table 22 – Performance metrics calculated for Long, Short, and Global trades ..................... 94
Table 23 – Performance metrics calculated exclusively for global trades ............................... 94
xiv
xv
List of figures:
Figure 1 – Triangle formation in EUR/USD, 4H time frame ....................................................... 8
Figure 2 – Example of a resistance that becomes support. ....................................................... 8
Figure 3 – Example of trading with Parabolic Stop and Reverse (PSAR).................................... 9
Figure 4 – Types of trends ........................................................................................................ 11
Figure 5 – Basic flowchart of a Genetic Algorithm ................................................................... 14
Figure 6 – Tree structure of a GP computer program .............................................................. 14
Figure 7 – Example of the crossover breeding operator ......................................................... 15
Figure 8 – Example of the mutation operator. ........................................................................ 15
Figure 9 – A tree genome and the equivalent portfolio. ......................................................... 22
Figure 10 – Overall architecture of the proposed solution ...................................................... 27
Figure 11 – Sliding Window Optimization ................................................................................ 28
Figure 12 – Global flowchart of the optimization of an asset .................................................. 29
Figure 13 – Flowchart of the Genetic Programming optimization of one window ................. 30
Figure 14 – Example of algorithm to eliminate similar genomes ............................................ 31
Figure 15 – S&P500 chart with the signals of a trend following trading system with two
subsystems ................................................................................................................... 32
Figure 16 – Example of a genome structure ............................................................................ 34
Figure 17 – Structure of a Chromosome .................................................................................. 34
Figure 18 – Gene details ........................................................................................................... 35
Figure 19 – Process of obtaining the trading signal vectors .................................................... 39
Figure 20 – Lake Ratio graphical representation ..................................................................... 44
Figure 21 – 100-day Exponential Moving Average applied to the close of the S&P500 index 47
Figure 22 – Price Channel applied on S&P500 ......................................................................... 48
Figure 23 – MACD applied to a chart of S&P500 index ............................................................ 48
Figure 24 – Relative Volatility Index in an S&P500 chart ......................................................... 50
Figure 25 – SAR Line applied to the S&P500 index .................................................................. 51
Figure 26 – Results for S&P 500 ............................................................................................... 68
Figure 27 – Results for Brent .................................................................................................... 70
Figure 28 – Results for Palladium ............................................................................................. 71
Figure 29 – Equal weight portfolio ........................................................................................... 72
xvi
Figure 30 – Returns of portfolio with optimizations obtained with setup 10. ........................ 73
Figure 31 – Portfolio with setup 10 optimizations – medium-term only ................................ 74
Figure 32 – Portfolio with setup 10 optimizations - long-term only ........................................ 75
Figure 33 – Generic structures of the cache ............................................................................ 92
xvii
List of Acronyms:
TA Technical Analysis
FA Fundamental Analysis
GA Genetic Algorithms
GP Genetic Programming
EA Evolutionary Algorithms
MT Medium Term
LT Long Term
DJI Dow Jones Industrial Average (American Stock Index)
S&P500 Standard & Poor’s 500 (American Stock Index with 500 stocks)
ETF Exchange Traded Funds
B&H Buy and Hold strategy
MPT Modern Portfolio Theory
ROI Return On Investment
Related to Technical Analysis:
S&R Support and Resistance
MA Moving Average
EMA Exponential Moving Average
SMA Simple Moving Average
WMA Weighted Moving Average
MACD Moving Average Convergence-Divergence
RVI Relative Volatility Index
RSI Relative Strength Index
O Open price of a particular bar (intraday, daily, weekly or monthly)
H Highest price of a particular bar
L Lowest price of a particular bar
C Close price of a particular bar
V Volume of a particular bar
Related to Fundamental Analysis:
PER Price to Earnings Ratio or P/E Ratio
EPS Earnings Per Share
PEG PER to Growth
ROE Return on Equity
DY Dividend Yield
PBV Price to Book Value
xviii
ROA Return on Assets
CCI Consumer Confidence Index
CPI Consumer Price Index
GDP Gross Domestic Product
1
1 Introduction Markets fulfill a basic necessity of mankind and exist in many parts of the world since
thousands of years. They allow the trade of goods and services between those who have them
and those who wish to acquire them. As time progressed, markets became more regulated,
sophisticated, liquid, global and more accessible. At present time, stock, commodity, financial
and currency markets have a crucial role in our economy and life, moving incredible amounts
of money each day. With the democratic access to markets, the general public can also
participate (mostly in stock markets) which means that crises in the markets can directly affect
millions of people.
In consequence of the markets being so central and important in our economies and societies,
they have been studied for many decades and many theories have been proposed. Some
argue markets are efficient [1] meaning that all relevant information is already incorporated
in the prices and it’s impossible to predict its future movements. This supposedly leads to a
market behavior known as Random Walk [2]. Others refute this view and defend the market
is not efficient and it’s possible to predict the future behavior of the markets to some extent
[3]. Increasing evidence point to markets not being efficient which means that methods can
be developed to use relevant information to increase profits and/or reduce risks [3], [4].
When market participants try to predict the future behavior of an asset, two very different
approaches have been used. One, called Fundamental Analysis [5] looks at the quality of the
asset or its intrinsic value and determines if it’s undervalued or overvalued. And the other
major market analysis method used is called Technical Analysis [5], [6], which studies past
prices on the premise that all relevant information is reflected there and that it’s possible to
predict it’s the future behavior by analyzing the patterns formed by price.
In the field of technical analysis, several techniques were developed in order to extract the
useful information contained in past prices. Some try to identify graphical formations that
correspond to a higher probability of a particular outcome. They are called Chartists. Others
use technical indicators calculated with past prices, which filter relevant information that can
then be used to decide what asset to trade and when.
Each market has its own particular characteristics and certain trading techniques used in one
market may not work well in other markets. Also, each market may exhibit different behavior
in different phases, consequence of underlying factors, trader psychology of simply as a result
of system dynamics. In summary, each market is different and markets change over time. This
leads to the necessity of adapting the trading approach or the trading rules as the market
changes, to maximize profits and minimize risk. In the past decades, much attention has been
devoted by academics to the problem of optimizing trading rules. Several optimization
algorithms have been experimenting, like Neural Networks, Fuzzy Logic, Monte Carlo, Particle
Swarm, Hidden Markov Models, Support Vector Machine and Evolutionary Algorithms.
2
Evolutionary Algorithms are particularly suited to optimize trading rules because of their
power and flexibility and is the optimization method used in this thesis. Trading rules
exclusively based on technical indicators will be optimized with Genetic Programing, a form of
Evolutionary Algorithm. Several chosen assets (stock indices and commodities) will be
independently optimized and portfolios will be simulated using those optimized trading rules.
In order to increase the robustness of the solutions, the optimization will be restricted in order
to produce solutions which follow market trends, since that is one of the most structural and
frequent market patterns. Since markets can exhibit different behaviors in different time-
scales, rules tuned to different time frames will be combined in order to increase the
robustness of the solutions.
1.1 Works Purpose
The main goal of this work is to develop a method of creating trading rules adapted to a
particular asset. This rules change over time as the asset conditions change in order to have
the most effective rules in each moment. The rules will use technical analysis and be based on
several technical indicators. Restrictions will be enforced on the type of rules that can be
generated in order to enforce trend-following rules. Two time frames will be used meaning
that two sub-sets of rules will be combined, in order to make the global solution more robust.
A secondary goal is to combine the individual and independent optimizations of the assets
and simulate a portfolio using the best generated rules.
1.2 Related Publications
During the development of this work, a paper was accepted in the 2015 Genetic and
Evolutionary Computation Conference – GECCO. The conference took place between the 11th
and 15th of July 2015 in Madrid, Spain [7].
1.3 Document Structure
Chapter 2 addresses the relevant theory to this works, namely Market Analysis,
Portfolio theory and Evolutionary Algorithms. It also reviews existing solutions relevant
to this thesis.
Chapter 3 presents the theory and general architecture of the proposed solution. Some
important technical indicators are explained and some trading rules are discussed.
Chapter 4 focus on more specific and practical aspects of the implementation of the
proposed solution.
Chapter 5 presents the tests and simulations performed and analyses the results
obtained.
3
Chapter 6 summarizes the results obtained and presents some conclusions that can be
drawn from the results. Limitations of this solution and future work are also addressed.
4
5
2 Related Work This section presents several fundamental concepts necessary to better understand the work
that was developed. Some theories regarding the behavior of financial markets are addressed
like the random walk theory and the efficient market hypothesis. Technical Analysis (TA) and
Fundamental Analysis (FA) are briefly described and compared since they are important tools
in the analysis of financial assets and portfolio optimization. Evolutionary Algorithms (EA) and
in particular Genetic Algorithms (GA) and Genetic Programming (GP) are presented as good
choices for the optimization of trading rules and portfolios. Several relevant papers are
reviewed and help to contextualize the work developed here.
2.1 Market Analysis
When an investor, speculator, fund manager or some other market participant has to make a
decision regarding which asset to trade or include in his portfolio, or has to decide the best
timing to do it, in most cases he will use a method he believes will give him consistent results.
There are several tools to analyze the markets but they usually fall into two major categories:
Fundamental Analysis and Technical Analysis. The later will be addressed in more detail since
it’s the one that will be used in this work.
2.1.1 Efficient Markets and the Random Walk Theory
An efficient market is defined as having a large number of rational participants, competing
between themselves, trying to predict future market values and trying to optimize their
profits. Relevant information must also be easily available. In an efficient market, this
competition between the market participants originates a situation where, at any given time,
the current value of an asset reflects all the relevant information available and the future
expectations for that asset. In practice this means that at any given point in time, the actual
asset price is a good estimate of its intrinsic value [1].
In an efficient market, price variations are independent from each other and past data can’t
be used to predict future data - asset prices assume a random walk behavior [2]. This has
serious implications for technical analysis since its premise is precisely that past data can be
used to predict the probable future behavior of the market [1].
In an efficient market, fundamental analysis can still be useful if the investor can interpret the
relevant information faster than the other market participants and take advantage of the
difference between the current asset price and the intrinsic value he calculates. After the
market integrates the new information, it’s allegedly impossible to beat the buy-and-hold
strategy.
6
The random walk theory in financial markets was first studied by French stockbroker Jules
Regnaul in 1863 [8] and was popularized in 1973 by Burton Malkiel [2]. Many researchers
defend markets have a high degree of efficiency and prices follow a random walk behavior,
like in [1] and [2].
The random walk theory is criticized by several researchers like in [3] where the study of
weekly stock returns can’t be explained by this theory. Most market returns don’t follow a
normal distribution having fat tails [9] which is evidence that, at least in certain periods,
markets have a non-random behavior.
2.1.2 Fundamental Analysis
Fundamental Analysis (FA) studies all the relevant facts affecting the price of an asset in order
to determine its intrinsic value. The intrinsic value of an asset is what the fundamentals say it
is worth based on the laws of supply and demand [5]. If the asset’s current price is lower than
its intrinsic value, the asset is said to be undervalued and its price is expected to increase in
the future. If the current price is greater than its intrinsic value, it’s said to be overvalued and
its price is expected to decrease in the future.
The relevant facts considered to determine the intrinsic value of the asset depends on the
type of asset. For stocks, financial statements (business’s assets, liabilities and earnings),
management, business concept, markets and competition are analyzed. Several ratios can be
used to objectively evaluate the situation of the company. These ratios allow for comparison
between stocks and also with historical levels. Some examples are: Price to Earnings Ratio
(PER), Earnings per Share (EPS), PER to Growth (PEG), Dividend Yield (DY), Return on Equity
(ROE) and Price to Book Value (PBV).
If the economy as a whole is being analyzed - important for stocks, stock indices and currencies
– economic indicators issued by central banks and other institutions are studied. Some of the
most important are the Unemployment Rate, Consumer Price Index (CPI), Consumer
Confidence Index (CCI), Gross Domestic Product (GDP), Interest Rates, Public Debt, and New
Home Sales.
For the readers interested in more information on Fundamental Analysis, the classic book
from Graham and Zweig on value investing may be a good option [10]. And the book from
Bernard Baumohl [11] has many information on economic indicators, useful to understand
the economy as a whole.
2.1.3 Technical Analysis
Technical analysis (TA) studies past market data in order to predict the most probable
behavior of the market in the future. The raw data used for the analysis is the asset monthly,
7
weekly, daily or intraday prices: Open (O), High (H), Low (L) and Close (C), the Volume (V) and
Open Interest (OI) in the case of futures. In special cases, other data may be used, like
commitment of traders (futures) and prices from other assets that influence the one being
studied [5], [6].
TA assumes all relevant information for an asset is already reflected in its historical prices and
it’s the best source of useful information to predict its behavior. In the case of stocks, the
company situation and future expectations are considered to be already reflected in the stock
past prices. TA studies certain repeating patterns formed by price, consequence of
human/trader behavior, system dynamics and fundamental underlying factors.
TA was used for many decades by traders, allegedly with much success in certain cases but
wasn’t taken seriously in the academic world and by many professional fund managers.
Gradually, empirical evidence and academic work showed some forms of TA are helpful in
predicting future price action and in the past two decades it has received much attention from
researchers [4].
TA can be used in many different ways and has several sub-categories. For example, Chartists
are technical analysts that look for graphical formations in price that indicate a higher
probability of some price behavior in the future, like continuation or inversion of a trend. Some
of the most popular formations include double or triple tops and bottoms, head-and-
shoulders, triangles and cup-and-handle. Detailed information on chart formations can be
consulted in the book by Bulkowski [12]. Figure 1 shows a triangle formation which is
composed of an upper and lower line that converge as time advances. Price bounces between
those two lines until it consistently crosses one of them. When this happens, price is expected
to continue in that direction and a price projection can be calculated using the initial height of
the triangle, applied to the breaking point. Using a computer to automatically identify this
graphical formations can be a challenge since rigid rules may not be the best way to deal with
price formations that exhibit noise and ambiguities. An experienced technical trader or a
computer neural network can probably better manage these ambiguities.
Some technical analysts pay close attention to price levels where the asset has recently
reversed direction and where many investors have entered or exited the market. These levels
may have psychological and emotional relevance to the market participants and influence
future trading decisions. Since stop loss and profit taking orders may also be entered near
those levels, the market may react strongly near them and reverse direction in the short-term.
These levels are sometimes called “support” if they are below the current price and
“resistance” if they are above the current price. Figure 2 shows a chart of currency pair
GBP/USD in the 4H time frame, with three horizontal lines representing price levels where
price clearly reacted. These horizontal levels can form in all types of assets and time frames
but are more common in Forex, especially in intraday charts. With the study of relevant past
levels and the way price reacts to them, it’s sometimes possible to gain important information
regarding the probable future behavior of the asset.
8
Figure 1 – Triangle formation in EUR/USD, 4H time frame
Figure 2 – Example of a resistance that becomes support.
Technical indicators calculated from raw prices is another very popular tool used by TA. Those
indicators can extract useful information from the historical data and highlight it. The list of
technical indicators is very long and include moving averages, indicators that show if there is
a trend in progress and how strong it is (like the Directional Movement Index [6]) and
indicators that show if the asset is overbought or oversold in the short-term (like the Relative
Strength Index [6] and Stochastic Oscillator [6]). Kaufman compiled many of the most
important and most used indicators in [6]. Figure 3 shows an example of a chart with a TA
9
indicator superimposed (Parabolic Stop-and-Reverse [6]) which was used to generate trading
signals.
Several technical indicators used in this work will be discussed in 3.3 as well as algorithms that
generate trading signals based on those indicators.
Figure 3 – Example of trading with Parabolic Stop and Reverse (PSAR)
The two most commonly repeating market patterns, which can be identified and exploited,
are:
Trends: Price tends to move preferably in one direction for an extended period of time
(autocorrelation of price). The strategy to profit from this behavior can be called “trend
following” and consists in opening positions in the same direction of the trend for as
long as it exists.
Mean Reversion: The price tends to overshoot a mean line that can be considered as
the consensus of a fair price. The resulting behavior is an oscillating price around the
mean line. The strategy to profit from this behavior can be called “swing trading” and
consists in catching these shorter-term oscillations of the market, buying at the bottom
and shorting at the top.
These patterns tend to alternate between themselves of coexist, alongside random noise
created by the chaotic trader activity. These two patterns form repeatedly in most markets
and in most time frames. In this work great care will be taken so that the developed trading
solutions have a trend following philosophy.
10
2.1.3.1 Nomenclature and definitions
It’s important to clarify the meaning of certain words or expressions that will be used in this
text. When dealing with stocks trading, there usually are only 2 market operations: Buy and
Sell and most the investors/traders can only earn money when the markets go up. When
dealing with commodities, stock indices, financial assets or currencies, it’s possible to earn
money when the market goes up but also when it goes down and there’s no negative stigma
associated with earning money in falling markets since contrary to stock markets,
commodities, currencies and financial assets have a cyclic nature.
To profit from a rising market, the trader must “buy” to open the trade and “sell” to close it –
that is called a “long” trade. To profit from a falling market a trader must “short” to open the
trade and “cover” to close it – that is called a “short” trade. In this work the terms “buy” and
“short” will be used to refer to the action of opening a trade and the terms “long” and “short”
will be used to refer to types of trades. The term “short” will therefore be used in two similar
contexts.
2.1.4 Technical or Fundamental Analysis?
TA and FA operate on different premises and historically there has been mutual rejection
between their proponents and users. FA dominated for many decades, especially in academic
circles and TA was considered an esoteric art with no solid bases. As empirical evidence
gradually showed TA had some predictive capabilities, its popularity grew. Academics began
to study the subject and provided possible explanations on why the random walk theory didn’t
accurately explain the markets.
TA and FA try to achieve the same using different methods: identify good trading or
investment opportunities to maximize profits. The question of which is better is probably the
wrong question since research points to both techniques being at least partially
complementary [13], [14]. This work will deal exclusively with technical analysis.
2.1.5 Trend-Following
A trend in a market is a price pattern that consists in price going consistently up or down for
some time. This movement is normally done in successive swings and not in a straight line.
Mathematically a trend corresponds to high auto-correlation of price. The slope, duration and
regularity of a trend can vary greatly. A trend can be upwards and can be referred as an “up-
trend” or “long trend” and it’s said the market is “bull” as a reference to a bull which attacks
with his horns in an upward movement. In an up-trend prices usually form higher highs and
higher lows as prices rise in a zigzag fashion, Figure 4. The trend can also be downwards and
can be referred as a “down-trend” or “short trend” and the market is said to be “Bear”. In a
down-trend prices usually form lower highs and lower lows as prices fall in a zigzag fashion,
11
Figure 4. Finally the market can have no defined trend and can be referred as a sideways,
lateral or oscillatory market, or can be said to be consolidating. Figure 4 exemplifies the 3
situations.
Figure 4 – Types of trends
Trend following is a trading approach used by technical analysts who take advantage of
market trends and in simple terms consist in:
1) Identifying a trend (upwards or downwards) in an asset;
2) Opening market positions in the direction of the trend to profit from the movement;
3) Closing or inverting market positions when the trend finishes or reverses.
Trends exist in almost all markets but some are historically more prone to be trending than
others, which translates in better returns from trend following strategies.
Brock et al. studied two of the most popular trend following strategies: moving average (MA)
crossover and trading range breakout (TRB) [15]. The authors applied these strategies to the
Dow Jones Index and the positive results obtained contradicted the once prevalent theories
about the markets (1992), namely the Random Walk Theory.
Ülkü and Prodan studied another very popular technical indicator used in trend following
strategies: the “Moving Average Convergence Divergence” (MACD) along with MA crossover
[16]. The trading rules were applied to stock indices and results confirmed these strategies are
viable, specially the MA crossover. The authors also found volatility to be beneficial to these
trend following strategies.
12
Trends are present in all time frames, from intraday to monthly but, the higher the time frame,
the more reliable are the trends. Intraday prices usually have too much noise and trend
following strategies can be a bad choice for trading in those shorter time frames. Lohpetc and
Corne used GP to develop trading rules for S&P500 and compared several time frames: Daily,
Weekly and Monthly [17]. Their results show that solutions obtained with monthly data easily
outperform the Buy-and-Hold strategy, and as the time frame decreases to weekly and daily,
the results become weaker. Using daily data, the results were still positive but equivalent to
the B&H. This confirmed previous works using different time frames and is a strong indication
that at least in some assets the longer time frame is important and should be taken into
account when generating trading rules. These results coincide with the empirical knowledge
of experienced traders who do not trade against the longer-term trend.
2.2 Portfolio Theory
A financial portfolio [18] is a group of financial assets help by an individual investor, hedge
fund, bank or other financial institution. The assets held can be currencies, stocks, bonds,
commodities, Exchange Traded Funds (ETF) or other. The owner of a portfolio expects to profit
from the assets it contains and, by diversification of the investments, expects the risk to be
lower when compared to holding single assets.
Portfolios are usually constructed with a desirable risk level as reference and portfolio assets
are chosen in order to maximize the expected returns maintaining the pre-chosen risk level.
Greater returns are usually obtained at the expense of greater risk and that tradeoff must be
defined according to the risk profile of the portfolio owner.
2.2.1 Passive management
In passive management [18] it’s assumed the market expectations are essentially correct and
that there is no advantage in reacting to market or asset price changes, therefore the name
passive. This philosophy is linked to the random walk theory and assumes a high market
efficiency. Some passive fund managers just mimic the stock index of reference and achieve
similar results in terms of returns and risk.
Passive management is also strongly linked to the “Modern Portfolio Theory” (MPT) whose
major contributor was Harry Markowitz. In his 1952’s work [19] he proposed the “Mean-
Variance Model” and mathematically demonstrated that the risk of a portfolio depends on
the covariance of returns from its components and not on the average risk of each component.
This model gave managers a mathematical tool to precisely construct portfolios which in
theory have the desired balance between risk and return (highest return for a desired risk or
lowest risk for a desired return).
13
2.2.2 Active management
In active management [18] it’s assumed the market is not efficient and the portfolio
components and/or its weights are constantly updated in order to reflect the changes in the
markets. The objective it to have, at all times, a portfolio whose components have the best
potential. The methods used to evaluate the assets and construct the portfolio vary and can
be based on FA, TA or a combination of the two. The process of managing the portfolio or fund
may be totally discretionary, based on the analysis made by a person or group of persons, or
can be partially automated with information coming from TA and FA algorithms or can be
totally automated, where complex algorithms select assets, quantities and the best timings to
update the portfolio.
2.3 Evolutionary Algorithms
Finding good trading rules or optimal portfolios is an optimization problem that can be solved
with several different optimizing algorithms. The ones inspired in nature and selection of the
fittest are of particular interest in this field for their characteristics.
Evolutionary algorithms (EA) are inspired in nature, they follow Darwinian principles and have
gained much popularity in recent decades [20]. They use a population of elements, each one
representing a solution to the problem being optimized. All the elements in the population
compete between themselves and are evaluated with a fitness function. The best scored
elements usually have a greater chance of reproducing with other elements or may move
intact to the next generation (elitism). Reproduction is made using one or more operators,
usually crossover and mutation.
Evolutionary Algorithms perform global optimization with a meta-heuristic character and are
trial and error problem solvers. They are random processes and results are never guaranteed,
but they deal better than other search algorithms with the problem of getting stuck in local
maxima because they use a population with a diversity of solutions. For these reasons EA are
robust, flexible and practical, being successfully used in many optimization problems, namely
in finance [21].
2.3.1 Genetic Algorithms
Genetic Algorithms [22] are a type of evolutionary algorithm that uses a population of possible
solutions. Those solutions are usually represented as an array of bits, integers, doubles or
characters (other less common representations include tree structures). A fitness function
must be defined to reward the most desired characteristics of the solutions and allow them
to be ranked from most fit to most unfit. Crossover and mutation are the most used
reproduction operators that randomly mix the best solutions into new ones. The process is
iterative and stops when a stopping criteria is met. Some possible stopping criteria include the
14
best solution being good enough or a certain number of generations being completed. In
Figure 5 the flowchart of a generic Genetic Algorithm is represented.
Figure 5 – Basic flowchart of a Genetic Algorithm
2.3.2 Genetic Programming
Genetic Programming [23], [24] can be considered a specialization of GA where each individual
of the population is a computer program. These programs perform a user-defined task and
their efficacy is evaluated by a fitness function. The programs are usually represented in a tree
structure with nodes corresponding to operators and data as shown in Figure 6. In this
example, the represented function is f(x, y) = y ∗ 2 +x
3.
Figure 6 – Tree structure of a GP computer program
GP allows more complex solutions when compared with GA. This happens because the tree
representation is much more flexible and also because it can use powerful operators like if-
then-else. This makes GP much more powerful and flexible than GA and capable of finding
novel and sometimes unexpected solutions to problems.
Its flexibility and power can also have negative consequences because it may increase the
search space and curve-fitting, which is a real danger that must be addressed.
15
Programs from one generation breed and their offspring will integrate the next generation.
The most used breeding operators are Crossover and Mutation. In crossover, a child is created
using random parts of the tree programs of two or more parents, Figure 7. In mutation a child
is created by randomly changing parts of a parent’s program tree, Figure 8.
Figure 7 – Example of the crossover breeding operator
Figure 8 – Example of the mutation operator.
16
2.4 Existing Solutions
The academic work in this area can be divided in two groups:
1) Works that focus on developing trading rules for a specific asset (ex: S&P500) or groups
of assets (ex: stocks from S&P500). The optimized rules generate signals to enter and
exit the market in precise moments. Results are measured in an individual basis and
usually no attempt is made to construct an optimized portfolio of assets.
2) Works whose main goal is to construct an optimized portfolio.
Greater focus will be given to works of the first group since the main goal of this work is the
development of trading rules.
Evolutionary algorithms have been successfully used to create rules for automatic trading of
financial assets, in many cases able to beat the Buy-and-Hold strategy, which contradicts the
efficient market hypothesis. The assets chosen are mainly stocks, stock indices and forex
currencies. Usually, daily historic data from the assets is used but other time frames have been
tested, including intraday, weekly and monthly.
A sliding window methodology is frequent, meaning a portion of historical data is used for
training (training window) and a posterior portion of data is used for testing (testing window).
This procedure is repeated several times, sliding both of these windows in time.
2.4.1 Single asset optimization
In this type of work no attempt is made to construct a portfolio and the optimization is
performed on a single asset of group of assets. Even when several assets are optimized, the
optimizations are individual and independent. The purpose of an optimized solution is to
obtain the best results according to a fitness function, in the asset being used. The authors
may focus on several different points like:
Testing specific TA or FA rules to generate the trading signals, with different
configurations and parameters;
The possible advantages of different representations of the solutions (ex: tree or
array);
Testing different fitness functions, simple or complex, with several components
(including for example: return, risk, complexity of solutions). Multi-objective
optimization are also studied;
Comparing the results of optimizations with different time frames (Ex: daily, weekly
and monthly) and combining them as well;
Testing different global methodologies for the process of obtaining the solutions.
17
2.4.1.1 Solution using Genetic Algorithms
One possible methodology to optimize trading solutions is to use GA with a pool of pre-
selected rules (with fixed parameters) that can be chosen. The number of rules can be
relatively small or very large, including variations of a basic rule, for example: a MA crossover
rule with different parameters for the MA. In this case the variable to be optimized is an array
of values that will select some of these fixed rules or that will give weights to these rules. The
final decision can be made by majority vote of the signals from the selected rules, by weighted
averaging of the signals of the rules or some other similar solution.
This optimization method doesn’t allow the creation of new elementary rules which can be a
serious limitation but, on the other hand limits the search space, curve-fitting and the
solutions found may be more robust. Other more flexible methods usually have to employ
procedures to limit the search space, or the complexity of the solutions.
Korczak and Roger used GA with fixed rules to develop trading solutions for 24 French stocks
[25]. More than 200 different fixed rules were available for selection and include: price
channel breakouts, close/MA crossing, ROC greater/smaller than a constant and rules base on
the ease of movement indicator. The optimization selects the combination of rules that
produce the best solution and the final decision is given by averaging the outputs of all the
selected rules.
Each element in the population is represented by a binary array of size N, equal to the number
of rules available. Each position in the array corresponds to one rule, which is either selected
(1) or not selected (0), Table 1.
Index 0 1 2 3 4 5 … N-2 N-1 N
Value 1 1 0 0 1 0 … 1 0 0
Table 1 – Array to be optimized (element of the population).
Mendes et al. proposed a different approach, using a pre-selected set of trading rules whose
parameters are optimizable [26]. GA were used to create trading solutions for two forex pairs,
EUR/USD and GBP/USD. Each element in the population is a trading system and has a total of
10 rules, 5 for opening a trade and 5 for closing. In each of these groups of 5 rules, majority
vote is used in the first 4 to validate a signal and the fifth rules gives the price for
opening/closing the trade. Several intraday time frames were tested and positive results were
obtained in the majority of the simulations.
Fundamental Analysis can also be used to create rules that generate trading signals in the
same formats discussed so far. And it’s possible to mix TA and FA rules in the same
optimization, allowing the best combinations to be selected. This was experimented by
Contreras et al. in [14]. They used GA on 100 stocks from S&P500 with daily data from 1994
18
to 2004. Each element of the population has 23 genes/variables that can be optimized and
which are the parameters used in 8 predefined rules. 4 rules are based on TA and 4 rules are
based on FA. The TA rules use the indicators MA, MACD and RSI and also support and
resistance levels. The FA rules are based on the following ratios and indicators: PER, PBV, ROA
and Sales Growth. Each of the 8 rules has an optimizable weight associated which defines its
contribution to the final decision. The signals generated by each of the 8 rules are summed
and a final signal is validated if the sum goes above/below a threshold that can be adjusted to
obtain a more aggressive/conservative trading style.
Training was performed from January 1994 to December 2003 and the year 2004 was used for
testing. In this period the cumulative return was 830% which compares with the return of
180% obtained by the Buy-and-Hold strategy in the same period.
2.4.1.2 Solutions using Genetic Programming
Genetic Programming usually represents the solutions in a tree structure and can use powerful
operators, making it much more flexible than GA. It can find solutions that are not expected
and aren’t possible if a more rigid optimization method is used.
The operators used in the tree can be of many different types, namely, arithmetic, logic,
comparison, mathematical, based on TA and on FA. If basic arithmetic operators are used as
well as raw data (Open, High, Low, Close, Volume), the search space can become very large,
curve fitting will probably occur and solutions won’t be robust. Usually more sophisticated
operators based on TA and FA are used, extracting useful information from raw prices.
Working with processed information instead of raw historical prices reduces the search space
to areas with greater probability of containing robust solutions.
In GP, curve fitting is a real danger and must be addressed. Becker and Seshadri discuss
methods of decreasing curve fitting and methods for improving the comprehensibility of the
solution in [27]. They present three main approaches for decreasing curve fitting:
Penalizing complexity or biasing toward simplicity;
Limiting the number of models considered;
Using a validation data set.
Their approach was to include a factor that penalizes complexity in the fitness function. They
compare the results of this method with an equivalent one without this factor and obtain
solutions with less complexity and more generalization power. The simpler solutions obtained
better results in the out-of-sample data and beat the buy-and-hold strategy.
To increase the comprehensibility of the solutions in tree format, they suggest using known
and proved technical indicators calculated from the historical data, instead of more general
operators. And they also suggest decreasing the complexity of the solution by the method
described above.
19
Following on their previous work [27], Becker and Seshadri used GP with two chromosomes,
one to generate buy signals and another for sell signals [28]. They opted to reproduce the two
chromosomes independently, meaning that a buy and sell chromosome will never exchange
genes and evolve separately. But since a trading solution must have synchronized buy and sell
signals to be coherent, the evolution of the two chromosomes is linked and interdependent -
the authors named this process as cooperative coevolution. There are several options to
manage the two types of chromosomes and some were tested in this work:
1) Each buy chromosome is paired with a sell chromosome in a solution and they evolve
together;
2) Each chromosome of one type is paired with the best 5 chromosomes of the other
type, from the previous generation.
3) Each chromosome of one type is paired with 5 randomly chosen chromosomes of the
other type.
The results showed option 1) with paired chromosomes performed best and the authors argue
it may mean that compatibility between the chromosomes is more important than diversity.
Tests were performed in S&P500 from 1954 to 2002.
Allen and Karjalainen used GP with tree representation to generate trading rules for American
Stock Index S&P500, tested from 1936 to 1995 [29]. The operators used were arithmetic,
logical, SMA, maximum price in an interval, minimum price in an interval, >, <, If-Then-Else
and Lag (offsets a vector received from a child node). The authors tried to limit curve fitting
by training the population in a 2 year window and then validating the result in another 2 year
window. When the best element has a better fitness score in the validation window, it is
saved, and otherwise it’s discarded. Optimization stops after a predefined number of
generations with no improvement in the validation window. S&P500 historical prices are
normalized, dividing them by the 250 day MA. The results of this configuration did not beat
the buy-and-hold strategy.
Lohpetch and Corne experimented with multi-objective GP with chromosome tree
representation in order to trade S&P500 stocks [30]. Several fitness functions were used and
they include in their formulation: market return, stability of returns and complexity of the
solution. Comparisons are made between the several configurations, some using only one
fitness function (single objective) and others with several fitness functions (multi-objective).
In multi-objective, two methodologies are also compared:
Majority vote: the solution obtained with each fitness function votes with a buy or a
sell and the final decision is given by the majority vote of all simulations with all the
objectives;
Normal: the capital is equally allocated to all the simulations.
Results showed that multi-objective strategies outperform the single-objective strategies.
Simulations with monthly data produce the best results outperforming the buy-and-hold,
20
weekly data gives worse results but still outperforms B&H. Using daily data the results are
equivalent to B&H.
2.4.1.3 Conclusions
The reviewed works on single asset optimization are summarized in Table 3. It’s difficult to
make direct comparisons between the methods used since they tend to use different assets,
different date intervals and sometimes different data time-scales. Nonetheless it’s possible to
make several observations useful to the present work and some will be discussed in 2.5.
2.4.2 Portfolio Optimization
The Portfolio optimization problem is a particular case of the resource allocation problem,
which consists in distributing a limited amount of a resource by a number of jobs, in order to
satisfy one or several utility functions [31]. In this case, the resource is the capital, the jobs are
the assets in which the capital can be invested on, and the utility functions can be the portfolio
return to be maximized and risk to be minimized.
2.4.2.1 Solutions using the Classic Portfolio Theory
Markowitz modeled the portfolio optimization problem in the 50’s using the Mean-Variance
Model [19] which can be solved by Quadratic programming. However, in the real world, many
constraints are added and the search problem becomes too large, non-convex and unsolvable.
Some of this constraints are the large number of assets, restrictions in the weights of each
asset and trading costs.
Evolutionary algorithms are a good choice to solve the Portfolio Optimization problem since
they are flexible and can efficiently search a large solution space. An example can be found in
[32] where Lin and Gen use GA and follow a Mean-Variance approach, inspired in the work of
Markowitz. Each element in the population is represented by an array of weights, one for each
candidate stock to integrate the portfolio, Table 2.
Stock index 0 1 2 3 4 5 … N
Weight 0.02 0.04 0.00 0.11 0.02 0.09 … 0.04
Table 2 – Element from the population representing a possible portfolio.
21
Ref Paper
date
Optim.
Method
Operators /
Genes Representation
Fitness
function
Time
Frame
Assets
used
Test
Period Results
[25] 2002 GA More than 200
predefined rules Array Several tested Daily
24
French
Stocks
1997 -
1999 Beats B&H
[26] 2012 GA Predefined genes with
optimized parameters Array Stirling Ratio Intraday FOREX
2005 -
2010 Beats B&H
[14] 2012 GA Several based on TA
and FA Array Cumulative returns Daily
S&P500
Stocks 2004 Beats B&H
[29] 1999 GP ARITM, Logical, SMA,
MAX/MIN, IF, >, <, Lag Tree
Excess return over
B&H Daily S&P500
1936 -
1995 Doesn’t beat B&H
[27] 2003 GP ROC, MA, Logical, >, <,
O,H,L,C, price levels Tree N/A Monthly S&P500
1990 -
2002 Beats B&H
[28] 2003 GP
AND, OR, NOT, >, <,
Monthly O, H, L and C,
MA, ROC, S&R.
Tree.
2
chromosomes
Not mentioned -
includes complexity
penalization
Monthly S&P500 1954 -
2002
Some configurations
beat B&H; Paired
Chrom. Perform better
[17] 2010 GP
AND, OR, >, <, MA,
ROC, “Price Resistance
Ind., “Trend Line Ind.”
Tree Multiple
components.
Daily,
Weekly,
Monthly
S&P500
Sets
from
1960 to
2008
Monthly and Weekly
beat B&H.
[30] 2011 GP Multi-
Objective
MA, ROC, MA
Min/Max, Logical, >, <,
O, H, L, C, V, Trend Line
Indicator
Tree Multiple
Daily,
Weekly,
Monthly
S&P500
stocks N/A
Multi-objective
superior. Beats B&H
Table 3 – Overview of different approaches for single asset optimization.
22
The fitness function is calculated dividing the total expected return by the risk of the portfolio.
No attempt is made to predict the future behavior of the assets by using TA or FA and the
main difference from the Markowitz approach is the use of GA to perform the optimization.
Aranha and Hitoshi proposed a different representation of the portfolio in [33], using a tree
structure as seen in Figure 9. The weight in a node refers to the left side of its sub tree and the
complement of that value is the weight for the right sub tree. The final weight of each asset is
given by the sum of the weights of all occurrences of that asset in the tree. For example, the
weight of a2 represented in Figure 9 is equal to [0.5 * 0.2 * (1 - 0.3)] + [(1 - 0.5) * (1 - 0.7)] =
0.07 + 0.15 = 0.22. The authors argue this representation allows the preservation of building
blocks which accelerates the optimization process. The results showed the portfolios
generated with this methodology have fewer assets compared with the ones generated with
the array of weights representation. Having fewer assets may have some benefits like reduced
transaction costs and better understandability of the portfolios.
Figure 9 – A tree genome and the equivalent portfolio.
2.4.2.2 Solutions using TA and FA
Assuming TA and FA can be used to predict to some extent the future behavior of financial
assets, they can be used to select assets to integrate a portfolio. The most used approach is
to rank the assets with some function or algorithm and include the most scored assets in the
portfolio. The process must be repeated at regular intervals to update the portfolio, removing
assets that are no longer promising and replace them with assets with a better score.
Gorgulho et al. used GA to optimize a portfolio composed of stocks from the Dow Jones
Industrial Average (DJI) and S&P500. They represented the solutions as arrays of real numbers
which correspond to the weights given to 6 TA rules used in the scoring of stocks. The rules
are pre-defined and based on:
23
1) MACD Histogram crossing the 0 level;
2) Hull MA slope change;
3) EMA/Close crossing;
4) ROC crossing the 0 level;
5) RSI crossing overbought/oversold levels;
6) MA crossover.
The rules output is averaged according to the weights and used to score the stocks, as shown
in equations ( 1 ), ( 2 ) and ( 3 ).
𝑆𝑐𝑜𝑟𝑒(𝑋) = ∑ 𝑤𝑖 ∙ 𝑆𝑐𝑜𝑟𝑒(𝑋, 𝑖)
𝑁
𝑖=0 ( 1 )
0 ≤ 𝑤𝑖 ≤ 1 ( 2 )
0 ≤ ∑ 𝑤𝑖 ≤ 1
𝑁
𝑖=0 ( 3 )
Where:
𝑆𝑐𝑜𝑟𝑒(𝑋) is the score of stock X.
𝑤𝑖 is the weight assigned to the technical rule i.
𝑆𝑐𝑜𝑟𝑒(𝑋, 𝑖) is the score given by the technical rule i to stock X.
All the stocks with a score higher than a pre-defined level are included in the portfolio whose
performance is evaluated using the ROI. Three simulations were performed: the first using the
optimization described above, the second using a Buy-and-Hold approach and the last using
random portfolios. The optimization with GA obtained the best results: ROI = 3.49% against
0.49% for the B&H and 0.90% for the Random approach.
Kaucic proposed a different methodology using GA with solutions represented by arrays of
variable length [34]. TA rules were used and each rule has a predefined identification index.
Each chromosome/solution can have a variable number of rules, up to a pre-defined
maximum. Each chromosome is considered valid and used if it has at least two rules and they
are all different. The final decision produced by each element is based on the mean average
of the signals generated by all its TA rules. This form of representation is more flexible than
fixed chromosomes and allows more complex solutions.
Silva et al. proposed the use of GA with rules based on both TA and FA to optimize trading in
S&P500 stocks [35]. For a stock to be included in the portfolio, several conditions must be
verified: a TA rule must give a positive signal (crossover between price and a SMA) and be well
ranked among all the stocks. The ranking is done using a weighted average of several FA ratios.
The chromosomes include the weights for the FA ratios, the period of the SMA and several
24
other parameters like a minimum score to enter the portfolio, a Take Profit level, a Stop Loss
Level and the capital to allocate to each stock. The portfolios were evaluated using ROI and
the Variance. Several variations were tested in 2012-2013 and the results obtained beat the
index of reference (S&P 500) in the same period.
2.4.2.3 Conclusions
The works on portfolio optimization are summarized in Table 4 and again it’s difficult to make
direct comparisons since the assets and dates tested are different. Using the Markowitz Mean
Variance Model is the classic approach for portfolio optimization but there are indications that
using technical analysis to extract useful information may allow the creation of portfolios with
higher returns.
2.5 Chapter Conclusions
In this chapter several relevant works were presented, many using Technical Analysis and the
majority of them show positive results in comparison with a Buy-and-hold reference, giving
support to the hypothesis that Technical Analysis has predictive capabilities. This goes against
the Efficient Market Hypothesis and the Random Walk Theory. The current work will also use
Technical Analysis as a Market Analysis Technique, in particular Technical Indicator.
Evolutionary Algorithms have been successfully used in different configurations which
demonstrate its flexibility and power. In particular, Genetic Programming with rules coded in
a tree structure, as presented in [23], [28] and [29], allow a great deal of freedom in the
configuration of the solutions and will be the optimization algorithm used in this work.
There are indications that using data from higher time scales (like weekly and monthly) tends
to increases returns [17], and this should be taken into account in the present work (more
than one time frame will be used simultaneously). Majority vote was used in some of the
works as a way to process multiple sources of information [26], [30] and is a path worth
exploring when creating more complex trading rules.
Genetic Programming with rules encoded in a tree structure allows for many combinations
which may easily generate curve-fitting and measures must be taken to address this problem,
similar to the ones employed in [27].
25
Ref Paper
date
Optim.
Method
Portfolio
Analysis
Operators /
Genes
Chromosome
structure
Fitness
function
Time
Frame
Assets
used
Test
Period Results
[32] 2007 GA Markowitz (Asset weights) Array Total return /
portfolio risk Daily
Taiwanese
Stocks N/A Not specified
[33] 2008 GA Markowitz Asset weights and
asset indexes Tree structure Sharpe Ratio N/A
Nasdaq and
NIKKEY stocks
2004 -
2006
Portfolios with
fewer assets
[36] 2005 GP TA
Arithmetic,
Boolean,
Inequalities, LAG,
Min, Max, AVG and
others.
Tree structure Profitability Daily 24 futures
contracts
1980 -
2000
No evidence of
profitability
[34] 2012 GA TA N/A Variable
length array
Information
Ratio, Omega
ratio, Sortino
Ratio
Daily
Stocks from
Dow Jones
Industrial
2006 -
2011
Higher returns
and higher risk
than DJI
[37] 2009 GA TA
Weights of TA rules
used in the scoring
function
Array ROI Daily
Stocks from
DJI and
S&P500
2006 -
2007
Beats B&H and
Random
portfolios
[35] 2015 GA TA and FA
Parameters used
for ranking and
trading.
Array ROI, Variance
Daily,
Trimestral
(FA)
S&P500
Stocks
2012 -
2013
Beats reference
index
Table 4 – Overview of different approaches on portfolio optimization.
26
27
3 Solution’s Architecture This section explains the proposed solution, first in general lines and then in detail. The work
can be divided into two main components. The first consists in the optimization of trading
rules for individual assets (stock indices and commodities) and is the main focus of this work.
The second consists in simulating several portfolios using the trading rules optimized in the
first component and is a complement of the main objective.
The overall architecture of the solution can be seen in Figure 10. A user interface allows the
user to select which module to use and which configuration file should be loaded.
Figure 10 – Overall architecture of the proposed solution
When the Optimization module is chosen it loads the global settings and asset specific settings
from files and also loads the historical data for the assets to be optimized. After this
initialization process, the optimization begins, optimizing one asset at a time independently
from all other assets. The optimization produces detailed results that can be later processed
and analyzed. It also produces results formatted for use by the Portfolio module.
When the Portfolio module is chosen, it loads settings from a configuration file and also loads
historical data from the assets it will use in the portfolio simulation. Several portfolios are then
created:
1) Default buy-and-hold with equal allocation for all assets;
2) Portfolio with equal allocation for all assets. Each asset is traded with the best trading
rules optimized for that asset;
28
3.1 Optimization Module
The optimization module independently optimizes trading rules for several assets which is
done sequentially following a list specified in a configuration file. The solutions optimized have
a trend-following philosophy and combine two subsystems, one focusing on the medium term
and the other on the long term. The optimization is done with a sliding window approach, with
successive windows containing in-sample and out-of-sample data. The trading rules are
represented in tree structures and are evolved using genetic programing. Results are saved
for later visualization and processing and the trading signals of the best solutions for each
asset can also be later used to construct portfolios. Several aspects of the optimization process
are discussed in more detail in the following sections.
3.1.1 Optimization Windows
The optimization of each asset is divided in windows of data, in-sample and out-sample using
a sliding window method as described in Figure 11.
Figure 11 – Sliding Window Optimization
Optimization is first performed in an in-sample window and the best rules obtained are
applied to the corresponding out-of-sample window. Then the optimizer moves to the next
pair of in-sample/out-of-sample windows and the process is repeated until there is no more
data. A flowchart of this process is presented in Figure 12.
29
Figure 12 – Global flowchart of the optimization of an asset
The complete results of the optimization of an asset is composed of the results from the
several individual windows. Each out-of-sample window has associated a different set of rules
(best genome) that was obtained in the optimization of the corresponding in-sample window.
The complete trading results of an asset, are obtained after all windows have been optimized
and each out-of-sample window has its associated genome.
3.1.2 Optimization divided in generations
The optimization of a window using Genetic Programming is done in successive generations
where the population of genomes is evolved towards solutions that produce better results
according to a fitness function. Several steps are necessary to complete one generation as can
be seen in Figure 13. Generation 0 is different since there is the need to generate a random
initial population. After that, a cycle is repeated until the stop condition is met. All the
genomes in the population are firstly interpreted which means converting the trading rules
into vectors of trading signals. Next, the trading rules are simulated with the asset data using
the trading signals previously obtained. With the results from the simulation, all genomes are
scored and ranked from best to worse, using a fitness function. Genomes that are too similar
are eliminated to prevent that after some generations, the population is dominated by a few
solutions with many copies, which would drastically decrease diversity. While the stop
condition is not met, a new generation of genomes is created from the old one: the best
genomes are preserved (elitism) and the others are reproduced using different reproduction
operators (mainly crossover and mutation). Before the new generation can begin, a check is
performed on the newly generated genomes to ensure they all obey a set of requirements.
30
Figure 13 – Flowchart of the Genetic Programming optimization of one window
3.1.3 Elimination of similar solutions
The elimination of similar solutions (genomes) is very important because otherwise the most
scored ones would gradually create similar versions that would dominate the population. This
would strongly decrease the diversity of solutions and increase the likelihood of the
optimization being stuck in local maxima’s.
The elimination of similar genomes is not based on the tree structure of the chromosomes
since different chromosome trees can produce similar or equivalent results. The chosen
method is to look at the Mean Squared Error (MSE) of the vectors with trade signals produced
by the genomes, Figure 14. The algorithm starts at the most scored genome and compares its
signals with the ones from the less scored genomes until a distance of N genomes. If the MSE
is less than an arbitrary threshold, the less scored are eliminated. After having compared the
first genome with all the genomes below up to a distance of N, the second genome is
compared to the less scored ones in a similar fashion. The process is repeated with all genomes
until the end of the list.
The reason why genomes are not compared with all other genomes in the list is that it would
be computationally expensive because of the large number of Mean Squared Error
calculations. The presented solution seems to be a good compromise between efficacy in
removing similar genomes and the computational cost.
In this work the value chosen for N is 10 which demonstrated to be a good compromise
between efficacy in eliminating similar genomes and the computational cost associated with
the algorithm.
31
Figure 14 – Example of algorithm to eliminate similar genomes
3.1.4 Only Trend Following Solutions
Based on the review of the relevant existing work, it was decided to develop trading rules that
benefit exclusively from market trends. The main reasons are:
Market trends are patterns that are well studied and exist in almost all markets and
exist since stock exchanges were created.
Literature shows it is possible to profit from trends in a consistent way using
mechanical trading rules.
It allows to strongly reduce the search space and avoid curve fitting. The optimizer will
be discouraged to profit from short term patterns that probably won’t be present in
the near future.
The measures adopted to enforce trend-following solutions is discussed in section 3.1.10.
3.1.5 Combining several time frames
Some authors found that the use of longer time frames tend to produce higher returns,
namely [17] and [30]. Normally monthly returns are higher than weekly returns and weekly
returns are higher that daily returns. Having those results present, it was decided to combine
several time frames in the same trading system, giving to each time frame a weight in the final
decision.
It can be hypothesized that in the market data, there is a signal corresponding to the
underlying factors that produce a trend, and there is noise corresponding to the contradictory
activity of all the market participants. In that case, the higher the time frame, the higher the
signal to noise ratio (SNR) one should expect since the noise produced by market activity has
more impact in the shorter time frames. With a higher SNR the trading rules will produce less
false signals and generate higher returns.
32
This effect observed in longer time frames (weekly and monthly) can be emulated with daily
data but using different parameters in the technical indicators that generate the trading
signals. For example, a simple moving average of 10 periods using weekly data, can be roughly
emulated in daily data using a period 5 times greater: 5 x 10 = 50.
In Figure 15 we have an example chart showing the S&P500 index with data that includes the
2008 violent bear market. In the top part of the chart we can see the signals generated by a
mechanical trading system which follows trends. This trading system has two components or
subsystems whose signals are then combined with equal weights to generate the global signal
of the system. The top line in green is the medium term subsystem and is the most responsive,
reacting to some S&P500 mild corrections. The middle signal line in blue corresponds to the
long-term subsystem which is much more stable and only changes position when the
movement is confirmed. The third line from top in red is the global output of the trading
system and is calculated giving weights of 0.5 to both subsystems. Each subsystem is either
LONG (1) or SHORT (-1) and when the two subsystems disagree, the global system is neutral.
The main chart background color changes according to the global system signal: green for
long, pink for short and yellow for neutral.
Figure 15 – S&P500 chart with the signals of a trend following trading system with two subsystems
33
This example shows it is possible to create trading rules which are stable and immune to daily
noise despite using daily data. This simplifies the optimization process by using data from only
one time frame. And at the same time allows the combination of several sub-systems sensitive
to trends in different time frames.
Preliminary studies show this type of system gives good results in stock indices and in some
commodities. There are in fact two trading systems trading in parallel, with different trade
profiles, different return profiles and different risk profiles, which average and complement
each other. The long-term subsystem tends to produce higher returns since it’s more stable
but tends to have higher drawdowns since it takes more time to react to market change. On
the other hand, the medium-term subsystem tends to have a lower return since it reverses
positions more frequently but, on the other hand will probably have lower drawdowns since
it reacts faster to market change. The system resulting from the combination of these two
components will tend to be more balanced and tend to have lower drawdowns than either of
the two subsystems alone. This approach will be studied, testing several configurations and
trying to find optimized solutions.
3.1.6 Genetic Programming
Genetic Programming was selected as the optimizer algorithm since it’s a powerful and
flexible search algorithm that has produced good results in previous works [30].
3.1.6.1 Representation
An element in the population is referred as a genome and it contains all the information
necessary to produce trading signals and trade an asset. A genome is composed of one or
more sub-systems as can be seen in Figure 16. A sub-system is a set of rules that produce
trading signals with a certain profile (ex: long-term trades) and contributes with a certain fixed
weight to the global output of the system. In the example of Figure 16 the genome is
composed of 3 sub-systems with weights 40%, 30% and 30%. The first two sub-systems have
a specific chromosome for long trades and another for short trades, but the third sub-system
only has one chromosome that generates signals for both long and short trades.
34
Figure 16 – Example of a genome structure
Chromosomes contain the actual rules organized in a tree structure and are integrated into a
sub-system, Figure 17. Each element in the tree is called a node or a gene. There are operator
genes (ellipses in Figure 17) which perform operations and deliver the result to its parent
node. And there are terminal genes which feed information to operator genes (rectangles in
Figure 17). Some terminal genes perform operations on pre-defined inputs so they don’t need
to be specified, ex: Relative Strength Index (RSI) which always uses the Close prices. Some
genes have parameters like the RSI and others don’t, like the operator AND.
Figure 17 – Structure of a Chromosome
Genes have inputs and outputs of type double or Boolean. For example, the Exponential
Moving Average (EMA) has an input double and output double; the operator AND has both
input and output of type Boolean; and the operator Greater than (>) has inputs double and
output Boolean.
35
3.1.6.2 Genes
Each node in a chromosome tree is a gene and can either perform an operation or be data
used by operators, Figure 18.
A gene has several characteristics which define it:
Gene group – genes from the same group share several characteristics. Ex: Arithmetic
operators or logic operators.
Type of Input data – if the gene has inputs (child nodes) the data it receives can be of
type Double or Boolean. For ex, the Simple Moving Average (SMA) gene has one input
of type Double.
Type of output data – The gene outputs data of type Double or Boolean. For example,
the SMA output is of type Double.
Number of inputs (number of child nodes). Ex: The SMA has one input which is a vector
of type Double from which the SMA is calculated.
Parameters – Some genes don’t have parameters and others have several. Ex: The
SMA has one parameter which is the period.
A set of possible genes is previously defined from which they are chosen when a chromosome
is constructed or mutated. Not all combinations are possible since parent-child genes have to
be compatible: a child gene must output same type of data used by the parent gene as input.
Figure 18 – Gene details
Genes are grouped according to their function and each group share several characteristics,
see Table 20 in Appendix B – Groups of genes.
36
A gene/node with inputs of type double must obviously have child nodes with outputs of type
double and the equivalent if the inputs are of type Boolean. The genes of type Technical
Primitive assume a very important role since they are complete trading rules and a
chromosome can have a single gene of this type. They can have complex rules coded inside
and output the corresponding signals. For example, the gene with id string “TMP_110112”
produces signals which are the result of the majority vote of 3 sub-rules. Some of these genes
are based on rules studied by other authors with good results and others are the result of
preliminary tests which showed promising results. Some of these Technical Primitive genes
don’t have parameters and therefore are not optimizable but, the chromosome and genome
still are.
3.1.6.3 Reproduction Operators
There are 5 different reproduction operators. Elitism and Best Chromosome Merge are applied
first and the remaining genomes are reproduced with the other 3 operators. The reproduction
operators are:
Elitism – In each generation the genomes representing the 5% most scored are copied
unchanged to the next generation.
Crossover – Two parent genomes are randomly selected from the list of available
genomes ranked by decreasing fitness score. The probability of a genome being chosen
decreases exponentially with its decreasing position in the list. And the most scored
genome is 5 times more likely of being chosen than the last one. The crossover point
is chosen randomly in the first parent genome. The probability of a gene being chosen
as crossover point decreases exponentially from terminal node to root node and a
terminal node is 5 times more likely to be chosen than the root node. A compatible
sub-tree is then randomly chosen from the second parent and inserted in the crossover
point. Figure 7 shows an example of the crossover operator.
Hard Mutation – A single parent genome is randomly chosen as in the crossover
operator and a mutation point is randomly chosen in the same way as the crossover
point. Then a sub-tree is randomly generated and inserted in the mutation point. An
example of the hard mutation operator can be seen in Figure 8.
Soft Mutation – A single parent genome is randomly chosen as descried for the
crossover operator and only the parameters of the genes are mutated. The
chromosome tree structure remain unchanged. The probability of a particular gene in
a chromosome being mutated is proportional to the number of genes in the
chromosome. Each chromosome has the probability of having 1.0 genes mutated.
Best Chromosome Merge – In each generation, a special genome is created by merging
the best chromosomes of each position in the genome. Chromosomes are simulated
37
individually, and a ranked list created for each chromosome position. Then the best
chromosome of each list is picked and a new genome created.
Crossover, Hard Mutation and Soft Mutation are mutually exclusive and only one is chosen
with probabilities 0.5, 0.3, 0.2 respectively. In these 3 operators Genomes are reproduced
chromosome by chromosome and a chromosome from a particular position in the genome
never exchanges genes with a chromosome in a different position. In the Crossover and Hard
Mutation operators, not all chromosomes from a genome are reproduced and the probability
of a particular chromosome being reproduced is 0.1. If a chromosome is not reproduced, it is
copied intact from parent 1.
3.1.7 Data organized in vectors
To increase the efficiency of the optimization, data is stored and manipulated in vectors and
operations are performed with vectors. Each vector position corresponds to a trading day
(bar) and all vectors are synchronized meaning that a particular vector index is associated with
the same trading day in all vectors. The first element corresponds to the oldest date and the
last element to the earliest date.
Asset data is also stored in vectors, one for each of the following fields:
Open – Open price for the period;
High – Highest price in the period;
Low – Lowest price in the period;
Close – Close or last price of the period;
Volume – Volume of the transactions in the period (not used in practice).
When the rules in a chromosome are applied to a window of data (interpretation), the output
is a vector with the same dimension as the data window, containing the trading signals
produced by the chromosome. In this work chromosome outputs are always Boolean, being
either true of false.
What true and false means depends on the type of chromosome:
Long only chromosome – true means buy and false means neutral.
Short only chromosome – true means Short and false means neutral.
Long and short chromosome – true means Long and false means Short.
Some trading rules can be implemented in a sub-system with a single chromosome that
produces trades in the long and short side, alternating between the two and never being
neutral, Table 5.
38
Long/Short chromosome Output
False SHORT
True LONG Table 5 – Meaning of outputs from a chromosome that produces long and short signals
Other trading rules must be implemented in a sub-system with two chromosomes, one for
long trades and another for short trades. In this case the outputs are combined to give the
final output, Table 6.
Long chrom. Short chrom. Output Comment
False (neutral) False (neutral) NEUTRAL Signals agree - Both neutral
False (neutral) True (short) SHORT Signals agree - Short
True (long) False (neutral) LONG Signals agree - Long
True (long) True (short) NEUTRAL Contradictory signals Table 6 – Combination of outputs in sub-systems with two chromosomes
A typical genome will have two subsystems, one of them will try to detect the long-term trend
of the market and usually needs only one chromosome, alternating long and short positions.
The second subsystem is usually medium-term, using one or two chromosomes depending on
the trading rules.
3.1.8 Interpretation of Genomes
As can be seen in Figure 13, the process of optimizing a window of data is divided in
generations and in each generation the population of solutions (genomes) must be evaluated
using a fitness function. The fitness function uses several performance metrics obtained after
simulating the trading rules contained in the genomes.
The method used for simulating a genome is to previously obtain a vector of trading signals
generated by the rules of the genome when applied to a particular asset. Each element in
these vectors corresponds to a trading day and contains the market position that the system
should assume. This market position ranges from -1.0 (100% short) to 1.0 (100% long), being
0 a neutral position.
The process of obtaining these vectors is represented in Figure 19 with an example of a
genome with two subsystems (medium and long term), each one having two chromosomes,
one for long trades and the other for short trades. The interpretation process starts by
individually interpreting each chromosomes and generating a corresponding vector with
39
trading signals. The interpretation of a chromosome is also done gene by gene, starting in the
root node and working down the tree in a recursive way. After the vectors for each
chromosome are obtained, the interpretation moves to the subsystem level and combines the
signals of the chromosomes belonging to the same subsystems. After obtaining the vectors
for all subsystems, the interpretation moves to the genome level and combines the signals
from all the subsystems to obtain the final genome trading signals. Each subsystem
contributes with a predefined weight to the global signal.
Figure 19 – Process of obtaining the trading signal vectors
3.1.9 String representation of Genomes
It’s important to be able to easily interpret the solutions found in the optimization process
and thankfully, the tree structure of the chromosomes facilitates the process of converting a
genome to string format. The genome subsystems are presented by index order and after each
subsystem title, the respective subsystem chromosomes are listed with a preceding tab
character. After each chromosome the respective genes are listed with 2 preceding tab
characters:
Subsystem 0:
Chromosome k0:
[genes from chromosome k0]
[…]
40
Chromosome kn:
[genes from chromosome kn]
[…]
Subsystem N:
Chromosome p0:
[genes from chromosome p0]
[…]
Chromosome pn:
[genes from chromosome pn]
Each gene has an identification (id) string that identifies it, for example, the id of the Simple
Moving Average gene is “SMA”. Each chromosome is converted to string starting by the root
node (top) and going down the tree in a Depth-First-Search fashion: Tries to go down the tree
as much as possible, first by the left side. When a terminal gene is found, backtracks and tries
the branch to the right or backtracks again if there are no more branches. Repeats the process
until all genes are processed.
Each gene is printed in an independent line and the start position of the string is advanced
with TAB characters, as many as the depth level of the gene in the chromosome tree (0 for the
root gene). If the gene has parameters, they are inserted by order in front of the gene string,
inside round brackets.
For example, the chromosome in Figure 17 has 3 levels (0, 1 and 2) and would have the
following string representation:
AND
>
Close
EMA (200)
Close
<
RSI (14)
40.0
As can be seen, it’s easy to interpret the chromosome and understand its logic: To generate a
signal (root gene returns TRUE) two conditions must be verified (AND). The first is the close
must be greater (>) than the Exponential Moving Average of 200 periods of the close. And the
second condition is that the Relative Strength Index of 14 periods is smaller (<) than 40.
41
3.1.10 Enforcing Trend-following rules
Since the goal of this work is to study the optimization of trading rules that use a trend-
following philosophy, it’s necessary to implement measures to enforce that outcome. This is
done by limiting the search space, eliminating many combinations of genes that would
produce other trading patterns different from the desired one. In essence, we want rules that
are capable of detecting a consistent movement of prices in one direction and open a trade in
the same direction. The criteria to decide when a consistent movement/ trend is present and
what technical indicators to use, in what configuration and with which parameters, is what
the optimization will do.
In practice, we want to eliminate, for example, a rule that buys when the close price drops
below a moving average of the same close, because that means the price is dropping (the
moving average is slower that the close). The optimization setup presented in 5.2, with a large
search space, allow for such rules. In preliminary testes, it was determined that the best way
to proceed was to use simple chromosome trees, with few levels or even just one level and
use mainly technical primitive genes. These genes allow a greater control on the creativity of
the optimizer because the basic format of the trading rule is already defined and only the
parameters can be changed. Even with this restrictions, the optimizer has several genes to
choose from and several parameters to adjust.
Another important restriction that was adopted was to define allowed ranges for the gene
parameters. And to define different ranges for medium-term and long-term subsystems. What
this means is that the same genes will have different allowed ranges of parameters depending
on the subsystem on which they are used.
Since it was decided to combine several subsystems that respond to trends in different time
frames (ex: medium and long-term), it’s necessary to adapt the parameters to that behavior.
A possible option to achieve this result would be to use the same parameters but use data
with a different period, for example daily data for the medium term and weekly data for the
long-term. The solution used in this work was to emulate the behavior of the indicators with
weekly data by changing the allowed parameters and making them less reactive. In practice
the ranges were empirically chosen using some preliminary tests. For example, the gene with
id “C>SMA” which compares the Close with the Simple Moving Average of the Close allows a
period for the SMA in the range [50, 200] for medium term chromosomes and [100, 400] for
long term chromosomes.
A different approach to enforce the differentiation of trading rules of the several subsystems
(medium term and long term) was to enforce a range for the average number of trades
produced by each subsystem. When a genome is simulated with the asset data, its subsystems
are also individually simulated. From this simulation, many performance parameters are
obtained and one of them is the annualized average number of trades. If the subsystem has
an average number of trades than is outside the allowed range, the corresponding genome is
penalized in its fitness score. The greater the departure from the allowed range, the greater
42
the penalty. In this way, the offending genomes are not completely removed from the survival
race but are strongly discourage to go in that direction. The allowed range for the annualized
average number of trades is [3.0, 8.0] for the medium term and [0.0, 3.0] for the long term.
These empirically defined ranges are not critical and small variations have little impact on the
results.
3.1.11 Evaluating the performance
Evaluating how good the elements of the population are is a fundamental step in evolutionary
optimizers. In each generation, all solutions must be tested to gauge how good they are in
solving the problem, so that the best ones are favored in detriment of the worse when
reproduction is performed. The biggest problem is deciding what criteria should be used to
evaluate the solutions. In some cases there is only one performance measure to take into
account and the task is simpler. But in the present case where we evaluate a trading system,
what characteristics do we want? Ideally The biggest possible return with the lowest possible
risk (maximum drawdown, average drawdown, negative volatility, etc). The problem is that
these two performance measures can be contradictory. This leaves the question of how to
balance the two in a fitness function that rewards higher returns but penalizes the solution if
those returns are obtained with too much risk. Several ratios have been proposed and we’ll
review some of them.
3.1.11.1 Sharpe Ratio and Sortino Ratio
The Sharpe Ratio [6] was proposed in 1966 by William Sharpe originally as “reward-to-
variability ratio” and it had a big impact in the industry since it allowed the results of funds to
be compared. It measures the excess return per unit of deviation in an investment asset or
portfolio and it’s formula is presented in equation ( 4 ).
𝑆 =
𝑟𝑝 − 𝑟𝑓
𝜎𝑝 ( 4 )
Where:
𝒓𝒑 is the expected portfolio return;
𝒓𝒇 is a risk free rate;
𝝈𝒑 is the standard deviation of the portfolio.
The Sharpe ratio is criticized because it doesn’t differentiate between positive and negative
volatility, and positive volatility shouldn’t be penalized. This lead to the creation of the Sortino
ratio [6] which is similar to the Sharpe ratio with the difference that in the denominator
43
appears an equivalent of the standard deviation that only accounts for the negative volatility:
Target Downside Deviation. The calculation can be seen in equation ( 5 ).
𝑆 = 𝑅 − 𝑇
𝐷𝑇𝑇
( 5 )
Where:
R is the portfolio average realized return;
T is the target return, chosen by the investor as the reference;
TDD is the Target Downside Deviation.
The Target Downside Deviation is calculated as shown in equation ( 6 ).
𝐷𝑇𝑇 = √1
𝑁 ∑(𝑀𝑖𝑛(0, 𝑋𝑖 − 𝑇))2
𝑁
𝑖=1
( 6 )
Where:
𝑿𝒊 is the 𝑖𝑡ℎ return;
N is the total number of returns;
T is the target return.
3.1.11.2 Risk Return Ratio
The Risk Return Ratio (RRR) is a ratio between the return of a portfolio in a certain period and
the maximum drawdown produced by the portfolio over the same period [38]. The calculation
is presented in equation ( 7 ).
𝑅𝑅𝑅 = 𝑅𝑒𝑡𝑢𝑟𝑛 (𝑖𝑛 𝑝𝑒𝑟𝑐𝑒𝑛𝑡𝑎𝑔𝑒)
𝑀𝐷𝐷 (𝑖𝑛 𝑝𝑒𝑟𝑐𝑒𝑛𝑡𝑎𝑔𝑒) ( 7 )
Where:
𝑹𝒆𝒕𝒖𝒓𝒏 is the percentage return in the period;
𝑴𝑫𝑫 is the Maximum DrawDown in the period. The DrawDown on a particular day is
the difference between the peak return achieved in the past and the current return.
An account that had 1000 units as maximum and drops to 500 has a drawdown of 500
or 50%.
The Risk Return Ratio is similar to other ratios like the Calmar ratio, the Sterling ratio and the
Burke ratio.
44
3.1.11.3 Modified Risk Return Ratio
In some situations the Risk Return Ratio is not appropriate, for example, let’s imagine we use
the RRR as the fitness function of an evolutionary algorithm trying to evolve trading rules.
What can happen is that Maximum Drawdown (MDD) values close to zero will produce very
high RRR values. In the extreme, if the rules produce a single short term winning trade, the
MDD can be zero and the RRR will be infinite. In practice what this fitness function may
encourage are very low MDD because it’s an easier way of getting high fitness scores.
A modification is desired to prevent this behavior that was observed in preliminary tests
performed by the author. What is suggested is the addition of a constant in the denominator
of the RRR formula, making impossible for it to be zero and creating a balance between the
importance of high returns and low MDD, equation ( 8 ).
𝑅𝑅𝑅 = 𝑅𝑒𝑡𝑢𝑟𝑛 (𝑖𝑛 𝑝𝑒𝑟𝑐𝑒𝑛𝑡𝑎𝑔𝑒)
𝐾 + 𝑀𝐷𝐷 (𝑖𝑛 𝑝𝑒𝑟𝑐𝑒𝑛𝑡𝑎𝑔𝑒) ( 8 )
Where K is a reference Drawdown. We suggest a value of 5 but it should depend on the size
of the test window amongst other things. In a larger window the MDD is expected to be higher.
3.1.11.4 Lake Ratio
The Lake Ratio was proposed by investor Ed. Seykota [39] and can be graphically explained
imagining that the return curve over time is like a mountain with its peaks and valleys, and
that rain water gets trapped in the valleys, Figure 20. The land in the yellow area is the sum of
the (daily) returns and the water in the blue areas is the sum of the (daily) drawdowns.
Figure 20 – Lake Ratio graphical representation
45
The Lake Ratio is calculated dividing the Water area by the Land area, equation ( 9 ).
𝐿𝑎𝑘𝑒 𝑅𝑎𝑡𝑖𝑜 = 𝑊𝑎𝑡𝑒𝑟
𝐿𝑎𝑛𝑑
( 9 )
Sometimes the inverse of the Lake Ratio is used so that higher returns correspond to higher
values of the ratio. In this work the inverse of Lake Ratio will be used.
This ratio may suffer from the same problem mentioned regarding the Risk Return Ratio when
applied as fitness function of an optimizer. It may stimulate trading rules that perform few or
no trades in order to have a very low drawdown and achieve a good score. Another issue with
this ratio is that, as the window size increases, the Land area tends to increase at a faster pace
than the Water area. This prevents the comparison of the ratios between different windows.
3.1.12 Fitness Function
The fitness function used to score the genomes is a modified Risk Return Ratio with a structure
previously suggested. The fitness score appears as “JR4” in result tables and is calculated as
shown in equation ( 10 ).
𝐹𝑖𝑡𝑛𝑒𝑠𝑠 𝑠𝑐𝑜𝑟𝑒 = 𝐽𝑅4 =𝑎𝑛𝑛𝑢𝑎𝑙𝑖𝑧𝑒𝑑 𝑅𝑒𝑡𝑢𝑟𝑛 %
5 + 𝑀𝑎𝑥𝑖𝑚𝑢𝑚 𝐿𝑜𝑠𝑠 %
( 10 )
Where:
𝑎𝑛𝑛𝑢𝑎𝑙𝑖𝑧𝑒𝑑 𝑅𝑒𝑡𝑢𝑟𝑛 % is the annualized return in percentage of the invested capital
(which is fixed).
𝑀𝑎𝑥𝑖𝑚𝑢𝑚 𝐿𝑜𝑠𝑠 % is the maximum percentage loss as described in 4.4.2.
The constant 5 in the denominator is empirical and not critical to the results. Its main
function is, as stated before, to discourage solutions that achieve a high fitness score by
making few trades resulting in a very small maximum drawdown.
3.2 Portfolio Module
The Portfolio module simulates what would have happened if a particular set of rules would
have been followed in trading a group of assets. First, a reference portfolio is simulated where
all assets are bought with equal capital, see 4.5.2. Then, the individual asset optimizations are
used to construct a portfolio. Each asset optimization produces a best genome for each out-
of-sample window and all those genomes produce a vector with trading signals for the entire
test period. The portfolio module loads these vectors from files and performs a global vector
46
synchronization described in 4.5.1. After all data and all vectors from all assets are
synchronized, a portfolio is simulated by trading all assets simultaneously using the
corresponding trading signals (using equal capital). Results are then compared with the
previously simulated reference portfolio.
3.3 Technical Indicators and rules
Technical indicators are the base of most trading rules used in this work. For this reason, the
most important ones will be reviewed and some of the setups in which they are used to
produce the trading signals. Some concrete genes that use those technical indicators will also
be mentioned.
3.3.1 Moving Averages
A moving average in technical analysis refers to a filter applied usually to close prices and it’s
used to filter noise or to have a reference of the trend in a longer time frame [6]. Moving
averages can also be applied to the Open, High and Low prices, to volume, a combination on
these vectors or even to a previously calculated indicator. The most used moving averages
are:
Simple Moving Average (SMA) which is a Finite Impulse Response (FIR) filter where all
N values used in its calculation have the same weight.
Weighted Moving Average (WMA) also called Linearly Weighted Moving Average
(LWMA) which is a FIR filter where the N elements used in the calculation have weights
that decrease in an arithmetic progression.
Exponential Moving Average (EMA) which is an Infinite Impulse Response filter where
weights decrease exponentially never reaching zero. The exponential curve of the
weights is controlled by a parameter (alpha).
Figure 21 shows a chart of the S&P500 index with a 100-day Simple Moving Average (SMA)
superimposed in blue. An upper and lower band calculated from the SMA are also plotted in
red (SMA value +/- 3%). The moving average rule structure used in this work consists in
triggering a buy/short signal when the close price crosses above/below the MA value (light
blue ellipses in Figure 21). A drawback of this method is that the close price can oscillate
around the MA line and generate several undesirable contradictory signals. One way to limit
this problem is to use an upper and lower band based on the moving average that follow it at
a distance and filter the signals. In this band configuration, a buy signal is generated when the
close price crosses the upper band from below (green circles in Figure 21) and a short signal
is generated when the close price crosses the lower band from above (red circles in Figure 21).
47
The distance of the bands can be fixed by a percentage, or dynamic, based on the volatility of
the asset.
All types of moving averages can be used in this type of rules and obviously the period can
be optimized as can the distance of the bands when used. The genes that use this rule
structure have ID strings “C>SMA”, “C<SMA”, “C>WMA”, “C<WMA”, “C>EMA”, “C<EMA” for
simple crossover rules. And “C>SMAB” and “C<SMAB” for band crossover rules. See
Appendix C – Technical Primitive Genes.
Figure 21 – 100-day Exponential Moving Average applied to the close of the S&P500 index
3.3.2 Price Channels
Price channels are a tool used in technical analysis used to detect a change in the trend
when the close price closes above/below the recent trading range/channel. This channel is
composed of an upper band and a lower band which can be calculated in several different
ways. The method used in this work uses the lowest low price in the past N bars to make the
lower band and the highest high price in the past N bars to make the upper band. A channel
calculated with this method can be seen in Figure 22, applied to the S&P500 index using a
period of 150 days. In this example, a buy signal is generated when the close crosses the
upper band from below (green circle) and a short signal is generated when the close crosses
the lower band from above (red circle). This type of rule is sometimes referred as Channel
Breakout. In this work the genes that use these rules have ID strings “CH_BRK_S” and
“CH_BRK_L” for short and long trades respectively, see Appendix C – Technical Primitive
Genes.
48
Figure 22 – Price Channel applied on S&P500
3.3.3 MACD
The MACD technical indicator stands for Moving Average Convergence/Divergence and was
created by Gerald Appel [6] in the late 70’s, being one of the most popular technical indicators.
The complete MACD is composed of 3 values which can be used in different ways. Figure 23
shows the MACD applied to a chart with the S&P500 index.
Figure 23 – MACD applied to a chart of S&P500 index
The main line of the indicator is called MACD Line (blue line in Figure 23) and is the difference
between two exponential moving averages (EMA) with different periods, calculated from the
49
close prices (the commonly used periods are respectively 12 and 26). The formula of the MACD
line can be seen in equation ( 11 ).
𝑀𝐴𝐶𝐷 𝐿𝑖𝑛𝑒 = 𝐹𝑎𝑠𝑡𝐸𝑀𝐴(𝑐𝑙𝑜𝑠𝑒, 𝑓𝑎𝑠𝑡_𝑝𝑒𝑟) − 𝑆𝑙𝑜𝑤𝐸𝑀𝐴(𝑐𝑙𝑜𝑠𝑒, 𝑠𝑙𝑜𝑤_𝑝𝑒𝑟)
( 11 )
Where:
𝑭𝒂𝒔𝒕𝑬𝑴𝑨(𝒄𝒍𝒐𝒔𝒆) is the EMA with the smallest period: fast_per (reacts faster).
Corresponds to the green line in Figure 23.
𝑺𝒍𝒐𝒘𝑬𝑴𝑨(𝒄𝒍𝒐𝒔𝒆) is the EMA with the longest period: slow_per (reacts slower).
Corresponds to the orange line in Figure 23.
The second MACD value is called the Signal Line and is an exponential moving average of the
MACD Line (red line in Figure 23). The formula is presented in equation ( 12 ). The most
commonly used period of the EMA is 9.
𝑆𝑖𝑔𝑛𝑎𝑙 𝐿𝑖𝑛𝑒 = 𝐸𝑀𝐴(𝑀𝐴𝐶𝐷 𝐿𝑖𝑛𝑒, 𝑝𝑒𝑟𝑖𝑜𝑑)
( 12 )
The Third MACD value is called MACD Histogram (gray bars in Figure 23) and is the difference
between the MACD Line and the Signal Line. The formula can be seen in equation ( 13 ).
𝑀𝐴𝐶𝐷 𝐻𝑖𝑠𝑡𝑜𝑔𝑟𝑎𝑚 = 𝑀𝐴𝐶𝐷 𝐿𝑖𝑛𝑒 − 𝑆𝑖𝑔𝑛𝑎𝑙 𝐿𝑖𝑛𝑒
( 13 )
The MACD indicator can be used in several ways, for example, the crossing of the MACD Line
and the Signal Line can signal a turning point and can be used to trade short term oscillations
(swing trading). It can also be used to detect trends by looking at the MACD Line that can be
above or below 0, signaling a possible up/down trend. In this work MACD rules will have this
later configuration and a buy signal will be generated when the MACD Line crosses the 0
level from below (green circles in Figure 23) and a short signal will be generated when the
MACD Line crosses the 0 level from above (red circles in Figure 23). The classic parameters of
12, 26 and 9 mentioned before can obviously be optimized. In this work, genes with this
MACD rule structure have the ID string “MACD>0” and “MACD<0” respectively for long and
short genes, see Appendix C – Technical Primitive Genes.
50
3.3.4 Relative Volatility Index
The Relative Volatility Index (RVI) was created by Donald Dorsey in 1993 and refined in 1995
[40]. It’s calculated in a similar way to the Relative Strength Index (RSI) but measures the
Standard Deviation of High and Low prices. It oscillates between 0 and 100, being 50 a neutral
value.
The RVI reacts fast when prices move with high volatility, signaling the probable start of a new
consistent movement. The calculations of the RVI can be consulted in the publication where
the indicator was presented [40]. Figure 24 shows the Relative Volatility Index in an S&P500
chart, in the configuration used in this work.
Figure 24 – Relative Volatility Index in an S&P500 chart
The figure shows three horizontal lines, a neutral one in the 50 level representing a neutral
situation, and upper threshold level (54 in this example) and a lower threshold level (46 in this
example). Instead of buy and short signals being generated by crossing of the neutral line,
which causes many undesirable false signals, the threshold levels are used instead. A buy
signal is generated when the RVI line crosses the upper threshold level from below (green
circles in Figure 24) and the system should remain long until a short signal is generated. A
short signal is generated when the RVI line crosses the lower threshold from above (red circles
in Figure 24) and the system should remain short until a buy signal is generated. This rules
produce a system that is constantly in the market, reversing positions, waiting for a longer and
stronger move. While the system waits for these longer movements, it’s likely it will sustain
several small losses.
The classic parameters for the RVI is a smoothing period of 14 and a standard deviation of 10
periods. These parameters can obviously be optimized, especially the smoothing period. In
51
this work, genes with this RVI rule structure have the ID string “RVI>X” and “RVI<X”
respectively for long and short genes, see Appendix C – Technical Primitive Genes.
3.3.5 Stop and Reverse
Rules based on Stop and Reverse indicators (SAR) are usually always in the market, alternating
long and short positions. A buy signal is generated when the close price crosses the SAR line
from below and a short signal is generated when the close crosses the SAR line from above.
When a signal is generated, a new cycle begins and the SAR line is recalculated making a
sudden jump from its previous value. For example, when a buy signal is generated, the new
SAR line will restart some distance below the close, calculated with a particular algorithm.
After that, the SAR line tracks the close price according to another particular algorithm. The
two algorithms previously mentioned may change but the logic of the trading rule remains the
same.
The most known SAR indicator is the Parabolic Stop and Reverse [6] where the initial distance
between the close and the SAR line is a parameter and the SAR line tracks the close price at
an increasingly faster pace until a reverse happens.
Figure 25 shows a SAR Line applied to the S&P500 index, with buy signals in green circles and
short signals in red circles. The rules of this particular SAR can be consulted in [41] and they
will be the ones used in this work. The main difference from the Parabolic SAR is that the
Line doesn’t accelerate towards the price and it adjusts to the volatility of the asset. The
genes using this SAR line have the ID strings “SAR_S” and “SAR_L” for short and long genes
respectively, see Appendix C – Technical Primitive Genes.
Figure 25 – SAR Line applied to the S&P500 index
52
3.3.6 Composite indicators
So far all the mentioned rules are based on a single indicator but it is possible to combine
several indicators into a composite indicator. Each of the single indicator rules focus on a
particular characteristic or behavior of the price, like momentum, volatility, price action and
when a trend forms, all give a signal, some sooner and some later. All of them will also produce
false signals some of the time that result is losses. The advantage of combining several
indicators into a single rule is to filter some of the individual false signals by waiting for the
consensus of at least the majority of the individual indicators.
A practical way to achieve this result is to use majority vote on the outputs of the single
indicator rules. For example, if we use 3 single indicator rules and apply majority vote, we have
the results observed in Table 7. Another option is to only validate a signal when all 3 inputs
have a signal, which corresponds to being more selective.
Single Indicator Rule 1 neutral signal neutral signal neutral signal neutral signal
Single Indicator Rule 2 neutral neutral signal signal neutral neutral signal signal
Single Indicator Rule 3 neutral neutral neutral neutral signal signal signal signal
Composite Output neutral neutral neutral signal neutral signal signal signal
Table 7 – Logical table of using Majority Vote on 3 single indicator rules
This majority vote operation is performed by genes with ID string “MVOTE3” and “MVOTE5”
corresponding to 3 and 5 inputs respectively. These genes have a single parameter which is
the number of active inputs necessary to validate the output.
There are two composite genes which use this majority vote technique. The first is a long-term
only gene without optimizable parameters, with ID string “TLP_100223_S” and
“TLP_100223_L” for the short and long versions. This composite rule is the result of previous
preliminary tests performed by the author. Internally, there are 3 single indicator rules:
Relative Volatility Index in the configuration presented in 3.3.4. The RVI period is 52
and the thresholds are 42 and 58.
Simple Moving Average with a band crossed by the close, as presented in 3.3.1. The
SMA period is 290 and the bands are separated by a distance equal to 2.5 times the
Average True Range [6] of period 60.
Based on the Aroon indicator [6]. Two 190-day SMAs are calculated from the Aroon up
line and from Aroon down line. The Aroon period is 16. Signals are generated when
the two SMAs cross.
The second composite gene is medium-term only, without optimizable parameters, with ID
string “TMP_110112_S” and “TMP_110112_L” for the short and long versions. This gene is the
result of preliminary tests done by the author. Internally it has 3 single indicator rules:
53
Relative Volatility Index in the configuration presented in 3.3.4. The RVI period is 35
and the thresholds are 48 and 52.
Simple Moving Average with a band crossed by the close, as presented in 3.3.1. The
SMA period is 60 and the bands are separated by a distance equal to 2.5 times the
Average True Range [6] of period 60.
Based on the Dynamic Momentum Index (DMI) [6] in an equivalent configuration used
for the RVI presented in 3.3.4. The DMI uses a period of 14 and the result is smoothed
by a SMA of period 30. The threshold levels used to generate signals are 48 and 52.
3.4 Chapter Conclusions
In this chapter the global solution was presented. The detailed representation of solutions
used in the Genetic Programming was explained, using a structure with Genome -> Sub-
systems -> Chromosomes -> Genes. An explanation was given on why the solutions will only
use the market pattern of trends and how this will be enforced. A fitness function which gives
importance to both return and risk was also presented. Some specific details were given on
how the data is organized and the trading signals are managed (using a vector organization).
There was an important goal when the architecture of the proposed solution was developed,
which was to have a tool with great flexibility, allowing the simulation of many different types
of setups and not being limited by technical constraints. As the work later developed, practical
decisions were made relative to the solutions used and only a fraction of the potential of this
solution was actually used.
Several technical indicators employed in this work were addressed in this chapter as well as
some algorithms that can be created using those indicators, in order to generate trading
signals. Those algorithms detect trends and generate trading signals that allow profits to be
made by following the trends.
54
55
4 Implementation In this section detailed information is given on the setups used to produce the results and how
they were implemented.
The trading rules were optimized using genetic programming coded in C++. All code was
developed with the help of the Integrated Development Environment (IDE) “Qt Creator”
version 3.3.0 (based on Qt version 5.5.0) [42]. Library MathGL version 2.3 [43] and library FLTK
1.3.3 [44] were used to produce charts with results and the Boost C++ library was used for
several auxiliary tasks [45]. The code was developed and run on a Linux machine with a core
i3 2350M CPU, 4GB of RAM and running Ubuntu Desktop 14.10 LTS 64 bits [46].
Depending on the setup used, the optimization of trading rules for the 30 assets took between
8 and 30 hours. The simulation of portfolios only takes a few seconds since all trading rules
have already been optimized and all the trading signals were saved to files.
4.1 Assets
It was decided to use classes of assets that tend to produce good results with trend following
strategies: stock indices and commodities. The selection method was composed of several
steps. Firstly, stock indices from all major markets with quality data available were pre-
selected as were all major commodities with quality data. A total of 34 stock indices and 18
commodities were obtained and from this group the final selection was made. The criteria
used for the stock indices was to select the major ones from each continent. From Europe, all
the EURONEXT indices (CAC40, BEL, AEX and PSI20), the British FTSE 100 and the German DAX.
In North America, the S&P500 and NASDAQ 100 from USA, the TSX from Canada and the IPC
from Mexico. In South America only the BOVESPA from Brazil and MERVAL from Argentina
were available and were selected. From Oceania the Australian AORD was chosen and from
ASIA 6 major indices were selected including the Japanese Nikkei. A total of 19 indices were
selected which are listed in Table 18 from Appendix A - Used Assets. To select the
commodities, the used criteria was the trade volume [47] amongst the ones available in the
pre-selection. The four most traded commodities from each of the following categories were
selected: Energy, Metals and Agriculture. (Only 3 options were available from the Energy
category). A total of 11 commodities were selected and are listed in Table 19 from Appendix
A - Used Assets.
The assets historical data was collected from several sources and include values for Open,
High, Low, Close and Volume. Volume wasn’t always available with quality and is not used in
the trading rules. The majority of data from stock indices was collected from “Yahoo Finance”
[48] using the software tool “ML Downloader” [49], saved in Comma Separated Value format
(csv). Data was visually inspected with software “MetaStock” [50] in order to detect possible
56
errors. When important errors were found and it was not possible to correct them, the asset
was discarded from the pre-selection.
All the Commodities historical data was obtained from the website “Quandl” [51] and consists
of continuous futures contracts created from individual contracts. The creation method used
was Backwards Ratio Adjusted, with Rollover on the first month. Some commodities had data
which wasn’t good enough in terms of quality (only close information in some periods) and
those commodities were discarded. Open interest was ignored.
4.2 Cache
Since many genomes have repeated chromosomes and genes, many calculations would be
repeated when the genes, chromosomes, sub-systems and genomes are interpreted. To
increase the efficiency of the optimization, a cache for vectors was implemented and is
described in Appendix D – Cache of vectors.
4.3 Optimization parameters
The optimization of an asset in successive windows is performed with a set of base parameters
which is constant unless stated otherwise. Some of these parameters are addressed next.
4.3.1 Window sizes
After preliminary tests with different sizes for the In-sample window (optimization) and the
out-of-sample window (test), a size of 3 years was adopted for the in-sample window and a
size of 3 months for the out-of-sample window. If the in-sample window is too small, it will
probably only capture part of the market patterns (only an up-trend, only a down-trend or
only a sideways market) and the best optimized solutions will not be prepared to trade in all
market conditions. On the other hand, if the window is too big, the best optimized solutions
will probably not be well adapted to the current market conditions. There seems to be a trade-
off between the advantages and disadvantages of bigger and smaller in-sample windows and
3 years is a good compromise based on preliminary tests using different window sizes. For the
out-of-sample window, theoretically, there is an advantage of it being small since it means
that the best solutions are re-optimized more frequently and will be more adjusted to the
current market condition. Because of processing power limitations, the out-of-sample window
used will be 3 months and not smaller. Changing the windows sizes affects the results but not
dramatically meaning results are always positive and in the same order of magnitude.
There is a third window not mentioned so far, the warm-up window, which corresponds to
asset data that must exist before the in-sample window. It is required by indicators that need
a certain amount of data to calculate its first value. For example, a simple moving average of
57
100 days, needs 100 days to calculate the first value. The warm-up window has a fixed size of
504 bars/days which corresponds to around 2 years in trading days. The existence of this
window doesn’t affect the qualitative comprehension of the presented solution and is a
technical detail.
4.3.2 Size of the population of solutions
The first generation has a higher number of genomes since they are randomly created and it’s
beneficial that those initial solutions cover the search space as much as possible. For this
reason, 300 genomes are initially created and only 100 are used in subsequent generations. A
lower number of genomes would compromise diversity and a higher number didn’t show a
significant improvements in results and would increase the computational effort. The best 2
genomes at the end of a window optimization are also copied to the initial population of the
next window. Since the two windows overlap a big percentage, it is probable that the best
solution in a particular window will be a good starting point on the next window.
4.3.3 Method of chromosome creation
The chromosomes in the initial genomes are created 30% with the full method [24] and 70%
with the grow method [24]. The grow method presents better results with most problems [23]
and for that reason it’s used in a higher percentage. This parameters did not show a relevant
impact in the results obtained.
4.3.4 Stopping criteria
The optimization of a window of data consists in successive generations through which the
population evolves, and there must be a point where the optimization is stopped according to
some criteria. Usually the initial generations produce a fast increase in the fitness score of the
best elements, and as more generations pass, the rate of increase diminishes. Sometimes the
optimization gets stuck in a local maxima for some time until a new successful
mutation/crossover produces a sudden increase in the fitness score and the improvement
resumes. There is no perfect stopping criteria and there is always the chance that the
optimization is stopped before an important improvement. But since the optimization can’t
go on forever, the most important is to have a stop criteria that allows for the solution to be
close to the optimal solution most of the time. Letting the optimization go on for too long can
also be counter-productive since it may produce over fitting.
A first condition that must be verified for the optimization to stop is a minimum number of
completed generations, which is set to 150. After this point, the optimization stops if the
58
increase in the fitness score is less than 5% in the last 150 generations. Both parameters were
obtained empirically based on repeated testing and are not critical to the results obtained.
4.3.5 Penalizing bigger solutions
In optimizations where the chromosome trees are allowed to have more than 1 level,
genomes which have chromosomes with higher tree depths receive a penalization in the
fitness score. The aim is to allow more complex solutions if they produce better results but
put some pressure towards simpler solutions if the results are similar. The used method is to
give cumulative penalizations to a genome for each chromosome they have with a higher
depth. The penalizations grow arithmetically from 0% for a chromosome with depth of 1 to
10% for the maximum allowed depth. For example, in an optimization that uses chromosome
trees with a maximum allowed depth of 3, the possible penalizations are: 0% for depth 1, 5%
for depth 2 and 10% for depth 3. Chromosomes created with more than 3 levels would be
deleted it this example.
These penalizations are not essential to any of the results obtained and are used mainly to
decrease the complexity of the solutions and decrease the computational effort.
4.4 Individual Simulations (optimization)
A simulation consists in using a vector of daily trading signals to simulate the result of following
those signals when trading an asset. The vector of trading signals can be generated by a
chromosome, a sub-system of a complete genome and the simulation can be performed on a
limited window of data or on the entire asset data. The daily vector of trading signals is created
when a chromosome, sub-system or genome is interpreted using data from an asset.
The main parameters used by the simulation function are always the same (unless stated
otherwise) and will be discussed next.
To annualize some of the performance metrics, like the percentage return, it is considered
that one year has 252 trading days on average.
The start and end dates of the simulations differ depending on the window of data.
4.4.1 Fixed capital
All simulation are performed with a fixed capital that remains the same throughout the
simulation. This means that there is no reinvestment of the profits and losses do not affect
the capital either. The main reason for using this method is to have gains and losses that are
always proportional to a fixed capital and that can easily be compared throughout the
simulation. The returns are therefore linear and not exponential and allow an easier analysis.
59
One counter-intuitive consequence of using a fixed capital is that the maximum drawdown,
when measured as a percentage of the invested capital, can be greater than 100%.
The fixed capital used in the simulations of individual assets is 1000000 monetary units.
4.4.2 Maximum Drawdown
In all simulations the absolute drawdown is calculated using the daily vector of cumulative
returns. This absolute drawdown is then compared with the fixed invested capital resulting in
a percentage value. This value is designated as Maximum Loss percentage (ML%) and is the
maximum drawdown of the returns as a percentage of the fixed capital invested, equation (
14 ).
𝑀𝐿% =𝑀𝑎𝑥𝑖𝑚𝑢𝑚 𝑑𝑟𝑎𝑤𝑑𝑜𝑤𝑛 𝑜𝑓 𝑟𝑒𝑡𝑢𝑟𝑛𝑠
𝐹𝑖𝑥𝑒𝑑 𝐼𝑛𝑣𝑒𝑠𝑡𝑒𝑑 𝐶𝑎𝑝𝑖𝑡𝑎𝑙 ⋅ 100
( 14 )
4.4.3 Position sizing
The cost of acquiring one share/contract/unit of an asset is considered to be its price in
monetary units. For example, if the S&P500 index is quoted at 1200, the cost of buying or
shorting one contract of the index is 1200 monetary units (US Dollars in this example).
Leverage is always 1 meaning the invested money will never be greater than the capital
available (there is no money borrowing). The formula to calculate the number of
shares/contracts/units of the asset to buy or short is given by equation ( 15 ).
𝑢𝑛𝑖𝑡𝑠 = 𝐹𝑙𝑜𝑜𝑟 (
𝑀𝑎𝑟𝑘𝑒𝑡 𝑝𝑜𝑠𝑖𝑡𝑖𝑜𝑛 ⋅ 𝐶𝑎𝑝𝑖𝑡𝑎𝑙
𝐴𝑠𝑠𝑒𝑡 𝑃𝑟𝑖𝑐𝑒) ( 15 )
Where:
𝑴𝒂𝒓𝒌𝒆𝒕 𝒑𝒐𝒔𝒊𝒕𝒊𝒐𝒏 Is the percentage of the capital to be invested, according to the trading
rules and the information is contained in the trading signal vector. Its values are in the range
[-1.0, 1.0].
𝑪𝒂𝒑𝒊𝒕𝒂𝒍 Is the fixed capital used in the simulation.
𝑨𝒔𝒔𝒆𝒕 𝑷𝒓𝒊𝒄𝒆 Is the price of the asset.
4.4.4 Commissions and Slippage paid
In every trade (opening or closing) a commission of 0.1% of the trade value is deducted. This
means a complete trade consisting in opening and closing will result in a total commission paid
of 0.2% of the trade value. No slippage is considered.
60
4.4.5 Position size recalculation
Since some trades can last for a long time (months or years) it is possible that the asset prices
change considerably in that period meaning the trade size no longer respects the desired
capital (since simulations use fixed capital). To minimize this effect, when prices increase or
decrease more than a threshold, the ideal size of the trade is recalculated using equation ( 15
) and the adjustment is made. The threshold used for this recalculation is 10%. This value
doesn’t have a critical impact in results.
4.5 Portfolio Simulations
Portfolio simulations are performed using all available assets, using the trading signals
generated by the best genomes obtained from the individual optimizations. The parameters
used are always the same unless stated otherwise. A total fixed capital of 100 million monetary
units is used and divided by the individual assets. A commission of 0.1% is used by trade (open
+ close = 0.2%) and slippage is ignored. It’s considered that a year has 252 trading days and
this value is used to annualize certain performance metrics.
4.5.1 Time synchronization of asset data
Each asset is traded in a particular market in a particular country with specific holidays. This
means that there are assets which are not traded in some days and those days differ from
asset to asset. When an asset is not traded in a particular day, there is no information for that
day in the historic data. This is very important in portfolio simulation since all assets are traded
at the same time and results must be synchronized. To deal with this issue, data vectors from
all assets and the vectors with trading signals are synchronized before the simulation. The
synchronization consists in inserting the missing weekdays in the vectors which assume the
values of the previous trading day.
4.5.2 Equal weight portfolio
It is important to have a reference portfolio to compare the results and the buy-and-hold
(B&H) strategy is frequently used as the reference. It’s the opinion of the author that the B&H
is a very limited strategy which only makes sense in assets with an upward bias like stocks and
stock indices. Even in those assets, the B&H is very sensitive to the chosen window: If the asset
is in a strong uptrend, it’s almost impossible to beat the B&H and if the asset is in a strong
down trend, the B&H strategy surely has a very poor performance and it’s probably very easy
to beat it. It’s thus a poor reference.
61
Still, graphically visualizing a portfolio with equal weights (assets always bought) can help
understand what happened with the assets in the test period (from 2006 to 2014) and serve
as a reference to the tested portfolios. This equal weight portfolio is not technically a buy-and-
hold portfolio since simulations are performed with fixed capital and periodic readjustments
are made in the individual asset positions as described in 4.4.5.
4.6 Chapter Conclusions
In this chapter many practical implementation details were addressed relative to the
optimization algorithm, the simulation of the trading systems evolved and the creation of
portfolios.
62
63
5 Results This section presents the results of the individual optimizations using several setups. These
results are then analyzed and interpreted. The solutions obtained by the best setups are then
simulated in portfolios will all assets traded simultaneously. The setups use the parameters
specified earlier unless stated otherwise.
5.1 Format of the results
Two types of results are presented in this chapter. The first type are results obtained from
individual asset optimizations. In most cases, the individual results are averaged to obtain a
global indication of the effectiveness of the setup. In one setup, individual results are
presented for each asset. The second type of results presented comes from the simulation of
portfolios using the individual trading signals optimized for the assets.
The performance metrics contained in the tables are explained next:
aRet % - Is the annualized return in percentage of the invested capital (which is fixed).
Avg Trade % - Is the average gain or loss per trade in percentage.
Avg Bars – Is the average duration of trades in bars (days).
Avg W/L – Is the ratio between the average winning trade and the average losing trade.
Win % - Is the percentage of winning trades.
Avg Loss % - Is the average daily drawdown in percentage.
Max Loss % or ML% - Is the maximum percentage Loss as described in 4.4.2.
Daily StdDev – Is the daily Standard Deviation of daily returns.
Lake Ratio 2 – Is the inverse of the Lake Ratio.
RRR – Is the Risk Return Ratio.
Fitness or JR4 – Is the fitness score.
aRet% [year] – Is the annualized percentage return in a particular year.
In the bottom of the table, there is a row which averages the results of all assets and works as
a summary of the setup being simulated. There is also a bottom row which shows the standard
deviation of the results from the different assets and is a measure of the dispersion of these
results.
64
5.2 Setup with a large search space
For the first test, a setup that allows high optimization freedom was chosen. The range of
genes are all those in Table 20 with the exception that only the “AND” and “OR” logic
operators are used and genes specific to one time frame are prohibited in other time frames.
The genomes have two subsystems, one for the long-term time frame and another for the
medium-term. Each sub-systems contributes with a weight of 50% to the final output trading
signals. Each of these subsystems uses two chromosomes, one to generate short signals and
the other to generate long signals (buy). Each chromosome can have a maximum of 3 levels
and increasing penalizations are applied to genomes with chromosomes with depths greater
than 1 (according to the rules previously mentioned). The Results are presented in Table 8.
Avg Avg Avg Avg Max Daily Lake Fitness
aRet% Trade% Bars W/L Win % Loss % Loss % StdDev Ratio2 RRR JR4
Average of all assets results
1.39 0.24 22.88 1.73 44.5 -18.8 -43.6 0.00817 1.40 0.069 0.057
Std. Dev. of all asset results
4.22 0.64 4.62 1.28 6.9 8.3 15.1 0.00174 2.61 0.142 0.118
Table 8 – Results from optimization with all operators (3 levels)
The first conclusion that can be drawn is that the overall results are very poor, with an average
annualized return of only 1.39% which means the trading signals are not very far from random.
There are several possible explanations for these results, one of them is that the big search
space allows for curve fitting, resulting in non-sense solutions that work well in the
optimization window (in-sample) but poorly in the test window (out-of-sample). This can be
exemplified with the following observed short chromosome:
IFB (Boolean IF operator)
C<EMA (143) (IF condition)
C<SMA (184) (the output if condition true)
>= (the output if condition false)
RVI (214)
CLOSE
The main problem with this chromosome is the comparison between the RVI and the close
price. Since the RVI oscillates between 0 and 100, it should never be compared with the close.
It’s therefore an undesirable non-sense chromosome.
Another issue with this setup is that it allows for solutions that are not trend-following, in
particular, mean-reversing solutions appear in some periods, meaning that the genome will
short when the price rises and buy when the price falls. This may produce good results in a
particular optimizing window but will probably fail in the corresponding test window.
Setups that bypass or mitigate these issues must therefore be explored.
65
5.3 Setup using only Boolean genes (3 levels)
Using genes with inputs/outputs of type Double allow for many undesirable rules like in the
example shown in 5.2. In this new setup, the allowed genes are restricted to the ones with
Boolean inputs and outputs. In particular the following genes were used, “AND”, “OR”,
“MVOTE3”, “IFB” and all the “Technical Primitives”. This setup should not only reduce the
search space, but also concentrate on trend-following solutions since the Technical Primitive
genes are already mini-systems with rules that favor that trading philosophy.
All the remaining parameters and formats are identical to the previous setup presented in 5.2.
Looking at the results presented in Table 9, we can observe that the average of the annualized
returns, 3.91%, is considerably higher than the value observed in the setup with a larger search
space (5.2). Still, the Boolean If gene (“IFB”) and the 3 levels permitted for the chromosome
tree, allow for complex solutions that may represent over fitting and degrade the out-of-
sample results. The path of further reducing the search space should therefore be explored.
Avg Avg Avg Avg Max Daily Lake Fitness
aRet% Trade % Bars W/L Win % Loss % Loss % Std Dev Ratio2 RRR JR4
Average of all assets results
3.91 0.59 25.29 1.97 44.9 -16.3 -44.9 0.00915 2.06 0.118 0.102
Std. Dev. of all asset results
4.45 0.66 8.86 0.88 9.8 6.6 20.2 0.00223 2.29 0.139 0.119
Table 9 – Results from optimization with Boolean operators (3 levels)
5.4 Setups with restricted Boolean genes
In order to find the best combination of maximum depth level and gene pool, several setups
were tested for which summary results will be presented. Half of those setups use
chromosome trees with a maximum of 2 levels and the other half use chromosome trees with
only 1 level. The setups with 2 levels include the operators AND, OR and MVOTE3 but not the
setups with 1 level since it’s not possible. Setups with increasingly less technical primitive
genes were tested, progressively eliminating those which in preliminary tests showed worse
results individually.
Starting with a maximum tree depth of 2, the following setups were tested (only genes used
in long chromosomes are shown):
Setup 3 with genes: AND, OR, MVOTE3, C>EMA, C>SMA, C>WMA, C>SMAB, RVI>X,
MACD>0, SAR_L, CH_BRK_L, TLP_100223_L, TMP_110112_L, TMP_090212_L,
MT_OSC_L, RSI>X, STOCH_D_s>X.
66
Setup 4 with genes: AND, OR, MVOTE3, C>SMAB, RVI>X, MACD>0, SAR_L, CH_BRK_L,
TLP_100223_L, TMP_110112_L, TMP_090212_L, MT_OSC_L.
Setup 5 with genes: AND, OR, MVOTE3, RVI>X, MACD>0, SAR_L, TLP_100223_L,
TMP_110112_L, TMP_090212_L.
Setup 6 with genes: AND, OR, MVOTE3, RVI>X, MACD>0, TLP_100223_L,
TMP_110112_L, TMP_090212_L.
Avg Avg Avg Avg Max Daily Lake Fitness
Setup aRet% Trade % Bars W/L Win % Loss % Loss % Std Dev Ratio2 RRR JR4
Setup 3 2.81 0.46 24.84 2.05 43.0 -17.2 -40.4 0.00921 1.80 0.091 0.078
Setup 4 4.74 1.18 44.48 3.47 34.5 -20.4 -48.6 0.01132 2.18 0.128 0.112
Setup 5 4.87 1.14 41.35 3.50 32.9 -21.0 -48.2 0.01140 2.00 0.124 0.109
Setup 6 6.94 2.15 54.48 3.11 37.1 -25.3 -59.3 0.01365 2.42 0.141 0.127
Table 10 – Average results for setups with depth 2 and Boolean genes.
Table 10 presents the average results from all assets and as can be seen, the average returns
increase as the pool of genes is reduced to those which presented best results individually.
The increase in returns may be explained by the reduced search space and complexity of the
solutions, producing less curve-fitting. Another possible explanation is related with the
characteristics of the removed genes: some of them generate trades when there is a cross
between the price and a moving average (ex: C>SMA) or between an indicator and a threshold
(RSI>X). And since the close prices and indicators have noise, this generates successive trades
alternating long and short positions which generates losses.
In practice the structure of the chromosomes will either have a single technical primitive gene
or will have “AND”, “OR” or “MVOTE3” in the root level and technical primitives as terminals
in the second tree level. See example of a long chromosome:
MVOTE3 (3) (Output true when 3 inputs are true)
RVI>X (50,48,52)
MACD>0 (20,50)
SAR_L (3.2)
The next setups tested use a maximum tree depth of only 1 level meaning that only technical
primitives exist in the gene pool. They approximately replicate the previous setups (3 to 6) in
terms of the technical primitive genes used. The following setups were tested (only genes used
in long chromosomes are shown):
Setup 7 with genes: C>EMA, C>SMA, C>WMA, C>SMAB, RVI>X, MACD>0, SAR_L,
CH_BRK_L, TLP_100223_L, TMP_110112_L, TMP_090212_L, RSI>X, STOCH_D_s>X.
67
Setup 8 with genes: C>SMAB, RVI>X, MACD>0, SAR_L, CH_BRK_L, TLP_100223_L,
TMP_110112_L, TMP_090212_L.
Setup 9 with genes: RVI>X, MACD>0, SAR_L, TLP_100223_L, TMP_110112_L,
TMP_090212_L.
Setup 10 with genes: RVI>X, MACD>0, TLP_100223_L, TMP_110112_L,
TMP_090212_L.
Avg Avg Avg Avg Max Daily Lake Fitness
Setup aRet% Trade % Bars W/L Win % Loss % Loss % Std Dev Ratio2 RRR JR4
Setup 7 6.31 1.59 38.63 2.79 37.9 -22.8 -55.0 0.01352 2.55 0.151 0.133
Setup 8 6.09 1.59 41.44 2.57 37.6 -27.1 -62.8 0.01391 2.06 0.119 0.107
Setup 9 6.89 1.99 45.67 2.42 40.4 -27.4 -62.9 0.01382 2.45 0.142 0.128
Setup 10 8.08 2.28 48.33 2.71 40.2 -24.3 -58.7 0.01429 3.04 0.171 0.153
Table 11 – Average results for setups with depth 1 and Boolean genes.
Table 11 present the average results from all assets and, as was observed with the setups of
Table 10, the average returns tend to increase as the worst performing technical primitive
genes are removed from the pool. There is an increase in the average number of bars per
trade corresponding to less trades and less reversals. This means the trading rules are more
stable and more immune to noise, changing position only when there is a stronger indication
of a trend change. The same progressive increase can be seen in the average trade return.
Given this results, a more detailed analysis will be made of setup 10.
5.5 Setup 10 – Best Genes with 1 level depth
As was previously shown, the average results tend to increase when the search space and
complexity of solutions is reduced, probably due to a decrease in curve-fitting and the effect
of removing worse performing genes. Setup 10 uses two chromosomes, one for the long time
frame and one for the medium time frame. Each of these chromosomes produce both long
and short trades. The specific genes used for each chromosome/time frame are detailed next
since some of them are time frame specific:
Long-term time frame: RVI>X, MACD>0, TLP_100223_L;
Medium-term time frame: RVI>X, MACD>0, TMP_110112_L, TMP_090212_L.
Table 12 presents the individual results for all assets as well as the averages and standard
deviation at the bottom. These are averaged results from 8 different optimization runs.
To graphically exemplify the results produced by this setup the graphs of three assets are
presented in Figure 26, Figure 27 and Figure 28. The horizontal scales shows the dates and the
vertical scales shows the percentage return. Each figure has a black line representing the asset
price percentage variation and a red line representing the percentage return. On the top,
68
three sub-windows represent the trading signals: Medium Term (MT), Long Term (LT) and
Final (F). Each of these trading signals range from -1.0 to 1.0 corresponding to being 100% long
or 100% short, respectively. The Final trading signal is calculated by averaging the MT and LT
signals with equal weights of 0.5.
Figure 26 shows the results of one optimization with setup 10 in the S&P500 index from 2006
to the end of 2014. We can observe the returns follow the price variation closely until mid-
2008, when the price starts to fall violently and the trading system profits by being short. This
results in a very positive performance in the 2008 bear market. Between 2010 and 2012
S&P500 had an upward bias but with some corrections which translated into the trading
system having difficulties, alternating periods of profits and losses. After 2012, the index
entered a period of steady gains and the trading system is able to consistently profit again. In
summary, the trading system was able to profit when the market had a clear direction and
was able to, not only prevent big losses in the 2008 bear market, but strongly profit in that
period. The average annual return of the trading system in the 9 year period was around 9%.
Figure 26 – Results for S&P 500
69
Avg Avg Avg Avg Max Daily Lake Fitness aRet% aRet% aRet% aRet% aRet% aRet% aRet% aRet% aRet%
Asset aRet % Trade% Bars W/L Win % Loss % Loss % StdDev Ratio2 RRR JR4 2006 2007 2008 2009 2010 2011 2012 2013 2014
CAC -0.82 -0.20 47.07 2.02 31.64 -38.55 -87.76 0.01307 0.608 -0.009 -0.009 15.61 -3.94 47.00 0.07 -31.86 -21.30 2.28 -1.39 -16.43
AEX 2.21 0.52 43.91 2.07 37.40 -38.23 -77.11 0.01253 0.733 0.029 0.027 11.10 -3.57 62.95 -6.83 -29.09 -17.44 5.94 5.77 -5.92
BEL 6.18 2.11 64.48 2.60 42.82 -23.66 -54.94 0.01160 2.150 0.117 0.107 21.16 -17.83 64.99 2.97 -15.71 -8.87 -7.02 4.91 1.05
PSI 7.96 1.85 40.15 2.27 44.12 -20.63 -55.23 0.01112 2.971 0.152 0.139 28.05 15.83 42.62 1.41 -19.57 -3.46 -7.16 0.87 7.32
DAX 4.70 1.29 51.07 2.18 41.58 -25.49 -61.38 0.01303 1.603 0.085 0.078 16.92 16.47 22.60 -10.57 1.24 -23.72 12.95 19.18 -15.83
FTSE -1.23 -0.33 49.97 1.22 40.63 -19.78 -41.49 0.01082 0.169 -0.027 -0.024 4.62 -4.14 3.13 9.31 -11.62 -13.65 0.60 5.34 -11.96
SPX 9.26 3.06 66.41 2.35 54.63 -7.43 -27.24 0.01210 6.067 0.342 0.289 11.13 -1.58 25.71 23.59 -0.78 -12.29 7.85 27.23 1.76
NDX 6.64 1.77 47.98 2.26 44.52 -11.31 -35.68 0.01204 1.686 0.187 0.164 -13.17 1.93 14.98 10.42 16.66 -18.30 13.20 28.39 0.71
TSX -0.42 -0.10 43.96 1.84 34.33 -19.30 -38.34 0.01023 -0.180 -0.009 -0.008 8.40 -28.74 14.78 -12.64 3.49 1.60 -5.45 4.92 2.52
IPC 7.03 1.86 44.17 3.21 36.17 -12.38 -34.98 0.01127 4.047 0.202 0.177 30.49 0.86 6.66 24.60 3.60 -15.03 6.64 -2.91 4.21
AORD 2.34 0.63 47.15 2.17 36.69 -24.54 -63.61 0.00992 1.318 0.041 0.038 10.87 2.05 40.28 16.83 -12.63 -31.17 -8.79 6.17 -2.36
NIKKEY -2.30 -0.52 38.78 2.07 28.74 -51.58 -105.29 0.01393 -0.271 -0.021 -0.020 2.28 0.42 25.34 -40.84 -9.89 -19.11 2.42 27.67 -2.36
HSI 4.62 1.21 41.67 2.70 35.77 -20.67 -49.24 0.01296 2.645 0.102 0.092 25.96 26.01 -2.89 18.71 0.54 -10.54 0.33 -8.23 -8.29
STI 8.45 1.75 37.36 3.53 37.27 -12.73 -30.17 0.01026 4.981 0.287 0.245 22.03 9.72 42.61 20.63 -6.20 -16.64 7.82 1.53 0.09
SENSEX 9.91 2.52 45.82 2.86 43.70 -20.91 -49.95 0.01338 3.216 0.203 0.184 38.75 41.23 1.88 16.50 4.02 -32.27 -5.91 2.42 24.25
JKSE 15.04 3.87 45.00 3.00 48.76 -12.14 -32.60 0.01235 7.975 0.463 0.401 29.61 18.00 19.73 42.75 29.35 -6.32 3.57 0.61 -4.73
TWII 11.15 2.70 42.87 4.85 35.30 -11.39 -34.01 0.01046 7.363 0.330 0.287 13.49 15.45 51.94 35.37 5.88 -11.83 -0.82 -11.82 -2.85
MERV 20.79 6.51 58.93 3.64 43.51 -25.10 -84.35 0.01768 3.098 0.265 0.249 23.54 0.51 36.88 42.19 20.38 -27.13 -13.87 51.00 68.21
BVSP 3.70 0.94 42.37 1.73 43.94 -31.00 -62.83 0.01487 1.243 0.061 0.057 16.62 33.41 3.97 7.63 -3.70 -7.88 -11.63 0.54 3.83
B1_FR 25.82 6.29 47.22 4.42 46.87 -12.96 -56.66 0.01809 9.299 0.456 0.419 4.34 42.60 102.95 18.07 -25.24 11.95 21.27 5.70 61.14
HO1_FR 16.15 5.07 60.85 3.43 41.53 -37.29 -79.26 0.01690 2.297 0.206 0.194 4.77 15.59 126.22 -38.04 15.34 -3.71 12.62 -19.23 41.60
NG1_FR 25.52 7.71 55.58 3.37 46.68 -30.91 -89.68 0.02828 5.536 0.298 0.281 43.54 23.22 93.81 57.83 4.05 15.61 6.10 -28.45 -8.16
GC1_FR 1.15 0.32 50.51 1.45 43.71 -33.89 -61.23 0.01166 0.301 0.022 0.020 9.68 19.10 -25.40 -12.76 18.06 2.74 -12.85 16.11 -3.38
XAGUSD 11.27 3.02 47.31 3.16 38.97 -35.41 -66.67 0.01992 1.554 0.170 0.158 16.59 -0.35 -6.77 17.69 55.38 -11.03 -3.69 15.84 11.25
PL1_FR 4.50 1.17 51.54 2.91 33.26 -25.70 -70.87 0.01330 1.915 0.064 0.059 2.94 23.14 40.33 11.26 12.21 -38.73 -6.27 -1.87 -3.29
PA1_FR -2.76 -0.59 36.36 2.90 23.57 -54.83 -114.19 0.01747 0.073 -0.022 -0.022 -12.95 -2.76 6.73 40.12 57.05 -46.77 -33.19 -11.61 -5.00
S1_FR 3.18 0.70 37.36 2.67 31.30 -17.85 -44.18 0.01403 1.689 0.073 0.066 10.78 13.38 14.50 10.90 20.77 -17.48 21.66 -5.98 -19.78
SM1_FR 15.27 4.03 52.21 2.92 46.35 -9.91 -43.49 0.01618 7.494 0.355 0.318 -7.82 29.87 23.16 38.50 3.54 3.84 43.79 17.10 -6.07
W1_FR 3.14 0.78 40.80 2.15 35.58 -29.42 -72.23 0.01956 0.785 0.046 0.042 -16.34 10.56 43.44 -14.97 -7.95 9.93 0.79 13.12 -12.10
SB1_FR 23.99 8.44 71.06 3.29 56.88 -13.39 -37.60 0.01968 8.707 0.650 0.571 31.00 3.39 24.85 22.00 99.08 -6.62 -6.22 21.63 29.48
Average 8.08 2.28 48.33 2.71 40.21 -24.28 -58.74 0.01429 3.036 0.171 0.153 13.47 9.99 32.30 11.76 6.55 -12.52 1.57 6.15 4.30
Std Dev 8.12 2.43 8.82 0.81 7.27 12.04 22.67 0.00401 2.837 0.169 0.150 14.77 16.16 33.26 22.81 27.10 14.42 13.72 15.92 20.96
Table 12 – Results from optimization with setup 10 – best genes with tree depth 1
70
Figure 27 shows the graphical results of one optimization in Brent, which is as asset that
produces very good results with the trading rules generated with this method and setup. We
can observe the returns follow the price change until mid-2008 where Brent initiates a very
sharp fall. The system quickly reverses to a short position and strongly profits from the fall
that ends in the beginning of 2009. After that, Brent has a more erratic behavior despite having
an upward bias, and the trading system only manages a small gain in that period, until mid-
2014. After mid-2014 Brent starts a very strong fall and the system is again able to strongly
profit from the movement. In average, the trading system was able to gain 27% per year. In
this case it becomes very clear that the system specializes in following strong trends and earns
most of its profits from those movements, resulting in a return curve that can be almost flat
for several years and then have sudden jumps. This means it’s wise to use these type of
systems in a portfolio in order to have a smoother return curve and less risk.
Figure 27 – Results for Brent
Figure 28 shows the example of Palladium, which has a chart that is harder to be traded by
the trading systems generated by this method/setup. The system is able to profit from the
strong trend that starts in 2009 and ends in the beginning of 2011 but, in the remaining years
Palladium trades in a price range with no defined trend and the returns fall consistently. In the
entire period, the return is close to 0 and palladium is a case of an asset which was not
successfully traded by a system generated by this method/setup.
71
Figure 28 – Results for Palladium
5.5.1 Example Genome from S&P500
As an example, a genome is presented from the optimization of S&P500 with setup 10. It is
the best genome of generation 150 from the last optimization window, corresponding to the
period between 2011/10/04 and 2014/10/06.
Subsystem 0 (Long Term - Weight = 0.5)
Chromossome 1
MACD>0 (186, 400)
Subsystem 1 (Medium Term - Weight = 0.5)
Chromossome 3
RVI>X (37, 48.7321, 53.7321)
As was previously mentioned, setup 10 has two subsystems, one focusing on the long term
time frame (0) and the other on the medium term time frame (1). Each subsystem has a single
chromosome which generates both long and short signals. Chromosome indices are unique
and can have any order depending on what is defined in the configuration files (in this case
the chromosomes 1 and 3 are used). Each chromosome has a tree with a single technical
primitive gene in the root node. In the long term chromosome, signals are generated by a
MACD Line when it crosses the 0 line. The MACD line is calculated from Exponential Moving
Averages (EMA) with periods 186 and 400. In the medium term chromosome, signals are
72
generated by a Relative Volatility Index (RVI) with period 37 when its value crosses the upper
threshold line of 53.7321 or the lower threshold line of 48.7321.
5.6 Equal weight portfolio (reference)
An equal weight portfolio where assets are always bought will be used as a reference for other
portfolios. As previously stated this portfolio is not exactly a buy-and-hold portfolio because
position sizes are periodically adjusted to ensure the invested capital remains close to the
ideal (simulations use fixed capital).
Figure 29 shows an equal weight portfolio simulated from the beginning of 2006 to the end of
2014. As can be seen, there is an upward bias but in 2008 there is a sharp fall followed by a
sharp rise. If an investor had used this strategy in this period, he would have had a reasonable
final return but would have sustained a very high decrease in his capital in 2008 (around 60%).
Table 13 shows a summary of the results obtained by this portfolio.
aRet% Win % Avg Loss % ML% ML Date Std Dev LR2 RRR JR4
5.21 76.67 -9.27 -59.94 21/11/2008 0.00936 0.035 0.087 5.614 Table 13 – Results summary of an equal weight portfolio
Figure 29 – Equal weight portfolio
73
5.7 Setup 10 portfolio
This portfolio uses the optimizations obtained with setup 10 whose individual results were
presented in 5.5. Each asset receives an equal share of the portfolio capital and is traded
according to the trading signals generated by the best genomes optimized for that asset.
Figure 30 shows the returns obtained with this portfolio (red line) and the reference line in
black is the all-bought portfolio discussed in 5.6.
Table 14 shows the summary of results and Table 15 presents the annualized percentage
returns by year. As can be observed, this portfolio has a much better performance in the 2008
bear market than the reference. Not only it didn’t lose money but was able to strongly profit
from the fall observed in the majority of assets. The Maximum loss was 16.69% of the invested
capital and happened in the 4th of November, 2008. Analyzing the results by year, we can see
that the results were very good until the end of 2011. The next year, 2012 was negative and
the following years were positive but with lower returns. In fact, the last 3 years were not able
to recover from the 2011 loss, which may be explained by a different phase of the market with
more erratic movements from the majority of the assets.
We can conclude that this portfolio had an interesting annualized average return in the 9 years
of the simulation with a relatively low risk, but the returns were heterogeneously distributed
by the 9 years. There is no disadvantage in having very high performing years but there is a
disadvantage in having negative or poorly performing years.
Figure 30 – Returns of portfolio with optimizations obtained with setup 10.
74
aRet % Win % Avg Loss% ML% ML Date Std. Dev. LR2 RRR JR4
7.81 40.04 -5.96 -16.69 04/11/2008 0.00631 0.088 0.468 8.705 Table 14 – Results summary of portfolio with setup 10 optimizations
2006 2007 2008 2009 2010 2011 2012 2013 2014
13.10 9.66 33.89 12.26 7.05 -11.91 1.64 6.25 0.81 Table 15 – Annualized percentage returns of portfolio with setup 10 optimizations
5.8 Setup 10 portfolio - long and medium term components
In order to better understand the contributions of the long-term and medium-term
components in setup 10, two portfolio simulations were performed using optimizations
created with setup 10, but with the difference that only one time frame was active. Figure 31
shows the returns of the medium-term portfolio and Figure 32 shows the returns of the long-
term portfolio.
Figure 31 – Portfolio with setup 10 optimizations – medium-term only
Table 16 compares the results from three portfolios: the one presented in 5.7 (with medium
and long term components) and the equivalent ones with only medium term (MT) and only
long term (LT) components. As can be seen, combining both time frames increases the winning
75
percentage and decreases the Maximum loss. Each time frame produces a different trade
profile, a different drawdown profile (maximum losses occurred in different dates) and
combining these two subsystems tend to balance each other and reduce risk. In practice, when
an asset has contradictory trends in the two time frames, the corresponding trading signals
are contradictory and the global system remains neutral. Only when both time frames agree
will the global system assume a position.
Table 17 shows the annualized percentage returns of the same three portfolio simulations
detailed by year.
Figure 32 – Portfolio with setup 10 optimizations - long-term only
Portfolio aRet % Win % Avg Loss% ML% ML Date Std Dev LR2 RRR JR4
MT 6.28 33.68 -8.44 -19.49 31/12/2014 0.00678 0.058 0.322 6.816
LT 7.85 36.33 -7.41 -21.21 18/05/2009 0.00689 0.070 0.370 8.592
MT + LT 7.81 40.04 -5.96 -16.69 04/11/2008 0.00631 0.088 0.468 8.705 Table 16 – Results summary comparing portfolios with different time frames
Portfolio 2006 2007 2008 2009 2010 2011 2012 2013 2014
MT 13.16 11.22 34.74 11.67 2.15 -15.53 -0.15 3.52 -2.24
LT 19.35 16.04 21.23 9.28 8.60 -12.86 -4.41 12.31 3.57
MT + LT 13.10 9.66 33.89 12.26 7.05 -11.91 1.64 6.25 0.81 Table 17 – Annualized percentage returns from portfolios with different time frames
76
5.9 Chapter Conclusions
Preliminary simulations showed that having a large search space is counter-productive since
it generates solutions with curve-fitting that performs poorly in out-of-sample data. As the
search space was restricted by decreasing the maximum depth of the chromosome trees and
by limiting the allowed genes (and consequently the complexity of rules), the out-of-sample
results gradually improved. The best setup used a single level in the chromosome trees and
only technical primitive genes were employed (setup 10). This appears to indicate that since
the data available for the optimization (in-sample) is limited, the search space and complexity
of the solutions must also be small in order to produce robust solutions.
A possible method to increase the complexity of the solutions without producing curve fitting,
is to use more complex rules encapsulated in genes that perform a simple task. In this work
this is done with Technical Primitive genes using majority vote internally to manage several
sub-rules.
The trading systems obtained with setup 10 produces very good results in some assets and
the portfolio constructed with the individual optimized trading signals produce interesting
results, with a good average annualized return and a relatively low risk. The trading systems
perform well in trending markets but poorly in lateral markets, which maybe can be improved
in subsequent works. Despite the portfolio overall positive results, it shows an irregular
distribution of returns, which means that the portfolio may produce high returns in some
phases and struggle in other phases. This may possibly be mitigated combining more
subsystems with different profiles, making more probable that the several different
subsystems balance each other.
77
6 Conclusions and Future Work This work presents a methodology to develop trading rules for trading individual assets. The
resulting rules use a trend-following approach and combined medium-term and long-term
components in an attempt to increase the robustness of the solution. Genetic programming
was used as the optimization algorithm and the solutions are represented with chromosomes
organized in a tree structure. Several setups were tested in order to find the best configuration
and the portfolio simulations were performed to observe the combined result of trading
several assets with optimized trading rules. Several measures were employed in order to
reduce curve fitting and obtain robust solutions.
Several positive results were obtained, some limitations were identified and many research
paths remain open for further work.
6.1 Conclusions
The results obtained confirm that trend patterns in markets can be successfully identified and
explored in order to obtain profits. The portfolio simulation using the best setup produced an
annualized percentage return of around 8% with a maximum loss percentage of around 16%
in the period 2006 – 2014 which must be considered as positive. This results also confirm that
technical analysis can be used to predict the future behavior of the markets to some extent.
It also confirms that technical indicators can be used to extract useful information from asset
historical prices in order to generate successful trading signals.
This work combined rules with two time frames, medium-term and long-term and
demonstrated the improvement that can be obtained by such approach. The returns of the
combined rules are similar to the returns obtained by the single time frame components but,
the maximum loss is significantly lower. This is the result of having two sets of rules with
different trade profiles and different loss profiles, which tend to balance each other and
reduce risk. It’s a form of diversification: trading strategy diversification.
It was possible to use two time frames but without using data of different scales (i.e. daily,
weekly or monthly). Only daily data was used to produce rules with different time frames by
defining allowed parameter ranges for the technical indicators. It’s a method that can have
benefits over using several sets of data. To further differentiate the rules of each time frame
the following method was successfully employed: The rules of each time frame must generate
an annualized number of trades inside a certain allowed range. Solutions not respecting this
are penalized in the fitness function.
Several measures were successfully employed to reduce curve-fitting of the solutions and
improve robustness. This was done by reducing the search space, eliminating solutions which
didn’t use a trend following philosophy. In practice this was achieved by only using genes that
78
produced trend-following rules. The complexity of the solutions was also reduced by limiting
the chromosome trees maximum depth.
To prevent successful solutions to dominate the population with many similar copies, similar
solutions should be deleted in order to prevent a drastic decrease in the diversity of solutions
and an increase in the likelihood of the optimization getting stuck in local maxima. To
accomplish this, a novel algorithm was used which doesn’t look at the chromosome tree
similarities. Instead, it compares the Mean Squared Error (MSE) of the vectors with trading
signals generated by the solutions. If two solutions have a MSE lower than an empirical
threshold, the one with less fitness score is eliminated.
A special type of complex gene was proposed (technical primitive) which combines output
simplicity with internal complexity. The signals generated by these genes are simple in the
sense that they are binary and follow a very simple logic of detecting and following a trend.
But the way to achieve that end is more complex than common trading rules since they include
several sub-rules which are combined by majority vote. This approach has the advantage of
not increasing the search space since the gene doesn’t necessarily have many optimizable
parameters and, allows for more sophisticated rules. Majority vote also tends to suppress
false signals generated by the sub-rules and the output signals tend to be more stable, with
some immunity to noisy price action.
Alongside the several mentioned successful points, some limitations were observed and are
enumerated next.
6.2 Current Limitations
The pool of technical primitive genes is relatively small, especially considering those
who produce the best results. This may limit the creation of successful solutions in
some assets or in some market phases.
The solutions generated by the proposed method produce good results in trending
markets but poor results in markets which trade in a price range or with no defined
trend.
The portfolio simulated with the optimizations obtained by the proposed method
performs well in half of the test period (2006 – 2011) but struggle in the remaining
period (2012 - 2014). This limits the practical use of this approach in its current form
since it’s an undesirable return pattern.
6.3 Future Work
Expand the pool of genes, especially the technical primitive genes specialized in trend
following. Test rules using technical indicators like the AROON indicator by Tushar
79
Chande, the Dynamic Momentum Index [6], The Directional Movement [6], Moving
averages which adapt to market volatility like the VIDYA [6].
Test complex trading rules encapsulated in technical primitive genes. Use several sub-
rules and apply techniques like majority vote to filter false signals and increase the
resistance to noise.
Experiment with genes that generate signals in the direction of the main trend but also
wait for a correction in that trend. That approach would probably give better results in
periods of less clear trends.
Test other ways to articulate the individual optimization phase with the portfolio
phase. Some assets could be excluded from the portfolio using a criteria that checks if
their contribution would likely be negative. Alternatively an algorithm could be used
to give different weights to different assets.
Try using different setups for different asset types (i.e. stock indices, commodities,
etc.). This was technically possible in this work but wasn’t tested because of time
restrictions.
Experiment with setups that use genomes with more subsystems, with different
profiles, in order to have greater system diversification. More subsystems with
different profiles tend to balance each other and reduce risk.
Experiment the optimization of trading rules with other types of assets, like stocks,
currencies (FOREX) and financial assets.
80
81
References
[1] E. Fama, "Random Walks in Stock Market Prices," Financial Analysts Journal, vol. 21,
no. 5, pp. 55-59, 1965.
[2] B. Malkiel, A Random Walk Down Wall Street : The Time Tested Strategy for Successful
Investing, New York: WW Norton & Company, 2011.
[3] C. M. Andrew Lo, A Non-Random Walk Down Wall Street, Princeton University Press,
1999.
[4] C.-H. Park and S. Irwin, "The Profitability of Technical Analysis: A Review, AgMAS
Project Research Report 2004-04," University of Illinois at Urbana-Champaign, 2004.
[5] J. Murphy, Technical analysis of the financial markets, New York Institute of Finance,
1999.
[6] P. Kaufman, Trading systems and methods, John Wiley & Sons, 2013.
[7] J. Machado, R. Neves and N. Horta, "Developing Multi-Time Frame Trading Rules with a
Trend Following Strategy, using GA," in Genetic and Evolutionary Computation
Conference - GECCO, Madrid, 2015.
[8] J. Regnaul, Calcul Des Changes et Philosophie de la Bourse, Paris: Mallet-Bachelier et
Castel, 1863.
[9] J. E. Felipe Aparicio, "Empirical distributions of stock returns: scandinavian securities
markets, 1990-95," Working papers (Universidad Carlos III de Madrid. Departamento
de Estadística y Econometría).: Statistics and econometrics series, vol. 96, no. 58, 1996.
[10] B. Graham and J. Zweig, The Intelligent Investor: The Definitive Book on Value
Investing, HarperBusiness, 2006.
[11] B. Baumohl, The Secrets of Economic Indicators: Hidden Clues to Future Economic
Trends and Investment Opportunities (3rd Ed.), FT Press, 2012.
[12] T. Bulkowski, Encyclopedia of Chart Patterns - 2nd Ed., Wiley, 2005.
[13] J. Bettman, S. Sault and E. Schultz, "Fundamental and Technical Analysis: Substitutes or
Complements?," Accounting & Finance, vol. 49, no. 1, pp. 21-36, 2009.
[14] I. Contreras, J. Hidalgo and L. Núñez-Letamendia, "A GA Combining Technical and
Fundamental Analysis for Trading the Stock Market," in EvoApplications, 2012.
82
[15] W. Brock, J. Lakonishok and B. LeBaron, "Simple Technical Trading Rules and the
Stochastic Properties of Stock Returns," The Journal of Finance, vol. 47, no. 5, p. 1731–
1764, 1992.
[16] N. Ülkü and E. Prodan, "Drivers of technical trend-following rules' profitability in
world," International Review of Financial Analysis, vol. 34, no. C, p. 214–229, 2013.
[17] D. Lohpetch and D. Corne, "Outperforming Buy-and-Hold with Evolved Technical
Trading Rules: Daily, Weekly and Monthly Trading," in EvoApplications, Volume Part II,
pp 171-181, Istanbul, 2010.
[18] J. Maginn, D. Tuttle, D. McLeavey and J. Pinto, Managing investment portfolios : a
dynamic process - 3rd ed., Wiley & Sons, Inc., 2007.
[19] H. Markowitz, "Portfolio Selection," The Journal of Finance, vol. 7, no. 1, pp. 77-91,
1952.
[20] S. Sumathi, T. Hamsapriya and P. Surekha, Evolutionary Intelligence - An Introduction
to Theory and Applications with Matlab, Springer, 2008.
[21] S.-H. Chen, Genetic Algorithms and Genetic Programming in Computational Finance,
Springer, 2002.
[22] Z. Michalewicz, Genetic Algorithms + Data Structures = Evolution Programs (3rd Ed.),
Springer, 1996.
[23] J. Koza, Genetic programming: on the programming of computers by means of natural
selection, MIT Press, 1992.
[24] R. Poli, W. Langdon and N. McPhee, A Field Guide to Genetic Programming, Lulu
Enterprises, 2008.
[25] J. Korczak and P. Roger, "Stock timing using genetic algorithms," Applied Stochastic
Models in Business and Industry, vol. 18, no. 2, p. 121–134, 2002.
[26] L. Mendes, P. Godinho and J. Dias, "A Forex trading system based on a genetic
algorithm," Journal of Heuristics, vol. 18, no. 4, pp. 627-656, 2012.
[27] L. Becker and M. Seshadri, "Comprehensibility & Overfitting Avoidance in Genetic
Programming for Technical Trading Rules," Worcester Polytechnic Institute, Computer
Science Technical Report, 2003.
[28] L. Becker and M. Seshadri, "Cooperative Coevolution of Technical Trading Rules,"
Technical Report, Worcester Polytechnic Institute, 2003.
83
[29] F. Allen and R. Karjalainen, "Using genetic algorithms to find technical trading rules,"
Journal of Financial Economics, vol. 51, no. 2, pp. 245-271, 1999.
[30] D. Lohpetch and D. Corne, "Multiobjective algorithms for financial trading:
Multiobjective out-trades single-objective," in IEEE Congress on Evolutionary
Computation (p./pp. 192-199), New Orleans, LA, 2011.
[31] T. Ibaraki and N. Katoh, Resource allocation problems: Algorithmic approaches,
Cambridge: MIT Press, 1988.
[32] C.-M. Lin and M. Gen, "An effective decision-based genetic algorithm approach to
multiobjective portfolio optimization problem," Applied Mathematical Sciences, vol. 1,
no. 5, pp. 201 - 210, 2007.
[33] C. Aranha and I. Hitoshi, "A tree-based GA representation for the portfolio optimization
problem," in Genetic and Evolutionary Computation Conference (GECCO), Atlanta,
2008.
[34] M. Kaucic, "Portfolio Management Using Artificial Trading," in Genetic Algorithms in
Applications, INTECH Open Access Publisher, 2012, p. Chapter 15.
[35] A. Silva, R. Neves and N. Horta, "A hybrid approach to portfolio composition based on
fundamental and technical indicators," Expert Systems with Applications, vol. 42, no. 4,
p. 2036–2048, 2015.
[36] M. C. Roberts, "Technical analysis and genetic programming: Constructing and testing a
commodity portfolio," Journal of Futures Markets, vol. 25, no. 7, p. 643–660, 2005.
[37] A. Gorgulho, R. Neves and N. Horta, "Using GAs to balance technical indicators on stock
picking for financial portfolio composition," in Proceedings of the GECCO'09, 2041-
2046, Montreal, Canada, 2009.
[38] R. Johnsson, "A Simple Risk-Return-Ratio," 2010. [Online]. Available:
http://www.richardcbjohnsson.net/pdf/risk_return_ratio.pdf. [Accessed 17 September
2015].
[39] E. Seykota, "The Trading Tribe," 2003. [Online]. Available:
http://www.seykota.com/tribe/risk/index.htm. [Accessed 17 September 2015].
[40] D. Dorsey, "Refining The Relative Volatility Index," Technical Analisys of Stocks &
Commodities, pp. 388-391, September 1995.
[41] "Jornal de Negócios - Caldeirão de Bolsa," 17 February 2010. [Online]. Available:
http://caldeiraodebolsa.jornaldenegocios.pt/viewtopic.php?f=3&t=
71777&p=710740#p710740. [Accessed 15 September 2015].
84
[42] "The Qt Company," [Online]. Available: http://www.qt.io/. [Accessed December 2014].
[43] A. Balakin, "MathGL sourceforge page," [Online]. Available:
http://sourceforge.net/projects/mathgl/. [Accessed January 2015].
[44] "Fast Light Toolkit," [Online]. Available: http://www.fltk.org/. [Accessed November
2014].
[45] "Boost C++ Libraries," [Online]. Available: http://www.boost.org/. [Accessed October
2014].
[46] "Ubuntu," [Online]. Available: http://www.ubuntu.com/. [Accessed October 2014].
[47] "2014 FIA Annual Volume Survey – Charts and Tables," Futures Industry Association,
2014.
[48] "Yahoo Finance," [Online]. Available: http://finance.yahoo.com/. [Accessed May 2015].
[49] ML Downloader, ver. 7.1.0.9, Trading-Tools.com, 2012. [Online]. Available:
http://www.Trading-Tools.com.
[50] Metastock, ver. 11.0, Equis International, 2009. [Online]. Available:
http://www.metastock.com.
[51] "Quandl," [Online]. Available: https://www.quandl.com/. [Accessed May 2015].
85
Appendix A - Used Assets Table 18 lists all the stock indices used in this work and Table 19 lists all the commodities used.
Ticker Country Start Description Region
CAC France 1990 CAC 40 Europe
AEX Holland 1992 Amsterdam Exchange index Europe
BEL Belgium 1992 Belgium BEL 20 Europe
PSI Portugal 1997 Portuguese Stock Index 20 Europe
DAX Germany 1990 Deutscher Aktienindex 30 Europe
FTSE Great Britain 1984 Financial Times Stock Exchange 100 Index Europe
SPX USA 1950 Standards & Poor’s 500 North America
NDX USA 1985 NASDAQ-100 North America
TSX Canada 1979 S&P/TSX Composite Index North America
IPC Mexico 1993 Indice de Precios y Cotizaciones North America
AORD Australia 1984 ALL ORDINARIES Oceania
NIKKEY Japan 1984 Nikkei 225 Asia
HSI China (Hong-Kong) 1986 HANG SENG INDEX Asia
STI Singapore 1987 Strait Times Index Asia
SENSEX India 1997 S&P BSE SENSEX Asia
JKSE Indonesia 1997 COMPOSITE INDEX Asia
TWII Taiwan 1997 TSEC weighted index Asia
MERV Argentina 1996 MERVAL Buenos Aires South America
BVSP Brazil 1993 IBOVESPA South America
Table 18 – List of stock indices used
ID Exchange Exchange Ticker Start Sector Description
B1_FR ICE B Energy Brent futures
HO1_FR NYMEX HO Energy Heating Oil futures
NG1_FR NYMEX NG Energy Natural Gas futures
GC1_FR COMEX GC Metals Gold futures
XAGUSD - - Metals Silver spot
PL1_FR NYMEX PL Metals Platinum futures
PA1_FR NYMEX PA Metals Palladium futures
S1_FR CBOT S Agriculture Soybeans futures
SM1_FR CBOT SM Agriculture Soybean Meal futures
W1_FR CBOT W Agriculture Wheat futures
SB1_FR ICE SB Agriculture Sugar Nº 11 futures
Table 19 – List of commodities used
86
87
Appendix B – Groups of genes
Index ID String Operator /
Terminal Input Output Description Genes
0 O_ARTM Operator double double Arithmetic operators +, -, /, *, Max, Min
1 O_MATH1 Operator double double Math operators Pow, SQRT, LN, LOG10
2 O_MATH2 Operator double double Math operators 2 Round, Floor, Ceil, Abs
3 O_COMP Operator double Boolean Comparison operators >, <, >=, <=, ==, !=
4 O_BOOL Operator Boolean Boolean Boolean operators AND, OR, NAND, NOR, XOR, XNOR, NOT
5 O_TECH Operator double double Technical operators
EMA (Exponential Moving Average),
SMA (Simple Moving Average),
WMA (Weighted Moving Average)
6 O_MVOTE Operator Boolean Boolean Majority vote Majority Vote 3 inputs, Majority Vote 5 inputs
7 O_IF Operator double* double IF with double inputs IF with inputs of type double (* condition is
Boolean)
8 O_IFB Operator Boolean Boolean IF with Boolean inputs IF with inputs of type Boolean
9 T_CONST Terminal - double Constant series Constant double value (ex: 80.5)
10 T_FIXED Terminal - double Fixed series The vectors Open, High, Low, Close and Volume
from the asset data
11 T_PRIM Terminal - Boolean Technical primitives (auto-
sufficient mini subsystems)
See
Appendix C – Technical Primitive Genes
12 T_TECH Terminal - double Technical indicators with fixed
parameters (no child nodes).
Stochastic Indicator %K, Stochastic Indicator %D,
Relative Volatility Index (RVI), Relative Strength
Index (RSI)
Table 20 – Gene groups and characteristics
88
89
Appendix C – Technical Primitive Genes Technical Primitive genes are terminal genes with Boolean output that encapsulate more
complex rules which are auto-sufficient in generating buy and/or short signals. The internal
algorithm may be relative simple or more complex. In some cases the output is the result of a
majority vote of several sub-rules. They may or may not receive parameters. Its use in this
work is fundamental since tests showed the best results are obtained when the chromosome
trees have a single technical primitive gene.
Table 21 shows the list of all Long Technical Primitive genes, with its Identification string and
a description. Equivalent short genes exist but are not shown.
Gene ID String Description
C>EMA
A buy signal is generated when the price closes above the Exponential
Moving Average (EMA) and a sell signal is generated when the price closes
below the EMA. See section 3.3.1.
C>SMA
A buy signal is generated when the price closes above the Simple Moving
Average (SMA) and a sell signal is generated when the price closes below
the SMA. See section 3.3.1.
C>WMA
A buy signal is generated when the price closes above the Weighted
Moving Average (WMA) and a sell signal is generated when the price
closes below the WMA. See section 3.3.1.
C>SMAB
A buy signal is generated when the price closes above the upper SMA
band and a sell signal is generated when the price closes below the lower
SMA band. See section 3.3.1.
RVI>X
A buy signal is generated when the Relative Volatility Index (RVI) crosses
the upper threshold from below and a sell signal is generated when the
RVI crosses the lower threshold from above. See section 3.3.4.
MACD>0
A buy signal is generated when the Moving Average Convergence-
Divergence (MACD) line crosses 0 from below and a sell signal is
generated when the MACD line crosses 0 from above. See section 3.3.3.
SAR_L
A buy signal is generated when the price closes above the Stop and
Reverse (SAR) line and a sell signal is generated when the price closes
below the SAR line. See section 3.3.5.
CH_BRK_L
A buy signal is generated when the price crosses the upper price channel
from below and a sell signal is generated when the price crosses the lower
price channel from above. See section 3.3.2.
TLP_100223_L
Long-term composite indicator described in section 3.3.6. A buy signal is
generated when the majority vote of its 3 components give a buy signal
and the equivalent for the sell signal.
90
TMP_110112_L
Medium-term composite indicator described in section 3.3.6. A buy signal
is generated when the majority vote of its 3 components give a buy signal
and the equivalent for the sell signal.
TMP_090212_L
Medium-term gene which has some internal complexity and was
previously created by the author. It uses a volatility calculation based on
the difference between an upper and lower price bands. The upper band
is calculated with the past high peak values and the lower band is
calculated with past low peak values. A buy/sell signal is generated when
the present volatility line crosses a past volatility reference.
MT_OSC_L
Medium-term gene which generates a buy signal when 3 conditions are
verified. 1) An up-trend is detected with an RVI indicator; 2) A correction
on that up-trend exists; 3) Price gives an indication that the up-trend will
resume. This was an experimental algorithm with limited impact on the
results.
RSI>X
A buy signal is generated when the Relative Strength Index (RSI) [6]
crosses a threshold level from below and a sell signal is generated when
the RSI crosses the threshold line from above.
STOCH_D_s>X
A buy signal is generated when the Slow Stochastic D Index [6] crosses a
threshold level from below and a sell signal is generated when the
Stochastic index crosses the threshold line from above.
Table 21 – Description of LONG Technical Primitive genes
91
Appendix D – Cache of vectors The optimization module works with vectors of data in which the vector elements correspond
to trading days. There are many types of data and all corresponding vectors must be
synchronized with each other. Data from those vectors is accessed using an integer index
(associated with a trading day) and a particular vector position must correspond to the same
trading day in all vectors.
When a chromosome is interpreted for a particular asset and converted into trading signals,
the process is completed for all trading days, resulting in a vector with the trading signals for
that chromosome for that particular asset. Intermediary steps in the interpretation of the
chromosome (gene interpretation) also produce vectors. The asset Open, High, Low and Close
data is also hold in vectors with the same characteristics.
Since there is a lot of chromosome and gene repetition in a population of genomes,
simulations can be made much more efficient if a repeated gene or chromosome doesn’t have
to be interpreted several times. For this end a cache system was created so hold vectors
resulting from calculations, in order to reuse its contents. To identify the vectors, they receive
an id string constructed with all the relevant information: Asset, start index, end index and
contents. If the contents is the interpretation of a gene, the id string of the gene is included,
as all its parameters. If the gene has sub-nodes, the id string of the sub-nodes (other genes) is
appended. The resulting id string of the vector can be long, for example in the case of a root
node belonging to a chromosome tree with many levels.
When the optimization module starts, it creates the cache with a pre-defined number of
vectors, which are encapsulated in a C++ class. If later the program determines the need to
have more cache vectors, they are created. A vector from this cache are generic vectors than
can be used and reused for any task that requires a vector.
Each vector in the cache will be referred as a slot and each slot contains the following
information:
An ID string which identifies the vector it holds which is unique;
A pointer to the actual object containing the vector;
The time of the last access: generation index (smaller is older).
The cache is managed in a C++ class which contains two structures with the previous
enumerated information:
A C++ unordered map structure which associates the ID string with the pointer to the
slot object;
A bidirectional map defined in the C++ library Boost (the 2 fields can act as keys). This
structure relates the ID string with the age of the series in the cache.
The cache class can receive two types of requests:
92
Find a specific vector and return the pointer to the slot: Before any calculation, the
program searches the cache to see if the result is already there. For this is calculates
the ID string of the resulting vector. The cache either returns the pointer of the slot
object containing the vector or returns NULL indicating the vector is not in the cache;
Get a slot: If the program needs a vector to hold some calculation, it requests a free
slot. The cache manager will locate an unused slot or will free the oldest used slot and
return the pointer to it. The cache function receives the new ID string as parameter
and associates it with the slot.
When the information on a particular vector is no longer needed, the pointer to it, received
from the cache, can be deleted and that slot/vector will later be reused for another purpose
in another part of the program. The use of this cache system has a high hit rate (requests for
vectors that are in the cache) that depends of the types of genomes, genes,
crossover/mutation rates and other parameters. In some cases the hit rate is higher than 80%
which represents a large saving in terms of the time required to run the
optimizations/simulations. The structures and organization of the cache can be seen in Figure
33.
Figure 33 – Generic structures of the cache
93
Appendix E – Performance metrics Each simulation of a genome, sub-system or chromosome produces performance parameters
that can be used to plot results and analyze results.
Table 22 shows a list of performance metrics calculated 3 versions: for long trades, for short
trades and for all trades (there are 3 versions of each item).
Short Name Description
sim_bars number of bars of the simulation
n_trades number of trades
n_win_trades number of winning trades
n_loss_trades number of losing trades
annual_n_trades average number of trades per year
annual_n_win_trades average number of winning trades per year
annual_n_loss_trades average number of losing trades per year
win_trades_p percentage of winning trades
gross_profit gross profit
gross_loss gross loss
net_profit total net profit
profit_factor profit factor
avg_trade average trade
avg_trade_p average trade in percentage of capital
avg_win_trade average win trade
avg_win_trade_p average win trade in percentage of capital
avg_loss_trade average Loss trade
avg_loss_trade_p average Loss trade in percentage of capital
R_avg_win_loss Ratio avg_win_trade / avg_loss_trade
large_win_trade largest winning trade
large_loss_trade largest losing trade
max_cons_win_trades maximum consecutive winning trades
max_cons_loss_trades maximum consecutive losing trades
avg_bars_trade average number of bars in trades
avg_bars_win_trade average number of bars in winning trades
avg_bars_loss_trade average number of bars in losing trades
time_market_p percentage of time in the market
max_dd maximum drawdown
max_dd_p maximum drawdown in percentage of the capital
max_dd_bar maximum drawdown date (bar index)
94
max_dd_date maximum drawdown date (string)
return_p percentage return
annual_ret_weighted weighted annualized rate of return
annual_return annualized percentage return
equity vector with daily returns (equity)
ddown vector with daily drawdown
Table 22 – Performance metrics calculated for Long, Short, and Global trades
Table 23 shows performance metrics that are calculated only for the global trades and not for the special cases of long and short trades.
Short Name Description
daily_DD average daily drawdown
avg_daily_DD_p average daily drawdown in percentage
lake_ratio Lake Ratio by Ed. Seykota
lake_ratio2 The inverse of the Lake Ratio
risk_return_ratio Risk Return Ratio
risk_return_ratio2 Modified Risk Return Ratio with the average drawdown in the denominator
risk_return_ratio3 Modified Risk Return Ratio in which the denominator includes the average drawdown and the maximum drawdown
annual_n_win_trades average number of winning trades per year
JR1 Experimental fitness function 1
JR2 Experimental fitness function 2
JR3 Experimental fitness function 3
JR4 Fitness function used
JR5 Experimental fitness function 5
Table 23 – Performance metrics calculated exclusively for global trades