How To Measure Your Trading System Quality: By Justin Paolini

Your trading system needs to have a positive expectancy, but it is the SQN that determines your potential effectiveness and efficiency in applying position sizing strategies to meet your specific objectives. – Van Tharp

If a trading system makes money over time, that’s all that counts right? And amongst the trading systems that make money, the best one is the trading system that generates the highest overall profit, right?
Unfortunately this is how most retail traders rank trading systems. All they look at is the end profit or the profit factor. This is like saying that the best car to drive is the fastest, period. It doesn’t matter what your needs are (if you have a family, if you have one or more children, or if it’s just you). You get from A to B before everyone else, but how comfortable was the ride?
The industry standard for ranking trading systems takes into account “how bumpy the ride was”. It only makes sense to measure risk-adjusted returns, not absolute returns, because the long-term viability of the system itself depends on how much risk is taken. In this article we shall explore various evaluation methods.

Van Tharp’s SQN vs. Sharpe Ratio

In conversations with systematic trader Robert Carver, what emerged was some degree of difference between the language systematic traders utilize and the language that “old fashioned” traders utilize. Systematic traders talk about volatility, mean returns, and sharpe ratios. Old fashioned traders talk about expectancy, drawdowns and system quality.
Fortunately our programmer Tony (see his work here) has proven that SQN and Sharpe Ratio are the same thing. Here is the mathematical argument:
R = profit / risk for a trade (Van Tharp’s R-multiple).
Let’s assume risk is constant for all trades at some % of the account (if you don’t use a fixed fractional position sizing method, then you may want to use the average loss per losing trade).
The SQN = Average( profit / risk ) / StdDev( profit / risk ) x SquareRoot( number of trades )
Squaring both sides gives:
SQN^2 = Average( profit / risk )^2 / Variance( profit / risk ) x number of trades
With risk constant,
SQN^2 = ( ( Average( profit )^2 / risk^2 ) / ( Variance ( profit ) / risk^2 ) ) * number of trades
The risk^2 terms cancel, leaving
SQN^2 = ( Average( profit )^2 / Variance( profit ) ) * number of trades
Taking the square root of both sides to return to SQN gives
SQN = ( Average( profit ) / StdDev( profit ) ) * SquareRoot( number of trades )
And we have demonstrated that SQN is another version of the Sharpe Ratio. Here is a spreadsheet with some hypothetical system variables to prove the point:
Our London Open Signals current have a Sharpe Ratio (since inception) of 1.53.

Disadvantages of the Sharpe Ratio

While it has become the industry standard, the Sharpe Ratio has an evident flaw: it does not diiscern between upside volatility and downside volatility. For example a positive skew system (trend-following, with many small losses and infrequent but large gains) with a large dispersion of returns will have a smaller Sharpe Ratio, despite having a strong performance. On the flip side, risky strategies that have a negative skew (mean reversion, or scalping, with many small gains and very few but huge losses) may have great Sharpe Ratios that do not convey the risk of the strategy.
Moreover, most trading systems display a skew (fat tails) in the return distribution, so measures like the System Quality, Sharpe, Sortino, etc. cannot adequately capture the risk-adjusted performance of trading strategies.

The MAR Ratio

The Managed Accounts Reports ratio is preferred for evaluating CTA performance, Hedge Funds and trading systems in general. It is straightforward:

MAR = CAGR/MaxDD = Geometric Return/MaxDD

This measure can be used over any sensible timespan (1Month, 1 Quarter, 1 Year, Since Inception) and is actually interesting even for projecting worst case scenarios. The worst drawdown traders face, within the systematic community, is around 2-2.5 times the average return. So the MAR ratio allows to estimate the worst case risk-adjusted return.

Our London Open Signals have a MAR (since inception) of 1.19 with a CAGR of 17.5% (risking 2% per trade).

Over to You

Are you correctly tracking your trading system performance, or are you just focusing on making money? One of these attitudes has a future, and one does not. Without knowing how much risk you are taking, in order to generate certain returns, it is impossible to evaluate the quality of your work.

About the Author

Justin is a Forex trader and Coach. He is co-owner of www.fxrenew.com, a provider of Forex signals from ex-bank and hedge fund traders (get a free trial), or get FREE access to the Advanced Forex Course for Smart Traders. If you like his writing you can subscribe to the newsletter for free.

Leave a Reply

Your email address will not be published.