We all want to believe there’s a way to measure the forward strength of a model, but the best you can do is make predictions about the future based on strategy performance and then track that performance. We use profit factor as the primary basis for that prediction, but there are a host of other attributes we track in our search for the holy grail of automated trade strategy. In this update, I’m going to introduce another — average time in the market. I’m also going to introduce our new testing process.
Over the last 3 months those of us on the hunt have made many discoveries. I want to thank our subscribers for all your help and support in this project.
One report I publish every week is the Mudder Report. I use the report as a way to review weekly performance and discuss any updates or announcements for the coming week. The primary goal of the report is threefold:
- to compare weekly stats against annual stats
- to develop a rubric/framework for a weekly strategy selection process
- to test backtest results against actual results
It’s the last bullet point that’s recently stopped us in our tracks.
What we found has led to several changes including the introduction of the trading metric ‘average time in the market’. This metric is one way to measure a very special kind of risk that I like to refer to as backtest risk.
Backtest risk is not attached to the backtest engine or methodology, but rather the unique attributes of a strategy that make it more or less prone to inaccuracy in backtest results. It is something unique to each individual strategy. That is, each strategy has its own backtest risk.
What can ‘average time in the market’ tell us about backtest risk?
For some automated trading strategies (not all), the more time you’re in the market, the less risky the strategy is from a backtest perspective. This might sound counterintuitive — it certainly does for me as a day trader — but it makes sense when you’re using backtests as a test of performance for an automated strategy.
We’ve spent a lot of time improving the accuracy of our backtest methodology — you can read more about that in the post: What Are We Doing To Ensure Backtest Accuracy, but we’ve come to discover that there’s a need for further due diligence. To that end, we’ve hired someone to add additional testing to the hunt.
- The backtest is the first test.
- The second test is to run the strategy on a simulated live account on a virtual server that is run continuously. The server is located in Chicago to reduce data latency.
- The third test is to run the strategy on Collective2, a third-party app used for copy-trading. To learn more about Collective2, click here.
I like to work backwards, so my goal is to have at least 10 strategies that pass these three tests. And, I’ll be reporting out on the weekly performance of these tests every week in the Mudder Report. We’ll return to the original intent of the Mudder Report once we’ve vetted at least 10 strategies.
I’m not suggesting that strategies that don’t pass this ‘master’ test should be thrown out, but our main tool to assess strategy performance is the NT8 backtest engine. So, if we’re going to use the backtest as a way to identify high performing strategies, we need to focus on those strategies that perform like the backtest. This is just one way to better identify those strategies. I can show you an updated version of our performance chart, and then compare that performance against the performance of our strategies one year ago, but what good does that do if the backtest results cannot be duplicated in live trading? As much as I don’t want to acknowledge it, there’s a gap and we’re turning that gap into something we can control. These two additional tests are the best way to do that without risking large amounts of capital.
The good news is that in our preliminary Collective2 tests, we’ve found that this issue only plagues certain strategies. In particular, those strategies with:
- a high number of trades; and,
- a short average time in the market.
Other strategies tend to be less prone to backtest inaccuracy; that is, they have a lower ‘backtest risk’. For example, Strategy 40 is performing exceptionally well on Collective2.
What does this mean? It means that some strategies, especially those strategies that enter and exit the market quickly, and by extension, have more than 3-4 trades per day, may be more prone to issues with backtest accuracy. We already track the average number of trades made per day. In our next performance update, we’re going to start tracking the average time trades are in the market.
Key takeaway: Those strategies with a higher ‘average time in the market’ and a lower ‘average trade count’ tend to have more reliable backtest results.
Now, let’s get back to our regularly scheduled programming.
We are on the hunt…
We are on the hunt for the holy grail of automated trade strategy. We define the holy grail of trade strategy as having the following attributes:
- Profit factor greater than 3
- Annual drawdown less than 3%
- Annual return greater than 500%
- Maximum daily low of -$1,000
- Avg Daily profit greater than $1,000
- Less than 5,000 trades annually
- Greater than 253 trades annually
Every two months I like to look at how our top automated trading strategies are performing and update you on key findings, highlights, takeaways and what’s in the pipeline. The following post is our March 2022 update. You can view past updates here:
- January 2022 Update
- November 2021 Update
- September 2021 Update
- July 2021 Update
- May 2021 Update
- March 2021 Update
What Are We Looking For In These Updates
In our last update we culled the field a bit. At the end of the update we found ourselves with 27 out of 43 strategies:
Doing this allowed us to focus our efforts for future ATS Research. However, in light of our new testing process, we’re going to take a step back. I’ve asked for testing on all strategies on the virtual server. Those strategies that pass the test (and aren’t considered high frequency strategies like Strategy 3, 8 and 10), will be tested on Collective2 as well.
With that, let’s dive into the update.
January 2022 Vs. March 2022
So where do we stand in comparison with our last update?
The table below provides the backtest results from our last update: 01/01/2021 - 01/01/2022
For the most recent update and links to all strategies click here.
The following table provides the backtest results for the current or most recent update: 03/01/2021 - 03/01/2022
Strategies with less than 50 trades for the year were not included.
First, let’s review what the trend was coming into the March update.
In general, we had a deterioration from July to September of 2021. Profit per day decreased from $388 to $355 and profit per trade decreased from $145 to $116. Most of the deterioration occurred with optimized strategies. When we removed optimized strategies the metrics improved. What we’ve found over the year is that as much fun as optimized strategies are to ‘hunt’, they are more prone to overfitting and therefore have highly inconsistent results.
Moving into November of 2021, net profit for the total portfolio was basically flat, decreasing slightly from $2,788,075 to $2,746430, but the portfolio profit factor increased from 1.24 to 1.35 on fewer trades. Profit per day decreased from $355 to $304, but profit per trade increased from $116 to $230. Even though net profit decreased, it was a leaner and more efficient portfolio.
Moving into January of 2022, net profit for the total portfolio increased from $2.7 million to $3.3 million, and the portfolio profit factor increased from 1.50 to 1.54. In addition to a higher profit factor, we also have a slightly lower max drawdown percentage. Profit per day continued to decrease from $304 to $252.
Coming into March, net profit for the total portfolio decreased slightly from $3.3 million to $3.1 million and the profit factor decreased from 1.54 to 1.27. Trades increased by 25%. Drawdown increased dramatically from 6.67% to 10.82%. Profit per day continued to decrease from $252 to $230, but profit per trade increased from $233 to $261 due almost entirely to Strategy 45, the only strategy that incorporates a risk management strategy that trades more than one contract at a time.
So what’s going on?
On the whole, and as predicted, what we’re looking at is a much more volatile market. From an automated strategy perspective that translates into:
- higher trade count
- higher commissions
- higher drawdowns
- greater overall risk
While not evident when comparing the performance of the entire portfolio—our entire portfolio includes strategies that we would not consider in the weekly Mudder Report—on a weekly basis we’ve also noticed that volatility tends to create higher returns. Indeed, over the last four weeks we’ve seen a huge increase in volatility. Those of you that trade probably saw your margins increase by 2 to 3x. In that same time period, our weekly portfolio made nearly $600K in net income based on backtest results.
What’s causing this volatility?
Everyone has their own theory about what’s causing this volatility, and this is mine…
Central banks are stuck between a rock and a hard place. They are being forced to end asset buying programs (in hopes of tightening or slowing economic growth) while increasing rates. The programs were put in place to provide liquidity to an unstable market. But now growth, disguised as inflation, is fueling the push to increase rates. It is the economic equivalent of being catfished. Still, it seems that not even the uncertainty and volatility caused by ‘Russia’s invasion of Ukraine’ will deter the Fed from hiking. In total, the street is looking for six 25bp hikes this year and two more in 2023.
It’s important to remember that the Federal Reserve is the governing body for a banking cartel, not the United States. They only care about the state of the economy as it pertains to their bottom line, and we can’t fault them for this because we gave them this power. They are the anointed demigods of modern economic theory — too big to fail. And, the only thing that really hurts banks is inflation because fixed loans are paid back with dollars that have less value. It’s only natural to assume that the Federal Reserve will continue to raise rates even if it flies in the face of stable markets.
So the big question on every banker’s mind right now is: how do we maintain liquidity while avoiding inflation? I don’t personally know how it can be done, but I’m sure they’ll invent something that pushes the inevitable down the road.
The good news is that corporate CFOs have all gotten together and are currently attending a summit entitled: How To Adjust The Financial Statements So That The Effects of Inflation Can Be Readily Identified By Investors. No they aren’t. They don’t care as long as earnings are up. We’ll talk more about how to take advantage of these markets in a future Mudder Report.
One thing is certain, the hunt has changed since we started this project. The dark and scary forest of trading is full of volatility, rising rates and inflation, oh my. For now, the best thing we can do is continue to monitor results and adapt to a new norm. There is no going back to Kansas.
Update Summary
 |
Scene from the movie “The NeverEnding Story” |
One of my favorite teachers is Joseph Campbell. Perhaps his most famous work is the Power of Myth in which he outlines the hero’s journey as a mythology that is shared across time and cultures. It appears to be something that is ingrained in the human psyche. Every now and then the hero will find himself at a crossroads. Do you proceed or retreat?
Those of us that trade are on our own journey. The hunt for the holy grail of automated trade strategy is an extension of that journey. It is a journey that I take very seriously and we are currently at a kind of crossroads. We are not retreating. We are moving forward, but we have to be intellectually honest in the process.
I am aware that the new testing process may be seen by some as a setback, but it’s not. It may feel like we’re going backward, but we’re making sure we’re ready for what’s to come.
For the next two to three months we will be focused almost exclusively on testing each strategy. This pushes many of our other projects back a bit. For example:
In the first six months of 2022, we planned on including backtests for:
- additional futures contracts like CL, FDAX, YM and bonds
- FX and cryptocurrency
- single stock picks
We were also looking into other platforms for our strategies as well as a few portfolio analysis options that might help to pull out additional insights for better strategy formulation.
All of these projects have been pushed, and I hope you understand why. Ultimately, we believe our new testing format will get us closer to the holy grail.
The key to finding the holy grail is to document, narrow and refine our approach, which is exactly what we’re doing. Our bi-monthly updates—like the one you are reading now—along with The Mudder Report and additional backtest research, are meant to share the process with you.
One thing is certain, we’re closer than we were last year and we couldn’t have done it without your support.
What’s in the pipeline:
This is where I usually tell you about the projects in the pipeline. This update is a bit different because we’ve already addressed that, but I can talk to you about what’s in the pipeline for strategy formulation. We post new strategies for subscribers on the 1st and 15th of every month.
If anything, what the Mudder Report has taught me is that events drive markets, not trends. Trends are nice to ride when you find one, but events are more reliable for planning purposes. To that end, I’ve ordered several event studies, the last of which was delivered today. Specifically, I asked the analyst for research pertaining to the NDX — the underlying instrument to the NQ — for the past 11 years. For example, did you know that:
- Tuesday has the highest return (followed by Wednesday).
- July has the highest return by month (followed by April and November) and that the only year that did not experience a positive month in July was July of 2014.
- July also performs exceptionally well in election years (2012, 2016, 2020) at 76% returns and post elections cycles (2013, 2017, 2021) at 80%.
- By contrast, July performs poorly in midterm election cycles (2014, 2018, 2022).
What’s one possible strategy to employ based on these observations? Go long on Tuesday’s in July, April and November of election years or post election years.
There are over two dozen observations like these in the report and I’m going to summarize and share them with all subscribers over the next two to three weeks. We are working on creating a strategy that is driven by these event cycles. At some point, we would like to focus on event studies for specific stocks, but that project has been pushed to the end of 2022.
What else are we working on? We’re looking at:
- creating strategies that attach orders to indicators.
- strategies that have an input that is driven by a non-indicator based optimization, like day of week. So one part of the strategy is static and the other is based on an aspect that is updated on a daily or weekly basis.
- creating more strategies that ramp up from 1 MNQ contract to 5 NQ contracts based on net income, like Strategy 45.
- the individual performance of strategies at the weekly level to see if we can find more trends like we did with Strategy 40.
We’re also looking at creating a strategy based on the average true range (ATR). In most cases, prices tend to fall quickly and rise slowly. That is, the market is lopsided in its action. This is due to a kind of gravity in the markets — no one wants to catch a falling knife. The measure of gravity is not a constant, like in Newtonian physics, but a measure of the speed of price movement. Just like in physics, where force equals mass times acceleration, in the markets, force is a function of the instrument, volatility and range. And, it is stronger going down than going up. That is, there appears to be a kind of market friction associated with accumulation, and a kind of market acceleration with distribution. In the same way that gurus of yoga and meditation will tell you to breathe in with the nose and out with the mouth, the market would appear to breath in with accumulation and out with a distribution.
From a trader perspective, distributions are lovely. Manipulation/traps are harder to set up in a falling market. What’s more, you can get in and out of the market quickly. It might take you 2 hours to make 100 points in a rising market, but only 2 minutes to make 100 points in a declining market. This isn’t always the case, but it occurs more often than not, especially when there’s a lot of market or geopolitical news like there’s been over the last four weeks.
Going back to the physics of trading, perhaps a longer, time-based series is better for catching rising movements and a shorter time-based series, or a series that is independent of time, is best for catching declining movements. That is, getting in and out of a long trending position should be done on longer dated time frames, whereas getting into and out of short positions should be done with greater precision, using a data series that is not dependent on time (range, tick, etc). For example, you can use a 5 minute chart to get in and out of a long position, but a 36 range chart to get in and out of a short position as we did with Strategy 44.
We’re also still working on the possibility that the holy grail of automated trade strategy could be opposite sides of the same coin. It could be that the holy grail is knowing when to flip any strategy based on market conditions, or it could be the same strategy based on complementary markets, i.e., gold and equities.
We’ve also got Strategy 46 coming up on April 1 and our first “Revised” Mudder Report coming up on April 3rd.
Answers to Some Commonly Asked Questions:
- Perhaps one of the most commonly asked questions we get is, ‘how much do I need to get started?’ For more on what we think is the best way to approach this question for yourself, click here.
- We continue to receive questions about funded trader programs. In particular, some of you are on the hunt for a strategy that will pass the funded trader programs out there. If that’s what you’re looking for, this is not the place for you (read: do not purchase a subscription if that’s what you’re looking for). That said, our most recent strategy (Strategy 45) uses a risk management strategy that slowly ramps up contracts. This is the kind of strategy you want to employ when attempting to get funded as well. Start small on an MNQ contract and then increase to NQ. For more on what we think about the best way to approach funded trader programs click here.
- In general, one of the most common questions we get pertains to the use of our strategies on other assets like crypto. You can read more about that in the post: Do our strategies work on cryptocurrencies? We plan on updating this post, but this is one of the projects that’s been pushed. I hope to revisit this by the end of the summer.
- Another commonly asked question is in regards to overfitting. You can read the following posts to learn more about overfitting and what we’ve done to reduce its impact: Overfitting: What is it and what can we do about it and What Are We Doing To Ensure Backtest Accuracy? A few months after writing these posts I received a question regarding strategy strength and what optimization tools we’re using to increase robustness. We answered that question in the post: How Can You Tell If An Automated Trading Strategy Will Perform Well Over Time?
- Some subscribers have said that our strategies are too complex, while others have said that they lack complexity. To the former I say: we are here to help. Please reach out to me with your questions. To the latter I say: our aim is not complexity, it is a certain level of consistent performance based on the attributes listed above. That’s what these updates are all about. We’re all trying to figure out what works and what doesn’t over time. What we’ve found so far is that complexity is often unreliable. If you have any questions, please ask us or reach out. It might take a few days, but I try to respond to every email.
- One common critique is that the strategies we give away (Strategies 1 and 5 ) don’t perform well. That’s why we’re giving them away. The goal is to show you how our process works.
- One of the most commonly asked questions we get is: How do I learn how to create automated strategies for myself? My advice is to use the resources provided by Ninjatrader. Just go to their Youtube channel and sign up. It’s free and there are many videos to learn from.
If you have any questions, feel free to contact me directly at automatedtradingstrategies@substack.com
No comments:
Post a Comment