Automated Trading Strategies: November Update

In The Last 12 months Our Strategies Have Made Over $2.7M Based on Backtest Results

We all want to believe there’s a way to measure the forward strength of a model, but as we’ve explained before, market data is not random. It is based on auction mechanics. So, the same tools we use to validate and test the quality of data in a random data set can’t always be applied to the market. This is why we say ‘don’t use the simulated data feed within Ninjatrader’. It’s a random, internally generated market and has no correlation to real market data. So what can you do? The best you can do is make predictions about the future based on strategy performance and then track that performance, as well as the rationale for your predictions, over time.

This is why we publish updates every two months to track the progress of strategies. We like to look at how our top automated trading strategies are performing and update you on our key findings, highlights, takeaways and what’s in the pipeline. The following post is our November update. You can view past updates here:

• March Update

• May Update

• July Update

• September Update

What Are We Looking For In These Updates

We are looking for one thing in particular: consistency. When we say consistency, we mean strategies that perform like the backtest. We are tracking our best strategies to find the best strategy attributes. We can then use these attributes to create a strategy that gets us closer to the holy grail of automated trade strategy?

We define the holy grail of trade strategy as having the following attributes:

• Profit factor greater than 3

• Annual drawdown less than 3%

• Annual return greater than 500%

• Maximum daily low of -$1,000

• Avg Daily profit greater than $1,000

• Less than 5,000 trades annually

• Greater than 253 trades annually

These attributes are not only how we define the holy grail, but some of them help to strengthen the strategy.

We’ve also added two columns to this backtest result: gross loss and gross profit. Why did we do this? We are increasingly focused on transparency. As much as possible, we want you to be able to understand how a strategy works. In our May update we decided to add a column for profit factor because someone asked the following question:

It’s nice that you can make $100K on a strategy, but how much did it take to get there?

It’s a good question for both the novice and professional. Ultimately, it gets at reinvestment risk, which can be hard to understand if you’re not used to trading futures contracts. We tried to address this question in the post: Reinvestment Vs Total Investment.

Now, we’re adding one more level of transparency because not everyone is going to read the article above. Instead of just looking at profit factor, which is calculated by dividing gross profit by gross loss, we’re showing the actual values of gross profit and loss.

We’ve also added RTY for Strategies 17 and 33 to the update. Both backtested well when we compared the NQ futures contract with the RTY contract in the post: Which Equity Index Will Lead To The Holy Grail Of Automated Trade Strategy: NQ Vs. RTY. While both have less than 253 trades in the year, they also have a high profit factor so we’re going to continue to track/test these strategies under the RTY contract as well.

Finally, to make room for these changes, we’ve removed the ‘# of trading days’ column in lieu of a note at the bottom of the chart since the number of trading days for each strategy is the same at 253.

With that, let’s dive into the update.

September Vs. November

So where do we stand in comparison with our last update?

The table below provides the backtest results from our last update: 8/30/2020 - 8/30/2021

[Our apologies for the small font, but you can enlarge by clicking on the picture.]

For links to all strategies click here.

The following table provides the backtest results for the current or most recent update: 11/01/2020 - 11/01/2021

[Our apologies for the small font, but you can enlarge by clicking on the picture.]

First, let’s review what the trend was coming into the September update. In general, we had a deterioration from July to September. This is an excerpt from the September update:

“Average profit factor for the entire portfolio decreased from 1.26 to 1.24 and average net profitability decreased from $98K to $89K. Profit per day decreased from $388 to $355 and profit per trade decreased from $145 to $116. The number of profitable trades also decreased slightly from 44% to 43%.”

Most of the deterioration occurred with optimized strategies (marked in yellow on the chart). When we removed optimized strategies from the portfolio the metrics improved. What we’ve found over the year is that as much fun as optimized strategies are to ‘hunt’, they are more prone to overfitting and therefore have highly inconsistent results.

Now let’s get to what we found in the most recent update:

For the year, net profit for the total portfolio decreased slightly from $2,788,075 to $2,746430 with a total max drawdown of $689K. The portfolio profit factor increased from 1.24 to 1.35 on fewer trades. In addition to a higher profit factor, we also have a lower max drawdown percentage from 8% to 7%. We also saw a decrease in average profit per strategy from $90K to $78K along with a 2% increase in the number of profitable trades. The decrease in net income led to a drop in profit per day from $355 to $310, however, because the number of trades decreased and the profit factor improved, the profit per trade increased from $116 to $188. So even though net profit decreased, this is a leaner portfolio; it’s more efficient. This is often the tradeoff in performance.

The bigger question, or the question you should be asking yourself, is how can I guarantee this will happen again next year? Unfortunately, the answer is “you can’t” and don’t believe anyone that says you can. All we can do is track how these strategies perform over time. Specifically, we are looking for consistency, which is why we conduct an update every two months like the one you’re reading now. We continue to mine and isolate the attributes of our top performing strategies with consistent results. The goal is to take the best attributes from the best performing strategies to create the holy grail of trade strategy.

In September, we published a post titled: Top 5 Strategies By Performance. It gives an overview of the top 5 strategies in each category (based on data from the September update). This is a summary of the post:

• Strategy 3 has the highest net profitability at $331K.

• Strategy 3 has the highest profit per day at $1,311.

• Strategy 3 has the highest daily cumulative profit at $331K.

• Strategy 8 has the highest percent profitability at 58%.

• Strategy 10 has the highest return on max draw at 1958%.

• Strategy 23 is the only strategy in this range with a drawdown of less than 1%

• Strategy 23 has the lowest daily net profit at $1,675.

• Strategy 25 has the highest daily net profit at $19K.

• Strategy 28 has the lowest daily cumulative profit at $9,690.

• Strategy 28 has the highest profit per trade at $525/trade.

• Strategy 32 has the highest profit factor at 1.93.

Subscribers, click here for a link to a discussion on how we used these top strategies to mine for the specific attributes that make these some of our best strategies. We will include an update to this post in the annual report for subscribers in the beginning of January 2022.

Profit Factor

We want to take some time in this update to focus on profit factor (gross profit / gross loss). It is our primary gauge of performance. Specifically, we want to track how consistent the profit factor is from the September update to the November update. Actual net profit might change, but all we really care about is profit factor from a consistency perspective. The following table provides an overview of the profit factor increase from the last period to the current period.

First, a bit on how to read this table:

• The right side is the strategy.

• The left side is the change in profit factor from the last update.

• All strategies marked in yellow have been optimized.

• All strategies marked in blue experienced a considerable change in profit factor (at least .03 points) from the last update.

• All strategies with an “*” have less than 253 trades for the year.

Based on our logic, everything in white would be considered “consistent” from the September update to the November update. So that’s strategies: 1, 2, 3, 5, 6, 7, 8, 10, 11, 12, 13, 14, 31 and 35. Everything in blue would not be considered consistent. Some strategies experienced an increase with the September update, like Strategy 4, but are now back to where they were with the July update. This could be a kind of consistency as well. We are in the process of creating a graph to visually track the profit factor over time. This should provide more insight into what ‘longer-term consistency’ looks like.

We also noticed the following:

• Almost all optimized strategies, except for Strategy 13, experienced a considerable change.

• Almost all strategies with less than 253 trades, except for Strategy 31 and 35, had a considerable change.

What does this mean? It means our previous observations are supported here as well: optimized strategies and strategies with a low trade count tend to have inconsistent backtest results. This does not mean that the strategy is not ‘good’, it simply means that we can’t trust the backtest results as much as we can for unoptimized strategies or strategies with more than 253 trades per year. It also means that these strategies will require more time than others to determine consistency. It also suggests that many of our earlier strategies are consistent, while our more recent strategies are not. Again, this does not mean that these strategies are poor performers, but our only means of testing a strategy is with a backtest so we must focus on those strategies that we know are more consistent with backtest results. Going forward, we will not be publishing/recording any optimized strategies or strategies with less than 253 trades for the year.

Some of you may be asking, why not just extend the backtest period until you get 500+ trades? It’s a good question. We did just that and conducted an audit to test the data. The further back you go, the more issues we found with the data and we used several data sources. So for now at least we’re only backtesting on the most recent 12 month period.

Answers to Some Commonly Asked Questions:

• Perhaps one of the most commonly asked questions we get is, how much do I need to get started? This is hard to say because it depends on many things, but ultimately you want to set your simulated account to an amount that is equal to the max drawdown plus some amount to buffer for negative attributes like a low profit factor. For more on what we think is the best way to approach this question for yourself, click here.

• We continue to receive questions about funded trader programs. There is no easy answer to this question, but we can tell you for certain that the answer does not lie with one of our strategies. I passed the test with 5 days to spare, but it wasn’t with one of our strategies. So there’s no need to subscribe if that’s what you’re looking for. That said, we can give you some guidance on what you need to think about in order to pass. For more on what we think about the best way to approach funded trader programs click here.

• In general, one of the most common questions we get pertains to the use of our strategies on other assets like crypto. You can read more about that in the post: Do our strategies work on cryptocurrencies? We’re also looking at other equity indices and asset classes. For a post on what we found when comparing NQ to RTY, click here. We also recently started a series that looks at futures contracts for other assets like commodities, bonds and forex. The first strategy in the series was Strategy 36 because it has the highest profit factor. You can view the results here. Going forward, each backtest will have its own post.

• Another commonly asked question is in regards to overfitting. You can read the following posts to learn more about overfitting and what we’ve done to reduce its impact: Overfitting: What is it and what can we do about it and What Are We Doing To Ensure Backtest Accuracy? A few months later, we received a question regarding strategy strength and what optimization tools we’re using to increase robustness. We answered that question in the post: How Can You Tell If An Automated Trading Strategy Will Perform Well Over Time? After writing these posts, we became interested in creating a strategy that optimizes in real time. That is, how can we can optimize a strategy as close to the event as possible. In this way, the optimization is still based on backtest results, but the data used in the backtest is based on a very short period of time. Theoretically, overfitting should be less of a problem the closer the optimization occurs to the event. You can read more about this here.

• One of our subscribers told us that the strategies were too hard to understand. To this we say, “Ask us anything. We’re here to help.” Practice on Strategies 1 and 5 (both are free) first and then ask us if you have any questions. Also, unlike most websites, we give you a full description of the strategy so you have the mechanics of the strategy, not just a license to use it.

• Another common complaint/question we get is: Your free strategies aren’t that good. Why is that? I agree, that’s why we’re giving them away. The goal is to show you how our process works.

• One of the most commonly asked questions we get is: How do I learn how to create automated strategies for myself? Our advice is to use the resources provided by Ninjatrader. We receive no remuneration for saying this. We are saying this because we like the platform and they have some great classes online. To learn more about their daily training programs, click here. See below for an overview of the kind of training they have and it’s all free.

  

What’s in the pipeline:

• We just started a weekly update series. The goal of the series is to track strategies on a weekly basis to see if we can discover how to select the holy grail of trade strategy. It is based on the premise that the holy grail of automated trade strategy is not static, but fluid or rather made up of multiple strategies. To read more about that, click here. We’re going to call the weekly update the ‘Mudder’ Report. If you don’t understand why, you will after reading the article. Some of you got upset because we used a horse race analogy. This is the ultimate hunt, not a horse race, we get it. But, we can gain insight by looking at the hunt from different angles. Looking at strategies on a weekly basis helps to understand what’s happening between each update.

• We’ve had several requests to publish the Mudder Report on Sunday and we’re going to make every effort to do so, but it also dawned on us that some people might be using the Mudder Report to actually trade live. Our goal is not to predict or recommend what to trade in the following week. It’s always fun to try your luck on a simulated account, but if you use the weekly update to trade live, you will get burned. Our goal in developing the Mudder Report is to aid in the development of a few working theories for the holy grail of automated trade strategy, which we believe could be any one or more of the following:

  • static

  • a moving target (uses multiple strategies)

  • made up of multiple contracts using a static strategy

  • made up of multiple contracts using multiple strategies.

• We also have several new strategies coming out in the next two weeks. Stay tuned.

If you have any questions, feel free to contact us directly at automatedtradingstrategies@substack.com.

2022 Could Be One Of The Most Volatile Years In Recent History

Calm Before A Storm, Henry Moore, 1883

Last week we published the results from a backtest of all strategies on the E-Mini Russell 2000 (RTY). We then compared those results with the E-Mini NASDAQ 100 (NQ) backtest. There were some interesting observations that you can read more about here.

In that post we talked about the impact of volatility on trading profits. We also reviewed the definition of volatility:

“A measure of dispersion around the mean.”

…and, discussed what our next steps were in the process: namely to compare our backtest results with other assets. In particular, those assets that may be more volatile than indices, like commodities.

Why are we doing this? Of course we’re curious about how well our strategies perform across multiple asset classes, but we also think 2022 is going to be one of the most volatile years in recent history. In general, higher volatility leads to better trading opportunities. And, we believe that volatility will be driven by inflation.

Why does inflation matter? Higher inflation or inflation volatility drives overall volatility in the markets, especially the commodity and currency markets. Since volatility is a trader’s best friend and we’re in the process of trying to find the holy grail of automated trade strategy (both moving target and static), it’s important to know how inflation will impact the markets. At the very least, we need to make an informed guess.

The good news (at least for traders) is that we believe 2022 will be one of the most volatile years in recent history and we are not prone to hyperbole. If this is the case, we want to make sure we’re looking at all options because there are assets that are even more volatile in price than the NQ and RTY. We are referring to commodities.

Inflation is Here

Unless you’ve been under a rock, you know that inflation is here. Prices for underlying inputs have gone up all over the world and those underlying inputs are commodities, quite literally. What does this mean? It means prices are going up because the value of the currency in which that asset is expressed in is going down. This spells trouble for all major currencies (this is a global issue), but assets that derive their value from something other than the backing of a central government will go up in both price and value. These are the assets you want to own.

Everyone has a different opinion as to why prices are rising. Some say it’s due to supply chain bottlenecks. Others say it’s due to enormous pandemic-related stimulus programs conducted by central banks. Still others say it’s the historically accommodative amounts of liquidity (cash) and historically easy financial conditions (rates near 0%), but we can all agree on the end result because it’s staring us in the face: inflation. Inflation is here and it’s not going anywhere.

As central banks print cash, managers are looking for a place to park it, which is pushing markets up even higher, especially those markets used to hedge risk. In other words, the premium on insurance (futures contracts) is going up.

While there may not be a consensus by the Fed on when it will start raising rates, the market is assuming at least three rate hikes before the end of 2023. And, while Jay Powell, US Fed chair, doesn’t want to link the start of tapering asset/bond buying programs with the onset of higher policy rates, we know they will be overlapping events. There are reports that show the Fed parring off its Treasury bond buys by $10 billion and its mortgage-backed security purchases by $5 billion each month beginning as soon as the middle of November.

The buyback scheme was put in place to address ‘dysfunctional trading conditions’ in the bond market during one of the sharpest contractions in history, but who’s to say that tapering the program won’t create another run on liquidity. If it does, the Fed will only be compelled to hasten its plans to raise rates in the face of increasingly higher inflation. As it stands, investors are challenging the view that the market is ready to stand on its own two feet. When you mix that with continued supply chain bottlenecks, which the Fed has already admitted has caught them off guard, it makes for fun times in the world of trading. This is especially true for commodities, which always increase with inflation and make equity indices have the appearance of treading water.

We are at the beginning of a perfect storm. And, it all makes sense because we are on the heels of what some would say has become a biblical event — the global pandemic.

The chart below provides an overview of the most volatile equity indices.

https://take-profit.org/en/volatility/indices/

Compare this with the chart for commodities:

https://take-profit.org/en/volatility/indices/

We are currently working on backtest results for some of these markets and will share them with subscribers first. This series will take some time to complete and we’re going to drop one backtest at a time based on each strategy. We’ll start with the strategies that have the highest profit factor for the NQ futures contract. The backtest will include the following futures contracts:

6B - British Pound (FX)

6E - Euro (FX)

CL - Crude Oil

ES - E-Mini S&P 500

FDAX - DAX

GC - Gold

NQ - E-Mini NASDAQ 100

RTY - E-Mini Russell 2000

YM - Mini Dow

ZB - 30 year Bond

ZN - 10Year Note

ZS - Soybeans

ZW - Wheat

Here’s the first backtest for Strategy 36:

For links to all strategies click here

This backtest is based on data from November 1, 2020 to November 1, 2021.

As you can see, Strategy 36 continues to perform well with NQ. It has a slightly higher profit factor than it did in the original backtest from August 31, 2020 to August 31, 2021, but it also performs well with CL (2.56 profit factor), ES (2.12 profit factor), ZS (1.82 profit factor) and YM (2.71 profit factor). It did not do well with currencies, gold, or bonds. In fact, it performed so poorly with bonds (.51 profit factor) that the inverse strategy might be worth looking at. We will be investigating this relationship further.

What’s Next

It is our goal to drop at least one backtest per week based on the instruments listed above. These will be short posts and not all posts will be available to the public, but our hope is to develop any trends we find among certain asset classes before the new year. We also plan on doing another backtest on the most relevant crypto crosses, micro contracts like MNQ, and certain stocks in the NQ index.

While the holy grail of trade strategy should work regardless of the market, our hope is to use the fundamental knowledge of what we can expect from the market in 2022 to make some predictions about the best markets and strategies to use to take advantage of what may become a once in a lifetime event. It will also help to understand/gauge the strength of each strategy and what it is mechanically that makes those strategies more robust than others. To learn more about what makes a strategy robust, click here.

Our Collective Freedom, The Ideal of America, Is Not For Sale

The Goldfinch – Carel Fabritius

If forced to label myself, I would say that I'm an egalitarian. I believe that each person has the right to live life in any way they choose as long as it doesn't hurt or infringe upon the rights of others. 

Do unto others as you would have them do unto you.

It's a deceptively simple doctrine to live by, because it takes strength to see yourself in the next man when he can't see himself in you. 

While I believe in local government, I also believe that like Gollum's precious ring, power corrupts. It turns God's best into greedy little trolls and makes it easier for the next man to dismiss you as being beneath him. And, while a federal government may seem like a progressive next step, it is actually antithetical to the movement started by the libertarians and egalitarians that created the ideal of America. None of us, including them, ever lived in its ideal -- a place where all men are created equal, endowed by their creators with certain inalienable rights, among them, life, liberty and the pursuit of happiness. 

Can you imagine thinking that our leaders were taking steps to secure our happiness? It is unimaginable to most, but the founders of this country took the first step. They stepped on the ideal of others to create it for themselves, but it was created nonetheless and we are, for better or worse, who we are because of it. Either way we have a debt to pay and it cannot be paid in dollars. It must be paid by your own pursuit of liberty and justice for all.

Money and freedom are not the same

It is wrong to believe that money and freedom are the same. The brainwashing begins early on in life, so it's hard to decouple the two, but you don't need money to gain your freedom. In fact, the pursuit of money is often at odds with your freedom. You don't need capital for liberty. Liberty is an unalienable right endowed upon us by our creator(s). And, if our creator(s) bestowed it upon us, why let a man, or even a nation led by a man, take that unalienable right away. 

So I fight for liberty for all. These aren't mere words -- they mean everything. They can seem lofty and ephemeral, but there are ways to anchor them to reality. The best way to do this is by asking the right question with every action. Is your goal to be rich or free? Do you strive for acceptance or freedom? Do you worship the dollar or liberty?

Many good people are conflicted by these questions and with good reason. What good is liberty if your child is hungry? It is this fear that causes delusional thinking. Instead of logic and reason, we turn to whatever works, which usually doesn't.

The beautiful thing about the system we live in is that there are many options. It may feel as though your path is pre-defined, but this is yet another delusion. In reality, your options, from minute to minute, are unlimited. And, you don't have to quit your day job to choose a different path.

The right path begins with the right question

José Clemente Orozco's painting, The Demagogue

 

I’m a big fan of Elon Musk. In an interview about how he and his team tackled challenges when creating Tesla, he lamented, "A lot of times the question is harder than the answer. If you can properly phrase the question, then the answer is the easy part." I like this quote because it exposes a truth that we often get wrong—finding your holy grail in life (whatever that may be) is about asking the right questions. If you ask the right questions, the answer will come. 

According to INC magazine,

Musk recalls that his first questions as a founder were pretty mundane, like, how can I make enough money to live and still have time for games and computers? His first few companies, like Zip2 and X.com, were designed to make money. As he got more money, he got frustrated with how money was transferred, and that led to PayPal. Today, the market cap of Tesla is around $40 billion and SpaceX is estimated at $15 billion. As Musk's founder questions have gotten better, his companies have gotten bigger. "To the degree we better understand the universe, then we know which questions to ask," he says.

Elon’s first question was “how do I make money”, but he didn’t start making real money until he asked the question “how can I make transferring money easier?” The answer to that question was Paypal and the rest is history.

So, the right question is not:

How can I make more money? (wrong)

It is:

How can I get more freedom? (right)

Another question I used to ask myself was:

Which political party should I donate to? (wrong)

But the real question to ask is: 

How can I create real change in my neighborhood and beyond? (right)

I haven't voted in an election for many years. I believe in mutual aid, not electoral politics. I donate to the People's Party and Jimmy Dore, but that's it. Any other donation goes straight to the people in need and the people most in contact with those people. 

Take the time to learn more about where you buy groceries. Is it a corporation or employee owned? Before supporting your favorite brands, find out if their employees are on strike and why. If you have the luxury, as much as possible, make your purchase decisions based on liberty rather than the lowest price and our collective freedom will come.

Automated Trading Strategies: September 2021 Update (In The Last 12 months Our Strategies Have Made Over $2.8M Based on Backtest Results)

Every two months we like to look at how our top automated trading strategies are performing and update you on our key findings, highlights, takeaways and what’s in the pipeline. Externally, it tells you where we are and allows subscribers to give their input. Much of our research has been a derivative of subscriber inquiry.

Internally, we are looking at two things in particular — alpha decay/consistency of published strategies — what can this tell us about the best strategy attributes and how can we use these attributes together to create a strategy that gets us closer to the holy grail of trade strategy. We define the holy grail of trade strategy as having the following attributes:

• Profit factor greater than 3

• Annual drawdown less than 3%

• Annual return greater than 500%

• Minimum daily net profit of -$1,000

• Avg Daily profit greater than $1,000

• Less than 5,000 trades annually

• Greater than 500 trades annually

Notice that we’ve added another bullet — the strategy has to produce more than 500 trades per year. We’ve noticed that strategies that produce less tend to have inconsistent backtest results.

We’ve also added two columns to the end of the backtest table - cumulative low and cumulative high (see below). Instead of just looking at the lowest and highest daily net profit, we wanted to look at the cumulative high and low over the year. While this is highly dependent on when you start the strategy, when combined with profit factor and max drawdown, it can provide a compelling risk/reward profile.

For example, Strategy 28 has the lowest daily cumulative profit at $9,690, which means that even though the strategy had a max drawdown of $16,365, the total account value never dropped below $9,690 on a rolling or cumulative basis. Yes, this means that on day 1, the strategy made $9,690. Again, this has more to do with when the strategy is started than profitability, but it can make all the difference in the world, especially with a highly volatile strategy. This is the main reason we include profitability per day of week and hour of day in our strategy results. We’ll discuss this more later.

7/2/2020 - 7/2/2021 Vs. 8/30/2020 - 8/30/2021

So where do we stand in comparison with our last update? The table below provides the backtest results from our last update: 7/2/2020 - 7/2/2021

The following table provides the backtest results for the current or most recent update: 8/30/2020 - 8/30/2021

[Our apologies for the small font, but you can enlarge by clicking on the picture.]

In general, when comparing the performance of both time periods, we had the same drawdown as the last update and a slight decrease in the return on max drawdown due to a decline in profitability. Average profit factor for the entire portfolio decreased from 1.26 to 1.24 and average net profitability decreased from $98K to $89K. Profit per day decreased from $388 to $355 and profit per trade decreased from $145 to $116. The number of profitable trades also decreased slightly from 44% to 43%.

Most of the deterioration occurred with optimized strategies (marked in yellow on the chart). Optimized strategies are more prone to overfitting and therefore have highly inconsistent results.

So what happens if we remove all optimized strategies from the portfolio? The chart below provides a comparison with and without optimized strategies.

Profitability increased from $89K to $110K on average. Likewise, average profit factor increased from 1.24 to 1.30, profit per day increased from $355 to $436, and; profit per trade increased from $116 to $155.

What does this tell us? Optimized strategies do not work, at least using the methodology we’ve used in the past — almost all optimized strategies performed worse except for Strategy 22. It also tells us that our unoptimized strategies are performing well against the previous update. Strategies 4, 7, 8, 10, 11, 26 and 31 all performed better. Strategies 4 and 26 had the highest increase in profit factor.

For the year, net profit for the total portfolio increased from $2,223,095 to $2,788,075 with a total max drawdown of $643,640. Specifically, the model is telling us that a $643K investment could have made $2.8 million between 8/30/2020 - 8/30/2021. This amount does not include commissions. We advise a fixed price commission plan for strategies with more than 1 trade per day.

The biggest question, or the question you should be asking yourself, is how can I guarantee this will happen again next year? Unfortunately, the answer is you can’t and don’t believe anyone that says that you can. All we can do is track how these strategies perform over time. Specifically, we are looking for consistency over time, which is why we conduct an update every two months like the one you’re reading now. We continue to mine and isolate the attributes of our top performing strategies with consistent results. The goal is to take the best attributes from the best performing strategies to create the holy grail of trade strategy.

We recently published a post titled: Top 5 Strategies By Performance. It gives an overview of the top 5 strategies in each category. This is a summary of the post:

• Strategy 3 has the highest net profitability at $331K.

• Strategy 3 has the highest profit per day at $1,311.

• Strategy 8 has the highest percent profitability at 58%.

• Strategy 10 has the highest return on max draw at 1958%.

• Strategy 23 is the only strategy with a drawdown of less than 1%

• Strategy 23 has the lowest daily net profit at $1,675.

• Strategy 25 has the highest daily net profit at $19K.

• Strategy 25 has the highest daily cumulative profit at $331K.

• Strategy 28 has the lowest daily cumulative profit at $9,690.

• Strategy 28 has the highest profit per trade at $525/trade.

• Strategy 32 has the highest profit factor at 1.93.

For links to all strategies click here.

Subscribers, click here for a link to a discussion on how we used these top strategies to mine for the specific attributes that make these our best strategies.

What’s in the pipeline:

Much is in the pipeline.

• We recently published a post titled: Do our strategies work on cryptocurrencies? In general, one of the most common questions we get pertains to the use of our strategies on other assets like crypto, so we’re also going to publish backtest results for other asset classes, including stocks.

• We’re also looking at various data series types outside of ‘range’ like minute, tick, volume and Renko. Subscribers can click here for the full post on our experience with Renko backtest results.

• Another commonly asked question is in regards to overfitting. You can read the following posts to learn more about overfitting and what we’ve done to reduce its impact: Overfitting: What is it and what can we do about it and What Are We Doing To Ensure Backtest Accuracy? After writing these posts, we became interested in creating a strategy that optimizes in real time. That is, instead of optimizing a strategy in 1 year backtest results, and then using those results as a way to place parameters on the future, we can optimize a strategy as close to the event as possible. In this way, the optimization is still based on backtest results, but the data used in the backtest is based on a very short period of time. Theoretically, overfitting should be less of a problem the closer the optimization occurs to the event. We’ll be publishing information about this indicator shortly.

• We’ll also be publishing a post looking at strategies that use divergence theory. We started this research with the post: Can We Maximize The Profit Factor By Focusing On Timing?

• We also plan on publishing a follow up post on market structure. We received some great feedback on this post and we’re hoping to incorporate it into our regular analysis.

• We continue to receive questions about the difference between Total Investment & Reinvestment: This is our answer: Reinvestment Vs Total Investment. In addition, we’ve just added cumulative net profit to our table as a key performance indicator. When used in conjunction with max drawdown and return on max drawdown, you can get a better understanding for how much you’ll need to invest/reinvest in a strategy. Some of our strategies have positive cumulative net profit, meaning that the account value never drops below 0. We take a deeper look at these strategies in the post: Top 5 Strategies By Performance Indicator.

• There are several other posts in the works, but it would take too long to discuss them here. Some research has been delayed and some has been pushed up. If you have any questions about anything in particular, please reach out to us directly.

What Can Auction Mechanics & Market Structure Tell Us About Trade Strategy?

The market is based on auction mechanics. This is why prices aren’t random.

An auction is a system of buying and selling goods or services by offering them for bidding—allowing people to bid and selling to the highest bidder. The bidders compete against each other, with each subsequent bid being higher than the previous bid. Once an item is placed for sale, the auctioneer will start at a relatively low price to attract a large number of bidders. The price increases each time someone makes a new, higher bid until finally, no other bidders are willing to offer more than the most recent bid, and the highest bidder takes the item. -Corporate Finance Institute

This is the nature of a simple auction, but this isn’t how the futures market works. In a futures market the auctioneer is on a never ending search for price discovery.

The futures market is actually the result of tulip auctions conducted by Dutch traders in the 1600’s. They wanted to buy and sell tulips year round, not just for 3 months of the year. This is the beginning of what is known today as the Dutch Auction.

A Dutch Auction is when the auctioneer starts with a high asking price and lowers the bid until one of the bidders accepts the price. Items for sale are allocated based on the bid order: the first highest bidder picks their items, followed by the second highest bidder, and the third highest and so on. Dutch auctions are often used for commodities and investment securities. Alphabet (Google) had a famous example of a Dutch Auction —Google’s Dutch Auction.

So what? This means that in a Dutch Auction new information can come in and change the direction of the market at any time. This new information acts like a wind that changes the direction of the price wave.

So, in a Dutch Auction, new information isn’t immediately digested into the market — it is a process that is based on bid order and that process usually plays out in threes. First, you will see an initial push or “price attempt” to change the direction. The first move is the initial test and is followed by a pullback. Then you will notice a second attempt, which I like to refer to as the bandwagon trade. Then there’s the last movement before the price moves in the opposite direction. The price will continue to trend in either direction until it finds equilibrium.

This is the framework or template for price discovery in the futures market. The variables that impact this process the most are speed (tick per second) and volume or market depth. So, if you had to create a formula for the best conditions to trade it might look something like this:

Price Potential = Speed of price * market depth * market divergence/market orders

Where price potential is the potential change in price, speed of price is measured as ticks per second, market depth is measured in terms of volume expansion/contraction and market divergence is measured as (number of contracts traded / number of orders). Keep in mind, there will always be more contracts traded than orders because traders often trade more than 1 contract. We are currently in the process of developing a strategy that makes use of this formula as a way to define the best time to trade. We’re also using it as a barometer for knowing when to “flip” the strategy (make the opposite trade).

In general, the higher the potential, the better the trade. Low potential is marked by quick pullbacks and choppy markets. High potential is marked by strong price movements. Those movements can be either sideways or trending up/down. They can also be tracked by lines referred to as “trend” lines. If a price is trading within known trend lines it will hit a price on the line and then go the other way. In other words, the lines act as support and resistance for the price. When the price is pushed outside of these lines, it is referred to as a breakout.

Breakout Trades

You can confirm that new information has entered the market with breakout activity. It is also marked by an expansion in volume as well as an increase in the speed of the market. The best way to take advantage of a breakout is by tracking the trend line. When the price breaks above the trend line, you’ve got a breakout, which means new bidders/seller have entered the room and have changed the direction of the trend. In general, breakouts only happen 1 out of 3 times that a price hits the trend line.

So the market is a combination of trends and breakouts. And these trend line and breakout attempts are patterns that repeat over and over again. You can find this structure (3 movements) at every time frame of the market. If you drill down, you’ll find it at the 1 minute or 150 tick level. If you drill out, you’ll find it at the 15 min or 5,000 tick level. The same patterns exist at every level of the market. In this way, the market moves like a fractal. That is, it creates a pattern that repeats on itself. One of the best Youtube videos that explains this is below.

The point is, you will find the same patterns at the 36 range level that you will find at the 18 range level.

Here’s an example of a 36 range chart and 3 price movements marked as 1, 2 and 3.

The trend line marked in light green supports all three movements. The price finally breaks below the trend line in a downward move. The movement starts with an upward breakout from a downtrend (marked by the blue trend line). If you’re a subscriber click here for instructions on how to use our favorite trend line indicator.

The chart above is a 36 range chart and the chart below is an 18 range chart. Specifically, this is a price chart of the first movement in the 36 range. It has three movements within it. And, if you look closely, you will see that the first movement in the 18 range also has three movements within it. Again, the market is a fractal.

This is a common pattern in trading due to auction mechanics.

From a candlestick perspective, this is referred to as the three bullish soldiers — each bar going higher than the last. To those that follow candlestick patterns, this is a very bullish signal.

And then of course there’s the great Richard Demille Wyckoff — one of the most well known traders in history. In the early 1900’s, Wyckoff developed a strategy around market mechanics and named each segment of the trade (1, 2 & 3) Spring, Throwback and Correction.

From a market mechanics perspective these movements are marked by an accumulation and distribution phase — accumulation leads to an increase in price while distributions lead to a decrease in price. Wyckoff’s book is still taught on Wall Street because market structure is a function of the underlying dutch auction. You can view a pdf of his original work on the Internet Archive by clicking here.

How Can We Use This To Help Us Find The Holy Grail of Trade Strategy?

We are still on the hunt for the holy grail of automated trade strategy. If we apply the rules of market structure, we can use it to look for two events that are certain to happen:

1) bandwagon movements

2) breakouts

Like most strategies, we are searching for an event that takes place within a condition and then we to need to define the action to take within that event. In this case, the event is a bandwagon movement or a breakout and the action is a buy or sell order with a trailing stop and possibly a limit order on entry. The trick (or the part we’ve failed at thus far) is to define the conditional that describes the price action just before the bandwagon movement or breakout occurs.

To learn more about who we are and our quest for the holy grail of trade strategy, click here.

---

A common question we get is, ‘do you teach’? The answer is no. None of us have any interest in teaching at the moment, but we can give you a recommendation. If you’re interested in learning more about futures trading or automated trading we recommend taking classes from Quantra. They’ve been around for over 10 years and have a mix of free and paid classes. Before signing up for anything, however, check out the credentials of Quantra’s faculty and compare it to other online schools. You’re looking for a mix of academic, practical, and institutional experience.

Automated Trading Strategies: Overffitting -- What is it and what can we do about it

First, happy reconstitution day traders! This is often the heaviest trading volume day of the year and this year it’s happening today (June 25). Get ready for a wild ride this afternoon. Click here to read more about what reconstitution is and how it impacts trading.

Second, there’s a mechanical elephant in the room and it’s called “overfitting”.

We aren’t programmers, statisticians or mathematicians in any way. So, we have to translate many of these terms into something that makes sense to us from a trading perspective.

In a nutshell, the definition of overfitting is: Good performance on backtested data, bad performance on future data. Specifically, it’s when a strategy learns the “noise” that’s specific to the backtest period, but that noise does not apply to future data.

So, the question is: what is noise in trading and how do you know the difference between a strategy that has been “overfitted” to the backtest data, but won’t perform well in the future and a strategy that performs well in backtests as well as in the future?

This is what we’re hoping to answer with this post.

Trading is both art and science

An illustration from Jules Verne's novel "The Steam House" drawn by Léon Benett.

I could limit this discussion to what we’re doing to prevent overfitting, which I do below, but I think it’s also important to understand how it applies to trading.

This is not an easy topic to discuss, primarily because the market is dynamic — statistical tools are not a fan of dynamic. Likewise, any attempt to forecast the market requires both art and science. Overfitting may explain why simple models don’t work in the future, but it has limitations on models that are dynamic and influenced heavily by outside forces, also referred to as noise.

I was amazed by how much literature there is on the topic of overfitting. It is of particular interest to those in the simulation or machine learning space, which makes sense. These strategies can be viewed as bots or machines and the holy grail of trading strategy is a bot that has “learned” the market. To do this, the bot would have to “learn” how to treat noise. Is it good noise or bad noise? This is where the comparison breaks down. Eliminating the noise in trading is extremely difficult because the market is primarily a reaction to noise, but we may be able to discern the good noise from the bad.

What does this have to do with overfitting? Overfitting happens when a model learns the detail and noise in the historical data that negatively impacts the performance of the model on new data. We know that we can optimize a strategy that then performs poorly in the future, but does it perform poorly because it learned the noise or because the noise is unlearnable? The only thing that’s truly learnable is the ability to identify the ideal conditions for a strategy to operate in. In other words, we’re looking for a strategy that knows when to trade. We can’t control the market, but we can control when the strategy takes action. The result is a strategy with fewer false flags (carving out the gross loss) and a higher profit factor.

So on one hand we can use the concept of overfitting to understand why a strategy does not perform well in the future, but on the other hand we must admit that the concept has limits for something as dynamic as market structure. It is almost impossible to not engage in overfitting when developing strategies, but there are things we can do to minimize it.

What can we do?

I want to go back to the original question. How are we making sure that our strategies aren’t the product of overfitting and will continue to do well in the future?

• Optimize on hyperparameters one at a time - When we make changes to a strategy we make them one at a time rather than optimizing several variables at the same time. This gives us an opportunity to isolate the benefit of each change to net profit and profit factor.

• Change the conditions of the strategy, not the hyperparameters: We don’t just want to create the machine, we want to change the surroundings or the environment of the machine. We want to create resistance to overfitting within the strategy itself. Put yet another way, we want to create the machine and define the conditional path it takes within the strategy. By doing so, we aren’t as concerned with the hyperparameters as we are the conditions of the strategy. For example, we aren’t as concerned with optimizing the hyperparameters of MACD (slow parameter, smooth, and fast) as much as we are with creating a condition that triggers an action (MACD cross when volume is higher than 3000). The trader can then update the volume accordingly rather than the hyperparameters of MACD. Volume sets the condition. In this way, the holy grail could be a moving target.

• Focus on scenarios rather than sensitivity - This piggybacks on the above. We’re more interested in creating an environment within the strategy. This environment is like a scenario (mid day, low volatility, low volume) rather than a single variable. So, while we optimize based on one variable at a time, the conditions within the strategy are based on a scenario.

• Use two sets of data in the backtest: Going forward, in addition to doing what we’re doing now, we’d like to use two sets of historical data, one to optimize on and one to test on, both are part of historical data. We’re thinking of using 3 months of validation data for the test along with a 3 month break. So that’s 18 months altogether. We’ll use the first 15 months in the backtest, followed by a 3 month break and then use the next 3 months as a test in two different data charts. This format may change, but this is the current plan.

• Use Gross Loss/Profit in the calculation of Profit Factor - We use gross loss and profit rather than a variant that uses average loss and profit when calculating profit factor. The average ratio may smooth out the impact of big winners and losers, but that seems to lend itself to overfitting, especially since this is one of the primary performance measures we track and optimize on.

• Use more than one instrument in backtest data - As a final check, we’re going to start running strategies through other instruments such as ES, GC, BTC, YM and stocks.

We’re hoping that these actions will help to ensure we’re picking up the “good” noise and not the noise that’s specific to a particular day or time frame when optimizing on backtests. That said, some of our strategies have no optimization at all and our focus is still heavily concentrated on NQ for its volatility.

Conclusion

I believe the market is full of both random and definitive events. Some things you do every day, some things are random (you know you eat lunch at around 12pm, but you don’t know what you’re going to eat). Like a simulation, everything is interconnected and random in the market and yet if you span out far enough some predictions can be made. It is this fractal nature of trading that makes it so interesting. The conundrum in trading is that the further you span out (i.e. increase the time series) the more susceptible you are to risk; the more predictable the market is, the more risk your trades carry. The predictable part of trading is the science (indicators, etc) and the unpredictable part is called noise. Indicators act different based on the noise. For example, most indicators are highly predictable in a rally, which is why traders love a good rally. So perhaps the best course of action is to create a strategy that is only triggered during a rally. This is much easier said than done, but just saying it gets us closer to the goal.

Trading is both art and science. Some of the most seminal papers in economic theory use formulas filled with assumptions that don’t make any sense (i.e., symmetry of information — an investor will never have access to the same information that a corporation would and investors will rarely behave rationally). These assumptions hold no relevance now, but they were used as a way to create a science out of economic theory and that science has little real world application. So it’s important to use statistics to track and measure progress and performance, but it’s also important to realize that statistical tools have a limit. Newton had to create the mathematical form we call calculus to express the laws of physics. Likewise, it could be that forecasting market structure requires a completely different way of thinking about “learning”, especially when the prevailing analytical framework dismisses relevant data as noise. As a result, data scientists are quick to blame poor performance on noise, but if we’re looking for a mathematical formula that can predict the true nature of the market it must include the noise. It isn’t that thinking about the market from a statistical perspective is bad, but it can limit our visibility and therefore our ability to find the holy grail of trade strategy if we get too attached.

Ultimately, while there are several key takeaways from this discussion, the main one is that not all noise in trading is bad. The key to discerning the good from the bad isn’t about changing the actions of the strategy, but the environment that those actions are taken in. And, we’re finding that those strategies that control the good noise through conditional equations and data series calculation (minute, range, tick, etc) seem to do better in the future.

What’s In The Pipeline?

There’s a lot in the pipeline, but we’re prioritizing something rather big right now. We’ve been impaled by a unicorn in the past, so we’re doing some due diligence before sharing, but the preliminary results are phenomenal. We want to say thank you to Pierre, one of our subscribers, for making a research request that has the potential to improve all of our strategies. Again, we’re compiling the data now and will share as soon as we feel it’s been properly vetted.

Happy trading!

To read more about our Automated Trading Strategies click here.

To subscribe to the Automated Trading Strategies newsletter click here. 

(this was originally posted on Automated Trading Strategies)