“Backtest” and “private markets” are not often used in the same sentence. CircleUp is changing that by building the first ever systematic VC fund in the private markets and utilizing backtests to validate the approach. In short, this means that we rigorously test the investment strategy using historical data in a way that has never before been possible in the private markets.
WHAT IS BACKTESTING?
Backtesting is an approach used in the public markets to test investing strategies on historical time-series data. If a strategy’s backtest generates strong ex-post returns, it gives the portfolio manager (and investors) the confidence to invest real dollars in that rules-based approach.
At CircleUp, we are backtesting algorithmically generated portfolios in the private markets, which has never been done before. For good reason.
WHY DO BACKTESTS MATTER?
In the absence of a backtest, private market LPs have relied on a manager’s track record to underwrite strategies. They are betting that if a manger has invested in good companies in the past, they should be able to invest in good companies in the future.
Backtesting offers a different way to build conviction in investing strategies, often with a more reliable (larger sample) and interpretable (data-driven decisions) method to evaluate repeatability. This isn’t to say that backtests are perfect – spurious correlations exist and some signals are nearly impossible to quantify. But still, any reasonable LP should be delighted by a backtest in the private markets as a new way to evaluate a manager and their strategy.
Reliability (Larger Sample)
- A traditional fund manager only makes a handful of investments in any given year. That means their track record is subject to small-sample bias. In many cases a fund’s performance is largely dependent on the success of a single breakout deal.
- In contrast, backtests consider the outcomes of thousands of companies to de-risk reliance on a single investment to drive fund returns. Portfolios may still rely on breakout brands but we can run Monte Carlo simulations to help quantify the risk.
Interpretability (Data-Driven Decisions)
- It is nearly impossible to quantify heuristic-driven investment decisions. How much of the decision was based on the team vs. the business model? Even the very best investors out there (and we have tremendous respect for Benchmark) admit that the discretionary process is intangible. A traditional track record shows objective company performance, but not an objective view as to why each investment was made. Without the ‘why’ it’s hard to determine whether it was skill or luck that yielded the desired outcome. Jeremiah Lowin says it best: “Investors [are] the ultimate black boxes.” (Invest Like the Best, 0:18)
- Backtests can be more interpretable (unless the model is black box) and can point to the exact economic intuition that selected a given investment in a given year – which factor had the strongest signal? An algorithm can provide unaltered interpretability of the historical decision.
Again, this isn’t to say that backtests are perfect. Signals of the past may not be relevant in the future. The role of a systematic manager is to identify a pattern, use historical data to better understand the potential value of that pattern, and then make investment decisions using that data. The private markets are ready for this level of data-driven conviction, but it’s not without its challenges.
CHALLENGES IN THE PRIVATE MARKET
There are three prerequisites for a backtest, each of which has meaningful structural barriers in the private markets.
To overcome some of these data and signal challenges, CircleUp is focused on one specific vertical: the consumer packaged goods (CPG) sector. In CPG there are treasure troves of data about companies – everything from distribution to product attributes to consumer sentiment – and the business models in the sector are standardized, creating the ideal environment for an algorithmic approach. Additionally, CPG is a massive market with stale incumbents and a historically inefficient funding landscape.
All that said, the above challenges are still present even in a sector like CPG. Let’s walk through how we are approaching these key challenges.
1. Collect Time-Series Data
We have written at length about CircleUp’s data moat, where the data comes from, the data acquisition strategy, data pipelines, and how we stitch the data together. In summary, we are working with a unique time-series data set. It is far from perfect – there are gaps and inaccuracies – but in aggregate, CircleUp has what we believe is the most expansive mosaic of the CPG landscape in the world. We typically run our backtest beginning in 2014 to allow for a 5-year performance observation window, which reflects the long hold periods in the private markets.
2. Define the Dependent Variable
Without observable pricing information on private securities we must use proxies as the dependent variable. We have conducted extensive studies on the strength of different proxies to optimize for the strongest correlation to company exit value.
An example proxy is revenue growth, which is highly correlated with exit potential in early-stage CPG and directly related to enterprise value. But with proxies come assumptions and with assumptions come a broader range of possible outcomes. The analysis is not bulletproof but it is much more precise than typical heuristic processes. We layer in assumptions around dilution, compression, and escape velocity to translate the backtest result into an implied net multiple on invested capital (MOIC) and perform sensitivity analyses to understand the impact of each assumption.
3. Identifying Signal in the Data
In the public markets there are countless data providers and countless hedge funds chasing the same data, which can quickly become commoditized. An information advantage is predicated on the firm’s ability to find unique data sources with orthogonal signal.
At CircleUp we have the advantage of working with a combination of data sets that others don’t have. We look for factors in four primary buckets: (1) distribution (2) brand (3) product (4) category. Each of these are components of a CPG company’s success story and each can be quantified.
As an example, Spindrift sparkling water was picked up by our 2014 backtest. Founded in 2010, the company had only eight employees and modest revenue in mid-2014. But our data highlighted the promising growth in the four buckets mentioned: distribution had more than doubled in a year, the brand was resonating with the consumer, the product was differentiated from the competition, and the overall category was showing promise. CircleUp’s data pointed to this potential in 2014 and with continued growth Spindrift closed $20M from VMG in 2018.
HOW WE STAY INTELLECTUALLY HONEST
It is well understood in the public markets that backtests can be manipulated and models can be overfitted to generate attractive results. To avoid hindsight bias and maintain intellectual honesty we have taken a page from BGI’s book (now BlackRock) by following the “SPCA” framework. When building the model everything must be sensible, predictive, consistent, and additive.
- Sensible means that there is economic intuition behind the investing rule. For example, if we find that companies that start with the letter ‘s’ are most likely to grow, we won’t incorporate that signal into the model. It simply isn’t sensible and is likely a spurious correlation.
- Predictive means we have demonstrated that a feature from the past is predictive of the future. Contemporaneous correlation, while often sensible, is far less actionable. For example, you might know that a kid’s arm length is likely to increase as they get taller, but this contemporaneous correlation doesn’t help predict how tall someone will be.
- Consistent means that the test is repeatable and resilient across time. Much like the scientific process, if the results of a test can’t be replicated, they aren’t deemed valid. This helps avoid strategies that are driven by idiosyncratic events.
- Additive means that factors and datasets are added to the model if there is orthogonal value to existing signals. If signal A and signal B are both predictive of future growth but are perfectly correlated with one another, then there is limited value to incorporating both into the model.
IN CONCLUSION
Private and public market characteristics are markedly different. Public markets are characterized by moderate, positive growth with low volatility. Most stocks see gradual growth over time. In contrast, private markets are known to have more binary outcomes, with meaty tails on both ends of the distribution – companies are more likely to fail and are also more likely to generate outsized returns. The implication is that there’s a huge upside to having high precision models and tremendous cost to low recall, or missing attractive deals. Backtests help us understand the probable precision and recall of a portfolio prior to investing. LPs can have more conviction around potential fund outcomes than ever before.
These market dynamics inform how our growth and risk factors are selected. In the coming months we will publish a more in-depth look at our portfolio construction process – from growth signal identification to risk factor incorporation to rebalancing. Stay tuned.