Alpha, Beta, Risk, And Performance Measurement - Again

 |  Includes: BRK.A, BRK.B
by: Braun Research

Some reader reactions to an earlier article that I wrote on this subject, titled "Basics of Alpha Revisited…For The Investor," prompted me to attempt a sequel.

Indeed, if you are an individual stock picker, as one reader sugggested, "all you have to do is buy good quality companies at a value price, hang on to them and use the dividends to add more 'free' shares."

But how about the person who does not have the time, the inclination or the talent to do it alone and hires a portfolio manager (PM) to do this job? Thus finding the "right" PM is therefore a useful exercise. It is not hard to find the performance of various managers calculated between a starting point and an ending point. The one who did better is "my PM", is s/he not?

Trouble is that even if you disregard the ubiquitous warning that "past performance is no guarantee for future results" - which is required by law and is a perfectly correct statement anyway - in order to get some confidence that a good past performance has a fair chance to continue, you must have some better ways to evaluate how that performance has been achieved. Which is exactly why Alpha, Beta and a slew of other measures were thought up, and which is why I took the trouble to write my above referenced article.

While you indeed can never be sure that any past performance will be continued, there are nevertheless ways to become more-or-less confident that a certain PM is more likely than another to produce for you too what s/he has produced before for others. As we have discussed, Alpha and Beta can help but, as you will see if you do me the favor to continue reading, even these measures can be somewhat "iffy".

Another reader observed that "these 'relative performance' measures have limited value." Absolutely correct: a client wants/cares about absolute returns, not "relative performances," as it is cold comfort that I lost "only" 15% of my money while the benchmark lost 35%!

But any useful measures can only be relative because what you need them for is to compare PMs to each other and decide which is the one most likely to be a good guardian of your money. Any comparison is, by definition, "relative." And, to be of some help, any comparison must be made looking at a common benchmark. Of course, it is useful to fully understand the thought process of the PM.

But what if the strategy is so complicated that few will understand it? What if it is a "secret" strategy, as in the case of most hedgefunds? Some "objective" measure of how that performance has been achieved is where Alpha and Beta come in. These, along with some other measures I am proposing in this article are useful but still limited. I will be attempting to explain what those limitations are and how they can be supplemented.

For this purpose, rather than using a couple of theoretical dreamt-up portfolios, as I did in my earlier article, I will take a real US-traded stock portfolio which I will call CAP (for Capital Appreciation Portfolio) which has been run since Sep. 1, 2000, in a highly disciplined mode, long only (i.e. no derivatives) without leverage, with a minimum market cap of $1 billion, and I will compare its performance after all fees, brokerage and expenses, with Warren Buffett's Berkshire Hathaway stock (NYSE:BRK.A). It is a daring comparison, but why not compare to the best?

Let us first take a look at the performance of the two portfolios using the inception date of the CAP on Sep. 1, 2000 to the most recent quarter ending on Mar. 30, 2012 (11 ½ years) and let's use S&P 500 (SPY) as the benchmark:

Click to enlarge.

Click to enlarge

Click to enlarge

It certainly appears that Buffett has beaten the CAP both in total performance and in terms of the portfolio statistics described in my previous article: better overall return, more PM contribution (Alpha) with less risk (Beta). Not by much, but by enough to tilt the decision in his favor - so far.

For the more detail-oriented reader, here is a little side note about the R2 (R-square) that is calculated and has not been explained in the previous article. R-square is a statistical measure that tells us how "scattered" are the points around the regression line. The more scattered the points, the less relevant the Alpha and Beta we calculate.

R-square can vary between 0 and 1, with 0 meaning that Alpha and Beta are completely irrelevant, random, and therefore useless, numbers, while 1 means that all points are on the line and therefore Alpha and Beta are very relevant. R-squares in-between, as we have here, give a measure of how much confidence can we have that Alpha and Beta are true measures.

For the comparison to be meaningful the two R-squares should be close and as high as possible. In our case they are not. So this is a first indication that using Alpha and Beta to compare these two portfolios is not a very good idea. They tell us that both managers contributed to the return being above the benchmark, in other words they deliver Alpha but we cannot tell with any degree of confidence that one contributed more than the other.

Let us now change just slightly the measurement period and see what happened in the 10+ years between Dec. 31, 2001, when the CAP's manager happened to decide to switch to another broker, and March 30, 2012:

Click to enlarge

Click to enlarge

We have an almost complete reversal: the CAP has done better than the Buffett portfolio, both in the overall performance and as far as Alpha (manager's contribution). Beta, which is "risk" as defined in the academia and accepted in the market, stayed lower and R-square for BRK.A is still too low to inspire much confidence in the results.

How is this reversal to be explained? First, it is clear that during the initial 16 months from Sept. 1, 2000 to Dec. 31, 2001, BRK.A outperformed the CAP, an advantage which it was not able to keep-up in the next 10 years or so. We have here a good way to demonstrate that point-to-point performances can be extremely sensitive to even slight changes in the measurement period, and thus they are misleading, as mentioned before. Therefore, all the usually encountered performance measures of 1, 3, 5 and 10 years, while useful, have to be taken with a big grain of salt. Some mangers give a table of monthly performances too - but the reader must figure out all by her or himself how to use those numbers.

Second, both Alpha and Beta which, as academics told us, were supposed to reflect the skill of the portfolio manager, are evidently dependent on the measurement period as well.

Third, some further analysis shows that the discrepancy might be explained by the difference in the number of negative quarterly results for the period since inception: 54% for BRK.A vs. only 27% for the CAP. The so called maximum "draw-down," i.e. the largest negative quarterly performance, was almost -14% for BRK.A vs. less than -5% for CAP. Similar data exists for the absolute performance comparisons and for the monthly comparisons and, if anyone is curious enough about those, the author can provide the appropriate graphs.


When evaluating a PM's past achievements, ask not just for performance but also for the number of negative quarters/months and for the maximum draw-downs included in any performance. They are relevant for the downside risk which is the real meaning of the word "risk" as used in common parlance, rather than the volatility or standard deviation, or for that matter Beta, as used by the academia, not incorrectly but somewhat against the intuition of the proverbial "man on the street." Some PMs and RIA volunteer at least some maximum draw-down numbers; others don't.

Before going any further, a "sidebar" clarification about "academia" may be in order. They are not attempting to mislead anyone (as a rule) and they are not oblivious of all this which, by the way, is called Capital Asset Pricing Model (CAPM), developed based on the so called Modern Portfolio Theory (MPT) by a few very bright scientists, some of whom were honored with Nobel prizes for their work.

In order to overcome these problems, a slew of ratios have been proposed in an effort to provide better measurement tools. To cite just a few of the more frequently used ones: Sharpe, Treynor, Sortino, Modigliani, Information Ratio. You can really go crazy trying to understand them all. What they have in common is that they are all measurement-period-dependent and the assumptions underlying them, which very rarely apply in real life, present problems whose mere listing fills a page in Wikipedia. If you are curious - or masochistic - enough, you can take a look here and scroll down to "Problems of CAPM."

All nice and good the reader will say, assuming that s/he did make it reading thus far, but the question remains: if total performance, as well as Alpha and Beta and all the other learned measures developed by Nobel Prize laureates, are so dependent on the measurement period and have so many questions relating to them, what is the poor investor to do when trying to decide which manager to choose? Are the negative quarterly or monthly periods and the maximum draw-downs all that this article adds to the issue of PM evaluation? Nice, but not very new. All that appears to be left is to toss a coin.

Before throwing in the towel, though, let us take a moment to consider the reality of the investment process. Few, if any investors start their association with a PM on the first day of a year or even of the quarter. The relationship can start on any day of any month of any year and it is from there that the investor will begin measuring her/his money's performance. So would it not make sense to measure the past results over equal sub-periods of several years (3,4,5, etc.), starting any month of the period?

That, in conjunction with the other measures described so far, will yield a more realistic reading of the PM's skill for the measured interval and will allow an estimate of the future potential results with a better likelihood that gross errors can be avoided. I have seen at least one Consultant do it on a quarterly-3-year-rolling basis but, as far as I can tell, this is not something in the mainstream.

Here is what such a method yields for 3 year rolling periods started in any month of the considered time span, for the two real portfolios that we show in this example:

Click to enlarge

Click to enlarge

Click to enlarge

So, is the CAP a better managed portfolio than Berkshire Hathaway? For 3 year holding periods, at 37% chances for a negative result for BRK.A compared to 0% for the CAP, it would appear so. A maximum draw-down of -40% for BRK.A vs. zero draw-down for the CAP seems to reinforce such a conclusion. For the investor's peace of mind ("true risk" anyone?), the CAP would be preferable. Quite a statement, indeed, considering Mr. Buffett's well deserved sterling and almost God-like reputation!

But not so fast. There are some other considerations here. The best 3 year performances for BRK.A at over +60% were clearly better than those of CAP at just over 36%. It is also fair to ask what's magic about a 3 year period? Answer: nothing! Indeed, a calculation based on a set of 5 year periods maintains a 100% outperformance over the S&P 500 for the CAP, but the Buffett outperformance raises to a very close 99%, with a max return of about +70% for BRK.A vs. just a tad over +60% for the CAP (I am sparing you the graphs).

Then again, the average of the 3 year returns is 20% for BRK.A vs. a slightly higher 22.6% for CAP and, as we have seen, from a return point of view, for the longer periods and from a statistical measures point of view, the comparison results are dependent on the measurement time-span chosen, so no judgment can be made solely on that basis.

Bottom line is that performance evaluation is not a black-and-white business. Both these portfolios are good investments: They both have positive Alphas and it is a matter of specific risk tolerance by the investor whether (i) a number of BRK.A stocks are bought or (ii) an investment in the CAP is preferred. The same may, however, not be true for every situation, so a thorough investigation of comparative performances is warranted, beyond and above what a normal mutual fund prospectus or most RIA presentations currently offer.

To answer the question posed by this example, when choosing between an investment in the CAP vs. BRK.A, the decision will have to be made going by some other considerations as well, tied to the need to understand what a manager really does:

  • The Buffett management style is to buy entire companies and to manage them as Berkshire Hathaway thinks fit - which has not been too shabby so far, to say the least. They rarely sell, which leads me to believe that this just might explain why their downside risk, as found through the analysis above, is slightly higher than it may be for some of the managers who can and do sell rather promptly stocks that disappoint them. But the Berkshire Hathaway people obviously compensate with excellent results on the upside. From an institutional investor's point of view Berkshire Hathaway also offers the ability to invest large amounts of money, of course to the extent that a good execution can be secured.
  • The CAP's management style is closer to what most PMs do, i.e. buy and sell stocks without taking ownership of entire firms. The approach used, permits scaling to as much as $1.5-2 billion in AUM (but not much more) and I will attempt to explain in one of the following articles the very solid and rather unusual approach being taken by the CAP's management. It is based on "Analyzing the Analysts" who, as any investor familiar with such matters will undoubtedly know, have a major impact on the price formation in the market. Specific stock selection examples, as well as sector allocation examples will be included.

Disclosure: I have no positions in any stocks mentioned, and no plans to initiate any positions within the next 72 hours.