There are a few ways to measure if Mr. Market is overvalued, undervalued or fairly valued. 3 can be easily used to give you a general sense of how hard it will be to find good prices on investments. While it may be harder or easier, there will be deals in every type of market, it just generally takes more work in overvalued markets.
The simplest is the P/E ratio of the S&P. We use the S&P as a proxy for the market and can take the historic P/E of the S&P and use that as a proxy for valuation of the average of the stock market. Then compare it with the current S&P P/E ratio to estimate whether the market is overvalued or under valued.
P/E is Price to Earnings Ratio. It measure the price of a stock or investment vs the earnings it brings in. So if company ABC is trading at 30/share and earns 3 dollars a share each year, then it would have a P/E of 10. If that same company was trading at 30/share but the earnings increased to 6 dollars, then the P/E of 5, would show that the company is undervalued.
The issue with this is that during boom/bust markets, the P/E ratio becomes quite skewed and will give incorrect information. During boom markets, most businesses perform well, have expanding profit margins (or are growing quite quickly and keep profit margins down temporarily), expanding profit margins = higher earnings. So if a stock is trading at 30/share and have 3/share in earnings at the beginning of a boom, and say at the peak of the boom before a bust, they are valued at 45/share with 6/share in earnings, their P/E is 7.5.
Right before a bust, it would give the incorrect indicator that the stock(s) are undervalued as P/Es are low historically. Keep in mind, there will be high flyers and darling stocks of the market where the price does outpace earnings, but taking the market as a whole and averaging it out will show earnings outpacing stock prices temporarily leading to misinformation. See below.
The flip side is also true. Take our most recent recession/depression. During the crash, stock prices dropped from 25-50%. With all the fear and restructuring and panic, earnings were down from say 3 dollars a share to 25 cents a share. So our stock price of 30/share goes to 15 a share and our 3/share earnings goes to 25 cents/share. This would show a P/E of 60. Meaning the company is vastly overpriced. But any good analyst would take a look at the fundamentals of the company, if the company is solid, has long history of profits and is only temporarily depressed due to the market and you believe it will normalize profits back again at 3 dollars a share over time, then you would know buying a stock that can earn 3 dollars a share in earnings for 15 dollars is a great deal as that would be a "real" P/E of 5 which is very low. But based on the boom/bust mechanics, the resulting P/E of 60 is what would show.
To compensate, a Yale economist by the name Robert Shiller, created what is known as the Shiller P/E. The Shiller P/E takes the average earnings of the S&P over the past 10 years and adjusts it for inflation using the CPI and then uses that as the proxy for the earnings to calculate the P/E. This results in a more realistic and accurate measure of the market valuation, whether it be over or under valued.
Lastly, the method mentioned by Warren Buffet is what we will use. We take total market cap and divide it by total GDP. This makes the most sense as well if you think about it. Total market cap, i.e. what we believe all our firms to be worth divided by the total actual production/output of our firms. If we are 100% accurate, then it should be somewhere around 1. Any higher is an overvalued market, and anything below around .9, is an undervalued market.