Simple Perspective On Through Silicon Via (TSV) And Hybrid Memory Cube (HMC) Technology

by: H. Bruce Campbell

I am long, long term, Micron Technology (NASDAQ:MU) and Apple (NASDAQ:AAPL), and hope to initiate a long Intel (NASDAQ:INTC) position soon. I also hold a very modest Micron $10 call option position which will expire, almost certainly worthless, this month. I'm not an investment professional - I earn no income from my investment perspectives other than whatever this contribution might earn from Seeking Alpha.

I'm an aging and now only modestly practicing electrical engineer. This article describes my personal view of TSV (Through Silicon Via) and HMC (Hybrid Memory Cube) technology as they seem to relate to the evolution of modern information technology and other electronic products. My hope is that investors with limited electrical engineering experience will find it helpful. I also hope it'll spawn commentary which might reveal more timely and precise information about TSV and HMC technology, especially including IP ownership insights.

As an electrical engineer, I'm personally very excited about TSV and HMC, not just because of the implications for Micron and Intel, but also for the electronics industry as a whole.

When my engineering career began in 1971, a profound transition from discrete components to integrated circuits was under way, but immature. Relatively shortly before, almost all circuits were implemented with individual resistors, capacitors, inductors, diodes, transistors, and other discrete components. ICs (Integrated Circuits) were available for only a few specialized tasks, and they were relatively simple and modest in performance. And packaging for all components, whether discrete or integrated, was relatively crude, and very bulky and inefficient by today's standards.

What followed was a long duration but nonetheless stunning IC revolution. ICs became the focus of attention and resources for the simple reason that they could accomplish far more with far less - more circuitry could be implemented in dramatically less space, and at dramatically lower cost, and yet yield higher performance. More ICs targeting more applications became available every day, allowing circuit design engineers like me to squeeze far more product functionality and performance into much less space, and at lower cost. Performance improved because microscopically tiny circuits had, by virtue of physical laws of nature, substantial inherent speed of function and conservation of energy advantages. And ICs could be more complex yet still cheap - relative to discrete circuit implementations, an IC's component count had a much smaller impact on overall cost.

That evolution continues to this day - as semiconductor fabrication processes continue to improve, more and better components are squeezed into ever smaller chips, yielding ever higher overall performance, energy conservation, and cost savings. And yet most end products still incorporate multiple ICs and some discrete components.

One might ask why electrical engineers don't simply design a single IC to accomplish everything any particular product requires, so as to fully minimize the product's size and assembly cost. Well, some such products do exist. Remarkably inexpensive hand held calculators, some small hand held games, remote controllers, clocks and watches, some appliances such as microwave ovens, and numerous other devices often contain only one IC to operate everything (or one IC and precious few additional discrete components which can't be fabricated within an IC due to their large physical size requirements).

But more complex products require multiple ICs (sometimes many of them) plus some discrete components. Product engineers want to combine all their ICs into just one IC in these products, but they can't - there are limitations. IC fabrication processes differ from one type of circuit to another. Microprocessor ICs are chemically and micro-structurally fabricated in a different way than RAM (Random Access Memory) ICs for example. And typical analog or mixed signal ICs, such as for amplification and measuring circuits, signal or power conversion circuits, and many others, are even more different. And some important scale limitations still remain - there are practical limits to how much circuitry can be combined into one chip while maintaining reasonable manufacturing yields. So for the circuit design engineer, it's still usually necessary to utilize multiple ICs in order to achieve all the functionality their product requires. We dislike that restriction, but we're stuck with it - it's imposed by current IC fabrication technology limitations. But for many products, it's a restriction which imposes very serious performance and economic barriers, so there's significant urgency to try to resolve it.

At this time we generally use circuit boards to interconnect all the ICs a product requires - the same basic method we used in 1971 when I graduated from college and began my electrical engineering career. Circuit board related technology has evolved of course, with the most significant advance being the introduction of surface mount technology, which displaced leaded component technology. But its fundamental nature and limitations haven't changed. The most serious limitation is that connection distances are far, far longer than in the microscopic realm within an IC.

And in high speed electronics, long connection distances are a very serious problem - they're performance assassins. State of the art microprocessors manipulate information at speeds faster than 2 GHz, which is a clock cycle of just 500 pS. (A picosecond is 10 to the minus twelfth second. 500 pS is .0000000005 seconds). The speed of light in a vacuum, Mother Nature's absolute unbreakable speed limit, is about 300,000,000 meters per second. The speed of electrical signals in naked wires in a vacuum is identical - both are electromagnetic wave propagation phenomena. In 500 pS light and electrical signals in a vacuum travel only 15 cm. That's a rather short distance - many circuit boards are longer than that. So you can see that our lust for speed has hit an unbreakable barrier. But it gets worse. Circuit boards are not a vacuum, but rather a solid or mixed transmission medium. The propagation speed of electrical signals (and light) is slower in solids (variably so, depending upon the physical properties of the medium).

In practice, the speed of electrical signals in circuit boards is, at best, very roughly 170,000,000 meters per second. That's only 8.5 cm in 500 pS. That's a very serious problem - signals which must travel 8.5 centimeters lose a full clock cycle of pace with a 2 GHz processor before the next IC, such as RAM, even has a chance to start using the signals. And a return signal trip must often be made, so another clock cycle is lost. And connection distances are often longer than 8.5 cm, causing even longer delays. 8.5 cm, which is about the length of your index finger, may seem like a trivial distance in everyday life. But in high speed electronics it causes very serious system delays - such distances are a very serious problem.

And we can't appeal to Mother Nature for help with this problem - she is absolutely unrelenting about enforcing her laws. No innovation will breach the speed of light barrier. We don't like it, and we dream extensively of overcoming this barrier in books and movies of all kinds. But it will never happen - on that issue, Mother Nature slammed her gavel down at least 13.7 billion years ago, and she will hear no further arguments, no matter how great the need nor how impassioned the plead.

So we have a very serious problem involving an immutable law of nature which can't be substantially solved by simply refining circuit board design and fabrication methods. Within an IC, component connection distances are microscopically short. That's why microprocessors and other high speed components have been able to achieve such remarkably high operating speeds. But individually packaged ICs soldered to circuit boards which connect to other ICs inherently involve connection distances which are enormously longer than those within an IC. And thus interaction speeds between ICs are much slower. It's a very serious bottleneck - its a system performance killer.

Given current IC fabrication and circuit board technology limitations, the only way to address this problem is to discover a novel new means to interconnect ICs as close to each other as possible. The nearly ideal solution would be to wed IC chips directly to each other in interconnected stacks like pancakes.

And that, quite dramatically, is exactly what TSV technology allows. It provides, for the first time ever, a means to connect IC chips directly (or through a very thin thermal dissipation intermediary) to each other in stacks, eliminating circuit boards and individual IC packages as intermediaries. That's a profound new capability - it's a substantial leap in technology which will lead to substantial advances in performance. And not just speed performance, but energy conservation performance too, because long distance connections are not only speed killers, they're energy hogs as well.

As Micron Technology, Intel, and every other enlightened electrical engineering centric firm knows very well, the ability to connect IC chips directly to each other in stacks is a major breakthrough with major implications. It wasn't easy to achieve TSV. And there are ancillary issues which must be managed, including an important heat dissipation issue. But these are challenges which aren't yet restricted by natural law barriers, so, though it won't be easy, they can and will be resolved. And in the case of HMC, the implication of corporate positioning and industry standards development suggests that these challenges have already been solved, or very nearly so.

For Micron, Intel, and similar firms, the first and most dramatic application for TSV technology is the HMC, which will allow computer system's RAM to be dramatically more compact, and placed much, much closer to the microprocessor it serves. That will allow the microprocessor to access its RAM much faster and with much less power dissipation than can be achieved with conventional technology. Engineers have heretofore been frustrated by the "Memory Wall" - the cyber system bottleneck caused by the comparatively slow speed at which a microprocessor can exchange information with RAM, which is a critical cyber system limitation which can't be significantly resolved in a circuit board based technology landscape. TSV and HMC technology breach a large proportion of the memory wall. As a result, cyber systems will be much faster yet use significantly less energy. They'll also be physically smaller.

It's a dramatic step forward. And, in my estimation, it's just the first step in a years long revolution which will restructure IC interconnection logistics from a longitudinal circuit board mediated connection of packaged ICs method to a direct stacking of naked chips method (often sandwiching a roughly one mm or less thick thermal dissipation or custom interconnect routing intermediate). IC chips will be stacked on top of one another like pancakes rather than individually packaged and then spread longitudinally on a circuit board like autumn leaves blown across a yard. The circuit board's role will ultimately be relegated to a much lower level in which it simply distributes electrical power and a few input and output connections (such as, in cyber products, USB, FireWire, Ethernet, Thunderbolt, and SATA, and WiFi and BlueTooth radio antennas, and audio and display connections), and provides a mechanical substrate to physically support the silicon cube.

The leaded component package to surface mount component package revolution is now essentially over. I suspect the torch will now start passing to a stack mount revolution. That will eventually take us much closer to the classically described 'Smoking Hairy Golf Ball', an idealized liquid helium cooled semiconductor sphere with wire connections on its surface, envisioned long ago as the final stage of evolution for maximum performance solid state electronics systems.

When HMC based cyber systems begin to hit markets, in my estimation eyebrows will rise, and lots of money will flow. Because preexisting systems will be rendered technically obsolete - they'll simply be too slow and energy inefficient to be competitive. Industry, now critically dependent upon refined high speed web server performance, will be forced by competitive pressures to upgrade servers and Internet infrastructure systems. And every cyber toy enthusiast, which is practically every living human being now, will lust for the faster and more energy efficient products which spring from TSV and HMC technology. And people will find ways, even in bad economic times, to allocate money to spend on this technology. (And if economic times seem good, they'll find ways to allocate quite a bit of money on it.) Big screen TVs and cell phones were purchased as if life depended upon them even during the last economic meltdown - even in very bad times, human lust for dramatic new technology is at least partially irrepressible.

But the TSV revolution isn't limited to HMC or cyber system applications. Ultimately it will minimize the role of circuit boards in most electronic products. Many or most ICs will be redesigned to support silicon cube based designs in a growing range of products. For the same reasons - to increase speed and conserve energy. Product fabrication will become more compact, efficient, and elegant as well, as silicon cubes minimize the role of bulky circuit boards. Performance will rise and size will shrink over a broad range of products.

This process won't happen overnight. It will be progressive, with initial focus on components used in high volume products. For example, in my estimation, after the HMC, an SSDC (Solid State Drive Cube) will be introduced. Then eventually complete silicon cube based cyber systems - systems fabricated with all of their ICs (or nearly so) stacked directly onto each other. Other commodity products will similarly evolve. This evolution will take years. But in my estimation it will, step by step, occur.

TSV strikes me as a remarkable technical development with very broad implications. It reminds me of the IC revolution in many ways. But in my estimation it will progress faster, because mankind's technology development pace is much faster now, and because enormous financial lubrication will be involved.

And although this is highly speculative, it might be possible for a firm with billions of liquid dollars at their disposal to very secretively skip a few steps in this revolution, then surprise the world by abruptly introducing a heavily TSV based ultra high performance product. But skipping steps would be no cake walk - no firm, no matter their wealth, can easily or almost instantly resolve numerous daunting technical challenges overnight. One might try, and might even succeed. It would require great human effort and dedication, and some time. But leapfrogging competitors can generate considerable rewards. So such a scenario is not inconceivable.

My guess is that some people or firms own some critical patents for TSV and HMC technology. But I don't know who - based upon my only modest investigation, that information seems clouded in mystery. It's very tempting to think that mystery is related to insider understanding of the enormous financial and corporate successes TSV patents might deliver. But I'm just speculating about that. And I have precious little idea whether any firm owns or co-owns critical patents, putting them in a position of significant control over this powerful new technology, or numerous important patents are spread across a broad range of firms or even individuals, such that none can dominate the technology. For an investor, the TSV patent landscape is critical, but to me it still seems mysterious.

However, I feel compelled to place my financial bets somewhere. And at this time Micron and Intel seem to me to be at the center of this revolution - recent events seem to suggest they might jointly own some critical patents. I've been long Micron with almost half my portfolio for many years, and will stay with them of course, at least until the now murky TSV and HMC technology intellectual property ownership landscape becomes more clear. But I've held Apple for many years too, and will continue to do so, because I suspect they're in the thick of this revolution as well. But I have no actual evidence to support that notion - I simply feel that Apple by nature has always been devoutly focused on superb design elegance, and TSV is a very powerful enabler of that design philosophy. And they have a legacy of rather dramatic innovation which they're under enormous pressure to maintain to survive and thrive. Still, Intel and Micron strike me as more likely to be at the very center of this revolution and its rewards. So if opportunities arise, I'll add Intel to my holdings.

As excited as I am about TSV and HMC, I'm not privy to a lot of key information, including technology issues which might impose implementation limitations in TSV and HMC based systems, the patent landscape, and other issues. And I'm biased by my affection for technology. And of course I'm often just plain wrong - I make many mistakes, and though I'd prefer to avoid them, I'll make many more.

And even if my vision of all this is reasonably accurate, lots of complex dynamics will be involved as the technology unfolds. There'll likely be critical patent wars. Some legitimate, some just dishonest greed motivated assaults, and some simply corporate distraction strategies. And guidance of the evolution of the technology will be painstakingly difficult. Skilled engineers will for example invest their very souls to create exquisite industry standards for TSV and HMC in the limited field of cyber systems, only to see them fall to dust as new issues come to light, demanding new solutions. And more skilled engineers and technologists will sweat blood for years in an effort to define broad TSV pattern standards which allow as many IC families of all kinds to be stacked directly to as many other like family ICs as possible, a supremely daunting effort which will never be perfectly successful, yet will be pursued to the limits of human ingenuity. None of the process will be anything like easy or fast. But dazzling progress will occur - substantial rewards will fuel an unrelenting march forward. In my estimation...

So the road, exciting as it may be, will be very rocky and serpentine. And rewards and their recipients will remain impossible to precisely predict. Every investor must steer their own ship. I'm simply being transparent about some personal perceptions which heavily influence how I steer mine.

2013 should be a fascinating year in many ways. Some because of man's constructive and innovative energies. And some because of man's dark side, including arrogance and greed for power which overwhelms ethics, not the least of which involves grave and very high magnitude dishonesty in national and global fiscal affairs. We live in a world of both realms, and they're playing out simultaneously. They create opposing pressures in the investment sphere which seem in constant conflict for dominance. And lately, the struggle seems unpredictable - it's usually very difficult to discern which side will dominate near or moderate term.

But ultimately, technology advances and enriches human life even as man's dark nature continues to wallow in jungle ethics dominated by lust for power, greed, and conflict. History seems to demonstrate that investment in the best technology yields significant rewards. My personal sense is that this hasn't changed, nor will, even long term. And I'm inclined to think that TSV and HMC technology is on the threshold of leading to dramatic advances in information technology and all other electronics realms. It's not the only game afoot of course - life sciences and other sectors are moving forward in very exciting ways too. But TSV and HMC technology is an area I better understand, and, as both a technology fan and an investor, am most excited about.

But again, I make many mistakes. And will continue to do so. Everyone must steer their own ship.

Disclosure: I am long MU, AAPL, STX. I wrote this article myself, and it expresses my own opinions. I am not receiving compensation for it (other than from Seeking Alpha). I have no business relationship with any company whose stock is mentioned in this article.

Additional disclosure: I hope to initiate a long Intel position soon. I also hold a very modest Micron $10 call option position which will expire, almost certainly worthless, on 19 January.