In this link we're going to look at what causes the gradual loss in testosterone, andropause, that occurs as a man ages. Although the current scientific understanding in this area definitely has many holes in it, several important interesting insights have emerged as well. And it is my hope that some of the younger guys will read this material, put it to use and possibly slow down their slide toward hypoganadism.
NOTE: For those of you who have forgotten your high school biology, testosterone is produced in the Leydig cells of the testes. On this page we are talking about factors that can slowly squeeze the output from these all-important cells.
Before I go on, though, let me point out that I'm not talking about a rather sudden, "catastrophic" loss in testosterone that occurs in some males. For example, I get several emails per day and most of them are from guys with low testosterone. A surprising number of these men are in their 20's and 30's and have likely lost their testosterone in just a few months or years. There can be many reasons for this kind of a sudden loss, including tumors and varicoceles. See my link on Rapid Testosterone Decreasers for other more "exotic" causes as well.
So what causes the steady, gradual loss of testosterone that almost all of us males experience to one degree or another? (I say "almost all of us", because there are men - or should I say 'lucky bastards'? - in their 70's with testosterone of 700 ng/dl.) You are immediately going to recognize the culprits as they can be blamed for a host of other issues besides just andropause:
1. Inflammation. Inflammation is arguably our greatest enemy as males: it sabotages erections, heart health and our brains. It also puts you at risk for autoimmune disorders, diabetes and cancer. Nice, eh? Well, it's probably no surprise then that it also is likely a root cause the loss of testosterone in middle-aged and senior men as well. IL-6 seems to be the primary culprit.  However, elevated C-Reactive protein has been implicated as well.  Several studies have also found that fertility and sperm production are related to inflammation. 
2. Mitochondrial Damage. There is a a whole field called The Mitochondrial Theory of Aging and for good reason: it seems to be a root cause for much of our decline over the decades. Your testosterone is no exception: researcher have found that mitochondrial damage in the Leydig cells corresponds to a loss in testosterone.  The bottom line is that mitochondrial health is likely critical to decelerating andropause. This translates to a diet high in certain antioxidants, avoiding fructose and perhaps taking certain supplements (CoQ10, ALA) as well.
NOTE: An interesting side note is that there is good evidence that some of the antioxidants, which again would guard and protect your mitochondria from damage, can powerfully boost testosterone. For more details, see my link on Antioxidants and Testosterone for more information.
3. Estrogen. Our old enemy estrogen, specifically E2 or estradiol, is at it again, this time correlating strongly with inflammation levels. Many men as they age and gain weight find their estrogen levels rising dramatically and this study shows they may be putting themselves at additional risk for inflammation as shown by elevated C-Reactive protein levels.  See my link on How to Lower Estrogen for additional information.
CAUTION: Some hypogonadal males have low enough testosterone to where they actually become deficient in estrogen and experience osteoporosis. See my book Low Testosterone by the Numbers for actual testosterone levels.
Again, I hope that any younger guys will read this and avoid the modern plague of declining testosterone levels and andropause. I have another link on How to Avoid Andropause that discusses the Okinawan's success in this area.
1) Clin Endocrinol (Oxf), 2007 Mar, 66(3):394-8, "Association of endogenous sex hormone with C-reactive protein levels in middle-aged and elderly men"
2) Diabetes Care July 2005 vol. 28 no. 7 1636-1642, "Relationship Between Testosterone Levels, Insulin Sensitivity, and Mitochondrial Function in Men"
3) Exp Gerontol, 2001 Aug, 36(8):1361-73, "Age-related increase in mitochondrial superoxide generation in the testosterone-producing cells of Brown Norway rat testes: relationship to reduced steroidogenic function?"
4) J Endocrinol Invest, 2005, 28(11 Suppl Proceedings):116-9, "The relationship between testosterone and molecular markers of inflammation in older men"
5) European Cytokine Network, Jun 2005, 16(2), "Testicular interleukin-6 response to systemic inflammation"