There's a lot going on in those testes of yours, including sperm and testosterone production. Yeah, it's a big factory down there and researchers have found increasing evidence that "oxidative stress", i.e. free radical damage, is one of the leading causes of lowered testosterone. Damage to the mitochondria appears to play a key role.  In an eye-opening study, researchers found that the Vitamin E, a antioxidant that quenches several free radicals, actually slowed down the age-related decline in testosterone in lab rats.  This may mean that one can significantly slow down andropause if one starts at a young enough age. Life is full of regrets, eh?
Researchers have also discovered reduced levels of key antioxidants, such as the heavy hitters SOD and glutathione, in aging rat Leydig cells, thus adding weight to the above oxidative stress theory and the need for adequate antioxidant dietary support.  However, here is what should really catch your attention:
Scientists discovered that, at least according to animal research, you can likely boost your testosterone levels through increased consumption of certain powerful antioxidants. It's analagous to adding lubrication to machinery in order to attain greater output and efficiency. Now some antioxidants appear to be more powerful androgen-stimulators than others and we outline some of those below.
Keep in mind that this is all relatively new and so studies on humans have not really occurred to date. However, animal studies do look promising as I have summarized below:
1. Pomegranate Juice. We have aleady covered how Pomegranate Juice Boosts Erectile Strength, but did you know that a study on rats showed a nice boost to testosterone as well?  In fact, the rats drinking the most pomegranate juice got a nice 27% in testosterone! In my link on Pomegranate Juice and Testosterone, I show just what dosage may achieve these kind of gains.
2. Quercetin. Quercetin is the powerful antioxidant flavonoid in onions, red grapes, tea and many other fruits and vegetables. It has powerful anti-cancer properties and has been the subject of extensive research. One of its lesser known superpowers (in rats at least) is increasing testosterone. Scientists found, for example, that giving 50 mg per kg of body weight gave an almost 80% in testosterone. 
Of course, the problem with this is that for the typical 160 pound male, you are talking about a megadose of over 3 grams of quercetin in us humans. Nevertheless, it shows the power of some antioxidants to power up testosterone production in the ol' Leydig cells. NOTE: I do not know the human-to-equivalent ratio for the rats used in this study.
3. Onion Juice. Based on #2, it probably will be no surprise that onions, if consumed in large enough quantities, can boost testosterone. Of course, the question of the day becomes "how in the world do I eat an onion or two and still live in polite society"? Researchers came up with a nice solution: onion juice. They just blended one standard onion and - shazam! - the juice boosted rat testosterone by 300+%. 
The only problem is that the amount of onion juice these rats consumed was 1 gram per kilogram, which translates to about 73 grams of onion juice for the typical 72 kg guy. However, it should be noted that even taking half that amount of onion juice created a nice testosterone boost as well, so it is likely dose dependent, i.e. even a smaller amount of onion juice may give you a little lift. NOTE: I do not know the human-to-equivalent ratio for the rats used in this study.
4. Melatonin. Data is very limited but the antioxidant and hormone melatonin appears to protect testosterone in Leydig cells as well. 
So should you start gulping gallons of juice? Well, that's a little premature for a variety of reasons. First of all, all of the above studies are on animals and may not translate to humanoids such as ourselves. Furthermore, perhaps it's possible to overdo antioxidant consumption. For example, some experts have posited that if you overdo ingestion of antioxidants, it shuts down the body's own natural antioxidant production. In fact, some studies have shown that antioxidants in supplement form may actually be dangerous to your heart or promote cancer as I document in my links Antioxidants and Your Heart and The Dangers of Antioxidant Supplements.
However, a solid diet loaded loaded with ample antioxidants will likely protect and possibly boost your testosterone and give you multiple additional health benefits as well.
1) Experimental Gerontology, Aug-Sep 2005, 40(8-9):728-736, "Vitamin E, aging and Leydig cell steroidogenesis"
2) The Journal of Steroid Biochemistry and Molecular Biology, Jan 2004, 88(1):61-67, "Aging alters the functional expression of enzymatic and non-enzymatic anti-oxidant defense systems in testicular rat Leydig cells"
4) Endocrinology, 2002, 144(7):2882-2891, "Reactive Oxygen Disrupts Mitochondria in MA-10 Tumor Leydig Cells and Inhibits Steroidogenic Acute Regulatory (StAR) Protein and Steroidogenesis"
6) J Endocrinol, 2004 Jun, 181(3):493-507, "Reduction of rat prostate weight by combined quercetin-finasteride treatment is associated with cell cycle deregulation"