“Total testosterone levels fall at an average of 1.6% per year whilst free and bioavailable levels fall by 2% 3% per year.” 
I guess the word “inevitable” comes to mind when I read descriptions like that. Losing your testosterone just seems like “part of life.” However, the Australian study mentioned above throws that way of thinking out the window. The particpants were 40+ year old men, and, again, the researchers noted that if a man reported very good or excellent health, then he simply had no decline in testosterone.
So what could explain this? Well, if you no me, I would like for their to be a natural, lifestyle-related answer to that question. And there is some evidence for it at least:
The Okinawans. Now I believe that sometimes this can be related to lifestyle. One of the most well-know of the long-lived cultures was the Okinawans. In another page, I document how the Okinawans had the highest senior age testosterone levels on record: How to Avoid Andropause. By the way, the Okinawans did not consume the typical “high testosterone diet” that most of the men’s health blogs push, i.e. abundant saturated fats. They were also not bodybuilders or into weights either, something I mention because so many of the blogs here in the U.S. push weights as a way to increase testosterone. Yes, there is likely a small effect there, something I document in my page on Testosterone and Weight Lifting. And, while weight lifting weights is my favorite form of exercise, the evidence just isn’t there that it is a big T booster.
NOTE: Toxins, such as pesticides and heavy metals, along with head injuries and concussion, are likely causes of an acclerated drop in testosterone. Obviously, avoiding these type of injuries, along with well done chelation, may be critical for many men as well.
Research-Backed Causes of Andropause. In my page on The Causes of Andropause I discuss how inflammation and mitochondrial damage are some of the suspected culprits. This seems reasonable, because there inflammation and mitochondrial dysfunction are so powerfully associated with aging in the research. This leaves the intriguing possibility that a well done anti-aging strategy could actually preserve one’s testosterone (along with DHT and estradiol) if begun early enough.
This leaves the very intriguing possibility that men, who are diligent from a somewhat early age, could actually keep their testosterone levels through a carefully designed anti-aging program. For example, we now have realistic strategies to boost the body’s key mitochondrial antioxidants, such as SOD, glutathione and CoQ10, to youthful levels as one ages. We even have strategies that can help create new mitochondria. And decreasing inflammation can be done nutraceutically with many supplements from turmeric extract, red tart cherry extract, quercetin, etc. Could this stop the decline?
Or Is It Just the Womb? However, I suspect that if you study the men with perfect testosterone, you will find that they are not living that much healthier of a lifestyle than their andropause-laden peers. This is what we find with centennarians. They are generally not health fanatics in any sense of the word, and I doubt any of their friends or family could have predicted their longevity. The same phenomenon may be true regarding testosterone and the secret may lie in the womb. Check out this research:
“A study has revealed how men s testosterone levels may be determined before they are born…Researchers have shown that the cells responsible for producing testosterone in adults – known as Leydig cells – are derived from a specific population of stem cells found in the testes…Leydig cells do not develop until puberty but the team showed that their function is impaired if their stem cell forefathers are exposed to reduced levels of testosterone in the womb.” 
Now this does not exactly show that testosterone must decline with age via an andropause like effect. However, it does illustrate how things outside of our control (including genetics and epigenetics) may play more of a role than we think.
CONCLUSION: We simply do not know yet if preservation of testosterone levels comes from variables within our control, such as lifestyle and anti-aging, or if it may be the result of events outside of our control, such as womb T levels. In the meantime, what we can reasonably conjecture (in my opinion) is the fact that we can at least slow down the rate of andropause through lifestyle and possibly nutraceutical methods. Hopefully, more studies will follow.
1) Clin Endocrinol, 2012; 77(5):755-763, (See Medscape), “Serum Testosterone, Dihydrotestosterone and Estradiol Concentrations in Older Men Self-reporting Very Good Health: The Healthy Man Study”
2) Clin Interv Aging, 2008 Mar, 3(1): 25 44, “Testosterone for the aging male; current evidence and recommended practice”