Some of us that were low testosterone just feel older. When I was first diagnosed with low testosterone, I had the testosterone of someone in his late 70's or early 80's. And that's about how I felt. I didn't just feel low in libido - I felt OLDER.
It turns out my gut feelings were right: low testosterone negatively alters almost every key marker of aging. In my case I am pretty sure I was low testosterone for decades and, as I'll show below, this was very likely very hard on my body.
Many people wonder why I very carefully watch what I eat and maintain a disciplined exercise routine. One of the primary reasons is that I am relatively certain that I was low testosterone for decades. Because of that, I think it is likely that I aged myself considerably and must now try to compensate for those years of "hard livin'".
Below we'll cover some of the ways that testosterone is associated with accelerated aging:
1. Telomerase. As you may know from My Link on Telomerase, there is a whole theory of aging that centers around our telomeres. Basically, our cells divide over the course of our lifetime and the lion's share of them suffer a slight shortening of the end of their chromosomes, the telomere, each time. This slight shortening eventually leads to aged, barely-functioning tissues. Furthermore, this area of research is considered one of the most promising ways to actually extend maximum human lifespan, although that is too big of a subject to include here.
One of the keys to slowing down the erosion of our telomeres is increasing the activity of the key enzyme involved in this process, telomerase. In the Above Link on Telomerase I clearly outline many natural strategies to increase telomerase. However, testosterone (and estradiol) has been shown to do just this as well in a number of studies and on a number of cells. Here are just a few examples:
a) Bone Marrow Cells. In one study of bone marrow cells, testosterone was found to increase TERT mRNA expression. (TERT and TERC are part of a whole suite of telomerase molecules.)  Interestingly enough, the researchers found that it was actually estrogen that increased telomerase and testosterone only got credit due to aromatization. Again, this just shows that "you can't fight your hormones!"
b) Stem Cells. Researchers took human stem cells and found the estradiol (E2) turned on and off TERT and telomerase.  In this study they even examined which type of estrogen receptor was involved. Again, this shows the estrogen to telomerase connection. (This has been seen in various cancer cells as well.) 
c) Testes. A study on rats shows a correlation at least between the lowered testosterone levels of aging with a decrease in telomerase activity. 
How does testosterone work its magic. Well, we get a double benefit, because testosterone and estradiol boost telomerase activity. Estradiol was found to do this through TERT expression. (You don't want to go too high on TERT activity based on what I have read.) Check out this summary of the situation:
"Their novel studies demonstrate that androgens increased telomerase activity via a transcriptional mechanism in normal peripheral blood lymphocytes, bone marrow CD34+ cells, and lymphocytes from patients harboring heterozygous telomerase mutations. Interestingly, estradiol had a similar effect. Further investigation of the molecular basis for increased telomerase activity demonstrated that androgens undergo aromatization to estradiol, which then binds to the estrogen receptor-a to increase TERT expression. This is likely through estrogen response elements in the TERT promoter." 
2. Insulin. One of the things I emphasize on this site is that as testosterone falls, insulin levels rise. And men that are low testosterone have been found to be greatly at risk for diabetes and Metabolic Syndrome (or prediabetes). The problem is this increase in insulin levels is accompanied with increasing insulin resistance.
Once insulin resistance sets in, the body cannot push the glucose into the cells and blood glucose levels begin to rise. And this is what will age many tissues in the body. Elevated blood glucose levels are very hard on the body and why diabetics almost always have a shorter-than-average lifespan.
3. Inflammation. Is there anything more dangerous to a man's long term health than inflammation? Elevated inflammation has been linked to heart disease, many cancers and many autoimmune diseases, so it has to be near the top of the list. In addition, inflammation is part of one of the primary theories of aging. One research summary stated that "we have proposed the theory of oxidation-inflammation as the main cause of aging." 
The key regulators of inflammation in the body are the "inflammatory cytokines" such as tnf alpha, IL-1, IL-6 and so on. As I show in my link on Testosterone and Inflammation, low testosterone is associated with rising levels of almost all of these inflammatory messengers.
4. Boosting the Body's Natural Antioxidants. One recent study found the following very important result:
"Extending our previous results, we show that treatment with testosterone, both in intramuscular or bioadhesive buccal formulations, increase plasma levels of Coenzyme Q10, lipophilic antioxidant, and total antioxidant capacity, measured with colorimetric method, in patients with secondary hypogonadism. Hypogonadism could represent a condition of oxidative stress, in turn related with augmented cardiovascular risk in such patients." 
5. Oxidation. As mentioned in #3, one of the primary theories of aging deals with oxidation (coupled with inflammation). An increased oxidative load, for example is highly associated with "immunosenescence", which is the steady decline of the immune system with aging.  As you may have guessed, some oxidative systems have been associated with decreasing testosterone levels. For example, castrated rats have been found to have greatly increased oxidative stress levels in the prostatic epithelium.
In my page on The Causes of Andropause, I point out that mitochondrial damage may be a cause of the age-related loss of testosterone. If so, this could be explained by the heavy oxidative load and increase of reactive oxygen species from falling testosterone levels, in part anyway. (Mitochondria, as the energy producers, are particularly vulnerable to free radical damage.)
1) Blood, Sep10 2009, 114(11):2236-2243, "Sex hormones, acting on the TERT gene, increase telomerase activity in human primary hematopoietic cells"
2) Mol Cells, 1999 Jun 30, 9(3):286-91, "Downregulation of telomerase in rat during the aging process"
3) Molecules and Cells, 2008, 26(5):454-458, "Estrogen receptor-alpha mediates the effects of estradiol on telomerase activity in human mesenchymal stem cells"
4) Cancer Res, Dec 1 1999, 59:5917, "Estrogen Activates Telomerase"
5) Curr Pharm Des, 2009, 15(26):3003-26, "An update of the oxidation-inflammation theory of aging: the involvement of the immune system in oxi-inflamm-aging"
6) http://www.discoverymedicine.com/ Nancy-S-Jenny/2012/06/25/inflammation-in-aging-cause-effect-or-both/
7) The American Journal of Pathology, Dec 2003, 163(6):2513 2522, "Androgenic Regulation of Oxidative Stress in the Rat Prostate : Involvement of NAD(P)H Oxidases and Antioxidant Defense Machinery during Prostatic Involution and Regrowth"
8) J Steroids Horm Sci, Published Jul 30 2013, 4:117, "Effects of Intramuscular or Bioadhesive Buccal Testosterone Treatment on Antioxidant Systems in Secondary Hypogonadism"
9) Blood, Sep 10 2009; Blood: 114(11), "HEMATOPOIESIS & STEM CELLS: TERTrific hormones promote hematopoiesis," by Laura S. Haneline