Testosterone affects everything in men, so it should be no surprise that it affects the appearance of your skin in multiple ways. Most guys aren't too worried about their appearance, but I occasionally receive questions about it. For example, on the Peak Testosterone Forum, I once received this question: "I have a reddish nose, dry flaky skin on nose and broken blood vessels on cheek...Can low [testosterone] give you bad skin?" 
Of course, skin condition is related to many things, including diet, sun exposure and growth hormone. That's right - lower growth hormone levels can actually thin the skin a little. So it's difficult to definitively say anything without some individual experimentation. However, as I'll show below, the flakiness could be partially related to low testosterone-related issues. But first I want to cover the three key mechanisms whereby testosterone has a big influence on the skin:
1. Sebum. Most of us know that testosterone can affect acne, but are not really sure how. Testosterone is actually converted by an enzyme called 5-alpha reductase to DHT (dihydrotestosterone). And it is DHT that controls the sebaceous glands in the skin and the production of sebum. Sebum is one of the things that you don't want too much or too little of. It is composed of a variety of cellular material including fat and keratin and is a natural lubricant for your skin and hair. However, too much sebum is a root cause of acne.
For this reason, there have been various anti-DHT solutions that have been developed to help with acne, including one herbal compound that did quite well according to one study.  Furthermore, sometimes HRT will result in acne as a reported side effect even in middle-aged men. (It is not common though: see my link on Hormone Replacement Therapy for more details.) And, at least theoretically, low testosterone could lead to low DHT, which could, in turn, lead to low sebum levels, "asteatosis" in its extreme case, resulting in dry, scaly skin. As always, there appears to be range that is best.
2. Estrogen (Estradiol or E2). Estrogen in males come almost entirely from the conversion of testosterone via aromatase. (The "bad" estrogen is estradiol or E2.)Aromatase is stored in fat cells and, as we age, we tend to put on weight and convert more testosterone to estrogen. This trend lowers testosterone and raises male estrogen levels over time. So that means that low T men have high estrogen, right?
In actuality, many hypogonadal men actually end up with low estrogen levels even if they are overweight, because their testosterone levels fall so far that no amount of aromatization can rescue the situation..In fact, it is a very common problem for men with low testosterone to actually develop osteopenia initially and later osteoporosis unless it is caught and treated early enough. (There are other nasty problems associated with low estrogen levels, which I cover in this link Do Men Really Need Estrogen?) Several studies have noted this. One in partcular found that low total testosterone was strongly correlated to bone loss and associated with an 88% increased risk of hip and 28% of non-spinal fractures, respectively. 
Theoretically, these low estradiol levels could also negatively impact a man's skin as well, because estrogen is tied to collagen production. Studies have shown, for example, that applying an estradiol cream to the skin of senior men increased collagen proudction.  Thus, it is not unreasonable to assume that men with lower testosterone levels could end up with decreased collagen production and more aged skin as a result.
3. AGE's (Advanced Glycation End Products). Testosterone is very much associated with insulin levels. In general, the more the testosterone, the lower the insulin levels. As I point out in my book Low Testosterone by the Numbers, low testosterone leads to decreased insulin resistance and elevated glucose levels.  This is why hypogonadism is associated so strongly with diabetes and Metabolic Syndrome. One HRT clinic that I talked to said that all their men on testosterone therapy had been able to completely get off of insulin. (A few still had to take Metformin but, still, it is remarkable that they could completely eliminate their insulin regimen.)
The point here is that lower testosterone will tend to raise both insulin and glucose levels, which could lead to accelerated AGE's. AGE's are glycated proteins that accumulate and age various tissues. This is one of the reasons that diabetes is such a debilitating disease: it ages many, many tissue due to this whole issue. The skin is no exception and can age overly rapidly from this as well. See my link on Advanced Glycation End Products for more information.
So, really, the bottom line is that, although high testosterone can lead to acne in some sensitive men, lower testosterone levels may be even more problematic, leading to accelerated aging of the skin via decreased collagen production and lubrication along with and increased levels of glycated proteins. As always, I urge men to Get Tested Regularly for Testosterone and Estradiol.
2) Clin Endocrinol (Oxf), 2005 Sep, 63(3):239-50, "Androgens, insulin resistance and vascular disease in men"
3) Arch Intern Med, 2008, 168(1):47-54, "Endogenous Sex Hormones and Incident Fracture Risk in Older Me: The Dubbo Osteoporosis Epidemiology Study"
4) Arch Dermatol, 2008, 144(9):1129-1140, "Induction of Collagen by Estradiol: Difference Between Sun-Protected and Photodamaged Human Skin In Vivo"
6) European Journal of Integrative Medicine, Nov 2008, 1(suppl 1):40, "Dermal application of a Cimicifuga racemosa (CR)-containing cream has beneficial effects on acne"