If your testosterone levels are above about 350, you should look at the Testosterone Lowering Factors, Mind, Sleep and Sex links to see if you can make some simple lifestyle changes and boost your testosterone naturally. These methods may boost your testosterone 20-50% and, if you're sitting in at about 350-400 ng/dl (or greater), may put you into safe territory without even going on testosterone therapy.
But odds are that if you're reading this link, your testosterone is quite low: 350 or less. If that is the case, then I recommend investigating testosterone therapy with your doctor. ( Free testosterone readings are critical as well, but most doctors test for both, at least initially.) Discuss with your doctor, but my experience is that guys begin to suffer erectile, mental and other distrubances once they starting getting into the 350 ng/dl and below range and this seems to match well with the studies as well.
So let's say you find you have low testosterone - it's a simple blood test ordered by your doctor - and he recommends testosterone (hormone replacement) therapy? Can it cause prostate cancer or other serious medical conditions? Well, the good news is that they have been studying this for years and the studies show no correlation with testosterone therapy and prostate cancer.  (Talk to your doctor, though, of course.)
The Oncology Times  summarized by stating that they "reviewed decades of research and found no compelling evidence that testosterone replacement therapy increases the incidence of prostate cancer or cardiovascular disease". And, if you have heard stories of men having their prostate cancer treated with low testosterone levels, consider what else the authors had to say:
"It has been known since the 1940s that severe reductions of testosterone can cause shrinkage of metastatic prostate cancer, and therefore there has been a concern that raising testosterone levels might cause growth of any hidden prostate cancers. But the study by Dr. Morgentaler and his coauthor, Ernani L. Rhoden, MD, found no connection between higher testosterone levels and prostate cancer, nor did they find evidence that testosterone treatment causes prostate cancer."
There were theories for awhile that "fast converters", i.e. guys that converted to DHT and estrogen easily, could develop prostate cancer. But, again, the research has not born this out as far as I know. In fact, one huge metanalysis that aggregated multiple studies and thousands of HRT participants found found no risk for high DHT, estrogen or testosterone.  (Again, talk to your doc for the latest info.)
CAUTION: Many doctors are still very cautious about the safety of HRT, because they know that if a patient gets prostate cancer, reducing testosterone is one of the proven, time-tested treatment protocols. In other words, if you have prostate cancer, testosterone exacerbates and accelerates it progression. However, most doctors do regular prostate exams along with PSA screening and, therefore, feel they can avoid putting patients on testosterone if they already have a latent cancer.
In fact, from all we know at this point, it looks like the real risk is not going on testosterone therapy if you have low testosterone. One study, for example, found that testosterone therapy actually lowered PSA levels and decreased most of the major symptoms of an enlarged prostate.  And, as I mentioned, low testosterone is tied to a number of serious medical conditions.
Furthermore, testosterone therapy will likely help cure, or even permanently cure any Erectile Dysfunction (ED) that you have been experiencing. For exmaple, one study from Taiwan found that 34% of men with erectile dsyfunction that were unresponsive to Viagra, which is pretty serious impotence, responded well to testosterone supplementation. And 38% of the man responded to both Viagra and testosterone therapy, so almost three fourths of the men with ED were significantly improved by simply taking testosterone. If you couple testosterone therapy, i.e. hormone replacement therapy, with the advice that I give on boosting nitric oxide here and here, you'll likely feel about twenty years younger in EVERY way.
Testosterone therapy can also improve your memory, brain, cognition and learning. Yes, testosterone is intimately linked to cerebral function. Several studies have shown that, in particular, visual spacial skills are tied to testosterone levels. And hypogonadal (low testosterone) men have been shown in several studies to have lower verbal skills.  You will also likely find that testosterone, and therefore testosterone therapy, improves mental outlook: low testosterone is correlated to depression, anxiety and other mental struggles. And by boosting your testosterone levels, you can help reduce your risk for diabetes and many other life-threatening low testosterone maladies.
So what are you waiting for? Get your testosterone read and find out if testosterone therapy is right for you. You can improve your mood, your sex life, your erectile stength and boost your memory and mental abilities. Yes, it takes several weeks (or even a few months) for testosterone therapy to "take effect": your tissues must literally rebuild themselves. However, once those tissues are rebuilt, the difference can be remarkable. Some of you will feel strongly that the minute you started taking testosterone was the minute you got your life back.
If you are interested, you should read about some of the More Common Side Effects of Testosterone (Hormone Replacement) Therapy and common Methods of Testosterone Delivery (such as gels and injections). These links should give you a lot of good talking points with your doctor.
1) Mar 25 2004,26(6):30-35
2) Saving Your Brain, Jeff Victoroff, p. 135
3) Clin Endocrin (Ox),1988;28:461-470
4) British J Clin Pharm,2008, 65:253-259
5) Intl J Andrology, 2002 Apr, 25(2):119-25
6) JAMA, 2006 Nov 15, 296(19):2351-61; N Engl J Med, 2004 Jan 29, 350(5):482-92
7) J of the Nat Cancer Inst, 2008, 100(3):170-183, "Endogenous Sex Hormones and Prostate Cancer: A Collaborative Analysis of 18 Prospective Studies"