Well, one concern with HRT (or TRT as some prefer) is that it could lower HDL levels. HDL is the “good cholesterol” that is responsible for cleaning up the “bad cholesterol” in your arteries. It does this via a mechanism called RCT, or Reverse Cholesterol Transport. RCT is a process – and HDL is the main player – that removes cholesterol from “foam cells” and then transports it to the liver for elimination. Most of the plaque reversing gurus like to see respectable HDL levels and you can see my link on HDL, LDL and Triglyceride Levels To Reverse Atherosclerosis for their actual thresholds. (Low HDL levels are also associated with a “pattern B,” pro-atherosclerotic lipid patterns.)
Although a surprisingly complex subject, clearly HDL is an important facet of arterial protection, including your penile ones. So let’s start with a older study on Seventh Day Adventists and Mormons, who were chosen because of their avoidance of alcohol and cigarettes, that found that “testosterone has a strong negative association with HDL-C in men.”  In other words, the higher the testosterone levels in these men, the lower their HDL levels.
Other observations and research have shown similar results, i.e. cases where increased testosterone has led to lower HDL:
a) Men on steroids have significantly lowered HDL. Yet another reason not to do steroids, eh?
b) Men taking the “old school” oral testosterones lead to lower HDL per one study. 
1. Oral Testosterone Undecanoate (Used in Andriol). This type of oral testosterone is different that the one mentioned above, because it effectively bypasses the liver. And one study showed that 120 mg/day led to no signficant decrease in HDL. Furthermore, the authors concluded, since the testosterone also lowered LDL and triglycerides, that “we conclude that TU [testosterone undecanoate] may be an effective drug for protecting coronary heart disease in healthy elderly men with lowered TT [total testosterone] and FT [free testosterone] levels.” 
2. Injected Testosterone Undecanoate (Used in Nebido / Aveed). One study noted that this resulted in a significantly improved total cholesterol/HDL ratio, which should, the authors noted, improve CVD risk.
3. HCG Monotherapy or HRT. Researchers gave hypogonadal men – non-Klinefelter (HCG), Klinefelter (HRT) – testosterone-raising treatments based on their underlying status concluded that “triglyceride, LpA-I:A-II, Lp(a), HDL cholesterol, HDL3 cholesterol, and apolipoprotein (apo) A-I concentrations did not change significantly after treatment.” 
4. Androgel. A study using 1% Androgel likewise showed no increases in HDL nor HDL’s “efflux capacity,” i.e. it’s ability to help macrophages get rid of their cholesterol. 
Just as important, I would argue that most men will have to self-monitor. Get an HDL (and other basic lipids) read before HRT and then afterwards and see if your HDL has changed. The studies deal with averages and you may be one of the unlucky ones that has their HDL lowered by HRT – who knows? For example, keep in mind the relationship of blood pressure to testosterone. Most studies that I have seen show a drop in blood pressure with HRT. However, some men have the side effect of elevated blood pressure and even have to go off of testosterone because of it. So there is no one-size-fits-all when it comes to HRT.
You may be thinking that your doctor is very unlikely to test this before or after unless you are covered via a physical or annual wellness check. I would agree with you.
The good news is that, at least here in the U.S., you can pull your lipid numbers very easily without a doctors orders in most states. (Thank God for freedom, eh?) I have a list of labs that Peak Testosterone Forum members have used. See my page Testosterone Labs for more information.
1) Metabolism, 1983 May, 32(5):428-32, “Relationship of plasma HDL-cholesterol to testosterone, estradiol, and sex-hormone-binding globulin levels in men and women”
2) JCEM, Published Online: July 01, 2013, “Physiological levels of estradiol stimulate plasma high density lipoprotein2 cholesterol levels in normal men”
3) Arteriosclerosis, Thrombosis, and Vascular Biology, 2004, 24: 1741-1742, “Estrogen and HDL: All that Glitters Is not Gold”
4) JAMA, 2009 May 13, 301(18):1892-901, “Circulating estradiol and mortality in men with systolic chronic heart failure”
5) J Clin Invest, May 15 2003, 111(10):1579 1587, “HDL-associated estradiol stimulates endothelial NO synthase and vasodilation in an SR-BI dependent manner”
6) Medscape, Clin Lipidology, 2012;7(4):363-365, Katya B Rubinow, Stephanie T Page, “Testosterone, HDL and Cardiovascular Risk in Men: The Jury Is Still Out”
7) Jpn Heart J, 1997 Jan, 38(1):73-82, “Beneficial effects of testosterone undecanoate on the lipoprotein profiles in healthy elderly men. A placebo controlled study.”
8) International Journal of Endocrinology, 2014, “Effects of Five-Year Treatment with Testosterone Undecanoate on Metabolic and Hormonal Parameters in Ageing Men with Metabolic Syndrome”
9) J Clin Endocrinol Metab, 1996 Sep, 81(9):3372-8, “Effects of gonadotropin and testosterone treatments on Lipoprotein(a), high density lipoprotein particles, and other lipoprotein levels in male hypogonadism”
10) J Lipid Res, Jul 2012; 53(7): 1376 1383, “Testosterone replacement in hypogonadal men alters the HDL proteome but not HDL cholesterol efflux capacity”