Vitamin D was once one of the superstars of the supplement world and was supposed to help solve virtually every chronic disease under the son. However, results have been lackluster to say the least, something I cover in my page on The Potential Dangers of Vitamin D. I try to keep my Vitamin D in the 40-45 ng/ml range, because of some of this research. The fundamental issue is that Vitamin D is involved in calcium metabolism, and so overly high Vitamin D can drive calcium into soft tissues. (Some argue that Vitamin D supplementation will overcome this dark side of Vitamin D, but, as far as I know, this has not been proven by research.)
On this page I will give you a profound reason that you do not want to go to the other extreme and let your Vitamin D sink too low: we now have a study that shows that Vitamin D is linked to erectile dysfunction. Furthermore, the authors of this study used the rich and extensive NHANES data, i.e. it was not based on a small sample. Their conclusion was as follows:
“After accounting for NHANES sampling, the weighted prevalence of 25(OH)D deficiency and of ED were 30% and 15.2%, respectively. 25(OH)D levels were lower in men with vs. those without ED. After adjusting for lifestyle variables, comorbidities, and medication use, men with 25(OH)D deficiency had a higher prevalence of ED compared to those with levels =30 ng/ml. CONCLUSION: In this cross-sectional analysis of a representative sample of U.S. men, vitamin D deficiency was associated with an increased prevalence of ED independent of ASCVD risk factors.” 
1. Nitric Oxide. The first link has to do with that fact that Vitamin D helps govern eNOS, the enzyme that helps create your endothelial (arterial) nitric oxide. I cover the research here: Vitamin D and Nitric Oxide. Basically, as your Vitamin D lowers, your nitric oxide levels will generally lower as well.
It is hard to argue that there are two more important molecules to erections than testosterone and nitric oxide and Vitamin D has some control over both of them. That said, more studywork is needed and the authors point out that “additional research is needed to evaluate whether treating vitamin D deficiency improves erectile function..” 
1) Atherosclerosis, 2016 Jul 29, 252:61-67. “Associated With Greater Prevalence of Erectile Dysfunction: The National Health and Nutrition Examination Survey (NHANES) 2001-2004”