It is now fairly well-known in the men's health community that correcting a Vitamin D deficiency can increase testosterone, something I cover in my page on Testosterone and Vitamin D. I have seen men on the Peak Testosterone Forum increase their testosterone with the simple supplement. However, I have also seen other men jump on this and assume, therefore, that the more Vitamin D taken, the better. Below I make the case that this is likely an unwise assumption and even show potential risks of driving your Vitamin D levels too high.
Measuring your Vitamin D levels is a simple, inexpensive test that you can do yourself (here in the U.S.) at any one of these Inexpensive Testosterone Self-Testing Labs. The test is called "25-Hydroxy." This is a good idea, because abundant research shows that a Vitamin D deficiency is likely NOT a good thing and has been tied to literally dozens of chronic and autoimmune diseases and conditions. A deficiency is usually defined as 25-hydroxy levels below 30 ng/ml. (25-hydroxy is just the name of the test for Vitamin D). And, as mentioned above, low Vitamin D can in turn lower testosterone levels . I take Vitamin D every day and am personally targeting 40-50 ng/ml 25-hydroxy levels, but you'll have to study the research and talk to a good doc and decide for yourself of course.
I would also recommend that you look at the 6 Potential Dangers of Increasing Your Vitamin D Levels Above ~50 ng/ml that I have listed below:
1. Decreased Fertility. When I first started understanding the importance of Vitamin D, I listened to a Vitamin D supplement representative claiming that 25-hydroxy levels above 140 are common in some indigenous cultures exposed to direct sunlight. He was clearly arruging for supplementation to very high plasma levels. Like many men, I assumed that I need to really boost my Vitamin D levels to achieve maximum benefits. However, one of the first clues that there was more to the story came from hearing about the fact that high Vitamin D levels actually lowered male fertility. One study found that "Sperm concentration, sperm progressive motility, sperm morphology, and total progressively motile sperm count were lower in men with '25OHD=50 ng ml(-1)' when compared to men with '20 ng ml(-1)=25OHD<50 ng ml(-1)'." 
This showed the obvious: all hormones have a range and Vitamin D was no exception. (Vitamin D is a considered a hormone by many experts, because it affects so many tissues and systems.) CAUTION: One study showed that lower levels of Vitamin D likely lower fertility and stated that "the pregnancy rates per patient and per cycle and delivery rates per patient and per cycle were all significantly higher in couples with normal Vit D levels."  This argued again for a "sweet spot."
2. Increased Estradiol. While it is true that some hypogonadal men are low in estradiol, the high prevalence of overweight and obese men often leads to unnecessarily high estradiol levels and "estrogen dominance" if you will. Men in this category do not need more estradiol, and it turns out that Vitamin D is one of the controllers of estradiol levels in both males and females. Men get their estradiol via the aromatase enzyme conversion of testosterone and Vitamin D governs the aromatase enzyme. One study on mice with deficient Vitamin D levels found greatly reduced aromatase enzyme levels. Estradiol levels were reduced correspondingly by almost 40%.  Going back to #1, this may explain the decreased fertility in men who take too much Vitamin D: higher levels of estradiol actually worsen sperm parameters.
3. Poor Study Outcomes. There have been literally dozens of studies giving patients Vitamin D supplementation. If raising Vitamin D to higher levels was beneficial, it can be (easily) argued that it would have shown up in the research by this point. However, the studies have actually shown the opposite, i.e. that going above 50 ng/ml either provides no additional benefits or, in some cases, even makes things worse. This journal summary says it all:
"Even if 25(OH)D values above 50 ng/mL, or even above 100 ng/mL, are unlikely to cause acute toxicity, achieving 25(OH)D values above 50 ng/mL has not been found to be beneficial, especially for the general population." 
4. Increased Mortality Rates. One study showed that death rates increased at either low or high values of Vitamin D. The authors wrote that "analysis of 24,094 adult patients showed that 25(OH)D levels less than 20 ng/mL and 60 ng/mL or greater before hospitalization were associated with an increased odds of 90-day mortality. Although previous reports have suggested an association between low vitamin D status and mortality, these data raise the issue of potential harm from high serum 25(OH)D levels, provide a rationale for an upper limit to supplementation, and emphasize the need for caution in the use of extremely high doses of vitamin D among patients." 
By the way, this verified what an earlier study had found:
"In this study from the general practice sector, a reverse J-shaped relation between the serum level of 25(OH)D and all-cause mortality was observed, indicating not only a lower limit but also an upper limit. The lowest mortality risk was at 50-60 nmol/liter. The study did not allow inference of causality, and further studies are needed to elucidate a possible causal relationship between 25(OH)D levels, especially higher levels, and mortality." 
5. Higher Cardiovascular Disease Rates. A 2009 study found that non-institutionalized seniors had an inverse association with cardiovascular disease risk. In other words, basically the lower the Vitamin D levels, the lower the risk of heart disease!  It is hard to know the cause for this but remember that Vitamin D controls the body's calcium levels and, therefore, potentially Vitamin D could push calcium into soft tissues. Yes, arterial plaque is calcium, so perhaps this is the mechanism.
6. Increased Pancreatic Cancer Risk. There is a recent study that shows 5000 IU of Vitamin D regressed prostate cancer. However, yet another study found that going too high seemed to increase risk:
"No significant associations were observed for participants with lower 25(OH)D status. However, a high 25(OH)D concentration (> or =100 nmol/L) was associated with a statistically significant 2-fold increase in pancreatic cancer risk overall (odds ratio = 2.12, 95% confidence interval: 1.23, 3.64). Given this result, recommendations to increase vitamin D concentrations in healthy persons for the prevention of cancer should be carefully considered.." 
CONCLUSION: We now have several studies, some even large and recent, that show that raising 25-hydroxy levels above 50 (or perhaps 60) ng/dl is risky and associated with increased risk of chronic disease. Results seem to indicate that there is an optimal zone for Vitamin D.
1) Asian J Androl, 2012 Nov, 14(6):855-9, "Association of 25-hydroxy-vitamin D levels with semen and hormonal parameters"
2) Endocrinology, 2000 Apr, 141(4):1317-24, "Vitamin D is an important factor in estrogen biosynthesis of both female and male gonads"
4) Reprod Biol Endocrinol, 2015, 13: 127, "Males with low serum levels of vitamin D have lower pregnancy rates when ovulation induction and timed intercourse are used as a treatment for infertile couples: results from a pilot study"
5) J Clin Endocrinol Metab. 2014 Apr;99(4):1461-9, "Evidence for a U-shaped relationship between prehospital vitamin D status and mortality: a cohort study"
6) Mayo Clinic Proceedings, May 2015, 90(5):577 586, "Changing Incidence of Serum 25-Hydroxyvitamin D Values Above 50 ng/mL: A 10-Year Population-Based Study"
7) J Clin Endocrinol Metab, 2012 Aug, 97(8):2644-52, "A reverse J-shaped association of all-cause mortality with serum 25-hydroxyvitamin D in general practice: the CopD study.
8) J Am Geriatr Soc, 2009 Sep, 57(9):1595-603,Epub 2009 Jun 22, "Prospective study of serum 25-hydroxyvitamin D level, cardiovascular disease mortality, and all-cause mortality in older U.S. adults"
9) Am J Epidemiol, 2010 Jul 1, 172(1):81-93, "Circulating 25-hydroxyvitamin D and risk of pancreatic cancer: Cohort Consortium Vitamin D Pooling Project of Rarer Cancers.