Elevated prolactin can lower libido and testosterone and affect your bedroom prowess. As I discuss in my link on The Male Refractory Period, elevated prolactin can also lengthen the recovery time post-intercourse. In fact, men with a high enough prolactin level can actually end up anorgasmic. Remember that right after the Big O, a surge of prolactin is released and this is meant by Mother Nature to counteract the effects of elevated dopamine. 
Elevated prolactin also tends to lower dopamine, your feel-good neurotransmitter, which can affect mood (negatively). Conversely, dopamine is a prolactin inhibitor.  In addition, high prolactin can decrease your fertility as well. So the picture can turn ugly quickly to say the least.
CAUTION: If you have elevated prolactin, discuss immediately with your physician. It can be the result of a prolactinoma, a pituitary tumor, that needs medical evaluation.
Of course, there are several well-known medications that powerfully lower prolactin. Cabergoline (Dostinex) and bromocriptene are the two well-known pharmaceuticals that have helped many men with clinically high prolactin. There is one big problem with these two drugs: they are rife with side effects. Cabergoline does a little better, but many men struggle with nasty issues such as nausea, constipation, dry mouth, fluid retention, heart palpitations and so on. Furthermore, the percentage of men who struggle with some kind of significant side effect is quite high, although that is dependent on dosage.
Are their decent natural alternatives? Well, yes and no. Nothing has the power of one of these medications, but, nevertheless, there are a few ways to lower your prolactin that could potentially help a man with mildly elevated prolactin. (Always discuss with your doctor first!)
1. Macuna Pruriens. Researchers in India found that an extract (5 grams of a certain product) from the legume called Velvet Bean not only gave a nice boost in testosterone from 449 to 572 ng/dl, but also dropped prolactin levels from 10.76 to 7.28 ng/ml, at least in men struggling with fertility.  For those without mathematical superpowers, that's a 27% increase in testosterone and a 33% drop in prolactin. Keep in mind that a specific subpopulation was studied in this case.
CAUTION: Macuna pruriens seem to have caused heart palpiations in a couple of Peak Testosterone Forum members. Of course, most men do not experience arrhythmias from macuna, but apparently the increase dopamine may affect some men. I would recommend discussing with your doctor first.
2. Zinc. Zinc may help some men according to one study of male hemodialysis patients. Giving these men 50 mg of zinc per day lowered their prolactin levels from 29 to 11 ng/ml! CAUTION: Don't go too high with zinc as some experts believe it is neurotoxic above a certain threshold. See my page on The Potential Dangers of Zinc for many other issues with zinc supplementation.
CAUTION: Believe it or not, low prolactin is a fairly common issue with many men and can be linked to a variety of medical conditions. See my link on A Guide to Low Prolactin for details.
3. Decrease Estradiol. Increasing estradiol, the "bad" estrogen, is said to increase prolactin by a number of experts. (Actually, you need some estradiol, or E2, as I document in my link on Do Men Need Estrogen?) However, I see no research evidence that lowering estradiol, through Arimidex (anastrozole), etc., actually lowers prolactin. In fact, a couple of studies in women show no significant change.  However, there is a chance that in some men lowering estradiol may help, which means things like weight loss may improve prolactin levels.
4. Sleep. Sleep disorders or lack of sleep will lower your dopamine levels, which can increase prolactin, as I document in my link on Sleep and Dopamine. One study of men with sleep apnea found that they had a significant decrease in prolactin levels from treatment with CPAP devices.  I'm not sure why, but they monitored this after about 1-3 years in these men, but, at least it shows that the positive changes look permanent.
5. Ashwagandha (Withania Somnifera). This popular ayurvedic herb lowered prolactin in a couple of subpopulations according to one study.  It should be noted that the effects were small - 10 to 15% depending on the subpopulation looked at. The dosage was 5 grams/day of the root powder.
6. Magnesium Deficiency. A magnesium deficiency is reported to lead to lower levels of dopamine. Often dopamine is antagonistic with dopamine and so it is possible that a magnesium deficiency could lead to elevated levels of prolactin, although I was not able to verify this. But it is easy to test this out.
7. Primary Hypothyroidism. This lowering of thyroid function is usually caused by Hashimoto's or an iodine deficiency. It leads to an increase in TRH and can increase prolactin levels, something I cover in my page on Hypothyroidism and High Prolactin.
8. CAUTION: Vitamin B6. Megadosing B6 can lower prolactin. I do not mention this as a solution, because megadosing with B6 can lead permanent nerve damage if done incorrectly. (I am cautious about megadosing vitamins in general.) The form of Vitamin B6 may matter as well.  Regardless, it would probably be prudent to make sure you are not deficient in one of these vitamins.
9. CAUTION: Chasteberry (Vitex agnus castus). Chasteberry has been shown in several in vitro studies to bind to dopamine receptors in pituitary cells.  It also has a history of being used in women for hyperprolactinemia.  Unfortunately, there appear to be no studies on men and, at least in women, it has some unusual affects such as increasing progesterone and LH (leutinizing hormone) while decreasing FSH (follicle stimulating hormone). In my opinion, this seems like an unpredictable mix of effects for long term usage.
1) Biol Psychol, 2006 Mar, 71(3):312-5. Epub 2005 Aug 10, "The post-orgasmic prolactin increase following intercourse is greater than following masturbation and suggests greater satiety"
2) Endocr Rev, 2001 Dec;22(6):724-63, "Dopamine as a prolactin (PRL) inhibitor"
4) Tumori, 1998 Jan-Feb;84(1):45-7, "Clinical efficacy of the aromatase inhibitor anastrozole in relation to prolactin secretion in heavily pretreated metastatic breast cancer"
5) Sleep and Breathing, Sep 2010, 14(3):253-257, "Infertility and obstructive sleep apnea: the effect of continuous positive airway pressure therapy on serum prolactin levels"
6) http://longwoodherbal.org/vitex/vitex.pdf, Clinicial Nutrition Insights, Black Cohosh and Chasteberry: Herbs Valued by Women for Centuries, BY JOSEPH L. MAYO, MD, FACOG
7) Drug Saf, 2005, 28(4):319-32, "Vitex agnus castus: a systematic review of adverse events"
8) Evidence-Based Complementary and Alternative Medicine, 2011, "Withania somnifera Improves Semen Quality in Stress-Related Male Fertility"
9) J Pharm Sci, 1979 Sep, 68(9):1179-81, "Effects of pyridoxine hydrochloride (vitamin B6) on chlorpromazine-induced serum prolactin rise in male rats"