As for plant-based diets, like veganism and vegetarianism, there was a decreased risk of prostate cancer risk.
"Our review highlighted a cause for concern with high consumption of dairy products," lead author John Shin said in a statement. "The findings also support a growing body of evidence on the potential benefits of plant-based diets."
Despite the results, the team acknowledged some limitations. They said the studies they evaluated used a range of methods, which could affect the overall findings.
They also noted prostate cancer is more prevalent in Western countries, where dairy is the main source of calcium. Asian countries have lower rates of prostate cancer and consume less dairy.
The scientists now hope to continue their studies with more randomized controlled trials that better report diet and lifestyle data.
Want to learn more about the results? Take a look at the full assessment here.
» RELATED: Study: These two foods could lower breast cancer risk