Researchers asked 258 healthy women across England who had been identified as having an increased risk of the disease whether they had agreed to take the drug to help prevent breast cancer developing, and interviewed 16 women to identify what influenced their decision to take it.
Women chose not to start taking the drug because they thought cancer was down to fate, they distrusted medication in general or they feared side effects would interfere with looking after their family.
But overall the team, based at the University of Leeds, Northwestern University, University College London and Queen Mary University of London, found women with children were more likely to take up the offer of tamoxifen.
The research, which is the first of its kind since the drug was approved to be used for prevention, also suggested that social class, educational attainment and ethnicity had no effect on uptake.
Tamoxifen is most commonly given to women who have been treated for breast cancer to lower the risk of it recurring.
But in 2013 the National Institute for Health and Care Excellence (NICE) also approved it for cancer prevention in women at increased risk of the disease due to a family history of breast or ovarian cancer, following research which showed it could lower risk by around a third**.
Dr Samuel Smith, study author from the University of Leeds, said: "While it's reassuring a woman's background doesn't seem to be a barrier to taking tamoxifen, only one in seven of those at increased risk of breast cancer are taking up the option. Therefore it's important doctors can discuss women's concerns and provide information to help them while they are considering their options.
"Further research is needed to understand if all women eligible to take tamoxifen for prevention are getting the help and support they need."
Dr Richard Roope, Cancer Research UK's senior clinical adviser and GP expert, said: "When an established drug like tamoxifen is found to work not only as a treatment for breast cancer, but is also shown to reduce the risk of the disease, it seems we're making real progress.
"It's valuable to understand why women might reject tamoxifen, and this research highlights there are a range of complex reasons behind the decision.
"It's vital more work is done to understand these barriers, improve treatments and ensure doctors are getting the support they need to help women decide whether preventative medication is right for them.
"Whatever a woman's risk of developing breast cancer, keeping a healthy weight and cutting back on alcohol are also ways of reducing it."