Human-animal relationships are essential for dairy farming. They affect work comfort and efficiency, as well as milk production. A poor human-animal relationship can result in stress and accidents to both animals and caretakers and needs to be improved. However, many studies have demonstrated the multifactoriality of these relationships. We aimed at assessing the relative importance of the various factors expected to be associated with poor human-animal relationships. On 118 dairy farms, we applied a standardized avoidance distance test to cows at the feeding rack. The sample of farms covered a wide range of situations: lowland versus highland, small versus medium size farms, cubicles versus deep-bedded systems, milking parlor versus automatic milking systems, and Holstein versus Montbeliarde breeds. We used Poisson regression to analyze the links between the number of cows that accepted being touched, and farm characteristics, animals, management, and farmers' attitudes. A multivariate analysis yielded a final model that explained 32.7% of the variability between farms. Calving conditions ("Main calving location" and "Cleaning or adding litter after calving") accounted for a significant part of the variability observed (respectively 25.8 and 13.6% of variability explained by the model, SSB). Fewer cows accepted being touched on farms where the main calving location was in the barn, and where farmers cleaned or added litter after calving. The proportion of cows that accepted being touched increased with the proportion of lean cows in the herd (18.8%), with worker/cow ratio on the farm (11.7%), when farmers considered "health" or "human-cow relationships" as most important issues for farm success (10.4%), and with farmers' years of experience (10.8%). Farmers with more negative behavioral attitudes toward cows had a lower proportion of cows that accepted being touched (8.9%). In conclusion, the human-animal relationship was not found to be associated with farm characteristics (e.g., housing or milking system) but varied with farmers' attitudes and management. We confirm that cows' fear of people is linked to negative attitudes displayed by caretakers toward cows, and is reduced in farms where several caretakers are present. Our study also suggests further exploring the key role of factors linked to calving conditions, as cows are more likely to be afraid of people when disturbed at calving.