Gene-culture co-evolution emphasizes the joint role of culture and genes for the emergence of altruistic and cooperative behaviors and behavioral genetics provides estimates of their relative importance. However, these approaches cannot assess which biological traits determine altruism or how. We analyze the association between altruism in adults and the exposure to prenatal sex hormones, using the second-to-fourth digit ratio. We find an inverted U-shaped relation for left and right hands, which is very consistent for men and less systematic for women. Subjects with both high and low digit ratios give less than individuals with intermediate digit ratios. We repeat the exercise with the same subjects seven months later and find a similar association, even though subjects' behavior differs the second time they play the game. We then construct proxies of the median digit ratio in the population (using more than 1000 different subjects), show that subjects' altruism decreases with the distance of their ratio to these proxies. These results provide direct evidence that prenatal events contribute to the variation of altruistic behavior and that the exposure to fetal hormones is one of the relevant biological factors. In addition, the findings suggest that there might be an optimal level of exposure to these hormones from social perspective.