The growth of a pig is not only affected by its own genes, but also by the genes of its pen mates. This indirect effect on a pig's growth is represented as social breeding value (SBV) in a newly developed breeding model. It has been hypothesized that pigs could affect their pen mates’ growth through their behavior. We investigated whether pigs selected for a relatively positive (+SBV) or negative genetic effect (−SBV) on the growth of their pen mates and kept in either barren or straw-enriched pens differ in fearfulness. Effects of coping style, as assessed in a backtest, and gender were also investigated. Pigs (n = 480) were subjected to a group-wise novel rope test and human approach test and individually to a novel environment test in which after 5 min a bucket was lowered from the ceiling. In the novel rope test +SBV pigs were faster than −SBV pigs to touch a rope (P < 0.01) and in the novel environment test +SBV pigs showed less locomotion than −SBV pigs after introduction of the bucket (P < 0.05). Furthermore, straw-enriched pigs were faster than barren housed pigs to touch a rope in the novel rope test (P < 0.10) and faster to approach (P < 0.05) and touch a person (P < 0.05) in the human approach test, suggesting that they are less fearful or more curious than pigs in barren housing. Straw-enriched pigs also had lower salivary cortisol concentrations than barren housed pigs (P < 0.001). Pigs classified as high-resisting in the backtest spent more time near the person in the human approach test (P < 0.10) and showed more locomotion (P < 0.10) and vocalizations (P < 0.001) after introduction of the bucket in the novel environment test than low-resisting pigs. Gilts appeared less fearful than barrows, because they were faster to touch a rope in the novel rope test (P < 0.05) and faster to approach (P < 0.05) and touch a person (P < 0.10) in the human approach test. In addition, in the novel environment test, gilts were more calm (P < 0.05) in the period before the bucket was introduced, paid more attention to the bucket once it was lowered (P < 0.10) and were overall more active (P < 0.01). Gilts also had lower basal cortisol concentrations than barrows (P < 0.001). Overall, these results suggest that +SBV pigs might be less fearful than −SBV pigs. Furthermore, the response of pigs in novelty tests seems to depend also on their housing conditions, coping style, and gender.