This paper explores the factors increasing the chances of thinking one’s cultural tastes and interests would be looked down upon by others, a sentiment put under the label of “symbolic violence” by Pierre Bourdieu. Using data from a survey fielded in the US in 2017–18 (n = 2,514), it examines the role of social position, lifestyle, family status and other properties. Multiple correspondence analysis is used to model both the US social space and the lifestyle system. Multinomial logit regression modeling then suggests the volume of economic and cultural capital possessed by individuals is key to anticipating symbolic violence, alongside age, gender and parental status, though capital composition plays a part in differentiating likelihoods of expressing neutrality on the question. Interactions are investigated using decision-tree analysis. This reveals that subtle interplays between social position and other factors are at work in shaping the likelihood of feeling one would be looked down upon by others.