Extensive research investigates how immigration shapes natives’ anti-immigrant sentiment. However, several areas require further scrutiny. This paper explores how processes of immigration affect anti-immigrant sentiment in a new immigration destination country—Japan—drawing on longitudinal data to examine these processes over time and explicitly testing the mechanisms of perceived threat and intergroup contact posited to underpin this relationship. Through this analysis, the paper aims to: examine how generalizable the immigrant share/immigration attitudes theoretical framework is to non-Western societies; refine our understanding of the mechanisms underpinning this relationship; and more robustly test its causal assumptions. To pursue these aims, the study draws on two sets of nationally representative Japanese data, designed to generate complementary insights, including: four waves of longitudinal panel data (2008–2014) and a unique cross-sectional dataset containing measures of perceived threat and contact. Applying multilevel and fixed-effects panel data approaches, the findings demonstrate that as immigration increases in Japanese prefectures and municipalities, residents become increasingly averse toward immigration (although there is some evidence of non-linearity at the municipality level, with sentiment improving again in high immigrant share environments). This overall relationship appears largely driven by two competing processes. In higher immigrant share environments, perceived threat is higher, increasing anti-immigrant sentiment. However, concurrently, intergroup contact also increases in these environments, reducing anti-immigrant sentiment. Therefore, despite the overall negative relationship (driven primarily by perceived threat), rising contact exerts a countervailing positive effect as immigration increases. Taken together, this research demonstrates that theories of attitudinal-change with higher immigration, developed within Western-contexts, also appear salient for newer destination, non-Western societies.