Taking its point of departure in two historically separate legal cases involving the right to care for kin this article compares the perception and treatment of two groups subjected to forced migration—enslaved Africans and their descendants in the former Danish West Indian colonial society and African refugees in present-day Danish welfare society. Drawing on Balibar’s notion of ‘internal borders’ it demonstrates the key role of ‘the Other’ in the management of displaced people in contexts of structural inequality and their contradictory position of being legally embedded but socially detached in the society in which they are placed. This comparative historical lens illuminates how, across time and place, family and kin ties can figure as sites of contention between universal, ideal, morally accepted human rights and the actual rights bestowed by local authorities, whether in colonial plantation societies based on enslavement or in modern welfare states.