Populist parties enjoy stable support in various European countries. The literature on the rise of populism argues that this support especially increases in times of crises. Surprisingly, the German right-wing populist Alternative für Deutschland (AfD) did not increase its support in the wake of the COVID-19 pandemic. Moreover, the party even lost 2.3 percentage points in the 2021 federal election. We address this puzzle and ask why the AfD has not been able to use the crisis to its advantage. Our main argument in answering this question is that, although the AfD pursued the classic populist strategy of fundamental opposition, the support base of the AfD is strongly divided on the preference towards measures containing the spread of COVID-19. This division is reinforced by individual affectedness by the pandemic. Introducing a novel weekly dataset on voter preferences, we show that the AfD support base is strongly divided on the issue with approval of the government measures being a significant and substantial contributor to vote switching away from the AfD. Using regional-level data and a difference-in-differences approach, we further show that western German regions hit especially hard by the pandemic display a lower AfD vote share than other regions. Our findings have important implications for the impact of exogenous shocks on electoral competition and also on the future of populist parties.