“Civilization has shrunk,” wrote Virginia Woolf in 1939. Scribbled in her diary on the eve of World War II, Woolf’s remark captures the fears about the decline of the British Empire that have afflicted the UK for decades. Thursday’s Brexit vote is in many ways the culmination of this perpetual crisis of self-worth. When Britain decides whether to leave or remain in the European Union, many will be swayed by nostalgia for the heady days of British supremacy. Voters will be making a statement not just about their country’s place in Europe, but about their self-esteem.

Nineteenth-century history looms large in Britain, and commentators have pointed out some of the imperial longing at play in Thursday’s vote. “Unlike other one-time imperial powers in the EU such as Germany, Spain or Portugal,” Peter Geoghegan observes in Al Jazeera, “a sepia-tinged vision of the colonial past is quite mainstream in the UK, uniting both elite politicians and an increasingly disenfranchised white working class.” The essayist Pankaj Mishra struck a similar though less forgiving note in Bloomberg, branding Brexit “little more than another instance of dead-end thinking by some empire fetishists, powered by the same delusion of a small island regaining its global power and influence.” The New Statesman framed the situation more irreverently: “Is Brexit a marvelous opportunity to renew our imperial contacts?”

These accounts may exaggerate contemporary Britain’s desire for “empire” but the memory of imperial dominance is important because it brings nostalgia for a stronger Britain, the Britain that supposedly once was. “They’re talking about Britain reasserting itself, Britain being a world power again, Britain not needing anyone else,” Oxford historian Margaret MacMillan recently told Christiane Amanpour. “They’re talking about the glorious Elizabethan Age; they’re talking about the time that Britain ruled the world. It’s a fake sort of nostalgia because of course it doesn’t take into account the complexities [of the situation].”

Ersatz nostalgia like this has led to an overly simplistic understanding of how to resolve the EU issue. MacMillan characterized the “Leave” perspective as: “Let’s disentangle ourselves from his ghastly organization called Europe and suddenly everything will be all right.” With so many connected issues at play—immigration, trade, sovereignty, employment, and wages, among others—Brexit affords a nation in flux a simple way to reassert its identity: negative definition. The UK is not Europe; the economic crises and hotbeds of extremism are over there, not here. This is not about trade agreements: Brexit is a reckless attempt to reclaim the UK’s self-esteem by going it alone.

Arguments for reclaiming national “sovereignty” are less about a boost in autonomy than about the jolt of self-confidence that going it alone would provide. Ex-London mayor Boris Johnson frequently indulges in this kind of rhetoric of national self-sufficiency. Fond of telling UK citizens that they don’t need Europe—and in fact are being held back by the continent—Johnson made the following plea for exiting the EU in Sunday’s Telegraph: “Now is the time to believe in ourselves, and in what Britain can do, and to remember that we always do best when we believe in ourselves.”

Characteristic of the Brexiteer message, Johnson’s text was less a policy speech than a shameless appeal to the wounded pride of Britons. “It’s about us taking back control of our own destiny as a nation and being free to blaze our own trail in the world,” Nigel Farage, the leader of the right-wing UK Independence Party, wrote in an editorial of his own. Only when liberated from Europe’s problems, the Brexiteer script goes, can Britain recover its former glory. This phenomenon is a psychological one: Britain’s insecurity about its status in world today has created a righteous fantasy in which the country no longer needs help from anyone else because, after all, it once ruled the world, and should be capable of doing so again.

In an essay adapted from The Future of Nostalgia, the late Harvard professor Svetlana Boym writes, “In extreme cases [nostalgia] can create a phantom homeland, for the sake of which one is ready to die or kill. … Unreflective nostalgia,” she adds, “can breed monsters.” Boym’s conclusion, which emerges from her flâneur-like strolls through European cities, pop culture, and the events of her own life, became tragically prescient last week, when a terrorist allegedly shouted, “Britain First,” before fatally shooting and stabbing the British MP Jo Cox, who was campaigning on behalf of the Remain camp.

To be sure, “the Brexit camp is not all nostalgists longing to return to British exceptionalism,” as Tanya Gold reminds us in the New York Times. “Mostly it attracts people who fear that Brussels excludes them and that there is a genuine deficit of democracy in Britain; that the political class, with a few exceptions, has forgotten them.” Other Brexiteers rightly see a correlation between EU membership and the combination of falling wages and rising house prices. These positions hold water. We cannot afford to caricature this debate as a battle between cosmopolitan elites and poor, white nativists. It’s much too complicated for that.

That said, the rhetoric swirling around Brexit has escalated to such an extreme that it is best understood by focusing on the affective or psychological currents at play, such as nostalgia. The fact that an otherwise deadlocked race has swung towards “Remain” after the murder of MP Jo Cox suggests that a collective feeling more than anything will determine the final vote. Current polls predict a very narrow victory for the “Remain” camp, though, again, one should never underestimate a mood. Come Thursday, it’s still possible that the pride of Britain may trump the need for a unified Europe. Civilization may yet shrink further.