I once wanted to become an atheist, but I gave up - they have no holidays.

The joke hinges on a pragmatic, self-interested reason for (not) adopting atheism: the perceived loss of holidays. Youngman treats belief as if it were a lifestyle subscription whose benefits include time off and communal festivities, reducing a profound metaphysical stance to a matter of perks. The humor comes from incongruity—choosing a worldview for vacation days—and from gently satirizing how people may value religion for its cultural and social rewards as much as for faith. It also pokes at the idea that atheism lacks ritual or communal celebration, a stereotype used here for a quick punchline.