Yeah it's interesting because a novel like The Stranger, like so much of existentialism, is actually a commentary about the loss of God (or Christianity). But everyone is so secular these days that we find it hard to see the problem (and so we sort of misread the points Camus, Sartre, Kierkegaard et al were making).
All this started with Nietzsche, who was among the first of the "great" thinkers of the 19th century to see that the consolations of Catholicism--Christendom--were sweeped away, gone. The entire foundation of the Western world was religion, and then in a short span of a hundred years or so, it wasn't. It was replaced of course with science, but again, Nietzsche saw that "science" was not really a meaningful replacement for religion. When he said "God is dead" he was being prophetic--he was saying "you people do not even understand the sea changes that are about to sweep through the Western world."
Existentialism was the philosophical response to nihilism; Kierkegaard was a Christian but thought it was absurd to be a "believer" and required a subjective, transformative experience that filled one with anxiety and dread (the "leap of faith",through darkness, into light, as it were). Sartre was an atheist and coined the phrase "existence precedes essence." This is, again, is a profoundly religious-inspired statement: we once got our "essences" from the religious world--the notion of a soul, a benevolent Creator, and a universe that had meaning for us personally. Suddenly there are no essences, as there is no longer "God" to give them to us. So in a void where nothing can mean anything, isn't existence (that is, without essences) terrifying and pointless? Who shall we be? And how? Sartre's answer is that we "create" our essences (his dictum means: we exist first, then we choose our essence). This all sounds warm and fuzzy today, but I think most of us don't really think through what he's saying. The freedom we gain once severed from our religious essences is, according to Sartre, a "radical" freedom. It's a gut-wrenching realization that every choice you make creates you (whereas you once had a "blueprint" to work with, so to speak). He would not understand (or certainly not agree with) our blue-sky attitudes about our existence. I think it's funny to reflect on existentialism's message in our modern techno-science world. It's sort of like: who has the time for all this fear and trembling? Huh? Like Huxley warned us in his Brave New World (an almost perfect commentary on scientific distopia), we can alienate ourselves from ourselves with distractions-- iPhones, money, Facebook, on and on. The big questions don't go away so much as they never can quite come up, busy as we are (and doing, really, what?). Existentialists would say we're guilty of a false consciousness (or in Sartre's words, a "bad faith"). I get it, but you know, existential angst has it's limits. :)
Anyway Camus makes this point well in The Stranger, and to my point above, note that it is the priest in prison that brings forth the main character's rage. Not accidental.