Adopt the Missionary Position
How exactly do evil cults recruit followers? There’s typically no shortage of cultists, who throw themselves in the path of adventurers. But how are they converted to a nihilistic or inherently evil faith? When the presence of gods is almost factual, how do you convince someone to abandon a formal church to worship a demon prince? What makes someone turn from the goddess of sunshine and puppies and decide to pray to an unspeakable lord of pestilence?
I’m not sure I have any great answers for these questions. And, really, filling out the backstory of cultists and making them reasonable human beings driven to faith in pure evil just makes the villains sympathetic. It humanizes what should really amount to cannon fodder.
I actually find it easier in Wharhammer 40K setting: being under an oppressive, demanding and demeaning government brings people to the brinks of despair. Any word speaking of freedom through rebellion gets a warm welcome in these stuations, even if it does mean worshipping an awful creature (provided its true nature is revealed to the would-be cultists at all – there is always the possibility of a “false flag” recruitment).
It is a lot more unlikely to happen in a medieval setting, especially in a kingdom under the fair and compassionate rule of a kind paladin… yet, there are a few persons whose life has been poisoned by envy, grief and bad luck, not to mention the ambitious ones which would sell their very soul to get access to power… I mean, those who didn’t take the adventurer’s path.
Evil cultists aren’t generally known as the highest quality group in your average Campaign Setting, so it seems to me that there would be three bases for recruitment into evil cults
1) Screwups. If you’ve made bad choices and ended up in a marginal role (or out of a role) in society, maybe doing the bidding of Karkus doesn’t seem like such a bad idea? When you’re an outlaw already, might as well start beseeching the forces that claim to have a plan and offer cool powers, especially when their plan generally involves upsetting the established order (after all, if the evil cult WAS the established order, as could happen with Bane, say, cultural assimilation could be an explanation)
2) The Lost and Destitute. Like #1, they have very little, possibly nothing to lose. Unlike #1, since it was through no fault of their own (or minor fault at worst), rather than being a mercenary decision to throw in your lot with a power greater than yourself, it could very well be a conscious choice to reject the gods favored by the society that dealt you a bad hand. In a sense, these are people who turn to evil after the Problem of Evil has disillusioned them with the shinier gods. True, the priests of Pelor say that Karkus is bad, but Pelor hasn’t done much for you, has he?
3) The overly ambitious. In D&D, “Good” is focused on the group, while “Evil” is focused on the individual. Therefore, those that strive for more than their fair share (in a philosophical sense) should logically feel at least a temptation towards “Evil”. Granted, since adventurers are a ‘thing’ there are paths to advancement that are more accepted by society, but in general the “good” way of uplifting yourself is going to be harder than what Evil forces offer. In essence, with Good you pay your dues upfront and get what you deserve, while Evil is more liable to give you what you want right now, at a cost to be paid later (the traditional “sell your soul” deal, whether literally as many fiends would bargain, or figurative in pledging yourself to an Evil higher power for the rewards they offer)
I do also think it depends very much on what Evil power and what assumptions about the Outer Planes and Afterlife are being made. It’s easy to see, in the context of some editions of Forgotten Realms, why somebody would worship Bane: Not only does Bane espouse a philosophy that is not without merits (after all, social darwinism has had its moments in real life), but depending on your edition, those who worship and live strongly within the precepts of an evil god will be accepted into their deity’s domain, not simply formed into a petitioner at the mercy of the fiends. So, if you worship the Evil God of Thieves, and live your life stealing everything you want and never repent your crimes, when your soul goes to judgment said God of Thieves will supposedly welcome you with open arms. I’ve always enjoyed this interpretation, since it seemed to answer why evil cults can sustain: they at least teach that you’ll go to Evil Paradise. Which might be more of a crab bucket than Good Paradise, but there’s at least some subset of people to whom it sounds like a good eternity.
It requires a bit of a diversion from the classic idea, but there are two ways I see of getting similar effective results to hundreds or thousands of cultists:
1) Like Razmir in Pathfinder and the Blood of Vol in Eberron, most people who follow it don’t realize the religion is actually a cult to an evil abomination. Of course, this works better in a setting with gods who are more on the quiet side, but I don’t think it’s impossible in settings like Faerun given a clever or powerful enough leader. For real life inspiration, imagine modern-day religious terrorist cells if an actual and literal demon or evil god was behind them.
2) Arguably a subset or variation of the above, but the Shadow Council in Warcraft would be another way to go about this. And this is probably similar to Cheliax’s history. Start with just a few cultists how are gifted great power by the evil entity they worship, then they get themselves into a place of great influence over the leader of a people, convince the leader into seeing the merits to their way, and then he’ll convert the populace. The majority of the populace doesn’t really need to follow or even really know much about the religion, and even if they do, they might not care/feel confident or safe enough to say anything. For real life inspiration, look to Nazi Germany.