Individuals and groups are often conflated and it's hard sometimes to make that separation. "Not all men" etc. It's true that not all men are actively woman-hating. It's true that not all cis people are transphobic. It's true that not all cops are evil.
But institutions and group dynamics create places where this behavior is unquestioned and unchallenged.

When the good ones do not question the bad ones, you create a situation where evil is allowed to thrive. When good does nothing, evil wins.

Good = Symbiotic, sympathetic, compassion, working together, cooperation
Evil = Parasitic, destructive, manipulative, taking for the few from the many

Do you know why I hate most morality systems in video games?
Because good is often incentivized over evil in a way that doesn't reflect the nature of the world. Makes good seem like it's always obvious, even when it isn't.
Evil becomes a caricature. Do you eat kids for power? Then you're evil. Do you... not? Then you're good
And evil is given a tiny fraction of power, and good is given a LOT of power. Or they're kept starkly equal.

Here's what good and evil look like in real life:

Good people work together, they create networks that foster teamwork and safety, allowing each specialist to go about what they're best at so that everybody prospers.
Evil people infiltrate this system and take, they destroy parts of it so that they and their in-groups can prosper.

(Using good and evil here mostly for shorthand, thus why I defined it there. Obviously there's more to it than that, but that should hopefully work as a 101 for talking about the nature of non-religious morality)

When good people do not oust evil people, because they find themselves on the in group, they enable the evil to strangle the spread of prosperity, until it becomes actively harmful. Evil people benefit much more than good people, here. But when we oust them, when we question their ethics and keep the systems working for everyone, we all gain more collectively, but not individually.

The strength of goodness is in numbers, in help, not in personal, individual prosperity/power.

Whenever saying "All" that's what it means. It means the systems of power and the lack of people questioning them, often benefiting from them. In institutions, it's closer to literal.... it's not that these people are without moral fiber, it's that they exist within a corrupt system and benefit from it while not rocking the boat.