a quasi political opinion about alliances
Any relationship will have places where it's not perfect and I think that applies to allyship. The closer you are to/pass as WASP and male, you *are* going to feel attacked (not everyone's a fantastic communicator, and our ability to communicate gets increasingly broken with the stress of living in a prejudicial society), and sometimes yeah, that's because some of the people you want to help straight up hate you (nobody's immune to getting bad baggage from bad experiences or convenient scapegoating).
This is on a bad collision course with actual privilege; you're already out of your comfort zone and suddenly you're in the middle of stuff which *is* uncomfortable while people are expecting you to be perfectly okay with it. This is the point where a lot of would be allies decide naw, they hate the idea completely.
But allyship is essentially a relationship with a more equitable society which has a lot of benefits for pretty much everyone, so it's worth sticking with.