The phrase "you have so much potential" stings like a slap in the face. I got it in school, when I was struggling with undiagnosed disabilities. I got it at home, when my Gran talked about pushing me into lucrative professions for which I'm totally unsuited. I got it in the workplace, from employers who pointedly didn't understand my needs.
From this, I conclude that the phrase is the interpersonal equivalent of pounding on a machine and shouting "just work, damnit!"