For me, the problem I can't get around is if they think he's inverted, why don't they try to fix him like they did the others?

Meanwhile, if they think he's uninverted, but are thinking that some of the things he did while inverted he did while uninverted, then they are thinking that he's jumped past the line he's always drawn, and therefore feeling really betrayed.