I've certainly wondered.

[Yeah in retrospect it was totally different but nevermind that!]

It seems like a stretch, for someone to remain loyal through all of this and then abandon him...but I suppose it is possible. It certainly explains why he may have some new vendetta against other people's freedom of choice.