Much, I suspect, depends on the particular abuses.
Enjoying free food from people who are trying to influence your opinion, because you are a person whose power affects the server's good is an abuse of power, but I'd hardly consider that a high crime. OK, with what I weigh, maybe my taking chocolates would be a crime, but not a high one.
Conversely, using your position of power to (say) arrange for the death or disgrace of an opponent--that's a high crime, as far as I see it.
Power, after all, can be abused in many ways... None of them would be a good thing, but some are, from the human standpoint, less horrid than others.
From the Divine standpoint, as someone pointed out, abuse of power is a sin, and all sin is infinitely abhorrent to God, but I believe the context here is the human perspective...