From the outside it honestly looks like half of what the federal government does is prevent certain states from abusing their people.

The problem isn’t large governments, you get abuse of power even on a very local level, think small town mayor or police or even in some hobby related clubs with just a few members.

Also, there is plenty of abuse when you don’t give any power to someone but they just take it because there is nobody to prevent them from taking it.