We should bring back the Committees Of Vigilance like they had back in the 19th century.
Basically peacetime militias. Armed townsmen who live regular lives and work regular jobs. When called upon they assist their local law enforcement agencies. Checks and balances. They both keep each other from power grabbing or getting out of control.