You're right, many policemen do carry guns, batons, tasers, and many other weapons. I honestly don't believe that their weapons are the problem as much as their training is. Police officers used to be trained to serve and protect, and used to be trained to shoot at an assailant's limbs. Nowadays many (not all, and not even most, but still a large amount) don't do that. Many police officers shoot to kill, and pull out their gun when they want to when they should only be pulling it out when they absolutely need to. Just look at all of the innocent children and young adults who have been killed by trigger-happy cops.
If we could change the training that police receive, and stop hiring corrupt cops, I definitely think that the cases of police brutality would decrease.