Recess duty

During the week I have recess duty 2 times. Which isn't bad, but what I don't like is when they put you on recess duty and leave you there to take care of things, just because you are the "gym" teacher. During recess, they let the kids play kickball, dodgeball, and football. I don't agree with some of these things, but I am only covering 2 times a week and each time I am at a different school. I told them that the kids should not be playing some of these games, but they said the kids like to play dodgeball and they are just going to let them go. Well a fight broke out over someone hitting someone else in the chest, so the other girl took the ball and threw it at the other one's face. Well, this made me mad and I took the dodgeballs and locked them up and said not more dodgeball. Now the teachers and parents are upset that I took it away from them. I don't think I am being unfair, but some people said I took it too far. Just wondering if anyone has recess duty and has come across any games or activities that the kids are playing that you don't agree with, and how you might have handled it?