User Group moderation help?

[mufar]

Basically, I'm looking to set up moderators for only a specific user group. I do not want them to be able to mess with other groups, and i plan on having several groups for different games (multi gaming mmo clan) Each with their own moderator that can:
-Add users to the group
-Remove users from the group
and of course moderate the appropriate forum they are given jurisdiction over.

These groups will be what the forum post, viewing permissions will be based on.
How would I do this? I don't want to give them access to admin control pannel if possible.

I looked at "Social Groups" but they are a little too simplistic and will not do what we want.

thanks.