Well, you could just do an if test in the reputation.php file for the user groups you wanted to allow (or dis-allow). The actual removing it from the templates would be a bit more work. (removing the link for them from the postbit or wherever you have the things to add rep.
After global.php require~
PHP Code:
if($bbuserinfo[usergroupid]!=6){print_no_permission();}
Would something like that do it? (6 = administrator usergroup on most boards)
Of course if you wanted something fancy from the admin panel that'd require more work as well.
EDIT: or whichever error message you wanted. You could always make your own and echo it or whatnot.