I've seen this posted about twice before over on vb.com. Unfortunately, neither user posted the fix (
When Editing Usergroup, vB Makes A New Usergroup of the Same Name Instead). Try disabling your modifications and see if it works then -
Note: To temporarily disable the plugin system, edit config.php and add this line right under <?php
PHP Code:
define('DISABLE_HOOKS', true);
If it's still happening, reupload all the original vB non-image files (except install.php). Make sure you upload these in ASCII format and overwrite the ones on the server.