It's the bitwise AND operator:
http://php.net/manual/en/language.operators.bitwise.php. $vbulletin->userinfo['permissions']['forumpermissions'] is a number which, when you look at it as a binary value, each bit is the state of one forum permission for that user. $vbulletin->bf_ugp_forumpermissions['custom_perm'] is a "mask" value with only one bit set, the bit corresponding to the "custom_perm" permission (whatever that is). So if you "bitwise AND" those two values, the result will be either 0 or non-zero, depending on whether or not the user has that permission.
Probably that's either too much or not enough of an explanation. If it's not enough, this might help:
http://en.wikipedia.org/wiki/Bitmask