Custom Field Viewable by ???

[Mookie]

Is there a way that you can make custom fields viewable by only members of certain groups. Kind of like a rights sections per custom field. Say for instance I would like

Custom field 1 - viewable by Usergroup 4, 5, 6
Custom field 2 - viewable by Usergroup 5
Custom field 3 - viewable by User 1, 2, 3
Custom field 4 - viewable by all usergroups

Reason why I am asking is my users want to be able to not share there location and stuff except with members of there own group. And with that group it is ok to display that info on the Members List area.

Is there a way to do that and how??? I am a total newb to the forums but I am getting the hang of this quickly. Not sure if this is the right area for this question.