Eh hehe I'm back again.
It seems I have a different problem with my the character MA this time.
For some reason I think there's some reason with the modifier for it. After I update my stats after changing class or something it seems to count my MA only from elements.
So basically it would decrease my MA to something it shouldn't be. I've checked it with other characters afterwards and found that everyone else who hasn't done this still has what seems like the right amount.