There is no option or single file or function that will allow you to change how the password is encrypted, but I believe there are only a few places where you'd have to change the code to do it. I think right now it uses md5(password . salt), or maybe md5(md5(password). salt), I'm not sure.
Anyway, It looks like the main check is in verify_authentication() in includes/functions_login.php, and there's code for changing the password in profile.php. There are a couple other places that I think just check for passwords the same as the user name (so that if you left those alone you'd just lose that security check). There may be other places that I didn't find.
One other note, I think when the password comes from the browser or a cookie it has another level of md5() that the browser does via js, so that just adds to the confusion.