`(new_maxhp * old_hp) / old_maxhp` avoids the original problem when using integers. Though you still need to make sure your type is big enough to not overflow.

Or using fixed-point (which are a relatively simple abstraction over integers): `old_hp * (new_maxhp * old_maxhp)`.