Using contacts has been better (for me) than the Zeiss inserts in terms of hand tracking and eye tracking accuracy. Often, I need to run a calibration step if changing between them, otherwise gaze typically targets the item below what I’m looking at, which is great for identifying one might have anger issues to resolve.
Have you tried w/ contact lenses? I've seen that work well for friends when they try them out.