I just wipe it off with a wet paper towel. I never notice any residue or flavor doing that. I figure if I'm not tasting the vinegar I spray on after, I'm definitely not tasting some dish soap.

I also use a very light natural dish soap. YMMV if you're using one with strongs scents/chemicals.