Spell it Xanthe. If people pronounce it wrong, they are the ones who look ignorant/unaware. If you spell it wrong, you and your husband are the ones who look ignorant.
I think it's cool and I like it. It is unique in the good sense of the word- an actual, albeit uncommon, female name.