I'm sure your DH is just being extra cautious. My DH and I didnt "wrestle" (as we call it) until cleared by the dr. and we still took it easy.
If you can get your DH to the doctor with you and have your dr explain that it's okay to "wrestle" that may help him get over his anxieties.
I don't think it's your hubby deliberately doing this, he just wants to be cautious. Plus, the thought of potentially harming your LO may put him off as well.