It's not that your husband doesn't want to spend time with you, but a) he's probably just as overwhelmed as you are at being a new parent, but b) he's trying to do right by your family and take care of the finances so you don't have to.
However, if you're that upset, you need to voice your concerns to him - either you feel alone, you don't feel he has as much time as you'd like with LO, etc., but don't just whine to him about it - offer some solutions.
And I'd highly advise you join a local mommy & me group or new mom's group so you can get out of the house with LO, and spend some time with other new moms. My new mom's group was my saving grace the first 3 months after DS was born...