I think you need to get out of the house. It sounds like you are getting a little bit of "cabin fever"
Here's what I do.
1x a week I go to a women's bible study with my friends. They have a nursery right off the study room, so they can get me if they need to, but I have an hour and a half to be a "grown up"
1x a week (at least) I make a plan to go out for a walk with one of my friends. Having a "date" with someone makes it more likely that I am going to actually get dressed and get out of the house.
1x a week I go to a "playgroup" with my friends at one of our houses. The kids play and we sit and talk.
If your friends don't have babies, look for a mommy group or a playgroup in your area.
What you are going through is totally normal. Let your mom watch LO, take a bath and then have a nap. Trust that your mom will get you if LO needs you, and it will be easier to sleep.
If you are BFing, maybe instead of going to dinner, just go for a walk or something first with your DH to show yourself that baby will be ok. Or just go out for dessert. Build up to dinner.
Also, like a PP said, the evening "witching hour" is normal. And will pass. My DD had hers from 6-9pm and would cluster feed the whole time.