I'm with ya, there, lady! I have one cup of coffee in the AM, but by the time the late afternoon hits, I'm exhausted and ready to call it a day. I try to get all my to-do's done during my caffeine buzz so that when that time hits, I can just relax. I have a 3 year old stepson whom I watch during the day, and a 4 mo LO who is teething and becoming more and more active. I'm 6 weeks along, and it can be tough (however, I don't feel like I've hit my full fatigue stage, yet)! But I gotta deal, so when the kiddos are taking their afternoon nap, so does this mama :) Then, when my lovely hubby comes home, he helps with the kids so that I can relax from the crazy day or finish getting my tasks done. When I just need a break until he does come home, my 3 year old loves puzzles and watching movies. And while I feel bad that he gets so much more TV time than usual, if we both benefit by a little down time, its a win. With my 4 mo old, however, she requires more attention because she's just growing so fast and going through so many milestones, right now. I endure through my fatigue because I'm afraid to miss anything!
My advice: take it easy when you can. Indulge in one coffee (or your choice of caffeine) a day at any point you choose to help. Relax/nap when the kiddos do. Take advantage of hubby when he gets home, and take some downtime for yourself.
Most of us are in the same boat, so take comfort that there are ears to hear and advice to be given. :)