Maybe also try spreading the caffeine you do drink throughout the day. I agree with PPs about slowly going down to 2 cups, and then maybe have 4 half-caff cups a day, and then if you can, increase the amount of decaf and decrease the amount of regular in the cups.
Good luck! It's a hard habit to break, especially with the fatigue that comes with pregnancy.
ETA: I just checked out the caffeine graph- also consider ice coffee- with ice and milk, it really waters down the caffeine!