As with many issues brought up on here, if your experienced doctor tells you something, you don't need to second guess his opinion on an Internet board: you need to ask questions to him, or ask for a second opinion.
We are not doctors, and it always makes me wonder when I see posters argue against your doctor's opinion because *they* never had a problem with a smaller baby.
Trust is important, and if you don't trust your OB, I don't know who you can trust on pregnancy matters, but certainly not Internet people... Call and ask, why the wait, get your doctor to explain things to you! I understand that you may have been in shock and not thought of those questions, but it is their job to answer them.