I think that everyone's pregnancy is so different....with both my first pregnancy and this pregnancy, all I did was lose weight the first half b/c I was sick. Then (with my 1st pregnancy) I really didn't gain any more weight except that of the baby (so like 15 lbs).
A big question is what your body type is like now. If you are underweight, it would be more of an issue vs. you being overweight. If your doctor is not concerned, I wouldn't be concerned.