I have never head of this. Infact my OB for my first and this pregnancy has ALWAYS recommended taking vitamins no matter what your eating habits. Prenatals contain a large amount of folic acid that is needed in order to prevent birth defects such as neural tube defect and spinabifida. I have never head of these defects being prevented from a "healthy" diet.
Also, in regards to large babies ... I took them every day with my 1st pregnancy, and my son was 7lbs 3 oz 19.5 inches. So I would say, no I don't think their is truth to it. But if I were you I would do the research for myself. Some dr's are more into "natural" medicine then vitamins, which is fine but I would definatley research and decide for yourself. Because ultimately you are the one who is going to have to live with the outcome.
Here is an article on folic acid:
http://www.marchofdimes.com/pregnancy/folicacid_before.html