I had a large fibroid with my last pregnancy. And i got a crash course on how many complications they can cause. Fibroids can grow outside the uterus, in the wall or inside the uterus, so they concerns for each type are different. I only had one fibroid and it was outside my uterus. What happened during pregnancy was extremely painful, but there were no real complications to the baby or myself. The hormones caused my fibroid to grow tremendously. Mine grew on a stalk and actually twisted in the process (i won't go on, but it caused me pain for 4 weeks which I could only take tylenol for pain management). It also made me show 10 weeks further along. Fibroids can also shrink in the process of pregnancy (nothing that happened with me), so it's all really case by case.
With my pregnancy, i saw an MFM in addition to my OB. Got u/s every month and was monitored very closely. I also had an MRI to see it better. I ended up having it removed 6 months after having my baby. She came out fine, but because of the lack of real estate to move her and her being transverse breech, i had to have a csection.
I would start asking where the fibroids are located and what they're approach is. Fibroids outside the uterus tend to grow in a stalk like mine and can distort the uterus shape. Other fibroids case bleeding problems and can be more complicated at time of labor. If your OB is not referring you to an MFM for special monitoring, I would ask why. Again, my case was kind of freakish. I had the largest fibroid my MFM had seen, so that should put it in perspective.
GL. Don't worry too much about it. Just get more information from your specific situation so you are aware.