I always took my kids with me to the eye doctor and dentist, regardless of their age until they were school age. I either kept them in the carseat bucket or used a stroller. Doctor's offices are generally pretty cool with helping you find a place to "park" the baby and stroller while you're in the examining room.
I took them with me to the gyno while they were babies, then got a sitter as they were old enough to know what was going on. Ditto with normal doctor's appointments, depending on the reason for the visit.
Sometimes you just can't hold/wear the baby because it's disruptive to the appointment or potentially unsafe. Then you have to use a stroller. If your child is not used to the stroller and you think that it will disrupt the appointment, get a sitter. Most doctor's offices will work with parents on this type of thing, though -- it's a customer service thing.