Yes, it can be hard to find. I have found plain, whole milk organic yogurt in the large tubs (32oz I think)- I have bought Nancy's Organic and Stonyfield Organic and like both. The only 6oz container of plan, whole milk yogurt I have found is a brand called Brown Cow. I would guess only higher end grocery stores carry it.
Also, I don't think it's that baby won't be "fine" with a low-fat yogurt, it's that the full-fat version is best for them. Babies burn a lot of calories everyday, and are growing and developing so quickly they need the fat to accomplish all that. Not only that, but the process to make a food low-fat usually depletes nutrients from the product.