This "someone" needs to quiet their mouth unless they are your OB. It does not "make sense;" how would it?
I worked a manual labor job when I got pregnant lifting things up to 40lbs and had a lengthy talk with my OB about lifting. He said there's no way, unless you already have bleeding, a placental issue or compromised cervix, that lifting an cause any harm to the baby. The idea that lifting something can harm the pregnancy is a myth. The myth stems from the truth that lifting can, once you are big and unbalanced, cause you to pull a muscle or strain something.
I was told I could keep working my job as long as it was comfortable for me; he guessed I would start to have issues around 25 weeks because of the big belly. He also told me that lifting my 30lb toddler is not an issue whatsoever. You can lift anything that your body is used to lifting. I don't plan to stop lifting my kid until I physically can't; as in I'm so big that there's nowhere to rest her comfortably. Pregnant women have been lifting previous kids since the beginning of time. Don't worry about it. Hold your toddler.