We brought him to an elective u/s at 10 weeks (done at an outside provider/not a doctor's office) and again at 30 weeks. They are family friendly and therefore had toys for him to occupy himself with. At the 10 week appointment, he could not have cared less about the baby. He was happy playing with the toys in which they provided.
At the 30 week, he was slightly more interested, and wanted to lay on the bed with me, which I allowed. He pointed at the large screen, said, "baby brother!" then hopped down to play with the toys once again.
I would never consider taking him into the OB office, though. I have had ultrasounds with the OB and have always made sure that he had a babysitter, as FI attends the ultrasound appointments with me. An elective ultrasound is different, and most OBs don't allow (or may not have a rule against, but certainly do not want) children in the office.
The ultrasound room at my OB is also incredibly small, and there is office/medical equipment within DS's reach. I wouldn't want to take the experience away from FI or myself when we'd be having to keep our attention on DS so he wouldn't let his curiosity get the best of him.