Had you been charting or doing anything that gives you an idea of when you ovulated?
Also, out of pure curiosity, do you know why you had an u/s so early? I mean 4-5 weeks is pretty early considering most OBs don't even do an u/s until the anatomy scan at 20w. Were you seeing an RE?
Don't worry and lots of luck at your next appointment!