Two week after we found out I was pregnant, my partner and I sat down and had a 20 minute conversation and decided the two names we wanted. We both had a couple choices and talked them out.
So I personally don't think it's silly to be thinking about it, or making lists early on. Naming decisions can be a very big deal.
However maybe your SO is worried about getting his own hopes up as well. So make your lists, research, and narrow down your choices to give you something to do while waiting for you SO to be ready to have the conversation a little down the road when he is a more comfortable that the pregnancy is going well.