I look at my son's professional pictures a lot (from Now I Lay Me Down to Sleep), but not the pictures we took on our camera. I find it nice to look at his pictures and remember what he looked like.
That being said, I also look at his ultrasound pictures a lot too. We had 2 elective 3D ultrasounds, and we have tons of pictures and videos from them. Like you said, I like looking at those because he was still alive when they were taken, and since they were 3D (and included videos), I can see his little facial expressions and small movements.
I think its normal not to want to always look at the pictures, but I wouldn't do anything drastic like delete them.