I am in a similar situation and just recently had a second opinion appt and switched REs.
For IVF #1 we did a long lupron protocol and it seems I was over suppressed. My E2 rose very slowly and they ended up getting 7 eggs, although only 3 were mature. All 3 fertilized, and 2 made it to freeze although they were not great quality. We did not do an ET because we have to do PGD and wanted to do another cycle and get more embryos to test since the cost of PGD is high and they will test up to 8 embryos for that cost.
I am 30 and my AMH is 1.17- I had asked my RE about it and he said it was fine. I felt given my age it was low, but went with what he said. For our next cycle I felt he wasn't being aggressive enough and sought a second opinion. The new RE is more aggressive and said that while my AMH is considered normal (above 1 is normal), that it is right on the edge and that for my age, it is low. I felt 100% more comfortable with this RE than I do with my previous one, and I am so happy I got a second opinion. (He is doing an antagonist protocol with estrogen priming for IVF #2).
IVF is so hard because you really have no control over the situation, so I think it is really important that you completely trust the RE you are working with. Initially I was kind of feeling "should I get a second opinion? And am I just doing this because our cycle failed?" I think that it is normal to kind of feel that way. I also know that if we did stick with our RE for IVF#2 and had similar issues, I would be kicking myself for not getting a second opinion. I sort of already feel like we "wasted time" with our first RE, and do not want to feel that way again.
I totally agree with PPs about getting a second opinion. I felt so much better after ours and feel excited about moving forward with a doctor I am comfortable with and confident in.
As far as requesting records or letting them know you are switching - they see it all the time. I was open with my RE and told him I was going to be seeking a second opinion because I just was not 100% comfortable moving forward at this time.
Phew, this was long! Good luck with whatever you decide, and keep us posted!