I work as an HR Manager. You need to go to HR and let them know. He may not understand FMLA laws, and to your point may have been stressed, short staffed, etc, and did not understand how to handle the situation. He really needs coaching from his HR Manager because he is a liability to your company.
Now, just because you are covered by intermittent FMLA for pregnancy does not mean your husband is. You will want him to get covered.
The company does have a right to ask for doctor/er notes & releases, but this should be collected by the fmla administrator to ensure it complies. Your manager does not even technically need to know you are pregnant, just that you are covered for an FMLA reason. If they suspect abuse of FMLA ( and trust me, intermittent fmla is abused widely and often looked down upon by both managers and hr...this is however typically for people who have fmla for migraines or difficult issues to truly see and we often see a pattern in when they take their days, i.e. monday/Fridays, extra long lunch hours to have an affair, etc), the manager should report it to HR/FMLA administrator and they will handle accordingly.
Honestly, the manager acted like a jerk and had no right to hold you there. If you work for a large company or company that has large presence in an area, can you imagine the headlines in the news, "Lady delivers prematurely at work and loses baby because manager from x,y,z wouldn't allow husband to drive her to hospital" or other horrible things. It's the news paper test and no company wants to be in it.
Take your note to the fmla administrator & HR manager and go talk to your HR manager quickly. I would also create a calendar of the days you have already used. If you are worried about damaging the relationship with him, just think about what would happen if you had really been in premature labor and he pulls this crap again.
There are things the company can reasonably request in this situation and might make sense, such as limiting the number of shifts the two of you work together to avoid situations such as these. You might word it to you HR Manager as "I'd like to discuss a situation related to my FMLA leave that occured on x evening with my manager. I was in extreme pain and can provide you with the relevant ER notes related to my condition. I have the utmost respect for my manager and have never had an issue such as this happen previously, but should something similar happen in the future, I really need to ensure it is handled differently. My husband and I are open to any reasonable accommodations necessary for the unit to ensure both the company and our family are not at risk."