If being rude is your only concern, then I would let him stay home. There is nothing rude about his not attending in whole or in part.
Possibly unrelated sidenote: I've known more than a few women who used the worried-about-rudeness excuse just to get their DHs to attend something with them; heck, I have done it myself, with disastrous results. In my experience, you're more likely to get a positive response from him if he knows he is helping you (instead of just following what he sees as arbitrary social rules). Yes, he may say no, but at least he knows what he's really saying no to!