I went to a similar one last weekend: no wedding party, held in a private home, bride wore a cocktail dress.
One of the pregnant guests wore slim black ankle-length slacks (cotton sateen, it looked like) and a dressy top (silk with an embroidered yoke) with heels. She looked lovely, and I heard her get lots of compliments on her outfit - I admired it myself! If you already have something like that, I say wear it. And if the bride doesn't care, and you want to wear a maxi dress, just make sure it's appropriate for the time of day and you'll be fine. (In other words, don't wear a sundress style to an evening wedding, and don't wear sequins at 10:00 a.m.) If your outfit is within the general parameters of a party held at that time of day, wear it with pride!
Sidenote: I wouldn't wear slacks to a church wedding, no matter how casual, unless it was a very last minute event on a weekday. To a church event, I'd wear whatever I'd usually wear to that kind of church at that time of day.