No, you didn't do anything wrong. You have every right to decline. If your dad asks again, just tell him that with the logistics and the ILs schedule it wouldn't work out. Otherwise, just leave it alone.
And I would not play into the childish antics of the silent treatment from your mom. If she doesn't return your call, then stop calling. She will get the hint that you are not playing that game.