If the camera sensor doesn't get that much light, the mode will be infrared and grayscale. Unless you have a bright light in the room, expect the colors to be muted. Not sure why your brown sheet would look white though, as our motorola monitor shows brown as black. We have the 3.5" model and I really like it, but the bigger screen makes a world of difference.
The video has been really useful for us, keeps us from running up the stairs for every little peep.
ETA: I have yet to find a real use for the intercom feature. Maybe it's more useful for older kids.