We basically swaddle for naps (and when we are trying to get him down for one) and take him out so he can play. If we want him to stay awake (usually after 5) we'll keep him unswaddled. So I'd say that he's unswaddled for about 4-5 hours during the day total + any diaper changes. If he's supposed to sleep after a bottle I keep him swaddled during the bottle too. This will gradually change as he gets older and stays awake more.
This is what we did with DS #1 as well and we cut the swaddle at night first cold turkey (we were sleep training, so we did it all during that time) and then during naps came about 3 weeks later when we had to sleep train for naps.