This has always been a tough one for me. When my big kids were little, I was really careful about what went to their mom's house and what stayed with us -- any clothing that was a gift from my parents, for instance, stayed here, and so did anything I bought for them. I was just as careful about sending back whatever belonged at their Mom's house, though, so maybe I'm an anomaly!
As they got older, and started growing like weeds, I decided that it was just silly to hold on to something that might not even fit them the next time I saw them (seriously, they grow fast. Our 14 year old daughter is now about 5 foot 9.). And it seemed just plain wrong to hold on to something they loved to wear just because it was a gift from my parents or because I bought it for them. So, now, whatever they wear most or whatever fits them perfectly I just sent with them. They also know, though, that if they forget to bring it to our house, that's on them... I'm not going to chase down their favorite shorts, they're old enough -- at 14, 12, and 9 -- to decide what they're going to wear where.