My follower unequips items when I take like items from their inventory

I gave my follower upgraded hat and shirt. I equipped them on him. I then removed the old ones from his inventory by clicking on the item then clicking transfer. When I did this he unequipped the other ones.