I take them out once the pupae/chryalis is fully hardened. I remove them from the plastic wrap or leaves (some pupate on leaves) by spraying with water, waiting a few seconds for the silk to loosen, and then pulling gently. If it doesn't come off with a gentle pull, I spray again.
Then I pin these with a stick pin (with plastic head only--I don't use metal heads for fear the butterflies will tear their wings on them). When the silk is still wet, I push the pin carefully through the silk above the cremaster. I weave the stick pin through the net container and leave the sharp pointed side on the outside of the container. I pin the pins vertically, not horizonally so they stay in place.
I find the pins easier to deal with than a cool glue gun or tying them off with string or dental floss.
Herndon, VA (USA)
Take care of the small things....