Yes, frost will kill tropical seedlings. But, those swan plants (africian milkweed, Asclepias physocarpa) I've seen more mature plants take a few frost in the fall and still looked green. Blood flower (tropical milkweed, Asclepias curassavica) is not as hardy.
Herndon, VA (USA)
Take care of the small things....