There's also the Milkweed Tussock Moth caterpillars.
In the past years, they have SWARMED my milkweeds. First time it happened, I was clueless and the armies stripped the milkweeds in my garden -- I had to go hunting milkweed for my cats later in the season
Last couple of years, I made sure to bag and discard mass hatchings of MTM's
This year, I've only found ones and twos of MTM's and those I just flung far away from my milkweeds. I reasoned that IF they make it back, then more power to them.
It's a good thing because the drought defoliated some of the plants in the driest part of my garden. They seemed fine, then all of a sudden, the bottom 18 inches or so of the leaves turned yellow and fell off in a matter of days.