We had an extremely cold winter as well. I have several places with milkweed. In one area, I am experiencing as you are where the milkweed has thinned to a great degree. In the other area they are really spreading and very thick. I have 3 other areas that just started from seed last year and they are spreading as well. I haven't a clue why the one area has thinned so considerably but if due to cold I would think all areas would have suffered the same fate.