I agree. However, the U-2 maintained high altitudes through efficiency on the ragged edge of its operating envelope. Relative to interceptors and SAMs, the U-2 was barely moving. It is still not known what brought down Gary Powers' U-2 but he did recall seeing another parachute on the way down.
On the other hand, the SR-71 employed the elegant, brute-force method. Short wings and gigantic afterburning turbojets employing ram recovery allowed for a much higher speed and altitude.
According to Wiki it was a Russian SAM S-75 that brought it down, it has a maximum range of 82K feet, enough to get to the U2.