Static discharge happens based on the relativity of potential. You can be charged up to 1000V (with respect to ground) and nothing will go wrong if you touch a sensitive device if the device is also charged to the same 1000V potential.
On the other hand, you may ground yourself and be at 0V potential. Then, you touch a sensitive device that happens to be charged (this can happen if you slide an IC inside a cheap plastic container) and damage the device instantly. That is why a wrist strap is used for grounding. A wrist strap grounds you through a resistor. That resistor limits the current flowing through your body and the device. That to some degree protects the device. Also, the devices must be kept on a special material that does not allow the device to charge up. You can see that kind of material on work benches in a semiconductor lab.
So, you are absolutely right. It does not help to ground yourself if the components are not connected to ground. If you have a hard drive on a table and you want to touch it, you should always first touch its case. Usually, the case of each equipment is connected to its ground plain.
If you have a PC, which is unplugged, nothing inside is grounded. But, you can discharge yourself safely by first touching the case of the PC. All ground plains (CPU, memory, hard drive, graphics card, ....) are connected to the PC case.
ESD kills (semiconductor devices)! Don't listen to advices like "ESD does not do anything". It's like saying "I have driven my car for 10 years and I have never had an accident; don't bother putting a seat belt on"!