Digital Sampling

[Bobthelost]

Originally posted by: smack Down

Besides the sampling at zero problem sampling a frequency at 2x its frequency will result in a sampled singal that has twice the amplitude as the orginial.

It will?

 

[Navid]

For the last time: You are supposed to sample at MORE than twice the highest frequency. If you sample exactly at double the frequency, you are not satisfying the Nyquist rate.

 

[Navid]

Originally posted by: Bobthelost
The higher frequency sine waves get attenuated more than the lower frequency sine waves. Look at the diagram provided at one of the sites you linked to: http://members.aol.com/ajaynejr/nyquist.htm. Look at the difference between the left hand diagram and the one on the right, attenuation.

Don't just look at the diagrams!
The author is explaining different problems that can occur during reconstruction if all the rules are not followed. The diagram on the right is not the reconstructed signal. If you only read it!

I am done here! The Nyquist theorem has been around for a very long time. The entire telecommunications platform on the planet (and in space) relies on it.

 

[Bobthelost]

No, not true, if you're sampling at 2x the frequency you might be able to reconstruct the signal precisely. If you're sampling a sine wave at the peaks and troughs then you can reproduce it exactly. But say you were to drop the frequency down by a third, that brings the ratio to 3x (and then play with the phase) so we're within the nyquist's theorem, you really want to see what godawful monstrosities i can make a wave into sampling around three times per cycle?

How many people know that, how many care? How many really think that sampling at 3x the signal's highest freq component means it's possible to reconstruct the signal properly? It doesn't work like that.

That is what i hate about the nyquist theorem and the way it's taught.

 

[Loki726]

Originally posted by: Bobthelost
Edit: Don't delete it! Crossposting happens and some/all the questions were good ones!

I hate it when people ask for proofs, but such is life...

Ok, example 1 you have a sine wave, it is a nice sine wave, all the curvy bits in the right places and it crosses the x axis exactly twice per cycle. As i said, it's a good sine wave. Now you decide to sample aforementioned wave, so you take it's frequency (Y hz) and then double it to furfill Nyquist's theorem (2Y hz). However there's a problem, you're sampling at 180 degrees and 360 degrees, or in other words at the zero crossing points.

Your signal has just made like elvis and left the building.

Of course this is slightly silly, you'd have to sample the signal at exactly the right phase to zero it all out, (on the flip side unless you're sampling at the exact right place you'll never get the max amplitude of the signal either. So your signal will be attenuated)

Remember our lovely little sine wave? Well now move the sampling points a tiny bit earlier, it's now sampling just before the wave crosses the axis (say 179 degrees and 359). The signal is back, but attenuated to near death. If you're just listening for a pure sine wave then that's ok, but if that sine wave is part of your latest britney collection then the higher frequency notes are going to be too quiet. So they will get lost compared to the lower freq stuff.

Ok, so at nyquist's freq we can lose the signal completely, or it can be attenuated so badly as to ruin the music, (remembering my example this may be a goal in itself). What happens if we increase the sampling frequency a tiny bit? Now the first sample is taken on a crossing point, the second one just next to a crossing point, the third slightly further away and so on. In other words you'll get the right frequency output, but the amplitude will be increasing as the sample points get closer to the peaks of the waves, and then smaller as they move further away. So britney is warbling a nice little solo bit, but the higher tones are even more unstable than a drunk on a tightrope.

Or what i've refered to as the AM bit.

Now if you want mathematical proof for all of this then i'm going to sit in the corner and sulk.

Nice try, but let's count the amount of samples you are taking. Say we have a sine wave at 1hz. And as you said, say we sample at 180degrees and 360 degrees. We get all zero points and no way of figuring out what the amplitude of the 1hz sine wave is. But we only took 2 samples over a period of 1 second; we only sampled at 2 hz, which was not greater than the nyquist rate.

Try sampling at 2.2 Hz

 

[Bobthelost]

Nice try, but let's count the amount of samples you are taking. Say we have a sine wave at 1hz. And as you said, say we sample at 180degrees and 360 degrees. We get all zero points and no way of figuring out what the amplitude of the 1hz sine wave is. But we only took 2 samples over a period of 1 second; we only sampled at 2 hz, which was not greater than the nyquist rate.

Try sampling at 2.2 Hz

We then get the AM effect that i mentioned. The first value = zero, the second a small + value the third a larger - value... That's what happens when you try to reconstitute the signal.

I've sat in a lab and seen it happening.

(Edit: Admittely i don't remember the exact values i was working with, but we were generaly using 1khz sine waves, if you want i can dig my log books out)

 

[Mday]

um, the MINIMUM nyquist frequency is 2x the max freq in a signal. you will need a PERFECTLY square filter to do any type of filtering with the sample. The reason why more than 2x is necessary is because a perfect square filter does not exist. And attenuation is a bad thing (which is the best any real square filter can do).

I suggest some of you read this:
http://cnx.rice.edu/content/m10791/latest/

I dont go to rice, but this is just a pure theory page. take a look at reconstruction on that page. some of you have to go back and read the sampling page as linked in the "introduction" section of that page.

 

[Navid]

Originally posted by: Mday
um, the MINIMUM nyquist frequency is 2x the max freq in a signal. you will need a PERFECTLY square filter to do any type of filtering with the sample. The reason why more than 2x is necessary is because a perfect square filter does not exist. And attenuation is a bad thing (which is the best any real square filter can do).

Mday,

The filter implementation is quite critical and should be considered as you point out. But, that is not why the sampling frequency MUST be higher than twice the highest frequency in the sampled signal. If it is not, there will be aliasing. The signal will be corrupted even before reconstruction!

You are right that it is not practically possible to design a perfectly square filter. But even if it was possible, you would still not be able to remove the sampling spur and keep the highest frequency of the sampled signal if the sampling frequency was equal to twice the highest frequency. Because they both would be at the same frequency. Even a perfectly square filter could not separate two components that are at the same frequency!

 

[BrownTown]

LOL, so much talk and yet i don't really see where it is leading. As to the origional post: no matter how good of a sample rate you have you are still going to have a distortion between the analog and digital signals (although in the real worls who really cares if its so tiny?). Even if you are able to decompose the signal into a few sinusoidal waves you still have to realise that they are not gonna be some simple formula like 2Sin(3x). They will bbe more like 35.13464583658Sin(.155568335x), and even if you have just 1 wave you still will not be able to calculate it exactly due to data precision problems....

 

[imported_Tick]

Back to the OP, hypothetically a digital wave can be as good as an analogue, remember,

(Lim(as N goes to Infinity) (sumation(as I goes from 1 to N) of (AI/N)(A/N)))=defenite-integral(x)(from 0 to A)

AKA Reimann sum is the same as integral, ergo, there can be a sample set with zero gap distance.

 

[Peter]

You just can't argue those things with people who don't grasp the concept of infinity. See also: 0.99999... = 1.

 

[BrownTown]

yeah, if you theoretically have infinite storage then you could get the same thing. Unfortuantely in he real world thats not really possible. I guess you can technically say that since there are only a finite number of energy states and a finite number of electrons in those energy states you can theoretically define what every single electron is doing in the analog signal and therefore have the signal completely defined.

Maybe its just me, but the whole argunment is stupid, in the real world you will never be able to exactly 100% replicate a signal, but you can get d@mn close. And since there is always tuff like interference and backround noise in the analog signal you are transmitting its not like the analog signal you recieve will even be the same one that was origionally sent.

 

[Navid]

Guys, let's take a step back and think about the original question. What is it exactly that you want to achieve?

Do you intend to capture the entire frequency spectrum from 0Hz to infinity? If that is what you want, you cannot do that with sampling since the samples will never be close enough together. If you have an infinite number of samples, you will be back to the original analog signal! An analog signal is a sampled signal with infinite number of samples.

On the other hand, if you want to capture an audio signal, you need to know that we can only hear a small band of frequencies from about 20Hz to about 25kHz.
http://www.sfu.ca/sca/Manuals/ZAAPf/r/range.html

If you capture a signal in an environment and only store the audio band and then listen to the original analog signal and compare it to the (reconstructed) captured signal, you will not be able to tell the difference even though the captured signal only has a portion of the frequencies that might have been in the original environment. But, our ears could not sense those frequencies in the original signal anyway.

The same goes for video. We can only see a limited band of frequencies. The same is true about radio frequencies. Each radio channel contains a limited band of frequencies.

To sample a band-limited signal and capture the entire information in it, you need to sample at a frequency higher than twice the highest frequency in it. This is not just a theory!

After sampling is done, the samples are often converted to digital format for ease of storage and transmission and immunity to noise. That (conversion to digital) is when the quantization error becomes an issue, which has already been covered in this thread.

Please don't forget to include in your post if you are talking about sampling a band-limited signal (audio, video, radio ....) or the entire frequency spectrum.

All of my posts in this thread were based on the assumption that we were talking about a band-limited signal.

Edit:
There have been several posts that talked about noise in one way or another. Any (active) circuit component like a transistor adds noise. So, the output of the circuit will never be identical to its input. If you look at the specs for an Audigy sound card for example, you will see a "Signal-to-Noise" ratio spec. That tells you how much noise the card adds to the signal. This is not limited to digital. Analog circuits add noise too. As soon as you use a circuit to amplify a signal, you add noise to it. You can build circuits that add less noise than other circuits. The same way that you can build a car that provides more protection to its riders in case of an accident. It costs money and the manufacturers do it and charge you for it.

My point is that the noise that is specific to digital is quantization noise, which I tried to explain in a previous post. But, I did not talk about noise in general since that is not specific to digital and sampling, which I thought was the point of this discussion. But, you are right, it is impossible to reproduce a signal exactly as the original signal (due to noise or distortion) whether the signal is sampled or not.

 

[imported_Tick]

Infinite sampling does not equal analogue. Same way a reimann sum is not an integral, their just equivelent.

 

[Navid]

Originally posted by: Tick
Infinite sampling does not equal analogue. Same way a reimann sum is not an integral, their just equivelent.

They are not "equal" but they are just "equivalent"!

Could you explain what you mean exactly?

Anyway, my point was that there is no point in sampling if you are going to sample at every point in time. You might as well have an analog system. There will be no advantage in sampling with respect to the original non-sampled system if you sample at every possible point in time.

 

[Peter]

Originally posted by: Peter
You just can't argue those things with people who don't grasp the concept of infinity. See also: 0.99999... = 1.

Originally posted by: Tick
Infinite sampling does not equal analogue. Same way a reimann sum is not an integral, their just equivelent.

Originally posted by: Navid
the samples will never be close enough together

See what I mean? :roll:

 

[BrownTown]

I think it is you who is missing the point. Yes, if you sample an analog signal an infinite number of time per second then you can perfectly replicate it. The problem is that "infinity" is just a silly word people use to refer to really big numbers. In reality no such thing exists, it just allows you to simplify things alot if you say you can do something infinite number of times because then you can just use the integral, or drop out some terms that might approach zero as the sample size reached infinity. But in reality you cannot ever reach such a point. You can just get so d@mn cloase that you might as well jsut assume you are there. So yeah, .99999... might equal 1, but unfortunately .999999... don't exist in the real world.

 

[imported_Tick]

Originally posted by: Peter

Originally posted by: Peter
You just can't argue those things with people who don't grasp the concept of infinity. See also: 0.99999... = 1.

Originally posted by: Tick
Infinite sampling does not equal analogue. Same way a reimann sum is not an integral, their just equivelent.

Originally posted by: Navid
the samples will never be close enough together

See what I mean? :roll:

Why is my comment included in this group? My comment implies a direct and thourough understanding of infinity.

 

[Mday]

Originally posted by: Navid

Originally posted by: Tick
Infinite sampling does not equal analogue. Same way a reimann sum is not an integral, their just equivelent.

They are not "equal" but they are just "equivalent"!

Could you explain what you mean exactly?

Anyway, my point was that there is no point in sampling if you are going to sample at every point in time. You might as well have an analog system. There will be no advantage in sampling with respect to the original non-sampled system if you sample at every possible point in time.

infinite sampling does not make sense. continuous sampling is possible. analogue signals attain any value. sampling may introduce rounding or digitization errors.

it is true that a continuously sampled signal may not necessarily be the same as the original signal.

 

[Peter]

In.fi.nite. If you have both an infinite sampling frequency AND infinite sampling precision, you get an EXACT replication of the original signal.

 

[Peter]

OP originally asked:

Given an analog waveform, and an infinite number of sampling points, ...

and why I included you, Tick, is because you keep saying things like,

The problem is that "infinity" is just a silly word people use to refer to really big numbers.

which is exactly what I was about when I said you can't argue these things with people who don't grasp the concept of infinity (in math). Thank you for proving my point.

 

[Navid]

Originally posted by: Mday
analogue signals attain any value. sampling may introduce rounding or digitization errors.

No it may not! It is not sampling that may introduce rounding or digitization error. You can sample and still keep it analog. You can store the sampled voltages stored in capacitors. The rounding error occurs because of converting the samples to digital. That is called quantization error.

 

[Navid]

The whole point behind representing a continuous signal with discrete samples is to allow the system to do something else in between the samples. For example, in a telecommunications system, multiple conversations can be transmitted on the same channel by interleaving the samples from those multiple conversations. This increases effective through-put. That is why increasing the sampling frequency towards infinity is funny. It defeats the purpose of sampling by keeping the system busy all the time!

There are multiple sources of error (noise) that people keep mixing up here.

There is going to be error (noise) no matter what (analog or digital, sampled or not sampled). There is going to be timing error (jitter) as soon as you sample the signal. There is going to be quantization error as soon as you digitize the samples. If the samples are not close enough (the sampling frequency is too low), there will be aliasing. The last one is the only one that can be improved by increasing the sampling frequency.

One question remains. What is the nature of the signal the OP wants to sample? Is it band-limited or not? If it is not band-limited, forget about sampling. It is not going to work. Of course I am talking about practical equipment doing it (not mathematical equations). And what is the point of sampling if the original signal contains frequencies that approach infinity?
If the signal is band-limited, there is no point in all this talk about infinity (LOL).

If the signal is band-limited and you keep increasing the sampling frequency, the quality of the signal improves up to a point (past twice the bandwidth). After that, you are just adding redundant information to your samples because the original signal does not change any faster (band-limited).

 

[sdifox]

Originally posted by: BigT383
I'm having an argument with this guy at work...

This is one of those "never possible in reality" things...

Given an analog waveform, and an infinite number of sampling points, can't you choose your sampling points so that there are no gaps between them- IE, produce a sampled function that, given any real number, returns sampled data without any interpolation?

My conjecture is that you can, and at that point your perfectly sampled function is the same as the original waveform. He believes that there would still be gaps between your sample points. I said that yes, you could have an infinite number of sample points and choose to still have gaps (only sample for rational numbers, for instance), but you didn't need to.

This all started from an argument we were having about whether it was possible for digital representations to ever be as good as analog. He said it was not possible, I said it was.

Correct me if I'm wrong about this: but in the real world there comes a point when, no matter how good your detector of the original waveform is there is still a margin of error (uncertainty principle), and that as soon as the distance between every two adjacent sample points becomes less than that margin of error, the digital representation becomes just as good as the analog.

Sampling is not your problem, aliasing is. The basic difference between analogue and digital will prevent you from attaining that. If you are sampling analog and keep it analogue, then you can. With a digital representation you can approach the analogue signal, but not attain it. Purely mathematical sense, once you move into the real world, the errors (not to mention lack of resolution) from your electronic devices will render the difference moot.

 

[Navid]

This link has a lot of good information relevant to this thread. Please read all of the sections (specially aliasing, anti-aliasing and quantization).

http://www.bores.com/courses/intro/basics/1_whatis.htm