Digital Sampling

[imported_BigT383]

I'm having an argument with this guy at work...

This is one of those "never possible in reality" things...

Given an analog waveform, and an infinite number of sampling points, can't you choose your sampling points so that there are no gaps between them- IE, produce a sampled function that, given any real number, returns sampled data without any interpolation?

My conjecture is that you can, and at that point your perfectly sampled function is the same as the original waveform. He believes that there would still be gaps between your sample points. I said that yes, you could have an infinite number of sample points and choose to still have gaps (only sample for rational numbers, for instance), but you didn't need to.

This all started from an argument we were having about whether it was possible for digital representations to ever be as good as analog. He said it was not possible, I said it was.

Correct me if I'm wrong about this: but in the real world there comes a point when, no matter how good your detector of the original waveform is there is still a margin of error (uncertainty principle), and that as soon as the distance between every two adjacent sample points becomes less than that margin of error, the digital representation becomes just as good as the analog.

 

[blahblah99]

Originally posted by: BigT383
I'm having an argument with this guy at work...

This is one of those "never possible in reality" things...

Given an analog waveform, and an infinite number of sampling points, can't you choose your sampling points so that there are no gaps between them- IE, produce a sampled function that, given any real number, returns sampled data without any interpolation?

My conjecture is that you can, and at that point your perfectly sampled function is the same as the original waveform. He believes that there would still be gaps between your sample points. I said that yes, you could have an infinite number of sample points and choose to still have gaps (only sample for rational numbers, for instance), but you didn't need to.

This all started from an argument we were having about whether it was possible for digital representations to ever be as good as analog. He said it was not possible, I said it was.

Correct me if I'm wrong about this: but in the real world there comes a point when, no matter how good your detector of the original waveform is there is still a margin of error (uncertainty principle), and that as soon as the distance between every two adjacent sample points becomes less than that margin of error, the digital representation becomes just as good as the analog.

You guys seem like you are arguing from two perspectives... the "perfect" analog world, and the digital world.

In order to recover an analog signal, you just need to sample at the nyquist rate (2x highest frequency content of the signal), assuming that quantization errors is negligible. However, in a perfect analog world, sampling at nyquist rate & interpolating will produce a small error when compared to the perfect analog signal.

In practice and reality, sampling at nyquist rate and interpolating seems to be working pretty damn well.

Take audio ADCs & DACs for example. Some are known to produce less than 0.001% THD & dynamic ranges in the order of -120dB.

Digital representation of an analog signal will never be EXACTLY the same as the analog signal, but will be close enough that the human senses (and sometimes test equipment) can't tell the difference.

 

[hypn0tik]

Originally posted by: blahblah99

Originally posted by: BigT383
I'm having an argument with this guy at work...

This is one of those "never possible in reality" things...

Given an analog waveform, and an infinite number of sampling points, can't you choose your sampling points so that there are no gaps between them- IE, produce a sampled function that, given any real number, returns sampled data without any interpolation?

My conjecture is that you can, and at that point your perfectly sampled function is the same as the original waveform. He believes that there would still be gaps between your sample points. I said that yes, you could have an infinite number of sample points and choose to still have gaps (only sample for rational numbers, for instance), but you didn't need to.

This all started from an argument we were having about whether it was possible for digital representations to ever be as good as analog. He said it was not possible, I said it was.

Correct me if I'm wrong about this: but in the real world there comes a point when, no matter how good your detector of the original waveform is there is still a margin of error (uncertainty principle), and that as soon as the distance between every two adjacent sample points becomes less than that margin of error, the digital representation becomes just as good as the analog.

You guys seem like you are arguing from two perspectives... the "perfect" analog world, and the digital world.

In order to recover an analog signal, you just need to sample at the nyquist rate (2x highest frequency content of the signal), assuming that quantization errors is negligible. However, in a perfect analog world, sampling at nyquist rate & interpolating will produce a small error when compared to the perfect analog signal.

In practice and reality, sampling at nyquist rate and interpolating seems to be working pretty damn well.

Take audio ADCs & DACs for example. Some are known to produce less than 0.001% THD & dynamic ranges in the order of -120dB.

Digital representation of an analog signal will never be EXACTLY the same as the analog signal, but will be close enough that the human senses (and sometimes test equipment) can't tell the difference.

Just step back and think of the definition of sampling for a moment.

x[n] = x(nTs) where n is an integer and Ts is the sampling period.

What you are suggesting is that Ts -> 0 so you have your points infinitely close together giving you back a 'perfect' digitized sample. However, you do NOT get a digitized result. Instead, your resultant signal will also be analog in nature since it is defined for ALL TIME.

As blahblah said, you need to sample at a minimum rate of 2x the highest frequency of the signal in order to recover the original by interpolation after sampling. You can increase this sampling frequency to recover a more accurate signal, but it will never be as 'good' as the analog representation. You will not be able to fill in the gaps perfectly.

 

[eLiu]

So you'd need an uncountably infinite number of points...if you sample at all the irrational values, you'd have an exact replica.

I believe that in some cases, sampling the rational points will work too, but these are not general.

So it depends on your definition of infinity.

Edit: I'm going for the purely theoretical & utterly useless standpoint...lol

 

[Gannon]

I think the argument assumes too much about the human ears ability to percieve sound, I wouldn't be surprised if human ears distort even the original auditory analog signals due to different ear shapes, biology, different sizes and makeups of ears, etc. The argument that analogue is better then digital is pure nonsense, since a person can only sample (that is sense and interpret) sound over limited period of time, we think they are 'continuous' but the brain has to do processing and cutting off chunks of data to interpret the sound so I doubt even our 'analogue' capturing of sound signals by our ears are close or even mathematically near to the original signal. You can see this in how there are people with perfect pitch and those without the ability, i.e. their brains have a lot to do with what they 'hear'. And expectations always color subjective perceptions.

 

[Matthias99]

Originally posted by: eLiu
So you'd need an uncountably infinite number of points...if you sample at all the irrational values, you'd have an exact replica.

I believe that in some cases, sampling the rational points will work too, but these are not general.

So it depends on your definition of infinity.

Basically, the margin of error is a function of the maximum frequency of the analog signal and the sampling rate. As the sampling rate goes up, the error goes down, although technically it will never be zero unless you have an infinite sampling rate. In practice, at more than two or three times the maximum frequency of the signal, you're close enough that nobody is going to complain.

There's also quantization error -- since each sample uses a finite (though not necessarily fixed) number of bits, and an analog signal can (theoretically) have an infinite number of different amplitude values, you cannot 'perfectly' represent an analog sample with a finite number of bits.

If you truly have an 'infinite' number of sampling points, then (ignoring quantization error), you will have a perfect reproduction of the original signal.

 

[djhuber82]

Given an analog waveform, and an infinite number of sampling points, can't you choose your sampling points so that there are no gaps between them- IE, produce a sampled function that, given any real number, returns sampled data without any interpolation?

Uhh, NO. Absolutely not. How are you going to sample at every moment in time? And why would you want to? As long as your sample rate is greater than twice the highest frequency component in the analog signal, you lose NO information from sampling. Loss of information happens as a result of the analog to digital conversion, i.e. when you quantize the sample to N-bit accuracy.

As far as digital being as good as analog, yes, it's possible. If you sample faster than the Nyquist rate and your quantization error is much smaller than the noise in the analog signal, then you lose no information in the analog to digital conversion. Note that in this case there is still some difference between the analog and digital signals, but that difference is much smaller than the noise that is present in both.

Correct me if I'm wrong about this: but in the real world there comes a point when, no matter how good your detector of the original waveform is there is still a margin of error (uncertainty principle), and that as soon as the distance between every two adjacent sample points becomes less than that margin of error, the digital representation becomes just as good as the analog.

Again, you seem to only be considering interpolation error, which absolutely does not matter if you've sampled above the Nyquist rate. What does matter is the quality of your samples.

 

[Howard]

Originally posted by: Matthias99

Originally posted by: eLiu
So you'd need an uncountably infinite number of points...if you sample at all the irrational values, you'd have an exact replica.

I believe that in some cases, sampling the rational points will work too, but these are not general.

So it depends on your definition of infinity.

In practice, at more than two or three times the maximum frequency of the signal, you're close enough that nobody is going to complain.

http://sound.westhost.com/cd-sacd-dvda.htm

 

[Matthias99]

Originally posted by: Howard

Originally posted by: Matthias99

Originally posted by: eLiu
So you'd need an uncountably infinite number of points...if you sample at all the irrational values, you'd have an exact replica.

I believe that in some cases, sampling the rational points will work too, but these are not general.

So it depends on your definition of infinity.

In practice, at more than two or three times the maximum frequency of the signal, you're close enough that nobody is going to complain.

http://sound.westhost.com/cd-sacd-dvda.htm

It sounds like what they are complaining about there is actually quantization error from the DACs (and/or shortcomings of the SACD format), not really sampling rate problems.

 

[0roo0roo]

there can be infinitely small gaps no matter how many samples you can take. so no, it won't be the same. its a load of bs but thats the answer

 

[Loki726]

There seem to be some misconceptions here.

We are talking about REAL signals, not mathmetatical idealizations.

IE

There are NO perfect square waves, NO delta functions, NO signals with sharp edges (discontinuous derivatives). These functions are mathematical idealizations that make calculations easier but do NOT exist in the real world.

As long as we consider real signals, a consequence of the Fourier Theorem is that any signal can be expressed as a FINITE sum of complex exponential terms (sine waves at different frequencies).

Now, as stated before, the Nyquist?Shannon sampling theorem says that as long as the sampling frequency is at least twice as big as any frequency in a signal's Fourier Series, it can be determined exactly.

So any real signal can be decomposed into a Fourier Series, and as long as we sample at twice the highest frequency, each compenent of the Fourier Series can be determined exactly.

The problem that a lot of posts above seem to be having with this topic is that they think that because the signal is continuous, the only way to get all of the information contained in it is to sample every point in it, ie have dt->0. You have to convince yourself that this isn't the case.

Take an example of a simple sine wave with frequency 1hz. Now say we sample at 2hz, obtaining two delta functions one at t=.5s and one at t=1s. Both have magnitude zero. To reconstruct the signal, we choose a sine wave with the minimum valid period to intersect both points, which is sin(2*pi*w*t+w*ts) where ts is zero and w is one.

See how I was able to reconstruct a (potentially) infinitely long sinusoidal signal with just two points. Fourier's thoerem says that any signal can be expressed as a sum of sinsoidal signals. So I can repeat the process for any real signal by finding each sinusoidal compenent and summing them together.

The only way that this breaks down is with signals whose Fourier series is infinite (then you would need an infinite number of samples). These are functions with discontinous derivatives. However, boundary conditions imposed on all waves (this includes E&M, quantum matter waves, sound, particle waves) make sure that all real (ie exist in the real world) waves can be decomposed into finite Fourier Series.

 

[smack Down]

I have to wondered did people even read the OP?

The answer is yes. If a wave is sampled for all (t) then the digitial sample will be equal to the analog wave when reproduced for all frequenies.

 

[Navid]

Originally posted by: Loki726
Now, as stated before, the Nyquist?Shannon sampling theorem says that as long as the sampling frequency is at least twice as big as any frequency in a signal's Fourier Series, it can be determined exactly.

Very well described.

Except, I believe that the sampling frequency has to be more than twice the highest frequency component present in the original signal. If the sampling frequency is exactly equal to twice a frequency component, that component will be unrecoverable.

Edit:
In your example, you have samples that are at the zero-crossings. These samples have no information that we can use to reconstruct the amplitude of the original sine-wave. Any 1Hz signal with any amplitude will still cross your sample points. You say that you choose sin(2*pi*f*t). But, 2*sin(2*pi*f*t) also crosses the sampling points. So does 3*sin(2*pi*f*t) etc. That is a failure in capturing the signal by sampling! That is why the sampling frequency must be higher than twice the highest frequency component in the sampled signal.

 

[Navid]

Originally posted by: BigT383
I'm having an argument with this guy at work...

This is one of those "never possible in reality" things...

Given an analog waveform, and an infinite number of sampling points, can't you choose your sampling points so that there are no gaps between them- IE, produce a sampled function that, given any real number, returns sampled data without any interpolation?

My conjecture is that you can, and at that point your perfectly sampled function is the same as the original waveform. He believes that there would still be gaps between your sample points. I said that yes, you could have an infinite number of sample points and choose to still have gaps (only sample for rational numbers, for instance), but you didn't need to.

This all started from an argument we were having about whether it was possible for digital representations to ever be as good as analog. He said it was not possible, I said it was.

Correct me if I'm wrong about this: but in the real world there comes a point when, no matter how good your detector of the original waveform is there is still a margin of error (uncertainty principle), and that as soon as the distance between every two adjacent sample points becomes less than that margin of error, the digital representation becomes just as good as the analog.

Any real signal has a limited bandwidth. If you know the bandwidth of the analog signal, the highest sampling frequency you need is a little bit more than twice that bandwidth. There is no need to sample at any higher frequency and you will gain nothing by doing so.

A source of error would be quantization noise, which depends on the number of bits your Analog to Digital Converter (ADC) has and how accurate it is. There is no perfect ADC. There is no ADC with infinite number of bits. That means there is no ADC with zero quantization noise. So, your friend may be right but not because of the sampling frequency.

At the end, it comes down to accuracy and tolerance. If the Nyquist rate is satisfied and the ADC is made accurate enough, there is no reason for the digital representation to be any inferior to the analog. Don't forget that the analog signal has noise too and in fact is more susceptible to interference.

Edit:
An example for quantization error would be using a single-bit digital word for representing 0.5. A single bit can be either 0 or 1. But it cannot be 0.5. So, we can have an error as big as 0.5.

Now, you can consider a 2-bit digital word, which can be 0, 1, 2 or 3. If we normalize that, the word can represent 0, 0.333, 0.666 or 1. Now, representing 0.5 can have an error as big as 0.166. So, we reduced the quantization error by increasing the number of bits.

Let's do it one more time. Let's try a 3-bit representation. Such a digital word can be 0, 1, 2, 3, 4, 5, 6 or 7. If we normalize that to 1 again, we can have 0, 0.143, 0.286, 0.429, 0.571, 0.714, 0.857 or 1. Now representing 0.5 can have an error as large as 0.071.

You see that the quantization error (noise) keeps going down as you increase the number of bits. But, it will never reach zero. However, the question is when it will become negligible compared to noise on the original analog signal. That's where you want to be.

 

[Bobthelost]

The answer depends on time as a variable, discrete or continuous. If you did sample at every possible increment (assuming time has a fundamental incremental value, it probably doesnt) then you'd be right, however if it's a continuous variable then you're screwed.

In the real world i'd say he's wrong. You'll get errors no matter what you're doing and it's possible (just expensive) to reduce those errors to imperceptible levels even for audio geeks with digital systems.
Note: Anyone who thinks that you'd be able to tell the difference between a signal sampled at 10Ghz as oposed to 20Ghz should be culled from the species.

 

[Navid]

There are thousands of web sites making references to the Nyquist theorem. This is one of them. It is related to this thread.

One thing that may be confusing about this sampling fact (Nyquist rate) is if you think that the samples will be connected together with straight lines to reconstruct the analog signal. That is not the case. There is a reference to that in that link.

Also, to understand the Nyquist rate, you need to at least have an idea about Fourier analysis. It basically says that any signal can be broken up into a series of sin and cosin waves. No matter what the signal looks like, it can be represented with the sum of a series of sin and cosin waves. Again, there are thousands of web sites on Fourier analysis. This is one of them.

 

[Navid]

Originally posted by: Bobthelost
Note: Anyone who thinks that you'd be able to tell the difference between a signal sampled at 10Ghz as oposed to 20Ghz should be culled from the species.

Harry thinks so!

 

[Bobthelost]

Nyquist's theorem is wonderful, but rather unhelpful (and misleading) at times. While you do only need 2x the max frequency to sample a signal you get amplitude problems (higher frequencies get attenuated and a lovely little AM effect that kicks in when you're sampling at slightly higher than 2x).

I dislike the way people tend to assume that as long as you've got 2x the frequency you'll get a decent signal, which is utterly wrong.

 

[Navid]

Originally posted by: Bobthelost
I dislike the way people tend to assume that as long as you've got 2x the frequency you'll get a decent signal, which is utterly wrong.

I agree it is utterly wrong. You need more than 2x the frequency as Nyquist says. If you only have 2x, you do not have enough to reconstruct the signal.

Read the theorem!

 

[Navid]

Originally posted by: Bobthelost
While you do only need 2x the max frequency to sample a signal you get amplitude problems (higher frequencies get attenuated ).

You are missing the point! The theorem says that your sampling frequency must be higher than twice the highest frequency present in the original signal. If you satisfy that, there will be no higher frequency.

 

[Bobthelost]

The higher frequency sine waves get attenuated more than the lower frequency sine waves. Look at the diagram provided at one of the sites you linked to: http://members.aol.com/ajaynejr/nyquist.htm. Look at the difference between the left hand diagram and the one on the right, attenuation. The lower the freq of the signal the more samples you get per cycle, and thus the greater the chance of sampling at or near the peak of the wave. So basically you're more likely to get attenuation at high frequencies than at low ones.

So even if you were to sample at 6khz a 3khz signal is going to be pretty crap and the 3khz components will be attenuated relative to the lower freq components, but it will still be slightly better than a signal with components at 2.99khz which will be all over the place because of the way the sampling will go in and out of phase. (Think of a sine wave carrier at X hz and then a singal sine wave at X/5 hz, that sort of effect. *Shudder*)

 

[Loki726]

 

[Bobthelost]

Edit: Don't delete it! Crossposting happens and some/all the questions were good ones!

I hate it when people ask for proofs, but such is life...

Ok, example 1 you have a sine wave, it is a nice sine wave, all the curvy bits in the right places and it crosses the x axis exactly twice per cycle. As i said, it's a good sine wave. Now you decide to sample aforementioned wave, so you take it's frequency (Y hz) and then double it to furfill Nyquist's theorem (2Y hz). However there's a problem, you're sampling at 180 degrees and 360 degrees, or in other words at the zero crossing points.

Your signal has just made like elvis and left the building.

Of course this is slightly silly, you'd have to sample the signal at exactly the right phase to zero it all out, (on the flip side unless you're sampling at the exact right place you'll never get the max amplitude of the signal either. So your signal will be attenuated)

Remember our lovely little sine wave? Well now move the sampling points a tiny bit earlier, it's now sampling just before the wave crosses the axis (say 179 degrees and 359). The signal is back, but attenuated to near death. If you're just listening for a pure sine wave then that's ok, but if that sine wave is part of your latest britney collection then the higher frequency notes are going to be too quiet. So they will get lost compared to the lower freq stuff.

Ok, so at nyquist's freq we can lose the signal completely, or it can be attenuated so badly as to ruin the music, (remembering my example this may be a goal in itself). What happens if we increase the sampling frequency a tiny bit? Now the first sample is taken on a crossing point, the second one just next to a crossing point, the third slightly further away and so on. In other words you'll get the right frequency output, but the amplitude will be increasing as the sample points get closer to the peaks of the waves, and then smaller as they move further away. So britney is warbling a nice little solo bit, but the higher tones are even more unstable than a drunk on a tightrope.

Or what i've refered to as the AM bit.

Now if you want mathematical proof for all of this then i'm going to sit in the corner and sulk.

 

[smack Down]

Originally posted by: Navid

Originally posted by: Bobthelost
I dislike the way people tend to assume that as long as you've got 2x the frequency you'll get a decent signal, which is utterly wrong.

I agree it is utterly wrong. You need more than 2x the frequency as Nyquist says. If you only have 2x, you do not have enough to reconstruct the signal.

Read the theorem!

Sampling is only part of the equation. To go from the analog singal from the digital one and not get errors the conversion is required to use every point in in the sample for all of time. In theory you can get the analog signal back from a signal sampled at the Nyquist rate but it can't really be done.

 

[smack Down]

Originally posted by: Bobthelost
Edit: Don't delete it! Crossposting happens and some/all the questions were good ones!

I hate it when people ask for proofs, but such is life...

Ok, example 1 you have a sine wave, it is a nice sine wave, all the curvy bits in the right places and it crosses the x axis exactly twice per cycle. As i said, it's a good sine wave. Now you decide to sample aforementioned wave, so you take it's frequency (Y hz) and then double it to furfill Nyquist's theorem (2Y hz). However there's a problem, you're sampling at 180 degrees and 360 degrees, or in other words at the zero crossing points.

Your signal has just made like elvis and left the building.

Of course this is slightly silly, you'd have to sample the signal at exactly the right phase to zero it all out, (on the flip side unless you're sampling at the exact right place you'll never get the max amplitude of the signal either. So your signal will be attenuated)

Remember our lovely little sine wave? Well now move the sampling points a tiny bit earlier, it's now sampling just before the wave crosses the axis (say 179 degrees and 359). The signal is back, but attenuated to near death. If you're just listening for a pure sine wave then that's ok, but if that sine wave is part of your latest britney collection then the higher frequency notes are going to be too quiet. So they will get lost compared to the lower freq stuff.

Ok, so at nyquist's freq we can lose the signal completely, or it can be attenuated so badly as to ruin the music, (remembering my example this may be a goal in itself). What happens if we increase the sampling frequency a tiny bit? Now the first sample is taken on a crossing point, the second one just next to a crossing point, the third slightly further away and so on. In other words you'll get the right frequency output, but the amplitude will be increasing as the sample points get closer to the peaks of the waves, and then smaller as they move further away. So britney is warbling a nice little solo bit, but the higher tones are even more unstable than a drunk on a tightrope.

Or what i've refered to as the AM bit.

Now if you want mathematical proof for all of this then i'm going to sit in the corner and sulk.

Besides the sampling at zero problem sampling a frequency at 2x its frequency will result in a sampled singal that has twice the amplitude as the orginial.