How do things like MD5 and SFV checksums work?

[konichiwa]

Eh?

*note that although this initial post may not be "highly technical," hopefully some of the responses will be.

 

[Mapidus]

MD5 is a provably hard to reverse hash algorithm. I believe it was created by Rivest (The R in RSA). Hashing is the process of taking a bunch of data (bits in this case) and combining them in a deterministic way to generate a smaller set of data. For example a simple and easy to reverse has would be to XOR all the bits in a file together to generate a 1 bit hash (generally called a parity bit). This XOR method is what is used to generate many checksums out there except they XOR sets of bits such as sets of 8 bits to generate an 8 bit checksum. MD5 is just a much more complex algorithm for generating hashes.

 

[CTho9305]

while you're at it, add hamming codes, which can fix any one-bit error and detect any two-bit error in a byte (I think). I dont know how many bits go to the code though.

 

[n0cmonkey]

<< Eh?

*note that although this initial post may not be &quot;highly technical,&quot; hopefully some of the responses will be. >>



I have never heard of SFV but Ill look it up when I get home.

Anyhow, MD5 is a one way hash. Any file or text is put through the md5 algorithm. The output is a 128bit hash of that file. The hash will always be 128bits (I am almost positive its bits...) and always the same as long as the file has not been modified. One of the uses for md5 is that as long as the file has not been changed the md5 hash will always be the same. tripwire uses this in thier product and many projects have md5 hashes of the tarballs so that when it is downloaded you check the hash of the file you downloaded and it should be the same as the md5 on the site... Heh, ok Im rambling. But check out the following link.

CounterPane Crypto-gram

 

[konichiwa]

Lemme see if I understand this. Basically you run a file through the md5 algorithm, and out comes a string of letters, numbers, whatever. Anyone who recieves that file can then run it through the md5 algorithm on their end, and if the output == the output on the other end, the file is still in tact.

Cool.

 

[Mucman]

More than you want to know

The above link is to the rfc for md5. I plan on reading it someday

 

[RedBeard0531]

Is that like the hash that winxp activation uses, meaning that they realy do have no way of knowing what hardware I have.

 

[Mapidus]

<< Lemme see if I understand this. Basically you run a file through the md5 algorithm, and out comes a string of letters, numbers, whatever. Anyone who recieves that file can then run it through the md5 algorithm on their end, and if the output == the output on the other end, the file is still in tact.

Cool. >>



Yeah, for all practical purposes, you can assume the file is intact. Of course there is always the small chance that there were mutations to the file that just happened to generate another file that has the same MD5 output, but chances of this are very, very low.

 

[n0cmonkey]

<<

<< Lemme see if I understand this. Basically you run a file through the md5 algorithm, and out comes a string of letters, numbers, whatever. Anyone who recieves that file can then run it through the md5 algorithm on their end, and if the output == the output on the other end, the file is still in tact.

Cool. >>



Yeah, for all practical purposes, you can assume the file is intact. Of course there is always the small chance that there were mutations to the file that just happened to generate another file that has the same MD5 output, but chances of this are very, very low. >>



From what I have read that is not possible. But I could be wrong

 

[Mapidus]

Of course it is possible to have two different files generate the same MD5 hash. Think about this - the hash is only 128 bits long, so that means at most there are 2^128 possible hashes. There are infinitely countable files, so that means there are collisions in MD5 hashes. The special quality in an MD5 has is that it generates hashes that are well distributed, so that it is hard to reverse the hash (generate another file with the same hash) and likewise hard for a corruption to generate a file with the same hash. So the result is a high probability of noncorruption if the hashes are the same, but this probability is not 100%.

 

[n0cmonkey]

<< Of course it is possible to have two different files generate the same MD5 hash. Think about this - the hash is only 128 bits long, so that means at most there are 2^128 possible hashes. There are infinitely countable files, so that means there are collisions in MD5 hashes. The special quality in an MD5 has is that it generates hashes that are well distributed, so that it is hard to reverse the hash (generate another file with the same hash) and likewise hard for a corruption to generate a file with the same hash. So the result is a high probability of noncorruption if the hashes are the same, but this probability is not 100%. >>



Ok that makes sense. Sorry its the end of the day so my brain aint quite working no more