Now, let's understand something here, "people who are telling you it offends them the same" perhaps don't have a frame of reference of being called the N word, do they? Am I missing something there?
Yes, they have similar definitions, but the facts as to why its not a double standard:
The N word is bound in history as a word bestowed on slaves and black people by the majority of the time, white people. Slavery, as we remember, was when black people were essentially stolen from their country, families were destroyed, you worked for no money, etc..
Later, after slavery, the word stuck around and still has unfortunately. You've probably noticed that black people have embraced the word amongst themselves in an effort to kill the word essentially, kill the historical significance of it. Nevertheless though, white people have never attempted to make the word mean less then it does or remove the conotation, for white people saying it to black people its straight derogatory. In my opinion its basically a word used to talk down, to spur back this master - slavery relationship implying that I'm better because of my skin color.
Cracker on the other hand; an attempt at a retort to the N word. No baggage I can outline anywhere near as clear.
Thats the difference. Way more weight in my opinion.