That's what I meant. Typically, full restore means start from scratch, unless you restore and then restore from a backup...
If it's not too much trouble, you can try something like iFunBox and such to save document, game saves, and so on onto your hard drive manually.
The automatic backup feature of iTunes backs up system configuration files, so if anything gets left over from iOS 4, it'll cause slow-downs, because iOS 5 introduces some new configuration options and deprecate some of the others.
Of course if you jailbreak, it's worse...