Sorry, this doesn't make sense and just indicates you're another person that never learned how to use the start menu. It's akin to asking, "well if the speedometer on your car doesn't take up a lot of space- instead of taking up the entire dash- then why use it at all?"
You're missing the idea here. If your start menu is small (aka: has few shortcuts/programs/etc), that means there isn't much you actually access with it; if it's not there, you can't get to it through there after all. And if that is the case, I think it stands to reason that you would be better served by taking those few important objects and putting there in a more immediately accessible location, such as the desktop or taskbar. You are correct in that I don't see the use for the Start Menu unless you have a lot of items in it, in which case it still requires large amounts of screen real estate and, imo, the Start Screen trades more screen real estate for improved organization.
The key here is in your opinion of course. Sorry, but no, having a desktop cluttered with shortcuts in place of a start menu is NOT more efficient than a logical start menu. It should be sort of a clue to you that 'your opinion' and your preference does not equal any sort of 'law of computing' as to what's a universally more efficient way to do something. That you prefer desktop shortcuts is fine, but it makes little sense to merge that with some false proclamation about the start menu.
In WinXP, opening a program through the start menu requires 5-10 mouse actions if you count each 'movement' and 'click'. In Win7, it's a little bit more organized and probably 4-8 is more accurate. An item on your desktop or taskbar is accessed in as little as 2-3, depends if you count a doubleclick as one or two; the start screen would probably rate about a 3-4 depending if you use the corner or the Windows key.
If you prefer to use your desktop as a "canvas", that's your opinion of course, but for most cases you're introducing inefficiency (at least in terms of mouse usage) by
not using it as actual workspace for shortcuts. Clutter is only a matter of putting a minute or two into organizing it.
In my opinion, the start menu is only useful/needed to access programs I use, say, less than once per month: a PDF editor, alternate browsers, malware scans, odds and ends 'utility' programs. Anything used more regularly than that I'm going to give a more immediately accessible route.