I don't litter, for the most part, but I don't see any real harm in throwing an apple core or banana peel into the woods on the side of the road somewhere. (But not on a showroom lawn or golf course.)
I also don't see harm in pulling small leaves or twigs (or ice or snow) and leaving them on the side of your car.
I also pour the remainder (1" or less) of a coffee or soda out onto the pavement in a parking lot, sometimes. As long as there's nothing solid in it, except for ice, which of course, melts in due time.
Those are my excepts to the rule of "do not litter", as I feel that they do no real harm.
Now, throwing trash, McD's wrappers, soda bottles, plastic bags, non-fully-bio-degradable food items, etc. - THAT would be a serious problem if everyone did that. So I don't. Basically, this is "Golden Rule" stuff, as applied to trash.