Never ever ever scrape the bottom of the heatsink with anything metal, or with anything hard enough to mark the surface. You will dig gouges into the surface, and seriously degrade the contact and cooling.
That said, scrape what you can off with a fingernail, then use nail polish remover or acetone or carb cleaner or any other solvent to dissolve what remains. Wipe clean, apply thermal compound, attach heatsink.