Depends on local custom and the agreement with the tenant.
Remember that you don't want electricity/heat shut off in the winter if you're in an area where it freezes. Frozen pipes can do a heck of a lot of damage.
Remember too that some utilities are a lien on the property if not paid-frequently this is water & sewer, but local law matters. For those utilities if you are passing them onto the tenant I'd suggest bill stays in your name, you present it to them and ask for a check when the bill comes.