Aluminum keeps well outside. It's good for the lake cabin you visit a couple times per year. They're noisy, they stick to rocks, and are frying pans in the summer.
Plastic is medium weight, and cheaper versions have too much flex. Sun can damage them. They're cheap, durable, and very slippery on rocks.
Fiberglass is heavy, and has medium slippage on rocks. Not abrasion tolerant, but easy to repair, or even to build from scratch if you wanted a project. Shape is more customizable. Price is midscale.
Kevlar is very light, but susceptible to damage, including the sun. Not bad to repair, but unless you're skilled, you'll ruin the good looks. Very expensive.
For a general recommendation, I'd go with plastic. It can be used in more environments, and doesn't cost much if you aren't sure you'll be a paddler. There aren't many deals in boats. You really get what you pay for. Buy a cheap plastic boat, and take it from there.