If you've never rode a scooter/moped before and think you can just ride around in the cities like taipei: You WILL crash and you WILL get hurt. The scooters and cars have a bitter rivalry and all the drivers are very aggressive in the cities. It's not as easy as it looks and you do need to present a license to rent.
On top of that: DON'T DRIVE. You'll go insane trying to avoid the mopeds.
Like others have mentioned, they have big banks (BOA, Chase, Citibank) so stick to using cash. None of the night markets stalls take credit card, CASH ONLY. Unless you go to one of the shops, and even then it'll be hard to get them to take cards. Best if you bring some cash and convert
Taipei central station has some ATMs that will allow you to take out cash allowances on your credit card. Most other ones are for local banks only and they don't work with your credit card.
For 28 days in Taipei: I'd recommend going to the following places.
-Hualien: Hualien city/night market has GREAT local cuisine you won't find in Taipei, and you can go to taroko national park. Unforunately there were a lot of rock slides last time I went so parts were closed, but the views are stunning. Tip: Rent a taxi driver instead of going along with a tour. You might be hard pressed to find one that speaks english but they let you look at the places at your own leisure instead of "ok 30 minutes here" and the price is roughly the same.
If the weather is a bit warmer you can go whale watching or snorkling in Hualien. There's also something called "river tracing" where you can follow one of the rivers up stream, one of the more popular places being the Golden Grotto.
These guys did something similar, but seems like without a guide.
http://taiwan-photography-blog.com/2009/11/10/the-golden-grotto/
The hike is usually led by one of the aboriginal tribe members as they're more familiar with the area. It involves lots of walking in a wet suit, crossing rivers and climbing over rocks and cliffs. It's almost like mountain hiking but right alongside the river and an absolute blast. On the way down you simply float down in the river. Bring a speedo, your swimming trunks won't fit in a wet suit.
This looks more like what I did. Warning: LOTS of stamina and climbing/jumping over rocks involved!
http://www.rivertracing.com/rivertracing.htm
All of that in hualien will take you about 3 days, 4 if you want to do a bit more looking around. You can stretch it to 5 depending on how much you want to see.
Taipei has a lot of stuff to do in general and others have covered that so I won't go into details. Shi-Lin night market is the largest one and right by a MTA station exit, and taipei 101 is also a must-see. Danshui is fun as well.
If you're into aboriginal stuff and traditional taiwanese culture, that's more central taiwan (taichung) stuff.
Kenting (southernmost part of taiwan) has a national park that is gorgeous in the springtime and summer. Huge range to hike/walk through and has a lot of good sights. Great seafood there as well, and the snorkling/scuba diving is nice when its not the hurricane season. Their night market is a bit small and leaves something to be desired though. Almost all the places you'll be staying in are hostels, but they're all super nice.
Need to get on a meeting but I can post what I did last year later today.