I was just there in July. It was awesome.
1) In the Upper Geyser Basin, there is more than just Old Faithful. You can walk around the platform at Old Faithful, or you can hike the 1/2 mile up to Observation Point and watch it from on-high. After that hike, a short hike from there brings you to Solitary Geyser. There are also other geysers and springs that you can see there if you walk around the boardwalks. You can also take a short drive over to Grand Prismatic - that one is pretty spectacular.
On top of it all, the park is very big. There are lots of trails and the NPS has a fair amount of guides listed on their site for different areas of the park. That's what we used for planning our days out.
1b) No swimming in the hot springs. There are some rivers you can swim in. If you want hot spring swimming, there is only The Boiling River near the border of Montana and Wyoming. The Boiling River swimming area is about a 1/2 mile from the parking area - a short hike you can do in swim gear. Here, the Boiling River dumps into the Gardner River and you can hang out, like a hot tub.
2) We didn't stay in the park because the hotels fill up real fast. Instead, we split our trip into 3 segments and stayed in hotels in the following places: 2 nights in West Yellowstone, MT to explore the middle/west area of the park, 2 nights in Gardner, MT for Mammoth Hot Springs, and 3 nights in Jackson, WY for Grand Teton and the northern area of Yellowstone (which we hit in the morning of our travel-south day).
3) We went in early July - no snow on higher trails at this point and the weather was fantastic (60s to low 70s). I've read that May and June will often still have snow hanging around, especially in mountainous trails. If you do go, plan for early days. If you get moving early in the day, you'll hit less traffic and get more in (we tried to be in the park before 7AM every day). Mountain weather tends to have pop-up thunderstorms in the afternoons, which can also be dangerous if you're hiking places like Mt. Washburn, where you'll be exposed above the treeline.
4) Besides Old Faithful; Grand Prismatic, Norris Geyser Basin, Mammoth Hot Springs, Tower Fall, The Grand Canyon of Yellowstone, Yellowstone Lake, swimming in rivers, hiking (trails of all skill levels can be found throughout the park), nature walks, scenic drives (lots of pullouts and some scenic roads off the main loop), Gibbon Falls, Uncle Tom's Trail, The Upper & Lower Falls of the Yellowstone... You can also try to pair a trip with Grand Teton National Park, since they're in the same general area.
Here's some photos I took on my trip this year:
https://flic.kr/s/aHskEtz9tR
And here's Grand Teton if you're interested:
https://flic.kr/s/aHskDbUgG5
Edit:
Also be prepared to drive a long way. We flew into SLC and rented a car. It's about 4.5 hours to West Yellowstone from SLC. And the park is also a big place. It was often 30-45+ minutes from the hotels to what we wanted to do.