As a person that's played both I can assure you Rugby is harder. Half the time in football people are barely even moving, or when they do, its for 5 seconds, then they get a 20 second break.. repeat a few times.
In rugby, especially rugby 7's, it is almost non stop running. In 15's it's more like regular football where there can be some dull moments in a ruck or something, but there is never any period in time when I played in college that I stopped moving completely for more than 10-15 seconds.
Don't forget that hitting somebody as hard as you possibily can, sans equipment is also allowed. Additionally, you need to wrap them up, or attempt to in your hit. Aka you can't just do some retarded NFL shoulder leading hit that all the safety's tend to go for now a days.
It combines all the toughness of the NFL, with the endurance and agility of soccer. No fat ass 320lb linemen hulking around. Sorry Vince.