When doing infrared spectroscopy in a chem lab, the only things we ever care about are water and CO2 screwing up the readings. Nitrogen (80% of our atmosphere) does not absorb infrared light, and neither does oxygen (the other 20% of our atmosphere).
This is part of the reason scientists care so much about CO2 and how it can potentially cause large global temperature changes. If N2 and O2 already absorbed all of the infrared light, adding CO2 to the atmosphere wouldn't do anything and nobody would care. Most of our infrared absorption comes from
water, not nitrogen or oxygen. This is the main reason global warming effects are not seen in places where water is in the air, but huge temperature changes are seen in places that do not have any gaseous water such as deserts and the poles.
Here is a picture of
predicted global warming trends over 100 years:
You'll notice the largest changes are expected in desert regions - north pole, south pole, Africa, and the Australian outback.
Venus would likely be colder than earth if it had a thick atmosphere made entire of nitrogen and oxygen with no CO2 or water.