No. Neither Nitrogen (N2) nor Oxygen (O2) are greenhouse gases and they compose most of the atmosphere.
Ozone (O3) is a form of oxygen that is considered a greenhouse gas but its concentration is small.
The following pages summarize quite well the situation: http://cdiac.ornl.gov/pns/curr...
More generally, the argument "it is only a small percentage of the whole atmosphere" is invalid. What is important is not the percentage of the various gases but their amount and their efficiency for trapping heat.
Also, people tend to underestimate the amounts of matter involved when talking about ppm or ppb. In https://en.wikipedia.org/wiki/... we find that "A column of air one square centimeter [cm2] (0.16 sq in) in cross-section ... has a mass of about 1.03 kilograms (2.3 lb)"
So the solar radiation that hits each cm2 of the earth surface has to go through about 1kg of air = 1000g.
The CO2 concentration is 400ppm so the solar radiation passes through 1000g * 400/1000000 = 0.4g of CO2 per cm2
Polycarbonate sheets used in most garden greenhouses has a density of 1.2g/cm3.
If atmospheric CO2 was compressed to that same density to form a hypothetical sheet of solid CO2 then its thickness would be 0.4/1.2 = 0.33 cm = 3.3mm
This is very comparable to the thickness of typical a polycarbonate sheet (3 to 6mm) so saying that 400ppm of CO2 cannot have any noticeable effects seems as stupid as saying that greenhouses are ineffective.