Unlike today's Peruvian, the Spanish of the time did know the difference between an Inca and a Native American, and the Incas were not very populous to begin with, making the Spanish records believable.

Why should I believe Garcilaso de la Vega over so many peoples claims?  First, because Garcilaso de la Vega had nothing to gain in stating his mother's people were not Native Americans; regardless of race, they were already at the pinicle of Andean Society, whereas the Native claimants of today have much glory to gain from such claims.  Second, because the facts supporting Garcilaso de la Vega are too numerous to ignore.

How does a warlike native tribe, surrounded by similar warlike tribes and simple hunter gatherers, suddenly transform itself into a full blown Theocracy with a revered preisthood, advanced civil engineering, advanced agriculture, and advanced metallurgy,- when all of these advancements take thousands of years to develop?  It doesn't.   It is taught by a people who already know how to build roads and other massive structures, who know how to store food in graineries, and how to build irrigated terraced fields on mountainous slopes.  Among many other things, it is the terraced fields, knowledge of Astronomy, and priestly worship of the sun, that points to Chinese teachers.