A good point.
A commentator at Sailer's blog writes:
Portugal is not a "mulatto society." The amount of sub-Saharan admixture is small, and it is not evenly spread throughout the population. Moreover, the populations of the Iberian peninsula likely already had small amounts of sub-Saharan admixture at their peak of power and influence. There have been genetic connections between North Africa and the Iberian peninsula for thousands of years, and the Moorish invasions would have brought additional, though dilute, sub-Saharan genes to Iberia, all before the Age of Discovery.
That's a good point. The low levels of admixture - real and/or artifactual - present in Iberia are most likely the result of (1) gene flow from North Africa over long time periods, and (2) the Moorish invasion, not from "Black slaves imported into Portugal's colonies."
In other words, the admixture was present before and during the Age of Discovery, and was not responsible for the subsequent decline of Spain and Portugal.
The same principle likely applies to Italy and Ancient Rome, particularly with respect to those genetics imported during the Neolithic farmer expansion. Much of the genetic differences separating the sections of Europe (North vs. South, East vs. West) were already in place at the beginning of the Classical Age.
Obviously, there have been changes since then, no doubt. But the naive view of "racial admixture = fallen Empires" has not been empirically proven, and is not always consistent with the actual time-lines.
Empires never last, because the efforts in creating and maintaining them exhaust the founding population in ways that include those other than the biological/genetic.
An analogy: the British Empire did not collapse because of alien immigration and admixture - the opposite is true; the bulk of the immigration, for the most part, occurred after the Empire collapsed for other (military, economic, geopolitical, etc.) reasons..