I completely agree we should talk about slavery—it's why I do—but we should not give a pass to any form of slavery. Some Igbo buried their slaves alive to display their wealth.
Slavery was driven by economics, as Eric Williams and Barbara Fields and others have shown. The theory of Ham was only a rationalization for Christian slavers.
Interesting question about why the African part of the Atlantic slave trade was forgotten by African Americans. I suspect it's simply that most slaves came here before 1800, so the reality for later generations was that most slave traders were white.