Privilege was originally used to describe the ability of the rich to do things that the rest of us could not. Its literal meaning is "private law"—the rich could buy things the rest of us could not, including the law. An example would be the OJ Simpson murder case. Poor people of any color would have been convicted in a day or two with that kind of evidence. Rich people buy the very best lawyers and either are found not guilty or get a sweetheart deal.

When Du Bois wrote about white skin privilege, it made sense to use the word because white people had legal privileges that people of color did not. That ended with Jim Crow.