If you want to think about colonization- and we all need to think about colonization- the Berlin Conference is a good place to start. Here's a wikipedia link about the Scramble for Africa. Don't forget to read between the lines and take some time looking at that map.
Have you heard of Yinka Shonibare? I think you should! He is a contemporary British-Nigerian artist whose works turn a lens on colonization. Here is a little bit on his piece (pictured above) "Scramble for Africa". He is important in this conversation because he highlights the connection between fabric manufacturing and colonization, which has an especially complicated history in Nigeria.
The fabric used in Shonibare's piece above is Dutch wax resist fabric, printed in traditionally African patterns. Dutch wax resist was a technique developed by the Dutch in the 1800's. The process was adapted from Javanese fabric production, when the Dutch were in Java fighting in the colonial wars (read that as: when the Dutch were colonizing Java through war). The Dutch took this production to Indonesia where they hoped to take over the fabric manufacturing industry with their "new" method. The fabric did not take off in Indonesia, where batik remained the more popular wax-resist technique, but when the Dutch inserted themselves into West Africa Dutch wax resist became enveloped in the culture there. In time it took on cultural significance separate from the colonizers and over the past century it has become African. However, it wasn't until the 1960's that the fabric was actually produced in Africa. Before that the Dutch succeeded in creating a cultural dependency between West Africa and European manufacturing.
So- the Dutch took it from Java, tried to use it to disrupt the Indonesian market, then brought it to West Africa where became a part of the culture. It is, in fact, the perfect example of the disruptive impact of colonization on indigenous cultures, set in motion by the Berlin Conference. Look again at the map of Africa split up into European land holdings. Imagine the cultures of each of those areas being appropriated by the Europeans and then sprinkled throughout their other holdings. Imagine too- European textiles being interwoven (maybe even literally) with traditional fabric production across colonized areas.
As you question the origins of your textiles (because it's not just clothes- look to your upholstery, towels, bedding and rugs as well) don't just ask who made them. Ask too where the designs originated and how they got there and out of what circumstances.