Expensive clothes are also likely to use exploitation but you are paying extra for the branding on top.
My daughter works for a high end fashion label and this is definitely not the case, they only use certified fabric and suppliers. the suppliers/manufacturers are regularly checked and audited, but it means that the landed cost of each garment is significantly higher than most High Street brands.