Will buying fair trade products improve the working standards of 3rd world labourers?

if everyone were to purchase fair trade certified products, wouldn't those that dont adhere to fair trade standards be forced to improve the wages of their workers in the 3rd world because otherwise nobody would buy their products (uming that people buy products based on ethical considerations. This is merely hypothetical as I realize many people consider cheap goods above the conditions that created the cheap good)? Businesses would still purchase labour overseas because it is cheaper, even when you pay them better, as it is cheaper to live in Bangladesh than it is to live in the US, thus the 3rd world would still get jobs, but they would be paid better. I realize that the increase in wage would be ped on to the consumer, but its worth it if it means workers in the 3rd world wont have to starve everyday.