The TUC says labour laws need "beefing up" to protect the rights of workers employed by outsourcing firms.
The union body says such workers cannot challenge the "parent" company over minimum wage or holiday pay abuses.
The TUC has called on the government to give subcontracted workers the right to challenge the end employer.
The Department for Business said plans had been set out to ensure employees, including agency workers, "benefit from enhanced rights and protections".
The TUC estimates that five million UK workers cannot enforce their basic rights with their "parent company". General Secretary Frances O'Grady said labour enforcement laws "urgently need beefing up" to deal with the problem.
The TUC said its research had found 3.3 million workers were employed through outsourced companies, 615,000 by franchise businesses and at one million by recruitment agencies, umbrella companies and personal service companies.
The trades union umbrella body said in such cases the employer using the service had a "duty of care" to the workers in their supply chains.
"This is an issue that affects millions, from fast food workers to people working on building sites," said Ms O'Grady.
"Joint liability must be extended to parent employers. Without it they can shrug their shoulders over minimum wage and holiday pay abuses."
The Department for Business said it had recently set out plans to enhance the rights of workers in such situations.
"Under the package of proposals, all 1.2 million agency workers will be able to request a more stable contract and receive a clear breakdown of their pay," a department spokesperson said.
"We are also considering repealing laws allowing agencies to employ workers on cheaper rates."