They are halting their use of the agency's entertainers after news conference in which the firm admitted its late founder, Johnny Kitagawa, had sexually abused possibly hundreds of boys for decades.
The Ministry of Agriculture, Forestry and Fisheries has decided to suspend its activities with Joshima Shigeru, a member of the group TOKIO, which belongs to an affiliate of Johnny & Associates.
The ministry has been using Joshima to encourage people with disabilities to work in agriculture.
Ministry officials visited Johnny & Associates on Tuesday to ask about the scandal and the company's policy going forward, but were not satisfied with the agency's response.
Burger chain apologizes
Mos Food Service, which operates the Mos Burger fast-food chain, says it has decided not to continue its contract with Johnny & Associates. The company says "any issues of sexual assault run counter to our basic human rights policy."
The company says it will change its current TV commercials and in-store advertisements as soon as possible.
Mos Food Service also apologized for how one store dealt with a poster featuring Johnny & Associates performers. An image that went viral shows company stickers covering the artists' faces.
The company apologized for making the performers, fans and customers uncomfortable, and said the offending poster had been taken down.
Business leader calls for cutting ties
Drinks company Kirin Holdings says it will stop making advertisements with entertainers under contract with the agency. Kirin says it won't do further business until the agency takes steps to compensate the victims.
Another beer giant, Asahi Group Holdings, says it will phase out advertising using celebrities connected to the agency, and will not renew existing contracts. Asahi is also urging Johnny's to make amends to victims.
The allegations prompted Japan Airlines to sever ties with the agency back in March. It says it is sticking with that ban.
And Tokio Marine & Nichido Fire Insurance says it is considering canceling its contracts with the firm.
Niinami Takeshi, the Chairman of the Japan Association of Corporate Executives, held a news conference on Tuesday and urged companies to take a strong stance in response to the scandal.
"Appointing the agency's performers is tantamount to endorsing sexual harassment, and that will draw international criticism," he said. "I feel very sorry for the performers, but there are various options for them, such as moving to other agencies."