Refuse workers were seen in a widely-circulated clip throwing the tents into the back of their lorry on Huntley Street, Camden, at about 3pm on Friday, November 10.
Around 10 tents were removed near University College London Hospital (UCLH) in an operation that had involvement from Camden Council.
The tents were cleared from the area after the Metropolitan Police issued a S35 dispersal order, which requires people to vacate an area for a maximum of 48 hours.
The force said it worked with UCLH and other partners in response to concerns for patient welfare near the entrance on Huntley Street.
The topic came up at Mayor’s Question Time on Thursday, shortly after Mr Khan had been criticising former Home Secretary Suella Braverman for calling rough sleeping a “lifestyle choice”.
Tory assembly member Andrew Boff said: “Will you be expressing that view to Camden Council, who have decided - just before Christmas - to remove the tents from homeless people, and declaring to them that homelessness is not a lifestyle choice. It’s a very appalling, only option for many people living in our city.”
The mayor replied: “I was appalled when I saw on social media the images of tents being collected and the police were involved as well, you’ll be aware. University College London Hospital (UCLH) was involved, you’ll be aware as well, and Camden Council.
“My team are looking into this. As soon as I saw those images, I asked my team to look into this, because I was [as] appalled as you are.
“You’re right to remind us at this particular time of year, but it should not affect our views on this - whatever time of year, it’s appalling. Nobody chooses to be a rough sleeper or homeless, living in the circumstances many people do.”
He added that the relevant authorities were looking into what happened, “to make sure it doesn’t happen again in any part of our great city”.
Labour-run Camden Council admitted earlier this week that it was involved in the matter. Deputy leader Adam Harrison said initial investigations found "there was operational involvement from Camden".
Mr Harrison said: "It's important to acknowledge what happened on Friday was unacceptable... it is clear that something has gone wrong here.
"Whilst this dispersal order was applied for by UCLH and enforced by the police, we acknowledge that waste collection services were arranged through the council and our rough sleeping support services were engaged to support people affected."
He said the incident did not reflect the council's values, adding: "We will make sure this is clear and understood at every level of the organisation.
"We are determined to make improvements to how rough sleepers are supported in the borough."
A spokesperson for refuse company Veolia has said it acted "on behalf of and under the strict supervision of Camden Council, and the police when needed".
"We follow all the procedures and requirements needed when it comes to removing sensitive items," they added. "We are working internally and with Camden Council to review how requests of this nature will be handled in the future."
A spokesperson for the Met Police said: "The hospital's security team had requested the assistance of police.
"This action was necessary due to the hospital's concerns for patient welfare. The tents were on private land close to the hospital entrance and had been at the location for up to nine months.
"Police had also received reports of drug use and drug-dealing, urination and defecation and of patients and staff being intimated or verbally abused by some of those begging and sleeping rough at the location."
The force said that on November 6 and 7 "partner organisations" told those at the site about "the hospital's wish to clear the site" on 10 November.
A UCLH spokesperson previously said the action was taken due to "public health concerns" outside "a hospital treating immunocompromised patients".
They added: "UCLH is deeply committed to improving the health and wellbeing of homeless people and we provide a number of services for this vulnerable group."