GetUp spokeswoman Zoe Edwards told AFP refugees inside the camp said "men are being forcibly removed by bus, apparently to other centres".
"They destroyed our food and damaged our accommodation too, they destroyed our water tank", one asylum-seeker said in a mobile telephone message from the camp.
A video shot by Sudanese refugee Abdul Aziz and posted on Facebook showed police using a megaphone to tell the asylum seekers to leave because their stay at the camp, located on land used by the PNG navy, was illegal.
"They are taking the phones and are very aggressive and are taking out some refugees who still remain in the rooms".
"They are destroying everything".
Media Entertainment & Arts Alliance chief executive Paul Murphy said Iranian-Kurdish journalist Behrouz Boochani appears to have been deliberately targeted by Papua New Guinea police because of his high profile as a journalist reporting from inside the detention centre.
"The refugees still are silent are watching them so scared". "They are clearly not safe in PNG".
Meanwhile, PNG Supreme Court is due to hear on December 15 an appeal against its ruling earlier this month that the camp's basic services were not to be restored, lawyer Ben Lomai told AFP Thursday.
Fellow detainee Ezatullah Kakar tweeted the asylum seekers were peaceful, but scared, and "everyone is crying".
"Immigration officials went in and explained to them that they really had no legal standing. voluntarily they packed their bags and moved across", he said.
The police operation was confirmed by Australia's Immigration Minister Peter Dutton, who said Canberra was "very keen for people to move out of the Manus regional processing centre".
"Some of them believe that if there is violence with police and the footage is broadcast back here that will twist our arm and change the policy position, but under no circumstance will these people be coming to Australia".
'The AFP has a liaison officer in Manus Province working in an advisory and mentoring capacity to the RPNGC on policing matters'.
Former Australians of the Year including Rosie Batty, Tim Flannery and Patrick McGorry have penned an open letter to Malcolm Turnbull and Bill Shorten urging them to address the "human disaster" unfolding on Manus Island.
About 50 asylum seekers have departed an Australian-run detention camp in Papua New Guinea after police moved into the complex, confiscating food, water and personal belongings from the roughly 310 who remain.
Australia closed the offshore detention centre on Manus Island at the end of October, but some 380 refugees and asylum seekers have refused to leave, citing safety fears outside the centre's walls.
Canberra has tried to resettle the refugees in third countries, including the United States, with little success.
The group warns Australia's reputation on human rights is deteriorating because of the failure to meet the obligations of the UN Refugee Convention.
The camp, and another on the tiny Pacific island nation of Nauru, have been the cornerstones of Australia's controversial immigration policy, which has been strongly criticised by the United Nations and rights groups. Refugees have been attacked and seriously injured by some members of the local population, who have made clear they do not want the men on Manus Island at all.