It happened about 3am on Saturday in Sydney's Domain but the damage went unnoticed until about noon on Sunday when police working at Australia Day celebrations realised what happened, according to NSW Police.
"The NSW Police Wall of Remembrance, dedicated to the memory of police officers killed in the line of duty, was significantly damaged by vandalism," the force said in a statement.
The words "evil", "dogs" and other markings were scratched into the stone monument.
The area has been taped off as a crime scene and an investigation launched.
The force is working with the Sydney City Council to fix the damage.
NSW Premier Chris Minns described it as "disgusting behaviour".
"Police put themselves in danger every day in the service of our state," he said.
"To deface a memorial that commemorates police officers who have served the state, and those who have lost their lives whilst on duty is lower than low."
Police are investigating after someone scratched 'evil', 'dogs' and other markings into the stone. (Bianca De Marchi/AAP PHOTOS)
Police Minister Yasmin Catley said no stone will be unturned as officers look for the "cowards" responsible.
"This is a sacred site which honours officers killed in the line of duty," she said.
"For it to be defaced, is the lowest of acts.
"Our police officers sacrifice their personal safety every day for our state – for that they should be celebrated, not subjected to vile attacks."
Anyone with information about what happened or relevant footage has been asked to come forward immediately.
It comes after a string of attacks on statues and monuments in NSW and Victoria.
A controversial monument to one of Melbourne's modern founders that was put up during the colonial era at Queen Victoria Market centre was cut in half and a nearby memorial to ANZAC soldiers covered in red paint on Saturday.
On Friday, a sandstone heritage statue of British explorer and naval captain James Cook in Sydney's Randwick had its hand and nose severed and was partially splattered with red paint.
Earlier, four people decapitated the statue heads of two Australian prime ministers at the Ballarat Botanic Gardens, causing more than $140,000 in damage.
The heads of former Labor prime ministers Paul Keating and Kevin Rudd were removed and stolen on Thursday, while the name plates of the 18 remaining statues were covered in spray paint.