A police critical incident response team was sent to the scene about 7.45am on Tuesday, as officers locked down the area and negotiated with the man.
The man was arrested and taken to hospital for observation under police guard. Police said the scene had since been declared safe.
Emergency service personnel earlier responded to reports the man was behaving erratically at the building while in possession of petrol cans and other hazardous substances.
Fire Rescue Victoria said the incident was declared under control at 8.15am after firefighters determined there was no flammable risk.
Witnesses Sam Di Palma and Daniel Safstrom said they were getting a coffee when they saw a man in the building with a trolley and officers were trying to talk to him.
"A bloke (was) behind a counter with a trolley full of chemicals or explosives," Mr Di Palma told AAP.
Mr Safstrom said the police "sent one of their guys in to sit down and negotiate, so they both grabbed a chair".
"I'm a bit shocked to be honest. It's something different," he said.
The building houses offices for CGU Insurance, the Commonwealth Director of Public Prosecutions, Court Services Victoria and legal firms including Gordon Legal and Baker McKenzie.
24°C