The ruling throws into doubt the future of the case, which includes Trump and 14 allies.
The appeals court in Atlanta, in a 2-1 ruling, found that a romantic relationship between Willis and a former top deputy, Nathan Wade, created "a significant appearance of impropriety".
"While we recognise that an appearance of impropriety generally is not enough to support disqualification, this is the rare case in which disqualification is mandated and no other remedy will suffice to restore public confidence in the integrity of these proceedings," the court wrote in its opinion.
Donald Trump says several criminal cases that targeted him were politically motivated. (AP PHOTO)
The case was one of four criminal prosecutions that Trump faced in the years since his 2020 presidential re-election loss.
Just one - a New York case over a hush money payment to a porn star during his 2016 campaign - went to trial.
He was found guilty in that case but has not yet been sentenced.
Trump criticised all four cases as politically motivated attempts to keep him from returning to power.
The Georgia court did not require that the case be tossed out but its decision will require a new state prosecutor to take over in order for it to continue.
Steve Sadow, Trump's lead lawyer in the case, called it "a well-reasoned and just decision".
The case was already expected to be at least paused against Trump when he returns to the White House on January 20.
He has said it should be dismissed in its entirety.
"The American people have demanded an immediate end to the political weaponisation of our justice system," a Trump spokesperson said in a statement reacting to the ruling.
Trump has publicly discussed using the federal government, including the Justice Department, to investigate his political rivals when he returns to power.
A spokesperson for Willis' office did not immediately respond to a request for comment.
Her office could appeal the decision to the state Supreme Court.
"We are very pleased the court of appeals agreed with Mr Roman and the other defendants that Ms Willis should not have been allowed to prosecute this case," said Ashleigh Merchant, a lawyer for Trump co-defendant Michael Roman, who initially sought Willis' removal from the case.
Trump and 18 of his allies were initially charged in 2023 in what prosecutors alleged was a sprawling conspiracy to undo Trump's narrow defeat in the battleground state in the 2020 election.
The defendants include former Trump lawyers Rudy Giuliani and John Eastman and former White House chief of staff Mark Meadows.
Four people initially charged pleaded guilty to lesser offences.
Trump and 14 others pleaded not guilty.
Trump and eight co-defendants moved to disqualify Willis from prosecuting the case early this year, alleging that Willis received a financial benefit from her relationship with Wade and had an improper motive to prolong the case.
Willis hired Wade, an Atlanta lawyer, to help run the investigation and prosecution of Trump and his allies.
Willis admitted to the relationship with Wade but denied allegations of wrongdoing.
She said she reimbursed Wade in cash for holidays the two took together.
After a multiday court hearing, Fulton County Judge Scott McAfee ruled in March that Willis could remain on the case as long as Wade stepped down, which he agreed to do.
But the appeals court found that McAfee's ruling did not address the appearance of wrongdoing that occurred earlier on, "when DA Willis was exercising her broad pre-trial discretion about who to prosecute and what charges to bring".
