(CNN) -- Calling the controversy over Bowe Bergdahl's release from Taliban captors "whipped up in Washington," President Barack Obama said Thursday he doesn't apologize for swapping the captured Army sergeant for five Guantanamo Bay detainees.
"I make no apologies for making sure we get a young man back to his parents," Obama said at a news conference in Belgium at the end of the G7 summit.
The issue has roiled the political debate in America, with conservative critics calling the price for Bergdahl's release too high and politicians in both parties complaining that Obama failed to notify Congress of the exchange ahead of time, as required by law.
Obama repeated his stance that he acted with legal authority to seize what might have been the last good chance to get Bergdahl out alive.
"We had a prisoner of war whose health had deteriorated and ... we saw an opportunity and we seized it, and I make no apologies" for that, the President said.
Asked if he was surprised at the backlash, Obama said, "I'm never surprised by controversies that are whipped up in Washington," calling such a reaction "par for the course" and reiterating what he called a time-honored U.S. principle that "we do not leave anybody wearing the American uniform behind."
Meanwhile, a knowledgeable source told CNN that efforts to free Bergdahl from Taliban hands were long and arduous, but the deal might pave the way for possible future talks between the United States and the Taliban.
"Often there could be a very long delay, even weeks, between passing a message and getting a response" from the Taliban, said the source, who spoke only on the condition of anonymity.
For example, the negotiation team would "go to Qatar for a few hours, pass the message, and it could be a week or more before we heard back."
The conversation over the past year was mediated by Qatar.
Then, on May 23, a U.S. negotiating team landed in Qatar and started another series of negotiations, the source said. That's when the team realized it was closer to a deal.
The pace of the talks picked up. Messages were passed back and forth on a daily basis. By May 27, the basic structure of the deal was reached, and President Barack Obama spoke with the Emir of Qatar by phone, the source said.
Implications of the swap
The deal showed that the Taliban are able to reach and execute an agreement, the source added. It showed that the Taliban's political commission can speak for the movement and can give orders to foot soldiers, which are then carried out.
The source said the real conversation going forward needs to be between Afghanistan and the Taliban, but did not discount the possibility that the U.S. would have future discussions with the Taliban.
Bergdahl's captors handed him over to the United States in exchange for the release of five prisoners held at Guantanamo Bay, Cuba.
The source said the five ex-prisoners could return to the Taliban movement but still not radically affect the strength of the Taliban on the battlefield.
Those five men, while seen as senior officials in the Taliban, have not been in touch with their brothers for a decade, and their ability to be operational and hurt U.S. troops if they go back to Afghanistan is believed to be minimal, the source said.
The comments stand in contrast to remarks made by Sen. Marco Rubio, R-Florida, who said the five members of the Taliban are going back to the battlefield "to kill Americans."
Both Democrats and Republicans have criticized the prisoner swap, arguing that it puts American lives at risk by releasing terrorists and accusing Obama of breaking the law by not notifying Congress 30 days in advance.
Since announcing Bergdahl's release over the weekend, Obama administration officials have repeatedly said they fast-tracked the operation over concerns about his health and safety, which appeared to be in jeopardy.
Recent intelligence, in addition to two proof-of-life videos of Bergdahl released in recent months, led to growing concern about his health and safety, a senior defense official said Wednesday.
Administration officials have said they consulted the Justice Department and acted legally. But some Republican lawmakers have said they're not buying that argument, and they want proof that Bergdahl's health was really in jeopardy.
On Wednesday night, Obama administration officials showed a proof-of-life video of Bergdahl at a classified Senate briefing. Lawmakers emerging from the closed session had mixed reactions as to whether his health was evidence that he needed to be immediately freed.
Some also continued their objections that Congress wasn't notified of the swap.
But Sen. Angus King, I-Maine, said the administration had intelligence that "had even the fact of these discussions (about the proposed swap) leaked out, there was a reasonable chance Bowe Bergdahl would have been killed."
"That was one of the pieces of information that we learned ... that gave it some credence in terms of why it had to be keep quiet so long," King told
CNN's "New Day" on Thursday.
Still, King said the leaders of the House and Senate intelligence committees should have been informed, if not all of Congress.
John Bellinger, a former State Department legal adviser in President George W. Bush's administration who was not at the classified briefing, told CNN that the prisoner swap deal was "defensible."
"This is one of those tough national security situations that presidents face," he said, "where all the options are bad."
Deserter or hero?
National Security Adviser Susan Rice said Bergdahl served with honor and distinction. But there's a growing chorus of criticism from some who served with him, describing him as a deserter.
"I believe that he totally deserted not only his fellow soldiers, but his leadership that wanted the best for him and for our country," said former Army Staff Sgt. Justin Gerleve, who was Bergdahl's squad leader.
Some soldiers involved in operations to find Bergdahl have said at least six soldiers were killed searching for him. Asked about this point, Defense Secretary Chuck Hagel told reporters Wednesday that he did not know of specific circumstances or details of soldiers dying as a result of the efforts to find Bergdahl.
Gerleve told "The Lead with Jake Tapper" that he believes Bergdahl is at least partly to blame for the soldiers' deaths.
"I can't really say I blame Bergdahl to fullest extent, but if he wouldn't have deserted us, these soldiers very well could have been in a different place at a different time," he said.
An Army fact-finding investigation conducted in the months after Bergdahl's 2009 disappearance concluded that he left his outpost deliberately and on his own free will, according to a U.S. military official briefed on the report. The official spoke to CNN on Tuesday on the condition of anonymity because of the sensitive nature of the information.
There was no definitive finding that Bergdahl deserted, because that would require knowing his intent -- something Army officials couldn't do without talking to the soldier.
Long road ahead
Bergdahl will remain at a U.S. Army medical center in Germany until he completes treatment, a U.S. defense official there told CNN. After that, he will return to the United States and go to a military base in San Antonio, Texas, the official said.
As the controversy continues, Bergdahl's hometown of Hailey, Idaho, canceled a "Bowe is Back" event planned to celebrate his return "in the interest of public safety."
The city said organizers expected a large number of supporters and protesters.
"Hailey, a town of 8,000, does not have the infrastructure to support an event of the size this could become," the city said.
CNN's Catherine E. Shoichet, Masoud Popalzai, Barbara Starr, Joe Johns, Jim Sciutto, Jim Acosta, Mariano Castillo and Dana Davidsen contributed to this report.
™ & © 2014 Cable News Network, Inc., a Time Warner Company. All rights reserved.