(CNN) -- Army Sgt. Bowe Bergdahl, the last American prisoner of war, returned home early Friday morning, his hero's welcome supplanted by a controversial prisoner swap and his reputation tarnished by accusations he was a deserter.
He arrived in San Antonio, Texas, from a military hospital in Landstuhl, Germany, where he'd been recuperating since his release May 31 in exchange for five Taliban figures held at Guantanamo Bay, Cuba.
The 28-year-old Bergdahl, the longest held American soldier since the Vietnam War, was taken to the San Antonio Military Medical Center.
"The Army will continue to ensure that Sgt. Bergdahl receives the care, time and space he needs to complete his recovery and reintegration," Pentagon spokesman Rear Adm. John Kirby said.
Bergdahl's full physical recovery may take months; his public rehabilitation will likely take longer.
The swap that freed Bergdahl has stirred up a political storm in Washington. And almost-daily revelations about Bergdahl's time in Afghanistan have not helped matters.
"Everybody has a piece of the story, and very few people have the whole story," a Defense Department psychologist told reporters.
The backlash has gotten so bad that a public celebration in his hometown of Hailey, Idaho -- one that the 8,000 residents there had waited five years for -- has been scrapped for fear of protests.
"It isn't over for us," Bergdahl's father, Bob, told reporters last week. "In many ways, it's just beginning for Jani and I, and our family. There's a long process here."
Bergdahl went missing on June 30, 2009, in Afghanistan's Paktika province, where he was deployed with the 1st Battalion, 501st Infantry Regiment, 4th Brigade Combat Team, 25th Infantry Division.
Several veterans and soldiers call him a deserter who walked off his base.
An Army fact-finding investigation conducted in the months after his disappearance concluded he left his outpost deliberately and of his own free will, according to an official who was briefed on the report.
The Army has no definitive finding that Bergdahl deserted because that would require knowing his intent -- something officials couldn't learn without talking to the soldier, a U.S. military official told CNN.
On Thursday, The Daily Beast published two letters the solider reportedly wrote to his family while imprisoned by the Taliban.
In the letters -- dated 2012 and 2013 -- Bergdahl discusses his life and partially explains why he disappeared.
"Leadership was lacking, if not non-existent," he wrote.. The conditions were bad and looked to be getting worse for the men that where actuly (sic) the ones risking thier (sic) lives from attack," he wrote.
"If this letter makes it to the U.S.A., tell those involved in the investigation that there are more sides to the cittuwation (sic)," Bergdahl said. "Please tell D.C. to wait for all evadince (sic) to come in."
Bergdahl was captured by insurgents with links to the Haqqani network in Pakistan, an Afghan Taliban commander not authorized to speak to the media told CNN.
Over the five years, the captive was transferred back and forth between Afghanistan and Pakistan, the Taliban commander said.
Citing an American official, The New York Times reported that Bergdahl told medical staff that the box he was kept in for weeks at a time was pitch black and like a shark cage.
CNN has reported that Bergdahl has said he was kept in a small box after trying to escape, according to a senior U.S. official, who spoke on the condition of not being identified.
A Defense Department psychologist notes that historically, those held by "less organized groups" -- such as those holding Bergdahl -- "have harsher treatment" than those held by representatives of governments recognized by the world.
In the half-decade that he was gone, Bergdahl's parents held rallies, maintained a website and lobbied congressmen.
The elder Bergdahl grew a long beard, immersed himself in books about Afghanistan and its Pashto language, to speak to his son's captors if only from afar.
The Rocky Mountain town of Hailey is just minutes from the renowned tourist destination of Sun Valley, famed for its skiing. Yellow balloons and American flags line the sidewalks of Hailey. And residents planted one tree for each of the five years Bergdahl was held captive in Afghanistan.
"There were times where we wondered, but (parents) Jani and Bob Bergdahl never once gave up faith that their son was coming home to them," family friend Stefanie O'Neill said.
With Qatar as the broker, the United States struck a deal with the Taliban to free Bergdahl. The price: the release of five Taliban fighters from Guantanamo.
The Obama administration didn't tell Congress of the releases. Under law, the White House has to give a 30-day notice to Capitol Hill before any terrorists are transferred from Guantanamo.
Believing that his health was deteriorating, the administration said it acted quickly to save his life.
special operations forces recovered Bergdahl without incident on May 31 at a pickup point in eastern Afghanistan, near the border with Pakistan.
The San Antonio Military Medical Center has a room ready for him and a support team is standing by.
The first meeting between Bergdahl and his parents may only last minutes depending on what psychologists recommend, said Army spokeswoman Arwen Consaul.
Bergdahl's daily routine will focus on four key areas: medical care, psychological support, debriefings and family support.
"This is to help a person who has had no control of their own life for years now regain that control step by step," she said.
Taumatizing experiences -- known only to Bergdahl -- may have distorted any sense of normalcy that he might have had, as well as mechanisms he might have used to cope with everyday life.
"They've been in a situation where everybody else has controlled them or something else has taken charge. And they have not been able to choose," the Defense Department psychologist said about long-held captives generally. "Now they're overwhelmed with daily living, and so we kind of start the process of giving them back the ability to predict."
There's no set time limit for how long the reintegration process will take, because the case for each returnee is different.
David Rhode, a former New York Times journalist who was held captive by the Taliban for seven months, said Bergdahl faces a long road ahead.
"Psychologically, there will be a long debrief," said Rhode, who was taken in Afghanistan and held in Pakistan until he managed to escape. "They're actually going to want intelligence (about) who held him."
Army human resources officials will determine where Bergdahl goes after he completes the reintegration process. He'll receive follow-up care and have access to a psychiatrist for the rest of his life.
Bob Bergdahl appears understanding about the delay.
He compared his son's situation to that of a diver going deep on a dive: "If he comes up too fast, it could kill him."
What was initially touted as America's commitment to never leaving a man behind, Bergdahl's release has been anything but universally celebrated.
In the partisan frenzy permeating Washington in an election year, lawmakers have accused the White House of overstepping presidential authority without consulting Congress.
Republican critics contend the cost of the deal was too high because it freed hardened terror suspects to possibly rejoin the fight against U.S. interests, especially in return for a soldier accused by some in his unit of deserting his post.
Defense Secretary Chuck Hagel has defended the deal -- in sometimes testy exchanges -- saying it was Bergdahl's last and best chance to get out.
Former soldiers involved in the operations to find Bergdahl asserted to CNN this week that at least six soldiers were killed because of the search for Bergdahl in 2009.
A U.S. official told CNN that Pentagon and Army officials have looked at the claims, and "right now there is no evidence to back that up."
But the damage has been done.
The FBI said it was investigating threats against Bergdahl's parents.
The controversy has forced Hailey to put its homecoming on hold. Town officials say it would be too hard to manage all of the supporters as well as all the protesters who might show up.
"I got a phone call from a person -- well, an attorney at an event organizer in California who didn't want to reveal the group, but wanted to know if they could bring 2,000 people up to protest the event, if they'd be allowed into the event," Police Chief Jeff Gunter told CNN. "So that was definitely a concern. There was also a gentleman in Texas who wanted to bring up a bunch of people to protest the event as well."
Bergdahl is still technically in the Army.
Up for promotion in June, he could return to his unit in Alaska, get reassigned somewhere else or be discharged.
Some members of his platoon want Bergdahl to face military trial for desertion under the Uniform Code of Military Justice.
Still, even if he deserted, a senior defense official said earlier this month that Bergdahl will not likely face any punishment.
As he told CNN, on condition of anonymity: "Five years is enough."
CNN's Ed Payne, Holly Yan and Mariano Castillo contributed to this report.
™ & © 2014 Cable News Network, Inc., a Time Warner Company. All rights reserved.