Earlier, amid national talk about the Petraeus scandal, Kelley, 37, and her husband released a statement saying they have been friends with Petraeus and his family for more than five years and asked for privacy.
The sprawling story continued to sprout new details Tuesday, including a revelation from a senior official close to Allen claiming that it was Allen who received an anonymous e-mail about Kelley, and tipped her off that someone was threatening her.
Other U.S. officials have said Kelley received harassing e-mails that concerned her and went to the FBI.
"There is no affair," the senior official said. "She is a bored rich socialite involved with every single senior commander at CENTCOM, because she worked as an honorary ambassador."
In another bizarre twist, a jogger running in Rock Creek Park in Washington on Sunday found Broadwell's North Carolina license, according to Lt. Bill Kellogg, spokesman for the Maryland-National Capitol Park Police. Police attempted to contact Broadwell, who lives with her husband and children in Charlotte, but were unsuccessful. Because her name has been in the news recently, they also reached out to the FBI.
Meanwhile, the FBI continues to look into the Petraeus affair amid congressional calls for an inquiry into why leaders were not notified of that matter sooner.
The Petraeus scandal also has raised questions about potential impacts on national security, including concerns that his paramour may have had access to his classified schedule and a New York Times report that she had classified documents on her laptop computer.
FBI agents were at Broadwell's Charlotte, North Carolina, home late Monday, said local FBI spokeswoman Shelley Lynch. She declined to say what the agents were doing there.
A source told CNN national security contributor Fran Townsend that Broadwell was acting as Petraeus' archivist, and that the FBI went to the house to look for any documents she might have. It was not clear whether any of the material was classified, the source said.
Also, a video has surfaced of a speech by Broadwell in which she suggested the Libya attack on September 11 was targeting a secret prison at the Benghazi consulate annex, raising unverified concerns about possible security leaks.
"I don't know if a lot of you have heard this, but the CIA annex had actually taken a couple of Libyan militia members prisoner and they think that the attack on the consulate was an effort to get these prisoners back," said Broadwell in a speech last month at the University of Denver.
A senior intelligence official told CNN on Monday, "These detention claims are categorically not true. Nobody was ever held at the annex before, during, or after the attacks."
Broadwell's source for that previously unpublished bit of information remains unclear, and there's no evidence so far that it came from Petraeus.
Administration officials have said the Benghazi assault was a terrorist attack.
Petraeus was scheduled to testify on Capitol Hill about the attack this week at closed-door hearings. Some Republicans have criticized the administration's response to the Benghazi attack and have speculated that the timing of Petraeus' departure was linked to the congressional inquiry.
Peter King, R-New York, said elements of the general's story "don't add up." King, who sits on the House Intelligence Committee, called Petraeus "an absolutely essential witness, maybe more than anybody else."
However, a senior U.S. official said Petraeus' departure wasn't connected to Benghazi hearings.
"Director Petraeus' frank and forthright letter of resignation stands on its own," said the official, who spoke on condition of anonymity because of the sensitivity of the topic. "Any suggestion that his departure has anything to do with criticism about Benghazi is completely baseless."
Senate Intelligence Committee Chairman Dianne Feinstein, D-California, said Tuesday that she hopes to bring Petraeus before the panel as early as Friday.
House Minority Leader Nancy Pelosi, D-California, added her voice to calls for an inquiry into the FBI investigation that unearthed the Petraeus affair, even though she does not believe formal reporting requirements were triggered.
"We have to find out what, who knew what when and why would Congress not have known," she said. "But again, again, if it doesn't involve national security, the notification requirement doesn't trigger. If it involves poor behavior, yeah, it would have been nice to know before we saw it on TV."
Section 501 of the National Security Act of 1947 spells out the requirements for the executive branch to inform the congressional intelligence committees of key intelligence-related activities.
"The president shall ensure that the congressional intelligence committees are kept fully and currently informed on the intelligence activities of the United States, including any significant anticipated intelligence activity as required by this title," the statute reads.
Panetta, who preceded Petraeus at the CIA, agreed with congressional calls for an inquiry.
"That's another issue I think we ought to look at, because as a former director of the CIA and having worked very closely with the intelligence committees, I believe that there is a responsibility to make sure that the intelligence committees are informed of issues that could affect ... the security of those intelligence operations," he told reporters while traveling in Australia.