PANAMA CITY » With a cordial evening handshake, President Barack Obama and Cuban President Raul Castro moved Friday toward a groundbreaking meeting on the sidelines of the Summit of the Americas in what would be a remarkable display of reconciliation between two nations.

The powerful symbolism of a substantial exchange Saturday between the leaders with the leadership of the Western Hemisphere gathered around them could signal progress. Both sides are still working through nettlesome issues that would lead to the opening of embassies in Washington and Havana, the first stage in a new diplomatic relationship.

The first visual clues of an improved relationship — at least among leaders — came Friday evening as Obama and Castro arrived at a Panama City convention center for the summit’s opening ceremonies. A reporter for a Venezuelan TV network posted video online showing the two greeting each other comfortably with multiple handshakes and extended small talk, while U.N. Secretary-General Ban Ki-moon and Cuban Foreign Minister Bruno Rodriguez looked on.

The White House said the interaction was informal and said they didn’t engage in substantive conversation. The two men were expected to speak further Saturday — the first extended conversation between the leaders of the U.S. and Cuba in more than 50 years.

Obama, speaking to a meeting of civil society groups, cast the move to end hostile relations as a triumph for the Cuban people.

"As the United States begins a new chapter in our relationship with Cuba, we hope it will create an environment that improves the lives of the Cuban people," he told the gathering, which included Cuban dissidents. "Not because it is imposed by us, the United States, but through the talent and ingenuity and aspirations, and the conversations among Cubans from all walks of life so they can decide what the best course is for their prosperity."

The White House was coy over the status of the State Department’s recommendation to remove Cuba from the U.S. list of state sponsors of terror. Removal is a top issue with Castro because it would not only eliminate Cuba’s status as a pariah, but also ease Cuba’s ability to conduct simple financial transactions.

Nevertheless, the pace of activity over the terror list suggested that even if Obama did not make an announcement Saturday, one would come soon.

The U.S.-Cuban outreach entered a new, accelerated stage in recent days, with Obama speaking with Castro by phone Wednesday and U.S. Secretary of State John Kerry holding a lengthy meeting with Cuban Foreign Minister Bruno Rodriguez late Thursday.

The Cuban foreign ministry issued a brief account of the Kerry-Rodriguez meeting, saying they met for nearly three hours in a "respectful and constructive atmosphere."

It was the highest-level, face-to-face contact between officials from the two countries since the Dec. 17 announcement that Washington and Havana would move to restore diplomatic relations that were severed in 1961.

Even as Washington talked up the historic shift toward Cuba, leftist leaders in Latin America took shots at the U.S. in solidarity with Venezuela.

Barely off the plane, President Nicolas Maduro of Venezuela laid a wreath at a monument to victims of the U.S. invasion of Panama in 1989. To shouts of "Maduro, stick it to the Yankee," he vowed to personally ask Obama to apologize to Panama and compensate victims of what he called a "massacre" that left more than 500 people dead during the invasion.

"Never again a U.S. invasion in Latin America," Maduro said.

President Evo Morales of Bolivia said he backs Maduro’s drive to end U.S. intervention in the region.

For Obama and Castro, their conversation Wednesday was the first since they spoke Dec. 17.

The flurry of diplomacy around the summit was recognition of the historic nature of the new relationship intended to end five decades of American presidents either isolating or working to overthrow Fidel Castro’s government. Officials hoped to make the most of the exchange between the two men.

Still, Obama made a point of meeting with about 15 Latin American activists, including two Cubans who have challenged Castro’s government. The White House identified the Cubans as Laritza Diversent, a human rights lawyer and independent journalist, and Manuel Cuesta Morua, a leader of a centrist opposition group. A large contingent of pro-Castro Cubans who were supposed to participate in a larger civil society forum left shortly before Obama spoke to protest the inclusion of Cuban dissidents.

Obama was already getting praise from allies in the Americas.

"President Obama is going to leave a legacy the way he is supporting Hispanics in the United States, and also his new policy for Cuba for us is very important," said Panamanian President Juan Carlos Varela.

Removing Cuba from the list of state sponsors of terror would be a major milestone and likely generate controversy in the U.S., given the political repercussions of any Cuba opening. The sensitivities over Cuba are especially acute in Florida, a key presidential battleground, and they are likely to ignite vigorous debate among the presidential candidates.

Congress would have 45 days from the day Obama removes Cuba from the list to issue a resolution of disapproval, but the margin of passage would have to be veto-proof.

Rep. Jeff Duncan, the chairman of a House panel on the Western Hemisphere, criticized taking Cuba off the list. And Sen. Lindsey Graham, who is among Republicans considering a presidential bid, decried the expected Obama-Castro meeting and called the Cuban leader an "entrenched dictator."

Associated Press writers Andrea Rodriguez and Joshua Goodman in Panama City contributed to this report.