Sharp differences erupted between the two parties during the Egyptian-sponsored reconciliation talks that began in Cairo earlier this week, they said.
Also on Thursday, the Palestinian Authority released 45 Hamas detainees who were being held in its West Bank jails without trial, a PA security source said.
Twelve of the detainees were from the Nablus area, while four were from Ramallah, the source said.
The decision to release the Hamas men was taken at the request of Omar Suleiman, the head of the Egyptian General Intelligence Service, who is hosting reconciliation talks between Hamas and Fatah in Cairo, the source said.
Five joint Hamas-Fatah committees that have been trying to forge compromises had made little progress in the past few days, representatives of Fatah and Hamas said.
Ibrahim Abu al-Naja, a senior Fatah official participating in the discussions at a secret Egyptian security installation, said the main dispute with Hamas was over the makeup and political program of the proposed PA government.
While Fatah supported an Egyptian proposal that called for a government of independent figures, Hamas was insisting that its representatives be part of the cabinet, he said.
"Fatah believes that it is in the interest of the Palestinians to have a government of technocrats who don't belong to any political faction," Abu al-Naja continued. "This way we won't have one faction imposing its political agenda on the government."
He said that both Hamas and Fatah must also take into consideration the international community's attitude, noting that the US and some EU countries have already made clear they won't deal with a new government that does not recognize Israel's right to exist, renounce violence and accept all previous agreements between the PLO and Israel.
According to the Fatah official, differences have also erupted with Hamas over the timing of presidential and legislative elections, as well as the reconstruction of the Palestinian security forces. The three thorny issues concerning the government, security and elections had been referred to a higher committee that included representatives of Hamas and Fatah, Abu al-Naja said.
Fawzi Barhoum, a Hamas spokesman attending the Cairo talks, confirmed that the two sides were still bickering over a number of issues. Unless the disputes were resolved in the coming days, the talks would end without agreement on a new government, he said.
Hamas opposed Fatah's demand that the presidential and parliamentary elections be held simultaneously on January 25, 2010, Barhoum explained. He said that the two elections should be held separately, saying that the Palestinian Legislative Council's term expires on that date, while the term of PA President Mahmoud Abbas already expired on January 9 this year.
With regards to the makeup of the proposed unity government, Barhoum said Hamas insisted that its representatives serve as ministers.
"Any new government should take into account the results of the [parliamentary] elections that took place in January 2006 and which led to Hamas's victory," he stressed.
Hamas was also strongly opposed to the reappointment of outgoing PA Prime Minister Salaam Fayad as head of the unity government, he said.
Barhoum and other Hamas spokesmen reacted angrily to US Secretary of State Hillary Clinton's demand that a new Hamas-Fatah government accept the conditions of the Middle East Quartet, namely recognizing Israel's right to exist, renouncing terrorism and accepting all agreements between the PLO and Israel.
"Who said that we are going to form a government just to appease the Americans?" Barhoum asked. "The main reason for schism in the Palestinian arena is US meddling in our internal affairs."
Taher a-Nunu, spokesman for the Hamas government in the Gaza Strip, said he wasn't optimistic about the Cairo talks. Fatah's demand that the next PA government accept the PLO's political agenda, which supports the two-state solution, was unacceptable, he said.
"Fatah wants a government that accepts the two-state solution, and this is something that Hamas can't and will never accept," Nunu said. "We are prepared to accept a Palestinian state in the 1967 boundaries only as a temporary solution, without recognizing the Zionist occupation of any inch of our homeland."
Senior Hamas official Mahmoud Zahar also talked about difficulties that were facing the negotiators. One of the main issues that remained unresolved was that of Hamas supporters who were being held in PA prisons in the West Bank without trial, he said, speaking before Thursday's release of 45 Hamas men.
"We are talking about nearly 600 people who are still in Abbas's jails," he said. "They have released only 70 so far, but in the last few days Abbas's men arrested another 35 Palestinians on suspicion of membership in Hamas."
Failure to release all the detainees would lead to the collapse of the reconciliation talks, Zahar said.