Over the years, an abundance of evidence had come to light that the North Vietnamese, while returning 591 U.S. prisoners of war after the treaty signing, had held back many others as future bargaining chips for the $4 billion or more in war reparations that the Nixon administration had pledged. Hanoi didn't trust Washington to fulfill its promise without pressure. Similarly, Washington didn't trust Hanoi to return all the prisoners and carry out all the treaty provisions. The mistrust on both sides was merited. Hanoi held back prisoners and the U.S. provided no reconstruction funds.
BTW, in the article John Kerry doesn't come out smelling too sweet. I have to say that I'm surprised that I heard nothing of this in the 2004 campaign. That it apparently wasn't bigger news during the 2004 election might be an indication that the claims made in the article are crap. Still, Schanberg seems like a fairly credible author.