Bruises had been spotted on the boys and council social services bosses initiated family court proceedings.
Judge Peter Greene concluded that one was unintentionally injured by his father's use of "excessive force" and the other probably hurt as a result of the "rough, abrupt manner" in which he was handled by his mother.
The judge also said the youngsters' mother had failed to act on professionals' advice against "co-sleeping".
He has outlined his conclusions in a ruling after analysing the case at a private hearing in Peterborough, Cambridgeshire.
The judge has not identified anyone involved. He said one boy was born in 2014 and the other in 2015.
Bosses at Cambridgeshire County Council had asked him to make decisions about the boys' futures.
Judge Greene said litigation had begun after the older boy's leg was bruised when a few months old.
The judge had concluded that the bruising had been "inflicted by the (boys' father) but unintentionally as a result of using excessive force".
He had also concluded that the boys' mother was "was ignoring advice against co-sleeping".
Social workers had been told to supervise and support the family.
Professionals had then become concerned about the younger boy. Checks revealed bruises and a broken wrist when he was months old.
Social workers said the children's mother was "continually ignoring their advice against having him in bed with her" - and rejecting advice about feeding.
Judge Greene said evidence showed that the couple loved their children and would not deliberately hurt them.
But he said evidence also showed the children's mother "always thinks that she knows best".
The judge said she was "brusque and physically forceful" with her children and dismissive of professionals.
He said he was concerned for the children's safety.