Snopes has an article on this topic, at
https://www.snopes.com/fact-check/lincoln-and-lees-views-on-... .
> Claim: A meme accurately reflects Abraham Lincoln's and Robert E. Lee's views on slavery.
> Rating: False
Concerning the quote by Lincoln, in the full letter:
"Lincoln drew a clear distinction between what he felt his political duty required of him (i.e., preservation of the Union and not a forced, unilateral abolition of slavery) and what his personal viewpoint on the slavery issue was (i.e., a "wish that all men every where could be free").
Concerning the supposed quote by Lee (which you have modified!) it was recounted by the Rev. Dr. Leyburn:
"Even this passage, which does not match the quote attributed to Lee above, is problematic in several ways: primarily that the purported conversation was witnessed by no one other than the person who reported it, that it was not disclosed until some sixteen years after the fact, and that Lee was long dead by the time it emerged and thus could not confirm or deny it. Moreover, even if it were correctly reported, we can't determine at this remove whether it truly reflected Lee's beliefs at the time, or whether it was part a post-war effort by Lee to resurrect his reputation in the North."
Further, "“[Lee] was not a pro-slavery ideologue,” Eric Foner, a Civil War historian, author and professor of history at Columbia University, said of Lee. “But I think equally important is that, unlike some white southerners, he never spoke out against slavery.”"
That viewpoint is quite different from Lincoln's, who was publicly anti-slavery: "I intend no modification of my oft-expressed personal wish that all men every where could be free."
EDIT: Leyburn's letter is at https://archive.org/details/sim_century-illustrated-monthly-... . You can see he's already pushing the false revisionist narrative that the Civil war was not about slavery:
> After dwelling with emphasis and with feeling on this point, the General then introduced another topic which also moved him deeply, viz., the persistent manner in which the leading Northern journals, and the Northern people generally, insisted that the object of the war had been to secure the perpetuation of slavery. On this point he seemed not only indignant, but hurt. He said it was not true.
I would not trust that letter any more than I trust the claim Darwin renounced evolution on his deathbed.