This section is from the book "The National Capitol. Its Architecture Art And History", by George C. Hazelton, Jr. Also available from Amazon: The National Capitol Its Architecture Art and History.
On the brackets about the semi-circular walls are arranged busts of the former Chief Justices. To the left, upon entering the chamber, are those of John Jay, the first Chief Justice, by John Frazee; Oliver Ellsworth, the third, by Auger; Roger B. Taney, the fifth, by Rinehart; and Morrison VV. Waite, the seventh, by St. Gaudens. To the right aYe similarly placed those of John Rutledge, the second; John Marshall, the fourth; and Salmon P. Chase, the sixth.
When first the question of so honoring Chief Justice Taney came before the Senate, the aversion to the author of the Dred Scott decision was still intense. An anti-slavery feeling—to which, no doubt, is due the fact that no appropriate funeral ceremonies were held in the Capitol at his death—is apparent in the debates. Mr. Sumner, in antagonizing the purchase of the marble bust, bitterly asserted that " Taney would be hooted down the pages of history, and that an emancipated country would fix upon his name the stigma it deserved. He had administered justice wickedly, had degraded the Judiciary, and had degraded the age." Mr. Reverdy Johnson defended the Chief Justice's memory, replying tartly: "The Senator from Massachusetts will be happy if his name shall stand as high upon the historic page as that of the learned Judge who is now no more." Mr. Sumner had the last word. He said that in listening to the Maryland Senator he was " reminded of a character, known to the Roman church, who always figured at the canonization of a Saint as the Devil's advocate" ; and, carrying out the figure, he added, that if it was in his power, " Taney should never be recognized as a Saint by any vote of Congress." On February 23, 1865, the bill was abandoned by its advocates as hopeless. On January 29, 1874, however, a bill passed the Senate without debate providing for the purchase of the bust of the Chief Justice, and at the same time of one of Chase. This occurred about a month before the death of Mr. Sumner. Can it be that his bitterness had perished? It would seem so; for, on December 2, 1872, he had proposed " that the names of battles with fellow-citizens, shall not be continued in the army register or placed on the regimental colors of the United States," and in the preamble to the bill had stated : " It is contrary to the usage of civilized nations to perpetuate the memory of civil war".
 
Continue to: