Main Content

Dolley Madison's House

Former Slave Shows Charity Toward an Impoverished First Lady

Copyright © White House Historical Association. All rights reserved under international copyright conventions. No part of this article may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system, without permission in writing from the publisher. Requests for reprint permissions should be addressed to

Did you know that after her husband's death, First Lady Dolley Madison was so poor that she had to accept money from a former slave and hand-outs from her neighbors on Lafayette Square?

The yellow house on the corner of H Street and Madison Place was Dolley Madison's home from 1837 until her death in 1849. Originally built by her brother-in-law, Richard Cutts, Dolley Madison moved into the house after the death of her husband, President James Madison, when she had to sell off all her other property and all but a few of the Madisons' 108 slaves.

The house in which Dolley Madison lived during her retirement still survives on Lafayette Square.

One of the enslaved people Dolley Madison brought with her to this house was a young woman named Mary Ellen Stewart. Stewart was among the African Americans, free and enslaved, who attempted to escape from Washington in April 1848 aboard the schooner Pearl. The escape attempt was not successful and as punishment for her participation in it, Ellen Stewart was confined by a slave trader in Baltimore. Ellen's fate was happier than most of the Pearl's captured slaves however, as her freedom was later purchased with funds raised by contributors in New York and Massachusetts.

Another enslaved person who accompanied Dolley Madison on her return to Washington was Paul Jennings. Like Dolley Madison, Paul Jennings had previously lived on Lafayette Square—in the White House—while James Madison served as President. Only 10 years old when Madison took office, Jennings began his tenure at the White House as a footman whose duties included serving, carrying messages, and assistant to the coachman. He later became the personal valet to James Madison, a role he filled for over 16 years until Madison's death. After her return to Washington, Dolley Madison hired Jennings out to work for friends and neighbors, including to President James Polk. In 1847, he was sold to her Lafayette Square neighbor Daniel Webster through an intermediary, Pollard Webb. Webster gave Jennings his freedom in exchange for working for $8 a month until he repaid the debt.

Title page from Paul Jennings' memoir "A Colored Man's Reminiscences of James Madison."

University of North Carolina, Documenting the American South

In 1865, now a free man and working in the Department of the Interior, Paul Jennings wrote the first memoir ever written about life in the White House, entitled A Colored Man's Reminiscences of James Madison. In this work he also described what it was like to see the former First Lady living in poverty on Lafayette Square after her husband's death. He wrote: "Mrs. Madison was a remarkably fine woman. She was beloved by every body in Washington, white and colored. … In the last days of her life . . . she was in a state of absolute poverty, and I think sometimes suffered for the necessaries of life. While I was a servant for Mr. Webster, he often sent me to her with a market-basket full of provisions, and told me whenever I saw anything in the house I thought she was in need of, to take it to her. I often did this, and occasionally gave her small sums from my own pocket. . ."

"Mrs. Madison's Slaves Again," 1848 newspaper article about the Madisons' slaves.

Albany Patriot

Mrs. Madison's Slaves Again (Transcription)

Eds. Albany Patriot:

GENTLEMEN – You have readily admitted to your columns my account of the manner in which Mrs. Madison deals with her people, called slaves. I have a correction or two which I wish to make. It is difficult to gather and state such facts as I had to deal with in this case, with perfect accuracy. Substantially I have stated the facts as they actually exist, but there are one or two particulars to be varied and enlarged. Congress actually paid thirty thousand dollars for the Madison manuscripts instead of twenty thousand, as I have before stated the sum. The man sold by Mrs. Madison was the confidential and personal attendant of Mr. Madison for sixteen years, and not twenty-five. Immediately after the death of her husband, Mrs. Madison promised to set the man free, as she knew was the wish and expectation of Mr. Madison. After she brought him to this city he worked a year and a half or two years on wages, which she took to the last red cent, leaving him to get his clothes by presents, night-work, or as he might. Thus situated, and fearing every day her wants might urge her to sell him to the traders, he insisted she should fix the price, which he would contrive to pay, whatever he might be. Two hundred dollars he actually did advance to her. The girl, Helen, she agreed to send to the street-pump at dusk. Two traders were actually there to seize her, but they happened not to recognize her, and she passed them and made tracks down street. At the time of Mr. Madison's death, there were one hundred and fifteen (115) slaves on the place. He is supposed to have left an estate equal to one hundred thousand dollars, and nearly all of it to his wife. The farm is gone, and all the slaves excepting a mother and three or four children. These it is said are levied upon by the officer.

If my first letter is copied into any paper, I will thank the editor to subjoin these additional facts in a postscript. Yours, sincerely, Hampden.

Washington, March 8, 1848. [Published in the Albany Patriot]

The document recording Dolley Madison's 1849 sale of Paul Jennings to Pollard Webb.

Courtesy of Michael Winston

Bill of Sale for Paul Jennings from Dolley Madison to Pollard Webb (Transcription)

Paul's bill paid

D.P. Madison


Pollard Webb

Rec'd 4th March 1847

To be recorded and the same day was recorded in [Leben?] W.B. No. 180 folio 287. One of the Land Records for Washington County in the District of Columbia and examined by W. Brewster