The home has a rich history as the birthplace of Henry Chapman Mercer and home of Miss Sarah M. James, the VIA member who donated the property to the organization upon her death in 1954.

In addition to serving as a meeting place for members of the Village Improvement Association of Doylestown, the James-Lorah Memorial Home is also listed on the National Register of Historic Homes and considered a historic place by the Bucks County Conservancy.