Peter Lucas purchased the property that began at the corner of Fauquier and Caroline Street in 1764 for the sum of 100 pounds. The property was originally part of the home, storehouse, kitchen, and stable. It's believed those structures were destroyed in the December 1862 siege of Fredericksburg.

The property remained in the Lucas family until January 1890, when Anna P. Green, Lucas' great granddaughter, sold a portion of the lot that would become 1303 Caroline Street. James and Mollie Musselman purchased the lot and, in April 1890, sold it to Mary E. Schooler. The following year, James Musselman had this Victorian home built for Schooler.

Mary E. Schooler (1833-1903) ran a school on Hanover Street, immediately behind the Masonic Lodge, known as the Schooler School. The Schooler House has many artifacts found that document owners over the years. Mary and her daughter Willie F. ran an advertisement in the Free Lance, dated August 26, 1892. A copy of that advertisement hangs on a wall in The Schooler House that reads:

The Female Charity School eventually moved to the brick building at 1119 Caroline Street (a law firm today) and remained in operation until 1930.

Upon her death in 1903, Mary Schooler willed her home to her 2 children, Willie F. and George Schooler. Willie retained ownership of the home until 1913.

The Schooler House is a Folk Victorian home with Italian influences with its bracketed and dentilled prominent cornice, paired hooded windows and shallow pitch roof.