Charles married Leah Morrison (born May 7, 1864 in St. Andrew's, NB) on May 16, 1888 at the parson’s residence in Victoria. They had one son, Frederick Morrison Rogers (born March 1, 1889 in Victoria) who unfortunately committed suicide just before his sixteenth birthday on January 29, 1905. All three are buried in the Ross Bay Cemetery in Victoria.
After their son passed away, Charles and Leah were devastated and devoted themselves entirely to their chocolate making, becoming reclusive and eccentric. As they made more money, they donated much of it to charity in the name of the son that they loved so much.
In 1891, the original shop was torn down, along with other buildings on the block, to make way for the new B.C. Land and Investment Agency Building. Mr. Rogers resumed his business in a new shop located at number 34 (later 916) Government Street. He remained at this location until 1917.
Mr. Rogers also dabbled in real estate and had architects Thomas Hooper and John Teague design (and George Snider build) the Queen Anne Revival Style building that houses Rogers’ today. In June of 1996, the building was registered with the Historical Sites and Monuments Board of Canada as a heritage building.
Rogers' Chocolates has operated continuously in the building up to the present time, although no longer owned by the Rogers family. While alterations and an addition have been made to the area at the rear section of the building, the retail shop and its furnishings remain unaltered from the time of Rogers' personal occupancy in 1917.
Charles passed away on February 26, 1927. After his death, Leah sold the business to Mr. William Alexander Lamb, a wealthy American customer. Shortly before Mr. Lamb’s death, he transferred the store and the priceless recipes to his wife who came
several times a year to look after her historic business.
Mr. Rogers left his wife an estate of $295,985 including twenty $10,000 government bonds bearing five percent interest and $53,000 cash in a bank. Leah was a kindly soul who led a reclusive yet philanthropic lifestyle. She was well known for her church work, contributing funds for the bells in St. Andrew’s Presbyterian Church on Douglas Street in Victoria and $25,000 to the Canadian Memorial Church in Vancouver to purchase stained glass windows.
Her generosity, however, eventually left her penniless and little was known about the final years of her life. She died on September 5, 1952 at St. Mary’s Priory in Victoria living on a pension and without enough money for a headstone. The current owners of Rogers’ Chocolates funded the refurbishment of the family monument and the addition of Leah Rogers’ inscription.