Rupert D'Oyly Carte and his wife Dorothy spotted the valley upon which the house and agrden now sits whilst out on a sailing trip and resolved to build their new home there. The family were owners of The Savoy Hotel in London and The Gilbert & Sullivan Opera Company.
Construction of the house began in 2925 and was completed in 1926 when the family moved.
The house remained in their family's posssession until 1947 when their daughter sold it as she felt it was too far from London where she had succeeded to the roles of managing the hotel and opera company.
The property was then owned by Rowland Smith, owner of the Palace Hotel in Torquay, until his death in 1979. After his wife's death in 1982 it passed into the hands of The National Trust.