Originally owned by the Vanderbilt family, Rough Point eventually fell into the hands of local philanthropist, tobacco heiress and art collector Doris Duke ... This mansion may not be as grand ...