You might think this to be the owner's right, but it apparently isn't, according to reporting by the A-B TAB: since the property is protected with a deed restriction and has landmark status within the Aberdeen Architectural Conservation District, he needed prior approval.
This is the owner who has applied first to demolishing the house to build a large development, then to move the house to another location. He was rejected both times. He has listed the property with a real estate agent among "commercial" or "land" properties, not residential.
Before (photo: Meaghan Ackermann of A-B TAB):

After (photo: Karen Elowitt of A-B TAB):

Yuck.