I would agree on your last point, but I would argue that enough of the original brick and stone fabric is still there to preserve the most important parts- the front facades.
Woodwork can always be restored, windows and doorways reopened/closed, and the ad-hoc additions removed. If we thought of heritage preservation as preserving immediate heritage value and not potential heritage value, many more buildings in this city would have been demolished long ago (think Paradise theatre and how tatty and compromised it was) and you'd never get a long-gone cornice restored.
The rear ends and interiors of the houses are architecturally unsignificant (going through the heritage report), and so I would agree that the rest of the houses are more or less tear-downs.
Again, I'm not against densification, but I am against this particular scheme- there's enough room for the accommodation of heritage and density, and the overall project would be richer than a clean slate approach.