(2)The Board shall make compensation to the owner of any building for any actual damage or loss sustained by him in consequence of the prohibition of the re-erection of any building or of its requiring any land belonging to him to be added to the street:Provided that the Board shall not be liable to make any compensation in respect of the prohibition of the re-erection of any building which for a period of three years or more immediately preceding such refusal has not been in existence or has been unfit for human habitation.