The owner of Fort Richmond has lost a bid for retrospective approval to put stone cladding on a Second World War gun emplacement.