The original two storey cottage had stone and brick walls and a shingle roof and was pulled down in 1883, however parts of the northern, western and southern walls of the building still remain. The southern wall of Samson's cottage formed part of the wall to Joseph Raphael's store and stable constructed 1853. The cottage has been rebuilt in 1991-2 in its original form.
The Samson's Cottage remnants and site are associated with the Chinese community (among others), which was established in The Rocks from the 1850s. The site was occupied by Lee Quan and Hong On Jang who operated a boarding house for Chinese travellers in the early twentieth century. The site was also used as a Chinese-run laundry from the 1940s up until 1974.