The lost castle once a palace and a prison you would never know was there