The original castle, reminded in 1337, was the seat castle of the Vartenberks, gentlemen of Dubá and the Šlejnic, the owners of the estate of Tolštejn. In 1642 it was burnt down by the Swedes. The castle has never been restored, but during the 19th century, from the extensive ruin a popular place of trips became with a restaurant and observation point.