It is located at one end of the walled enclosure. It was built between the 14th and 15th centuries in the Mudéjar style, using brick and masonry work. It is almost square in shape and is fortified by seven towers, one of which has a five-sided floor plan and the others with various different shapes. The towers have rows of openings crowned by semicircular horseshoe arches. It was the residence of the Marquis of Santillana and his descendants. The walled enclosure and the fortress were declared a National Monument in 1931. In 1993, Buitrago was declared a Property of Cultural Interest, in the category of Historic Artistic Site. The ownership of the castle passed into the hands of the municipal district of Buitrago in 2000.