Still, we all can dream. And if you've got enough cash, there's a surprising number of castles out there just waiting to become someone's home.
Some are enormous, some are almost cosy, and many are divided into smaller houses or apartments, meaning that you can enjoy the grand life without paying a king's ransom.
We look at seven castles - or bits of them - on the market.
The west wing of Dukes House is spread over five floors, with plenty of grand Victorian features, and is one of three homes since a 1980 conversion. There's an absolutely spectacular staircase in the baronial-style reception hall, which has a double-height, barrel-vaulted ceiling supported by gothic-style stone arches and corbels. There's a big farmhouse kitchen with range cooker and six bedrooms; you get a good-sized garden of your own. The property costs £1.1 million through Finest Properties.
This house isn't itself a castle - but it has one in the grounds. Fifteenth-century Denmylne Castle is a tower house that's now in ruins. The nineteenth-century house it comes with is in rather better nick - it has a roof, for a start - but needs a bit of work. There are four bedrooms and three receptions, set in an acre of gardens and orchards. It's up for sale with Auction House Scotland with a guide price of just £250,000.
Douglas, Isle of Man
Built around 1830, this gothic delight has prime position on Douglas Head overlooking the bay. There's a hexagonal tower, buckets of castellation and elaborate arches. The eight-bedroom property has been recently restored, with bespoke hand-made carpets and a designer kitchen. There's a separate two-bedrtoom cottagfe and three-bedroom house in the three acres of grounds. Harold Tower is up for sale with Chrystals Estate Agents for £3,950,000.
Dumbarton, West Dunbartonshire
Built in the 1880s, sixteen-bedroom Dalmoak Castle was restored in the 1990s, making the most of the extraordinary interior. The drawing room features sumptuous ceiling plasterwork and scagliola columns, and there are elaborate ceilings, hearths and mirrors everywhere. The stained glass windows are particularly interesting, with three round arched panels depict mythical characters, and one window is thought to be the largest in a private house in the UK. It costs £1,530,000 through agents House Simple.
Built 1520, Barsham Manor House was described by Henry VIII as 'my small country palace in Barsham'. It's got eight bedrooms and plenty of grand reception rooms, complete with enormous fireplaces, flagstone floors and stained glass windows. Outside there are some rather lovely formal gardens and a stunning gate house, as well as a separate three-bedroom cottage. The place does need a bit of care and attention, say agents Abbotts, who are asking £2,650,000.
Otterburn, Newcastle Upon Tyne
Otterburn Castle is currently run as a country house hotel, and has a wonderfully grand reception hall with a carved staircase and timber panelling. The 17 bedrooms and four entertaining rooms have stone-mullioned windows, oak wall panelling, huge fireplaces and intricate plasterwork. Outside there are 32 acres of formal gardens, mature woodland and parkland, including a lake. There's also staff accommodation, including three flats and a house. All this can be yours for just £1.4 million through agents Strutt & Parker.
You only buy a bit of Callaly Castle - but what a bit. You get a grand reception hall and an enormous drawing room that was once the castle's library, along with four bedrooms, a study and a mezzanine dining room. You get your own garage and a chunk of garden, but also share 27 acres of gardens and woodland with the owners of the other units. It's up for sale for £600,000 through agent Finest Properties.