Leaving the house you have lived in and loved for nearly 30 years can be a huge wrench. But what about leaving its garden – the garden you have made from scratch over that time, with its tall trees you put in the ground as knee-high saplings?

What about the idiosyncratic follies you personally hand-built to mark special anniversaries? Or the cuttings of rare plants given to you by friends, which are now whopping great shrubs that must be left in place for others, who don’t know the happy memories and associations rekindled each year, when they come into bloom?

This is the conundrum that David Wheeler and Simon Dorrell find themselves in, having made the decision to pack up and leave Bryan’s Ground, the substantial Arts and Crafts house whose garden they have made – and made famous – since they arrived there in 1993.

It is also the home of the Bryansground Press, David’s small publishing enterprise, which produces Hortus, the erudite gardening quarterly journal.

Tucked down a track off a long lane, surrounded by sheep-grazed hills literally on the border with Mid Wales, it is in just about as remote a location as anywhere in England.

It’s romantic, pastoral England, too; a bucolic working landscape of hops, fat cattle, apple orchards and huge pear trees rising out of billowing hedgerows. Yet many thousands of people have found their way to Bryan’s Ground in the last two decades, since the garden has matured into an attraction of inspiration and renown.

“We moved into Bryan’s Ground at the end of November 1993, and immediately there was a light dusting of snow,” recalls David.

“From the upstairs windows we could see in the snow-covered expanse of grass in front of the house, neat rows of indentations, where there had formerly been an orchard. So, we planted an orchard again, with Edwardian-bred varieties, which are contemporary with the house, and also traditional Herefordshire varieties.”

Today, the orchard and the wiggly margined canal that adjoins it, are surely the most photographed part of the garden in due season, when David’s now mature apple trees come into leaf, each tree set within its own blue sea of Siberian flag irises.

“The sibiricas always flower in the third week of May; you can almost set the clock by them,” he says. As we stroll through the gardens, which are separated into numerous “rooms” by tall, mainly yew hedges, the memories unfold. Near the house is the Spring Garden. “The first bit I planted,” says David.

“The jacquemontii birch trees came from our previous house, because in 1993 they were easily movable tiny saplings. Some years ago, I planted 50 bulbs of martagon lilies among them and we must have several hundred coming up every year now, they look wonderful,” he adds.

Almost every tree, every shrub, has a memory attached to it. The waist-high hydrangeas were from cuttings gifted by owners of unimpeachable collections in France and Belgium; the Portugal laurels that make topiary sentries by the front door were a gift, in little 6in pots, from Rosemary Verey.

The collection of succulents in the greenhouse all began as cuttings, a present from the gardener at Villa Ephrussi de Rothschild, at Saint-Jean-Cap-Ferrat on the Côte d’Azur.

The dovecote is an exquisite half-timbered oak building with a steeply pitched roof, designed by Simon and built by him in 2004, with help from two friends. Inspired by an original 17th-century building nearby, it terminates several important vistas and is the most significant of several buildings and follies created by Simon, deployed at carefully chosen locations within the formal grounds.

There are so many parts to Bryan’s Ground: the wisteria-garlanded terrace, the Beech Walk, the Dutch Garden, the Cuckoo Walk, the June Garden, the Formal Garden and, on the edge of a five-acre arboretum known as Cricket Wood, a boating pond, the haunt of numerous wildfowl. These are just some of the features made by the creative pair.

The large Edwardian greenhouse, with its attractive fish pond, is a treasure trove of tender plants, nurtured cuttings, and a collection of beautiful bonsai trees David created while recovering from cancer in 2008 (see “How My Garden Healed Me” in The Daily Telegraph Aug 1, 2008). How much of a wrench is it, deciding to leave all of this for pastures new?

As it turns out, the decision was not difficult. “The garden is getting too much for us at this stage,” says David.

“I had huge spinal surgery two years ago, and I can’t do anything like the amount of gardening I used to do. And Simon is not getting any younger – we really wanted to ensure that the garden didn’t disintegrate ‘on our watch’. It’s just time to move on, and we both came to the same decision at the same time, earlier this year.”

Simon agrees. “I felt that, if we went now, we would still have the energy to make something new and smaller, somewhere else. I’m 59, I’ll be 60 later this year, and David is approaching 76. I’ve obviously got a bit more energy left, but I think it would be nice for David, and for me, to have lots of fun, going into his 80s still planting things, and I can see there is a renewed spring in his step, now we are thinking about our new project.”

That project doesn’t have a definite name yet: Simon and David are looking locally, having many ties to the area. But confining the search area in such a rural location naturally restricts the number of potential houses on offer. To this end, there are just two possible properties in their sights presently.

“The two houses that we like are so extremely different from each other,” says Simon. “There’s the very sensible one with not a thing which needs doing to it; we could just move in and be very comfortable there – our heads say we should go for it, and we haven’t ruled it out.

But then there’s the insane one, which our hearts prefer. It’s larger and it is definitely a project. We’re hopeless romantics, so I think we might be complete idiots and go for the heart, not the head, if it turns out to be still available!”

This gives me the thought that they both would like, in an ideal world, a purchaser equally adventurous and romantic to take on the home they are leaving behind. But old houses and complicated gardens are always high maintenance investments, and David is pragmatic. “We’re quite philosophical about it; you’ve got to be bonkers to garden on this scale.”

Simon concludes: “I have said to everyone who has been to view, ‘please don’t be daunted by the garden; please don’t feel you’ve got to preserve it in aspic’. But I do want someone to love it, because we love it to bits. Obviously, people have to make changes to suit themselves but if they do it in the spirit of Bryan’s Ground, which we did when we came here, I won’t feel so awful about leaving it.

“And you know, we’re going to stay locally, so we’ll hear about it if a JCB is brought in and demolishes everything!”

Bryan’s Ground, Stapleton, LD8 2LP, is for sale via Bengough, Leominster.

Email ; 01568 720159

Packing up the garden 

David

“I’ll be moving about 250 plants in pots. I’ll also be taking some 1,500 empty terracotta pots of all sizes. They’ve been with me for years and have a seductive patina that doesn’t develop overnight.

“I’m disinclined to entrust my numerous mature potted plants, countless young propagations and delicate bonsai to the removal folks. I’ll hire a large van or small truck and ferry them to their new home myself.

“As for the goldfish in the greenhouse pond, I’m hoping the new owners will adopt them. If not, they, too, must endure a buggy ride to new waters – my fondness for them won’t allow their abandonment.”

Simon

“There are benches, sculptures and assorted bits and pieces in the garden, which the agent advises are ‘by separate negotiation’. We wouldn’t want to take everything.”

Starting afresh 

Simon

“When looking for a property, we seem to spend ten times longer in the garden than in the house. When we came here, we were two hours looking around outside before we even came into the house. We look at the land, the way it is situated, the outbuildings, the potential.”

David

“Other people go around potential house purchases with cameras making a record of rooms and so on; I go around the garden with a pH meter, that I prod into the soil. I’ve done a soil test on both properties we have been looking at – although we don’t know of course at the moment whether we will get either one of them.

But both of them seem OK for hydrangeas, which is good, because I’m presently madly propagating from the 250 or so varieties we have here.”

Five signature plants from Bryan’s ground

Siberian flag iris (Iris sibirica) 

Always in flower in the third week of May in the orchard around the pool.

Japanese maple (Acer palmatum ‘Osakazuki’) 

Prized for its unbeatable autumn foliage.

Martagon lily (Lilium martagon)

“We started off with 50 bulbs and must have hundreds of them now.”

Oleaster (Elaeagnus ‘Quicksilver’)

Silvery, olive-like foliage and tiny, highly fragrant yellow flowers in May.

Dog’s tooth violet (Erythronium dens-canis) 

Grown for their sulphur-yellow starry flowers, self-seeding everywhere.