Randy Travis grew up in a big family, but never had any children of his own. There's a reason for that.

Travis — who just released a new song called "Fool's Love Affair" last week — nodded in agreement as wife Mary Travis told Taste of Country about his raising in Marshville, N.C. and how that affected him as he got older. The couple also talked about his complicated relationship with ex-wife Lib Hatcher, whom the country singer divorced in 2010.

"His daddy was pretty tough,"Travis says. "A carousing sort of a guy, drank a lot, had a temper that couldn't be tempered ... was not real kind on a bad day, Randy said, but then there were these other days that were pure magic around the house and Daddy was the kindest, gentlest, sweetest man in the world."

Travis goes on to describe how Travis' mother had a soft heart, and it's that quality the now 61-year-old singer took with him when he left for Nashville in the early '80s. His inability to have kids of his own was one of circumstance, she says with Travis in agreement.

"He wanted children," Mary says.

See Randy Travis Pictures, Through the Years:

"He wanted children, and I think his relationship with Lib, who was so much older than him and they were not able to, is what he explained to me,"Travis says.

Hatcher — who is 16 years older than Travis — was nearing 40 when the couple moved out of North Carolina together. They married in 1991 and were together for 19 years before divorcing three years prior to the singer's devastating stroke, which is covered in quite a bit of detail in his 2019 memoir, Forever and Ever, Amen. His physical recovery has been a slow process, but mentally Travis is still sharp. Being around children is still something he enjoys quite a bit — he always has.

"Children are very special to Randy and he's always had a very — you know, some people are magnetic toward children and animals, and Randy was always that charismatic,"Travis says. "If a fan had a crying baby, he would take that crying baby and put it on his lap, and maybe it was just that honey voice of his, that the baby would just look at him and be quiet."

Author Ken Abraham is the third person seen in the above interview with Taste of Country, filmed in May 2019. See more from that interview — including details about Travis' life today — below.

More From Randy Travis' Interview With Taste of Country: 

See Randy Travis' 10 Best Songs: