The key moment occurs in the sixth verse of the third song on the tenth album proper by Nick Cave & The Bad Seeds. The tune is called ‘Hallelujah’, and its protagonist, like so many of Cave’s, sets out on a journey, leaving behind the familiar solace of his home, his nurse and her medications. In due course, our traveller comes upon a little house, from whence emerges a young woman. Seeing that he’s soaked to the skin, the girl invites him inside. So what does he do? He turns around and goes back home.

On the face of it, ‘Hallelujah’ seems like the odd song out on And No More Shall We Part in that it constitutes a return to Cave’s narrative ballads of yore. Yet stitched into the story is an abundance of autobiographical detail; mute typewriters, pianos with their teeth bared and the like. In fact, Bad Seeds obsessives will probably interpret it as the first-person parable of a guy who rejects the love of a strange woman for the clammy comfort of his own addictions.

“Well, I think that that particular song, of all of the songs, I feel the most connected to, for a lot of reasons,” Cave says. “It’s comic, but I do really feel, at this point in my life, very close to whatever’s going on in that song. The final verse of it is written very late in the piece, he’s gotten to the house and all of that sort of stuff…”

And he turned back…

“…Well, he turned back primarily because the song was already eight minutes long and he had to just beat a hasty retreat! They dictate the terms, the song. But at the same time it says a lot, I think, about the way things are. To me… it’s a retreat to safety, which I know is sneered at. But fuck ’em.”

Things have changed in Nick Cave songs. Ten years ago, our ‘Hallelujah’ man would’ve entered a room full of “wet-lipped women with greasy fists” all crawling the ceilings and the walls, voluptuous succubae who’d spike his drink and leave him for dead.

“Well exactly, it certainly echoes ‘Papa Won’t Leave You, Henry’,” he concedes, “and I quite intended that and it is supposed to be a classic Nick Cave song in a way, but a rejection of it as well. There is stuff about the creative process there, y’know, the typewriter and all that business, and it’s a sort of sorrowful choice of un-adventure and returning to a kind of known misery, and I quite like the way that went.”

Cave’s comment about the song dictating the terms suggests one of the more tantalising themes in The Secret Life Of The Love Song, the lecture he performed around Europe in 1998-99, and which was released on CD on the King Mob label last year. In that exposition, he held that “the love song holds within it an eerie intelligence all of its own”. This intelligence, he proposed, has the power to foretell and even manipulate events in its creator’s life, the love song singing the singer.

Right now, that singer is doing interminable rounds of interviews in the Kensington Gore hotel in London, sick of hearing the sound of his own voice. “You’re number 69 or something like that,” he explains, with weary good humour. I don’t know which is more disquieting, the fact that my subject may be all talked out, or that he associates me with that number.

This is the fourth time I’ve interviewed Nick Cave in as many years, so I won’t bore you with the usual pen pictures, the myth-mongering or myth-debunking. Rather, we’ll try to uncover the roots of the new record by understanding what the singer got up to after The Boatman’s Call and Best Of campaigns.

Following a handful of festival shows and an American tour in the fall of 1998, Cave sent the band home, intending to take the year off to write new material and generally keep a low profile. However, things didn’t go according to plan. First of all, he was invited to teach a class on songwriting at the Poetry Academy in Vienna, and also to deliver his Secret Life lecture. As an exercise in stretching his audience’s expectations and marrying rock ‘n’ roll and academia, it proved a successful endeavour, not least because it managed to throw considerable light on Cave’s various artistic credos.

“I believe the love song to be a sad song, it is the noise of sorrow itself,” he wrote. “We all experience within us what the Portuguese call saudade, which translates as an inexplicable longing, an unnamed and enigmatic yearning of the soul… saudade, or longing, is the desire to be transported from darkness into light, to be touched by the hand of that which is not of this world. The love song is the light of God, deep down, blasting up through our wounds. In his brilliant lecture entitled ‘The Theory and Function of Duende’, Federico Garcia Lorca attempts to shed some light on the eerie and inexplicable sadness that lives in the heart of certain works of art. ‘All that has dark sounds has duende,’ he says, ‘that mysterious power that everyone feels but no philosopher can explain’.”

Lorca was the poet’s poet, championed by Leonard Cohen, Patti Smith, Joe Ely and Shane MacGowan among others. His writings on the duende, deep song, gypsy ballads, Spanish folk art and lullabies resound across many musics, not least the arcane American tunes collected by Harry Smith and Alan Lomax, a well of songs from which much of Nick Cave’s earlier work drew inspiration. Lorca’s Andalusian fatalism had always been present in tunes like ‘Sad Waters’, but when Cave moved to Sao Paulo in Brazil in the late ’80s, his work became saturated in saudade, resulting in 1990’s The Good Son, a turning point in the artist’s career, and perhaps one which made his last couple of albums possible.

Mind you, the constant sorrow and suffering which Cave once reckoned “flows through life like water” is now tempered by a kind of grace, manifest in ‘As I Sat Sadly By Her Side’, the album’s title tune, and most acutely, in the closing three-song suite. Elsewhere, Lorca’s lullabies seem to inform the coda of ‘Hallelujah’, with its sad refrain of “The tears are welling in my eyes again/I need twenty big buckets to catch them in/And twenty pretty girls to carry them down/And twenty deep holes to bury them in”.

These lines are rendered by Montreal’s award-winning McGarrigle sisters Kate and Anna, last heard by rock audiences on Emmylou Harris’ Wrecking Ball, and who provide a counterpoint to Cave’s voice redolent of Dylan’s gospel choirs or the Greek choruses in Leonard’s songs of love and hate.

“They were so quiet and haunting,” Cave remarks, “these really strange harmonies. When I was writing the record I always just kind of had them in mind, kept hearing them piping up.”

Cave was introduced to the McGarrigles by Hal Willner at the Harry Smith night of the Meltdown festival, which the singer curated in London in the spring of 1999. By this time, Cave had begun performing sporadic shows with a sidebar ensemble formed around the nucleus of himself, Warren Ellis and drummer Jim White of the Dirty Three. This “kind of permanent line-up for a completely different sort of show” was completed by bassist Susan Stenger, drafted in because Cave was having “a bit of a problem with my bass hand”.

Cave: “It began pretty much being about that lecture, and the songs were just included to augment it, but I enjoyed doing the songs so much we just got rid of the spoken word side of it.”

On the eve of the Liss Ard 99 festival, I ribbed the singer over the fact that, not content with enjoying the services of possibly the finest live band in the world, he’d gone and poached the guts of the next best.

“I’m a greedy son of a bitch!” he joked, before describing how those shows taught him about spontaneity and improvisation.

“I’m not just saying this glibly, but it’s been an incredible process for me,” he said. “It obviously doesn’t have certain things that the Bad Seeds can offer, in regard to the power and that sort of thing, but there’s a freedom in this music that I can’t get from the Bad Seeds because there’s just so many Bad Seeds and you’re very much a cog in a greater machine.”

Not only did the extracurricular gigs open Cave’s eyes to a new way of performing, but it also showed him the stuff his back catalogue was made of.

“My relationship with a lot of the songs that I do has completely changed since doing them with this group,” he observed. “They’re actually pretty good, some of the songs I write – I’m feeling quite amazed. You oughta hear our ‘Papa Won’t Leave You, Henry’; it’s a fucking mindfuck. Well… it’s pretty good. It’s just a kind of (laughs) howl of despair.”

Through playing those old songs with his seedlings, and relieved of the highly stylised elements of the Bad Seeds’ sound, Cave found himself formulating a new approach to recording.

“I want to keep the sound very much stripped down,” he vowed. “I think the songs will end up much more feel orientated. In the recording sessions occasionally there’s a certain amount of Bad Seeds-by-numbers type thing going on, and we can’t kind of help ourselves in a way, and I’m trying to do something about that with the next record. I’ll get Blixa Bargeld to play the drums or something.”

It didn’t go that far, but now that “next record” has been written, recorded and released, it becomes clearer what the Cave was on about. Back at the Kensington hotel, he muses over the lessons learned since The Boatman’s Call:

“It was very important for me to record all of the songs playing an instrument, so that everything you hear on the record is the original take, the original vocal and piano playing, with very little doctoring of it afterwards. I really wanted to get away from the idea of writing a song, handing it over to the band and then me coming in and singing over the top. I just wanted to be in the guts of the music.”

Following various demo recordings, including some in Melbourne, the album sessions proper began last autumn at Abbey Road in London, with a string section and the McGarrigles supplementing the usual eight-piece Bad Seeds line-up. Mere months before recording commenced, Mick Harvey, Warren Ellis and Cave himself had all become new fathers, with the singer siring twin boys by actress and model Susie Bicks, whom he had married in October of 1999.

As interview number 70 is due shortly, I abandon my conversational approach and try some straightforward questions…

To what can we attribute last year’s frenzied bout of procreation in the Bad Seeds’ camp?

I have absolutely no idea, but it was a happy coincidence all round. It was a kind of baby boom.

Writing a song about your children is the last taboo in rock‘n’roll. It’s acceptable to document marriage and relationships so long as you also address their deterioration in forensic detail, but singing about your kids is seen as the preserve of mawkish country songs.

Yes, you’re destined to fail, and then deserve to. But I actually did do that, y’know. I wrote a song called ‘Sheep May Safely Graze’, which was a song to my son that I wrote for The Boatman’s Call. And I played it and Blixa took me outside and put his arm around me and said, “Look, that’s a very sweet song, I suggest you give that a present to your son when he’s a few years older and spare the world!”

That was quite a brave thing for Blixa to do.

He’s a brave man. And I’m very happy he did that.

The songs on the new record retain a solitary, meditative air, yet presumably in everyday life you’re surrounded by the din of wife and kin. Is that a contradiction, or can the two co-exist?

Well, first of all, I like this record because I think it says a lot about me, and you aren’t really required to know the subtext of my life to enjoy these songs, or at least they live and die without the need for that added information, which is perhaps one of the criticisms I’d make of The Boatman’s Call. But y’know, I do have two places of existence these days, one is my office and one is my home, and a quick kind of dash in between in the car, and they are incredibly separate from each other. I don’t do anything with my family in the office or the other way around. Now I go to the office from 9 o’clock ’til 6 o’clock five days a week and work.

It’s interesting that such a traditional structure can improve both areas of your life.

Well it does, y’know? They’re these two alternate worlds that I enter, that to me seem very separate from the outside world, and I like both those places. For a long time everything just leaked into everything else and it was just kind of confusing, and it just seems a little more clear these days.

In the Secret Life lecture you suggest that the creative impulses in us sometimes orchestrate catastrophes in our lives.

It read nicely, I thought! The question (being), do I still feel that way?

Yeah, do you have to sacrifice a certain amount of happiness in order to come up with good work?

Well I think in a lot of ways, if you don’t have a lot of happiness, sacrifice is a very easy thing to do. And because I very much value what’s going on in my family life, the kids and my wife, I feel I don’t want to be reckless with that, and I’ve had to separate it, because the creative process is very reckless, and it’s very self-absorbing, self-obsessive, and it is by nature chaotic – whether I work in an office or not has got nothing to do with what goes on in my head when I’m writing. And I think to protect both those environments, the workspace and my family life, I had to separate them, y’know, like two children who don’t get on, you put ’em in different rooms.

There are several wry references in the songs to the contract of marriage, the ring locked upon the finger, the wife’s friends awarding her courage with gifts. Did getting married alter your perception of yourself; did you take on the role of husband with any great gravity?

Yeah, I think I did. Y’know, to me marriage was just an enormous relief; it was just like, “Thank God for that”. I can’t even think of a relief from what, but that was the overriding feeling. And we love each other. I don’t know about my role of husband, but we’re just very happily married together and she likes me just the way I am, she doesn’t want me to be any different, doesn’t want me to change, doesn’t want me to fucking “shape up” and all that sort of stuff (Laughs). Hallelujah, that’s all I can say. Praise the lord! I’ve got a wife that just likes me. The way I am. Something like that. You can edit that!

Do you ever fictionalise details in the songs, in order to obfuscate or deflect attention from their autobiographical nature?

I’m not sure that I do actually.

Does that get you into trouble?

At home?

Yeah.

With the missus?

Yeah.

(Laughs) No. Well, I think she did very, very well on this record. Comparatively. To some others. I think she’s very happy about it. I think Susie, who is intensely private, worries sometimes that I’m too candid and values certain things that have happened between us, and they’re just between us, and they don’t have to be on public record. And I guess I’ve become more conscious of that. But at the same time I don’t feel I have much control over what I write and it just… often songs aren’t ever supposed to see the light of day, they’re just a line that you write and it’s quite naked, and then you write another line and it rhymes, and eventually you’ve got a song and then you put a bit of music to it. And if I’ve to put out a record, suddenly you think, “Oh shit!” “I married my wife on the day of the eclipse” or something like that, which is true… They just get out of control. Do you understand what I mean?

Yeah, I do. It never actually sinks in that it’s going into the public arena…

…until it’s there and you’re having to answer questions about it. And not only that, it’s there forever. Which is even more frightening.

The new album seems to reach a détente with the idea of happiness, or at least contentment. There are lines like “I’ll never be free/If I’m not free now” and “It seems we can be happy now/Be it better late than never”.

They sounded pretty good in an Irish accent.

Presumably you would reject the notion of contentment being the enemy of art.

Well complacency I would say is, but I do think that they’re different things. I’d worry if this thing that you were writing was some kind of: “This is how it is now and it used to be so bad back then”, because I’ve never had that view of things really, and I never trust those kind of interviews. It’s the beginning of the end.

No, I’m not chasing that angle. That’s a kind of fallacy of permanence, the idea that you reach a happy end and that’s it.

I have to agree with you. But I do think that the notion that you have to destroy yourself to create is highly overrated. I just don’t think it’s true. The abyss is overrated. It’s only two feet deep! Jump in and you’ll find it’s knee-high! You know, I’m aware that the idea of going into the office every day and working from nine o clock ’til six o clock, you’re not gonna be able to get a hell of a lot of good rock ‘n’ roll stories out of that, but fuck it, y’know, I’m 43 years old and I don’t feel I need to sort of… I just feel clear about what I want and what makes me happy, and I see no reason why I shouldn’t be allowed to pursue that. What is it the Americans say? The pursuit of happiness is our god given right.

Cave’s wariness at any death ‘n’ drugs-to-domesticity story is understandable enough: life just doesn’t stick to those lines. Equally, the idea of art out of chaos is a romantic conceit, one that conveniently shirks the degree of labour involved in bringing any artistic enterprise to fruition. Rock ‘n’ rollers – and their biographers – frequently favour dubious interpretations of Blake’s proverbs of hell over Flaubert’s belief that “you should be regular and ordinary in your life, that you may be violent and original in your work.” Similarly, CS Lewis once observed that the muse is most likely to come calling when you’re sitting at your desk working.

And Nick Cave’s last couple of years of work have yielded some strange fruit. If the prevailing image of The Boatman’s Call was of a guy sitting in a church, the smell of his lover still on his hands, looking sidelong at the stone idols and saying, well, it’s alright for them, then the new album is a different story. And No More Shall We Part opens with that same man sitting by a window with his lover, observing the still life of the street, reflecting on the human condition.

But despite the confessional nature of his work, anyone looking to read the runes of the singer’s private life in his songs can get thrown badly off track. Cavespotters swarmed all over ‘Do You Love Me Part II’ when it was released, reading into it a history of child abuse, when its author had merely appropriated ideas from Peter Straub’s short story ‘The Juniper Tree’.

The new album is a similarly tangled weave of autobiography and imaginative construct. The push-pull currents of ‘15 Feet Of Pure White Snow’ and the wintry melancholia of ‘The Sorrowful Wife’ herald not just the Bad Seeds most intense utilisation of dynamics since ‘The Mercy Seat’, but also a series of lyrical set-pieces. ‘Oh My Lord’ for instance, is a sequence of sprawling paranoid fantasies, one of which freeze-frames the narrator crawling around the floor of a hairdresser’s looking for the plot, while a guy in plastic antlers presses his bum against the glass.

One of my favourite parts of the new record is the comic-horror paranoiac episode in the hairdresser’s in ‘Oh My Lord’.

I like that too.

I don’t need you to dismantle the song, but I’m interested to know…

Which hairdresser?! That guy there! Well, I find hairdressers to be terrifying places. And it isn’t… this happened. Well, a lot of this happened. There is a particular hairdresser’s and it’s all windows and it’s in a very public area, and in all levels I think I found that to be one of the most painful places to ever be, and I just threw it in. What can I say? Y’know, hairdressers hangin’ around, scissors, everybody being able to look in, English people walking past, little bits of hair on the ground, mirrors everywhere, they’re terrifying places!

‘As I Sat Sadly By Her Side’, is a dialogue instead of a monologue, looking outward instead of in.

It took a long time lot to write that song, a lot of work. But I really enjoyed doing it, and I really liked the dialogue. Dialogue in songwriting, I mean it’s never been done better than ‘Isis’ or something like that by Bob Dylan, which is hilarious. In fact, just to fly off somewhere else, I’m enjoying dialogue more and more. I’ve just finished a film script for Johnny Hillcoat, he asked me to write a film script on a particular theme, which I’m not supposed to say what it is. And I said, “Look, I’ll write the story, but you get someone else to write the dialogue”, but once I started I found I really enjoyed that, just people saying things without revealing their true motives for saying them. So a book is on the way I reckon. I thought if I could write this fucking film script that came so easily, I don’t see why can’t I just bang out another novel. I haven’t done anything towards it yet, but the fact that I could say that in an interview . . .

In ‘God Is In The House’, there’s the suggestion of some empathy for the people you’re satirising, both the God-fearing white picket fence community and the city dwellers they fear so much, the “drug fiends in the crack house”.

It’s light and it’s comic I think, and it raised a kind of chuckle with some of the members of the group who indulge in that sort of thing. And that seemed like good enough reason to put it on it for me. I think there’s a certain depth underneath, I think there’s a kind of cry of woe towards the end with, “I wish he’d come out” which I always feel is my voice, but essentially I look at it as a comic song really.

There’s been some debate in the office as to who coined that word “teetotalitarianists”.

I did.

Actually I did.

What?

I came up with it in 1997. But then, my colleague Liam Mackey reckons he invented it about ten years earlier.

Well I came up with that entirely on my own. I’m the first person to put my hands up to ripping off a line here and there, but I’m sorry. You might’ve come up with it before me but… it’s mine!

‘Darker With The Day’ contains a tribute to “friends who had died of exposure” and other ones “who had died of the lack of it”. You and Bono had a mutual friend in Michael Hutchence. Have you heard U2’s ‘Stuck In A Moment You Can’t Get Out Of’?

I did hear it once on the radio. I remember thinking it was pretty kind of catchy, but I wasn’t concentrating on it. It sounded like it had a good melody. But I couldn’t comment on it really. It would be unfair.

One of the new album’s most dramatic moments occurs during ‘The Sorrowful Wife’, where she appears to be ruminating on some awful event, possibly an infidelity. The repeated piano notes seem to mimic her counting out the days, trying to work out the details. Then you and the band come in with this bellowing noise…

That’s probably best left obfuscated, or whatever the word is. I’m sure that song will reveal itself to me in the future. Sometimes they’re prophetic, my songs, I think.

That’s quite a frightening idea.

Yes, I know!

So, here we leave it. Perhaps these new songs do have, encoded in their genes, inklings of what’s in store for Nick Cave. Certainly, his best records tend to bloom over a long period, a kind of perfumed repenting at leisure. And No More Shall We Part is no different in that respect.

Uncork the fucker, and let it breathe.

And No More Shall We Part is out now on Mute. The second single off the album is ‘Fifteen Feet Of Pure White Snow’ and it’s out this week.

The Secret Life Of The Love Song/The Flesh Made Word – Two Lectures By Nick Cave is available through King Mob records.