The time was the summer of 1968. The Stones were topping the charts with a return to R'n'B via 'Jumpin' Jack Flash', Dylan had rediscovered folk with 'John Wesley Harding' and the music papers were burning up with the news that Elvis, recording a comeback special in L.A., also seemed intent on hauling rock'n'roll back to its roots.

Scott Walker, however, comfortably placed in key positions in Britain's singles and album charts, was looking back much further through the history of music. "I am not competing with pop writers," he said while recording Scott 3. "What I do is polish up the antique and bring something fresh to the work. I feel I must become highly stylised in my own way. A highly stylised composer would be Rawsthorne, Gerhard or Shostakovich."

Walker also revealed that he was "listening to classical music more than ever and it is this influence which permeates the work that follows." At the core of Scott 3, released in March 1969, is a song-cycle containing seven compositions by Scott. The other songs were three by Brel - the last he'd be recording, he explained because “He doesn't write good melodics and because I now am just as concerned with music as well as lyrics it is impossible for me to record any more of Brel's material."

He was similarly critical of Bob Dylan, "The thing wrong with most popular music," he declared, "is that you get someone like Dylan who writes marvellous lyrics and fraudulent melodies - just diabolical - then you get another type of writer who produces great melodies and bad lyrics. What we're trying to do is combine the two, with the use of an orchestra, and make them whole."

The egalitarian 'we' refers to Scott's long-time producer Johnny Franz, and the two hugely influential producers he'd used since going solo: Peter Knight and Wally Stott. Working with Wally Stott on Scott 3 was, said Walker, "like having Delius writing for you."

Certainly, in relation to the song-cycle on Scott 3, a unified whole is precisely what the team achieved. In line with ideals set down by Franz Schubert, the poetry, vocal melodies and orchestral accompaniment here are so thoroughly integrated that the words interpret the meaning of the music just as the music interprets the inner sense of the words, while the harmonies of the accompaniment add the necessary accents of light, shade and colour. It is truly other-worldly.

This is clear right from the opening track 'It's Raining Today' which softly sets the song-cycle in the limbo of retrospection. Similarly 'Butterfly' opens like Satie's 'Trio Gymnopedies' before floating into that lace-like world inhabited by composers such as Debussy. In keeping with Debussy's innovative experimentation in his opera 'Pelleas Et Melisande', the text of the song is faithful to natural patterns of speech and Walker delivers it accordingly. Here was a precursor of the form of "sprechgesang" (speech-song) which later came to dominate his work.

The attempt to 'polish up the antique' is also evident in Walker's exploration of contemporary themes and his use of lyrics that shift from the concrete to the impressionistic to the surreal. Striving to evoke the kind of fragmented perspectives that are the hallmark of Modern Art, Walker also abandoned set stanza forms and end-of-line rhyme schemes. Altering the rhythmical emphasis of the music as often as is dictated by changes on a thematic level he firmly sets these songs near their antecedents in art-song, rather than popular culture.

All of Scott Walker's aesthetic goals were realised and even surpassed, in 'Big Louise'. "She stands all alone/You can hear her hum softly From her fire escape in the sky/She fills the bags 'neath her eyes With the moonbeams/And cries/'cause the world's passed her by". It is a song-poem that could so easily have slipped from the doomed lips of Blanche Du Bois, Tennessee Williams female alter-ego in "A Streetcar Named Desire". Later, Scott revealed that it was written for "an ageing homosexual" but it could have been about any ageing romantic, the success the song finally proving that Walker writes from that artistic vantage point which transcends gender.

Similarly, 'Rosemary' reveals how he now was able to fully inhabit the female psyche, or the feminine side of his own nature, without instinctively calling forth the misogyny that mars many of Brel’s songs.

Lyrically too, the spectre of Brel's influence is finally laid to rest by the album's battle-anthem, the suitably ironic 'We Came Through'.
"Like the gothic monsters perched on Notre Dame/We observe the naked souls of gutters/Pouring forth mankind/Smothered in an avalanche of time" he sings, in a merciless distillation of his own existentialism set to song.

With its narrow emotional ambit and over-riding classical base, it was inevitable that Scott 3 would sit uneasily near the top of Britain's album charts, alongside The Best Of The Seekers. So, pragmatist as well as poet, Walker immediately released a selection of standards, called Songs From His TV Series, which proved to be his most commercial album to date. He also recorded a single 'The Lights Of Cincinnati' that unmitigated slush. To loud accusations that he was selling out, he retorted, "I need the money." That same need presumably led to one of the most embarrassing, and unprofessional moments in his career when he was booed off the stage in Blackpool.

According to a music paper report: "An estimated two hundred patrons, described as mostly middle-aged were undoubtedly dissatisfied and after booing Scott Walker's singing of the same song twice they demanded their money back." Scott subsequently explained that he had taken some “prescription pills" and that these "mixed with a drink or two" had led to his confusion. But no doubt he also had offended his audience with this nice line in diplomacy. "I am now going to sing a medley of my two hits of the past year. I am joining them together because they are boring for me and they won't take too long".

Boring to him they must have been, because what the audience didn't know was that only two weeks before, in Portugal, Scott had completed eight of the most cerebral, complex and perfectly realised compositions of his career. It was these he included on his first fully self-composed song-cycle Scott 4, his meisterwork.

With the release of Scott 4, Walker clearly chose to risk incurring further wrath from the likes of his Blackpool audience and all those fans who would limit him to the role of that heart-throb from the Walker Brothers or ‘The guy who sings Johanna’. Even the record's sleeve makes savagely clear the seriousness of his intentions for this album. It was illustrated with workman-like studies of Walker in the studio, juxtaposed against telling images of Cervantes' windmill, a war-torn soldier, a scene from an Ingmar Bergman movie, and portraits of Stalin and Albert Camus. A seminal quote from the latter also adorned its cover and all advertisements for Scott 4.

“A man's work is nothing but this slow trek to rediscover through the detours of art, those two or three great and simple images in whose presence his heart first opened."

If the previous album had been, primarily, a voyage of discovery through the hallowed halls of the history of music then Scott 4 clearly was a voyage of recovery, a trek through the hollowed shell of one's self in search of a true artistic base and core identity. And where better to start than by furrowing into the bowels of that fundamental fear which limits our lives from the moment we start breathing the fear of death?

“The Seventh Seal" is Walker's telescopic version of the Ingmar Bergman movie, which relates the tale of a knight whose sole crusade was a search for God. After watching a "young girl on a stake/Her face framed in flame" and hearing her cry "I am not a witch/God knows my name", the knight runs to a misty church to try to feel "God's breath" yet finds instead "that the face inside the booth is mister death". They enter into conversation, the knight declaring, "My life's a vain pursuit of meaningless lives/Why can't God touch me with a sign/Perhaps there's no one there/Answered the booth/And death hid within his cloak and smiles." In time they meet again and play chess for the soul of the knight. Death wins. God doesn't arrive. All is nothingness.

Proof that Walker was wholly conversant with his source can also be gleaned from the fact that in this folk-ballad he uses mostly lines culled directly from Bergman's script. In this, he was being true to the aesthetic principle defined when he'd said: "I am a formalist. I believe that an artist must work within walls, within boundaries. Greater originality emerges from this rather than from large sporadic leaps." It was an aesthetic distinctly at odds with the form-less and frequently masturbatory nature of art in the '60s, particularly so-called progressive rock-but, as evidenced by ‘The Seventh Seal’, it was an aesthetic that worked convincingly.

Though almost equally accomplished 'On Your Own Again' and 'The World's Strongest Man' both hark back to the decidedly romantic paradigm used on Scott 3. Yet if the musical base (here and throughout the album) is predominantly classical, with a deepening and wholly appropriate reliance on the kind of modal lines used by the likes of Bartok and Schoenberg, there is also evidence of a developing bias back in the direction of pop/rock. This was not entirely surprising since, prior to the recording of Scott 4, when asked about the relative failure of his previous album, Walker had said "I was getting too musical in ways, too technical. A lot of the lines went on too long.

People who haven't listened to so much music aren't ready for that." While stressing that classical music was still his major influence he explained that he was "getting more into country/rock" such as The Band and Dion, describing the latter's album Abraham, Martin And John as "the best pop album I've heard". Its influence in Scott 4 is evident not only in Walker's occasional and uncharacteristically loose rock scat vocals but, more explicitly in 'Angels Of Ashes' which reads "like a reply to a track from Dion's album, 'Sisters Of Mercy' by Leonard Cohen." Yet whereas Cohen's song is, in a central sense, a paean to two sisters who loved him without sex, Walker eulogises all women and men who give over their flesh to help another person to find their "passions, again and again" - sexual or otherwise.

Lovers reaching through each other to try and recover lost passions and roots, and thus rise above a seemingly unceasing night, is also a theme explored in the labyrinthine 'Boychild' - the definitive Scott Walker song. Here the narrator takes the boy who rides upon his back and pushes him through "forgotten courtyards" and "mirrors dark and blessed with cracks" towards a suggestion of dawn; the last symbol of that lost state of original innocence. "Go seek the lady/Who will give not take away/Naked with stillness/On the edge of dawn she stays/Night starts to empty/That's when her song begins/She'll make you happy/She'll take you deep within". However, the promise of light in a Scott Walker landscape is always tentative- and fleeting. And in the end, despite the positive assertion in the lyrics' second last line, the fear remains that Walker's earlier requiem for an ageing homosexual has now become a requiem for a boy damned to endless darkness from the moment he was conceived. There may, after all, be no lost state of original innocence to recapture. "Love catch these fragments/Swirling through the winds of night/What can it cost to give a boychild back his sight? Extensions through dimension/Leave you feeling cold and lame/Boychild mustn't tremble/Because he came without a name Boychild" also marks a musical turning point for Walker.

Where once his melodies formed, primarily, a harmonic echo for the lyric (as in 'The Bridge'), here, by setting his own liquid vibrato against a fractured, Schoenberg-like counter- melody his goal is one of maintaining the opposition of conflicting elements within a song rather than seeking synthesis. It is this musical sub-text which, finally, underscores the irreconcilable sense of dislocation which dominates 'Boychild'. This single realisation -that all the old harmonic/linguistic structures are at odds with such a text, and, even perhaps, with the world in which it is set - was to be the most far-reaching musical discovery of Scott Walker's career. It later led to another similar masterpiece, the Climate Of Hunter album.

In the second section of Scott 4, Walker turned from songs based primarily on self-concern and/or sexual politics to political commentary in the broadest sense. Yet tackling the broad base did not make Scott Walker forsake his belief in the universality of the particular.

Thus, in the highly satirical 'Hero Of The War', by honing in, via a merciless Bergmanesque close-up, on the effects the Vietnam war has on one veteran, he also highlights the effects of any veteran, of any war, political, domestic and/or psychological.

His political tour-de-force, however, is a slouching beast of a song, dedicated to a "neo-Stalinist regime" called 'The Old Man's Back Again'. "I seen a hand/I seen a vision/It was reaching through the clouds/To risk a dream/A shadow crossed the sky/And it crushed the ground/Just like a beast/The old man's back again”.

Yet if Walker is critical of Russian intervention in Czechoslovakia in 1968 'The Old Man's Back Again' should not be seen purely as a polemic in support of Dubcek and the freedom fighters who were crushed during that short-lived Prague Spring. The final verse may focus on a lone soldier, but the soldier is Russian. "He's standing in the rain/For him there's no old man To walk behind Devoured by his pain Bewildered by the faces/Who pass him by/He'd like another name/The one he's got is a curse/These people cry/Why can't they understand His mother called him Ivan And then she died".

One of the few occasions when Scott Walker was asked to define himself politically, specifically in relation to this subject, he said: "Both sides were wrong. I don't like some of the things that are happening in Russia at the moment -it's neo-Stalinism...The Czechs, on the other hand, went about things the wrong way for a socialist state. What was done to them was cruel but probably necessary in order to prevent the break-up of something for which so many people have worked so hard and suffered for so long. Socialism will not happen if there are dissidents within their own ranks, and neither will it happen if there is a dictatorship in the Soviet Union." Clearly, the collectivist emphasis of his socialism was intact. Scott 4 ends with three songs which take their author even deeper into the areas he'd explored on this and previous albums. There is the enigmatic Duchess', in which all linear and literal language fragments into a cubist text which could be addressed equally to a country, a woman, a man, or ultimately, one's self. A Sartre-based rejection of past selves and a plea for a new beginning, however, is the theme of the rock-based 'Get Behind Me'. And the song-cycle ends, fittingly and, as it transpired, prophetically with 'Rhymes Of Goodbye', "The rhymes of our passions Find beauty in loving love/The rhyme of our madness Burns cities in push and shove/And roaring through darkness The night children fly/I still hear them singing/The rhymes of goodbye".

With Scott 4, Walker soared to Olympian heights in terms of sustained, awe-inspiring creativity in the process leaping light years ahead of his peers in popular music. By digging deeply into the gutted shell of his many selves and fearlessly laying his sexuality, politics and philosophy on the line he also succeeded in producing, for the first time, a single, coherent artistic unit which, though transcendent, was indelibly stamped with his own name. If markedly fatalistic, it was nevertheless a violent death-defying and passionate assertion of the right to live, albeit in shadows. Or perhaps, especially in shadows. In short, on Scott 4 he'd found and reformed his core identity. Perhaps that's why he insisted that the album be credited to Noel Scott Engel and that nowhere on either the record or its sleeve should there be any mention of his previous persona: Scott Walker.

Needless to say, his Blackpool audience were not impressed! On the contrary they, and the bulk of his erstwhile admirers, on finally discovering that the true heart of their hero throbbed a God-less black, missed the glory in such an admittance and took flight. Sadly, the rock audience, still intent on dismissing him as the MOR wimp of 'Lights Of Cincinnati' infamy didn't want to know either. And so, Scott Walker's artistic rebirth resulted in commercial suicide. Scott 4 was his first unmitigated flop. Not surprisingly, because of the nature of the work, Walker took the rejection on an intensely personal level he was, it was said, devastated. He'd lost his game of chess. It would be at least a year before he'd surface again. But what was to come next scarcely merits the word surface at all. For if the hallmark of Walker's previous work had been the extent to which he was willing to invest himself in a song, it was a commitment noticeably absent from the suite contained on the 1970 album Till The Band Comes In. He now worked, to conceal rather than reveal himself, as he sang from behind the eyes of a set of disparate characters living in an apartment block.

But then both Walker and his new manager were, they said, eager that he should "escape from the doomy image he'd previously projected". All the new songs on the album were co-credited to ‘Scott Walker/Ady Semel'. And under pressure from his record company he also included relatively more material from other songwriters, a marketing strategy which edged all his own compositions out of the albums that followed, from The Movie-goer (1972) to We Had It All (1974). Looking back on that phase of his career, Scott Walker, the who once had said "money is a monster that offends me," admitted: "I had a new manager and he told me to get a big pad...and suddenly I had all the records I wanted to buy and I thought If they don't want me to write anything, fuck it' so I just sat back and copped money for whatever they wanted me to do."

However, in time he realised that the price he paid for copping-out and selling-out both artistically and politically, was a total loss of self-esteem and self-confidence as a writer. In 1976, after joining the temporarily re-formed Walker Brothers he said I'm completely dwarfed by other people's achievements. Like, what's the point in my even trying, when all that music is out there already? I'll put on Scriabin, or Scarlatti, or Bartok or even Joni and suddenly I'm not working anymore." He also added, rather tragically, and tellingly: "There was one point way back there when I turned back from what I believed in and was never able to get round again." But Noel Scott Engel did 'get round again', though it took him a further eight years. And Climate Of Hunter, which can justifiably be called Scott 5, also flopped commercially but it was a fiercely defiant and uncompromising artistic success for its author. Once more 'Roaring through darkness' he succeeded in soaring even higher than he'd done on the albums Scott 1-4.

Since then he has remained silent and is, reportedly, back studying Art, at a college in London. But, in the end, one abiding thought must dominate all in relation to the recording career of Scott Walker/Engel. Back when he'd just discovered Brel and was proclaiming that his "duty was to uncover hypocrisy" and "open people's eyes to the truth via the medium of songs", he said "I'll fail. Yet at the end of it all, I'll get self-satisfaction. And I won't fail completely. At least a minority, a strong minority, will listen and that's the important thing."

True then. Even more true now.