On 17 April 1953, Starkey's mother married Harry Graves, an ex-Londoner who had moved to Liverpool following the failure of his first marriage. Graves, an impassioned fan of big band music and their vocalists, introduced Starkey to recordings by Dinah Shore, Sarah Vaughan and Billy Daniels. Graves stated that he and "Ritchie" never had an unpleasant exchange between them; Starkey later commented: "He was great ... I learned gentleness from Harry." After the extended hospital stay following Starkey's recovery from tuberculosis, he did not return to school, preferring instead to stay at home and listen to music while playing along by beating biscuit tins with sticks.
Beatles biographer Bob Spitz described Starkey's upbringing as "a Dickensian chronicle of misfortune". Houses in the area were "poorly ventilated, postage-stamp-sized ... patched together by crumbling plaster walls, with a rear door that opened onto an outhouse." Crawford commented: "Like all of the families who lived in the Dingle, he was part of an ongoing struggle to survive."The children who lived there spent much of their time at Princes Park, escaping the soot-filled air of their coal-fuelled neighbourhood. Adding to their difficult circumstances, violent crime was an almost constant concern for people living in one of the oldest and poorest inner-city districts in Liverpool. Starkey later commented: "You kept your head down, your eyes open, and you didn't get in anybody's way."
After his return home from the sanatorium in late 1955, Starkey entered the workforce but was lacking in motivation and discipline; his initial attempts at gainful employment proved unsuccessful. In an effort to secure himself some warm clothes, he briefly held a railway worker's job, which came with an employer-issued suit. He was supplied with a hat but no uniform and, unable to pass the physical examination, he was laid off and granted unemployment benefits. He then found work as a waiter serving drinks on a day boat that travelled from Liverpool to North Wales, but his fear of conscription into military service led him to quit the job, not wanting to give the Royal Navy the impression that he was suitable for seafaring work. In mid-1956, Graves secured Starkey a position as an apprentice machinist at a Liverpool equipment manufacturer. While working at the facility Starkey befriended Roy Trafford, and the two bonded over their shared interest in music. Trafford introduced Starkey to skiffle, and he quickly became a fervent admirer.