This was the site of Clegg House, built circa 1865-1866, demolished in 1982 by St. Peter's Lutheran Church so they could put up a parking lot, just like in the song. Read the whole sad, sordid story here.
That's St. Peter's rear elevation on the far side of the lot. Rustic appearance notwithstanding, the church's English Gothic design (Cecil Burgess) dates to the early 1950s. I like to think of it as part of the Great Mid-century Ecclesiastical Architecture Schism — that decade or so when for every church built to look like it predated Tintern Abbey, another one appeared resembling a cement dove from outer space.
At any rate, this is what Clegg House looked like shortly before it was destroyed...
...and this is how Charles Goad depicted it a little over a decade after it was built (1878, sheet 43.) Blue indicates stone construction, which was more of thing back then, especially in Upper Town. Christ Church Cathedral and its rectory occupy the west end of the block.