A debut is a compilation of everything a musician has ever had to say, assembled over years without a deadline. A second record is what one person can make in fourteen months while being asked how the second record is going.
We judge these two objects by the same standard and then express surprise at the result.
The pattern in the disappointments
Go back to the records that were received as failures and a consistent shape appears: they are less immediate, more specific, and structurally stranger. They lose the audience exactly where the artist stops repeating themselves.
That is not always good. Ambition is not achievement. But a catalogue of second albums is a better map of what an artist actually cared about than any run of hits.