Although you hadn't asked why, it had less to do with you...

Although you hadn't asked why, it had less to do with you not noticing than with you not wanting to hear the answer.