There are subtle effects of hearing loss on memory and cognitive function in older adults. The effect of expending extra effort comprehending words means there are fewer cognitive resources for higher level comprehension.
This can seem like some age-related memory problem like mild cognitive impairment or Alzheimer's, but it's really a sensory problem.