Why are people afraid of ghosts? 'Ooh, no, I wouldn't want...

Why are people afraid of ghosts? 'Ooh, no, I wouldn't want to see one! I'd be too scared' - accompanied by a tremolo of fear in the voice - is the common reaction. This puzzles me. I'd think anyone would welcome he opportunity. I've never heard of a ghost hurting anybody.