Probability is expectation founded upon partial knowledge. A perfect acquaintance with all the circumstances affecting the occurrence of an event would change expectation into certainty, and leave nether room nor demand for a theory of probabilities.
In no other branch of mathematics is it so easy for experts to blunder as in probability theory.
Maybe we should teach schoolchildren probability theory and investment risk management.
One of the most striking and fundamental things about probability theory is that it leads to an understanding of the otherwise strange fact that events which are individually capricious and unpredictable can, when treated en masse, lead to very stable average performances.
Entropy theory, on the other hand, is not concerned with the probability of succession in a series of items but with the overall distribution of kinds of items in a given arrangement.