I often wondered about what is the nature and purpose of science. While researching what was to become the foundations of statistical economics, I stumbled across an author that I hadn’t heard of before, Edwin Jaynes.  His work has become a profound influence on my research and thinking.  In one of his works, he asks, “How shall we best think about Nature and most efficiently predict her behavior, given only our incomplete knowledge?”

This is a profound statement. What I saw in his question a humble acknowledgement of our capability and limitation.  They are not contradictory, but instead complimentary conditions of our existence. I see in his words a way of resolving the conflict between science and religion.

Jaynes was a pioneer of the application of systems evolving to a state of maximum entropy. This is the configuration based on our knowledge of the extensive parameters (pressure, temperature, etc) that we know the least about. This is to what he is referring in the quote: making predictions only using what we know and assuming nothing else.  It is a very humble proposition.

Our knowledge about the world around us revolves around one simple practice–we interact with the world and look for when our actions yield consistent outcomes.  This is the hallmark of human progress.  We study reproducible events. We want to understand how these events occur and turn them to our advantage.  I do not think humanity is alone in this endeavor, just that we are perhaps the most evolved at leveraging this idea.

Science is about reproducibility. We scientists seek to develop theories that consistently explain our observations.  If an experiment/phenomena is not reproducible, then it has less value than one that is.  This random noise, as Jaynes is the scientific garbage that may be another man’s treasure, who based on a different set of information can explain deeper into the noise.  An example of this is the perihelion of Mercury, which was unexplained noise in classical mechanics, but readily explainable in relativistic mechanics.

Just because something is noise doesn’t mean that we can’t someday explain it. Conversely just because we can explain one thing does not mean that we have the best explanation.  With this said, the definition I’ve settled on for what is science is simply that, “Science is the study of reproducible events.”  Science cannot presume more than what we can count.  In this regard it is actually very limited. This is also why mathematics is the language of science.

I think that the above definition of science is adequately weak to be non exclusive while being strong enough to provide a consistent framework for logical discovery.