The Role of Experimentation in Computer Science

Computing is evolving constantly. New principles are discovered; older principles fall out of use. Experiments play an essential role in science. However, the role of experiments in computer science is still unclear. Questions about the relevance of experiments in computer science attracted little attention until the 1980s. Among researchers in computer science, there is wide support for views of computing as an empirical or experimental science. Empirical science is used to refer to research that relies on observation based collection of data, while experimental refers to a specific kind of research, where experiments are carried out under artificially produced and deliberately controlled, reproducible conditions. The role of experimentation in computing became popular when Feldman and Sutherland published their report entitled “Rejuvenating Experimental Computer Science. ” In my opinion, experimental science is crucial in science and it should be used more often.

To begin with, W. Tichy mentions that experimentation is very important because it can accelerate progress by eliminating fruitless approaches while leading engineering and theory to promising directions. Moreover, with extensive experimentation and analysis, the possibility of a theory being falsified decreases. As Albert Einstein said ‘No amount of experimentation can ever prove me right; a single experiment can prove me wrong’. Furthermore, testing and observations can lead to new and unknown scientific areas. Here goes an example of how experiment of one thing discovered another whole field of research.

On the other hand, there are many researchers who think that experimenting is unnecessary or not practical. The first line of defence against experimentation is that it is too expensive and in order to find the funding, gather the data for the experiments and conduct them it would take a substantial amount of time and money. Thus, scientific progress will be dramatically decreased. Although in some cases this is true, testing is inevitable. In software engineering, for example, when a product goes into the market, it must be tested to verify its expected behaviour as outlined in the software specification. Otherwise, the product will be unreliable and won’t be used.

Another argument against experimentation states that experimenting can be successfully replaced by demonstrations said by Juris Hartmanis in his 1994 Turing Award lecture. I strongly disagree with that opinion. A demonstration aims to prove a methods feasibility and verify that it has practical potential. However, it cannot test all study cases rather than provide an incentive towards further experimenting. In order to test all scenarios, careful analysis including experiments data and replication is required.

The experimental emphasis of many disciplines suggests the importance of the experiment as a basic research tool. It is evident that experimentation has not yet penetrated computer science to the degree it has other areas of science. The empirical finding that less experimentation is employed in computer science than in other sciences suggests an important question: what level ofexperimentation is appropriate in computer science ? In my opinion, we should not accept experiments as a substitute for a more careful and general analysis, unless there really is no way to parameterize the input space suitably.