3. Getting it done

Programming is not an end in itself. We resort to programming to get things done we wouldn’t accomplish otherwise. While it depends strongly on your local context what you want to get done developing software, we focus here on the central aspects of scientific software development and writing code for scientific data processing and analysis.

Goal

Introduction of the central aspects of scientific data processing and analysis: reproducibility, traceability, testability, reliable code. Additionally, the Python scientific software stack – NumPy, SciPy, Matplotlib – gets introduced.

• 3.1. Key aspects of scientific data processing and analysis
  • 3.1.1. Reproducibility
  • 3.1.2. Traceability
  • 3.1.3. Testability
  • 3.1.4. Reliable code
• 3.2. Scripts vs. library
  • 3.2.1. A general decision
  • 3.2.2. The case for libraries
  • 3.2.3. Replaying is not reproducing
  • 3.2.4. Is there some middle ground?
• 3.3. The core: an intellectual model
  • 3.3.1. The large picture: data processing and analysis
  • 3.3.2. A smaller picture: fitting a model to data
  • 3.3.3. An even smaller picture: plotting data
  • 3.3.4. Things that usually do not belong here
  • 3.3.5. Wrap-up
• 3.4. Separation of concerns
  • 3.4.1. The role of scientific thought
  • 3.4.2. A strategy to cope with the inherent complexity
  • 3.4.3. Some applications to scientific software development
• 3.5. The Python scientific software stack

The first four chapters focus on more general aspects, only the last chapter gives an overview of the most important scientific software libraries available for Python. Why all this more general remarks in the part entitled “Getting it done”? Shouldn’t we focus on writing code, rather than spending time discussing how to write code? Frankly speaking, the biggest problem with scientific software is that most people spend much more time writing software than thinking about how and what to write.

Just to make this point once: You only learn how to write scientific software of decent quality by writing scientific software. It is always a bit of “learning by doing”. But as with all complex endeavours, a bit of planning ahead and realising at least to some extend how complex the things really are we are going to do helps with not being utterly surprised afterwards.