Graphical User Interfaces (GUIs)

Another topic that often pops up when developing software for scientific data analysis is graphical user interfaces (GUIs). The idea is as simple as valid: a graphical approach with its (direct) visual feedback often tremendously facilitates data handling and analysis.

The “only” problem with creating GUIs is that it adds an entirely different level of complexity on top of the already quite complex task of developing software that meets scientific standards. Furthermore, with GUIs the otherwise linear workflow becomes highly complicated and usually unpredictable, further complicating the task of tracking all individual steps, as required for reproducibility.

Note

The reason to not include GUI programming in the main part of this course is simple: Developing software that meets scientific standards is complicated enough for most scientists, and there are enough new topics to cover and strategies to learn. Those who are interested (or forced) to look deeper into GUI programming will nevertheless find a few starting points here.

Which GUI library to use?

One part of the Zen of Python obviously does not apply to GUI programming: there is clearly more than one rather fundamental way of creating GUIs with Python, and none of them obvious or largely agreed upon. Hence the first question to ask is: which GUI framework or widget toolset shall we use? The fair answer: it depends – on your needs, requirements, context, …

There is a rather nice and current overview of the different GUI libraries available for Python: Which Python GUI library should you use? On the same site you will not only find a very thorough introduction to using Python and Qt, but as well a similarly recent direct comparison of Qt and Tkinter: PyQt vs. Tkinter — Which Should You Choose for Your Next GUI Project?

The scope of the different solutions is tremendously different, hence a direct feature comparison is not really possible. You need to decide between a library, such as Tkinter, and a full-fledged framework, such as Qt. Both have advantages and disadvantages. Generally, if you aim at creating more complex GUIs – and displaying an interactive data table is already quite complex a task from a GUI perspective –, you are much better off using a framework such as Qt. If you “just” want to have a very simple graphical interface that is rather portable and with smaller (in terms of external libraries) dependencies, a GUI library may be a better option.

Important

Regardless of your choice between GUI library and framework, your GUI shall always be an unimportant implementation detail from the perspective of your data processing and analysis routines (search for “clean architecture” if you don’t understand what that means). Notwithstanding, GUI development is highly complex task that can easily outweigh implementing the actual science in terms of development time (and sometimes complexity). And for sure, creating user-friendly GUIs is kind of an art of its own.

From the perspective of scientific software development, a few key criteria seem important when choosing a GUI library or framework:

  • stable, robust, long-term available

    Scientific software, perhaps more than other software, has a long-term perspective. Hence, a library that is neither mature nor reasonably long-term available is usually a bad choice.

  • cross-platform available

    At least the three “major” operating systems (Windows, Linux, macOS) need to be supported. Whether mobile (or web) platforms are a target at all for more complex data processing and analysis software is a discussion of its own.

  • simple yet powerful

    “Simple things should be simple, complex things should be possible.” (Alan Kay) The library or framework should have a realistically shallow learning curve, but not restrict us in dealing with the inherent complexity of scientific data analysis.

Personally, I decided to use Qt for GUI development. Main reasons for this decision were the possibility to create (arbitrarily) complex GUIs supported by a mature GUI framework (not just a widget library), robust and stable cross-platform support, and the availability of well-established tools e.g. for graphically designing GUIs (Qt Designer), besides excellent documentation and a wealth of examples and resources available online.

Here, the next question arises, as there are two Python bindings for the Qt framework: PyQt and PySide. Which one to use? Eventually, it does not really matter, as they are mostly identical in their interfaces and as both provide a rather thin interface to the Qt C++ libraries. For a more elaborate discussion, see PyQt6 vs PySide6 from Martin Fitzpatrick. Here, I opted for PySide6, mainly for two reasons: the more permissive LGPL license and, more importantly, the direct support from the Qt company developing this library. Hence, if you use PySide6, at least for now, the Qt library and its bindings are developed by the same company.

Python and Qt: getting set up

Having decided upon using Qt as GUI framework (!) for Python and using the PySide6 Python package(s), its time to install the necessary requirements. As usually, do this in a dedicated virtual environment. One beauty of using Python virtual environments here: you do not depend on the Qt library version that may be installed on your system. Hence you can go with the latest Qt library regardless of the Linux distribution you are using… However, have in mind that installing Qt will create a virtual environment with an additional size of approximately 530 MB for the Qt libraries and tools only. Therefore, if developing Qt GUIs, you may not want to create an arbitrarily large number of Python virtual environments.

After all, installing Qt bindings for Python as well as all the convenience tools that come with Qt, such as the Qt Designer for graphically designing GUIs using a GUI, simply install the PySide6 package within your Python virtual environment:

pip install PySide6

As mentioned, this will download approximately 250 MB of data and result in a virtual environment with additional 530 MB for the PySide6 libraries alone.

But what about all the Qt tools that come bundled with the Qt framework, such as the Qt Designer? There seems to be some confusion how to actually obtain and use them. It turns out that things are quite simple. From within your virtual environment, call them with the pyside6- prefix. To start the Qt Designer:

pyside6-designer

Similarly, if you would like to use the UIC or RCC tools, the respective commands (from within the Python virtual environment you installed PySide6 into) would be pyside6-uic and pyside6-rcc.

Python and Qt: learning resources

Qt is not a library, but an entire GUI framework, providing you with many valuable tools and strategies for developing complex GUIs. This includes, i.a., signals and slots, models and views, and straight-forward ways to create complex user-defined widgets. Hence, learning Qt can be a bit daunting to start with. However, there are excellent learning resources available online. First of all, Qt and PySide are documented extensively. But to get started, you may have as well a look at the website (and books) of Martin Fitzpatrick that are a good starting point.

As Qt is intrinsically a framework written in C++ and mainly used to develop GUIs for programs written in C/C++, many examples (and the majority of the documentation) are available for C++ code. However, porting these examples to Python is not that difficult after all, and besides of an appendix in Martin Fitzpatrick’s books, there is a tutorial available from the official Qt documentation: Porting a C++ Application to Python

Note

This course will not go into any details about how to develop GUIs using Python and Qt. For this, see the references above, or use the web search engine of your liking. Here, we are more concerned with overall strategies in how to cope with the added complexity of a GUI.

GUI development: strategies

Perhaps the single most important aspect when creating any GUI is to understand: a GUI is an interface, and as such the GUI is an unimportant detail from the viewpoint of the business logic of your application, whatever this application may be. Of course, users of your application will always use an interface to interact with it, be it the signatures of the classes, methods, and functions of your code, be it a command-line interface (CLI), a GUI or whatever. Hence, good interface design is a crucial aspect of software development, and probably one of the aspects of software development that often lacks the attention it deserves. Nevertheless, a good software architecture means that you can easily replace the interface to your application, meaning, e.g. that you can use both, a CLI and a GUI, without having to change anything in the business logic that is at the core of your software.

Important

Always keep the GUI an implementation detail that can be easily replaced/swapped, without affecting the core business logic of your application.

As with many aspects not only in programming, this is mostly a way of thinking about your task and being aware of the separation of concerns. From that, naturally certain overall design strategies will emerge.

Structuring the code: overall project

Generally, Python code that spans more than a (small) module shall be organised as a package. This has a lot of advantages, one of them being that it can be installed as any other Python package (and eventually even published at PyPI). For a general introduction into packaging of Python code and the accompanying directory layout, see the chapter on packaging. For the additional tasks required to make a Python package including a GUI, see the section on packaging below.

Structuring your overall project including a GUI boils down to creating a separate directory within your project source directory containing all the GUI-related code. Such a directory layout separating GUI and rest of the application may look similar to the following:

mypackage/
├── mypackage/
│   ├── gui/
│   │   ├── __init__.py
│   │   └── main.py
│   ├── __init__.py
│   └── module.py
└── setup.py

Whether you name the module containing the code for your main window main.py as in the example above is a matter of taste. But generally, you should get the overall idea. Here, we have actually created a subpackage gui (subpackage due to having an own __init__.py file).

Structuring the code: GUI

The first and most important aspect of structuring the code of your GUI is to separate it from rest of the application, as shown above (creating a subdirectory or even subpackage). Next, for all except of the simplest (and hence trivial) GUIs, separate the different aspects of your GUI at least into separate files, if not subdirectories. One large file is usually hard to maintain. A few general ideas how and what to separate:

  • One file/module per window

  • Separate models and views

  • For Qt: place all *.ui files (from Qt Designer) in a subdirectory (similarly for resource files)

  • Place all additional files (such as graphics, icons, relevant data) in corresponding subdirectories

Generally, this question has been asked (and answered) several times. For some inspiration of how to organise a GUI consisting of a series of sub-windows/panels, the following thread on StackOverflow may be helpful:

There is even a GitHub repository available from the person answering the question in quite some detail, where you may draw some inspiration from:

Another example mentioned as good layout (and rather complex project) is the OpenShot video editor created using Python and Qt: https://github.com/OpenShot/openshot-qt/

Note

The following description is based on first (preliminary) experience with structuring code for developing GUIs using Qt for Python (aka PySide6). Hence it may change in the future without further notice.

Directory layout

Given development of GUIs in Python using Qt for Python (aka PySide6) and following the general advice given above, we may end up with a directory structure for the gui subdirectory of the package similar to what is shown below:

├── gui
│   ├── app.py
│   ├── data
│   │   └── splash.svg
│   ├── __init__.py
│   ├── mainwindow.py
│   ├── Makefile
│   └── ui
│       ├── __init__.py
│       ├── mainwindow.py
│       └── mainwindow.ui
└── __init__.py

A few comments on the structure shown:

  • The main entry point for the GUI is the file app.py in the gui directory. This file contains a main() function that serves as an entrypoint as well. More on this below.

  • Windows are designed using Qt Designer and stored as *.ui files in the ui subdirectory.

  • The Makefile helps with automatically creating Python files from the *.ui files and stores them next to the *.ui files in the ui subdirectory. While technically speaking, one could directly import the *.ui files, having the translation being made “on the fly”, this seems to slow down the startup even for extremely simple GUIs. Hence, they are converted into Python files beforehand, using the pyside6-uic tool.

  • Each window, including the main window, has its own Python module in the gui directory. Here, this is (only) mainwindow.py. Note that this file is separate from the file with the same name located in the ui subdirectory.

  • Data such as the graphics used for the splash screen are located in the data directory. Currently, no resource files are used.

  • __init__.py files are placed in each subdirectory containing Python modules to allow for their import and to make packaging work correctly.

  • As the Python files corresponding to the *.ui files are automatically created and depend fully on the corresponding ui file, they are excluded from version control (with an appropriate entry in the .gitignore file).

To document at least basically how each of the individual crucial files could look like initially, below stubs are presented for gui/app.py and gui/mainwindow.py.

The main GUI module: app.py

Usually, a package will have one main GUI as central user interface, and this GUI should be accessible by a single command via an entrypoint. A rather minimalistic example of a module app.py responsible for providing the entrypoint and starting the main GUI window is shown below:

import sys

from PySide6.QtWidgets import QApplication

from <packagename>.gui.mainwindow import MainWindow


def main():
    app = QApplication(sys.argv)
    window = MainWindow()
    window.show()
    app.exec()


if __name__ == "__main__":
    main()

Here, the crucial part is to import the main window using the full “path” specification, hence replacing <packagename> with the name of your package, to make it work when packaged and distributed. Furthermore, the main() function serves as central entrypoint, defined in the setup.py file (for details, see below).

More complicated GUIs will require some startup time before the window can be displayed. Hence, it has become an established practice to show a splash screen to the user, giving some immediate feedback, that is automatically removed once the GUI is setup and displayed. A slightly more complicated (and complete) example of an app.py module providing such as splash screen and demonstrating how to display messages on the splash screen as well is given below.

import os
import sys

from PySide6.QtCore import Qt
from PySide6.QtGui import QPixmap
from PySide6.QtWidgets import QApplication, QSplashScreen

from <packagename>.gui.mainwindow import MainWindow


def splash_screen():
    pixmap = QPixmap(
        os.path.join(os.path.dirname(__file__), "data", "splash.svg")
    )
    splash = QSplashScreen(pixmap)
    splash.show()
    return splash


def main():
    app = QApplication(sys.argv)
    splash = splash_screen()

    window = MainWindow()
    window.show()

    alignment = Qt.AlignmentFlag.AlignHCenter | Qt.AlignmentFlag.AlignBottom
    splash.showMessage("Loaded main window", alignment=alignment)
    splash.finish(window)

    app.exec()


if __name__ == "__main__":
    main()

Note that we do not use Qt resource files here for the splash image, but rather paths in the file system. This is probably mostly a matter of personal taste. Note, however, that Qt resource files that are converted into Python files will contain the image data translated into ASCII characters (probably base64 encoding), typically at leasat doubling the required file size.

A nice feature of the QPixmap class is to directly render SVG files. Hence, there is no need to create and store an additional derived bitmap file (such as PNG).

The main window module: mainwindow.py

As mentioned above, every window should reside in its own module. This is shown here for the main window and the corresponding mainwindow.py module in the gui directory. Make sure to not confuse this file with the autogenerated file mainwindow.py with the same name, but residing in the ui subdirectory. Actually, the module shown below imports the (auto-generated) class from the latter file:

from PySide6.QtWidgets import QMainWindow

from .ui.mainwindow import Ui_MainWindow


class MainWindow(QMainWindow, Ui_MainWindow):
    def __init__(self):
        super().__init__()
        self.setupUi(self)

While in this particular case, only the window is initialised, typically there will be all the slots defined that make your GUI work. Hence, it is clearly sensible to separate each window into its own module.

Making packaging work: setup.py and MANIFEST.in

In order to have your package including your GUI installable and working correctly, a few additional things need to be taken into account. One is to place an (initially) empty __init__.py file into every subdirectory containing Python files. This signals the pip tool that these directories are subpackages that need to be included in the package build as well.

As soon as you want to add data files, such as the splash image, you usually need to create a file MANIFEST.in in the root directory of your package. In our case, this file can be as simple as shown below:

# Include data
recursive-include <packagename>/gui/data *

Here again, you need to define the full path to the data files, hence replace the string <packagename> with the actual name of your package.

To make the packaging work, extend your setup.py file residing as well in the root directory of your package as follows:

import setuptools


setuptools.setup(
    name='<mypackage>',
    packages=setuptools.find_packages(exclude=('tests', 'docs')),
    entry_points={
        "gui_scripts": [
            "<mycommand> = <mypackage>.gui.app:main"
        ]
    },
    include_package_data=True,
)

Here, replace both <mypackage> and <mypackage> with the name of your package and the name of the command you would like to use to start your main GUI from the command line. And make sure to not include the angle brackets, as they are just used here to denote placeholders that you need to replace with your own content.

Automating things: Makefile

An established way to automate build steps in the unixoid world is to use a Makefile. Here, we use this approach to automate converting the *.ui files containing the description of our GUI windows to Python files, using the pyside6-uic tool. A working Makefile placed in the gui directory could look like this:

UIC = pyside6-uic
UI_DIR := ./ui

UI_FILES := $(shell find $(UI_DIR) -name '*.ui')
UIPY_FILES := $(UI_FILES:.ui=.py)

all: uic

uic: $(UIPY_FILES)

$(UIPY_FILES): $(UI_DIR)/%.py: $(UI_DIR)/%.ui
    $(UIC) $< -o $@

To automatically convert all (changed) *.ui files containing the description of our GUI windows to Python files, simply type

make

in a terminal from within the gui directory, and you are done. Note that in case you would want to use resource files and automatically generate Python files from those files as well, you could simply add another target, similar to what is shown above.

Version control: ignoring auto-builds

Regarding excluding the automatically generated Python files from the *.ui files from version control, add the following lines to your .gitignore file:

# PySide ui->py
**/ui/*.py

!**/__init__.py

The last line ensures the __init__.py file to always be included, as these files are crucial for the package to work properly.

Separating GUI and application

It has been mentioned several times already, but it is too important not to mention it explicitly once more: The GUI is just an implementation detail that you should be able to replace without affecting the core of your application. What does that mean in practice? None of your Python modules containing code of your core application should import any of the modules dealing with the GUI. But of course, the GUI-related Python modules will call the modules covering your core application logic – making sure not to duplicate code available there.

In terms of developing software for scientific data handling, this boils down to: data processing and analysis is not done within the GUI, the GUI always calls external routines. This separation allows for reproducibility, automation, and scriptability. Note that the separation is a necessary, though not sufficient, requirement for the three characteristics just mentioned.

Hence, the GUI does not contain any core logic, but only the logic necessary to interact with the GUI elements. Furthermore, you may need to implement models for the data displayed in/interacted with from within the GUI that are separate from the data models used in the rest of your application. However, again these models call the models available at the core of your application, not the other way round (dependencies always point inwards).

Python and Qt: unit tests

  • How to write unittests for your GUI code?

  • What actually to test?

A few helpful resources, even if none explicitly for Qt6:

From the official Qt documentation (note: there are still many remainders from the original C++ code in there, rendering it at least partly unusable):

Packaging & deploying

Being an interpreted language, deploying Python code to other computers than that it was originally developed on can be a bit tricky, as usually, a Python interpreter and a number of dependencies (read: Python packages) need to be present. In case of GUIs, things are further complicated by the GUI libraries used adding to the requirements.

Generally, there are two things that should be distinguished: creating Python packages that can be distributed and installed in the usual way, e.g. via pip, and creating self-contained standalone files for deploying your Python application to users that do not or cannot want to deal with installing a Python interpreter and alike. The latter can even involve converting the Python code into a compiled language (usually C or C++) and compiling the result into a (platform-dependent) binary. This does not necessarily mean that the application will run faster, but it will definitely be independent of a Python interpreter (and usually startup faster than in case of the usual packaging tools as PyInstaller that first start a small local Python interpreter and only afterwards the actual program).

Creating a Python package

For a more general overview of how to create Python packages, see the chapter on packaging. However, as a GUI typically serves as the main interface of users to your application, you will usually want to create an entrypoint allowing to easily call the GUI (application) from the command line. Furthermore, particularly in case of using Qt for Python and having either or both of *.ui and resource files available that should be converted into Python code, there are further tasks that should be automated as much as possible. In a unixoid setting, a Makefile is perfectly suited for this task, residing either in the project root directory or the GUI (sub-)package directory.

For some ideas on how to create working Python packages with GUIs, including entrypoints for conveniently starting the GUI from the command line, see the section Structuring the code: GUI above.

Creating a standalone executable

While usually for Python projects not containing GUIs standalone versions are less relevant, for GUI-based applications this is an entirely different matter. Here, again a whole list of different tools has been developed with different scope, compatibility and functionality. One fairly widespread way of creating standalone installable files from your Python project is to use PyInstaller. This will, however, not compile your Python code, “just” package it together with a (stripped-down) Python interpreter and all dependencies.

Note

Note that regardless which method you use to create standalone executable files for your Python project, cross-compiling (i.e. compiling on one platform for a different platform) is not possible. What that means: to build an installable, deployable executable for Windows, you need to run PyInstaller (or whatever tool you use) on a Windows machine, and similarly for macOS and Linux.

One interesting aspect particularly in context of Python packages using GUIs as their main user interface is to create a “true” binary executable, i.e. compiling the Python source code via an intermediate step of translating it into C/C++ code. This is the realm of Nuitka and similar tools.

The official Qt documentation has a whole section on deploying your GUI applications created with Qt for Python:

Here, it looks like PySide (aka “Qt for Python”) shines by means of the tooling made available quite recently (with version 6.4): pyside6-deploy is a (thin) convenience wrapper around Nuitka that allows for compiling your entire Python package into a binary.