Kicksecure How to test an OS
Testing OS installation and Post Installation
Overview
[edit]Testing the operating system after the first release cycle (for testers) is an important phase in development that every project producing complex software must go through. The testing phase contains a lot of steps that vary and depend on the project being tested.
Testing the operating system before installation
[edit]Supported Installation Medium
[edit]For Whonix and Kicksecure, we have:
Libvirt
[edit]KVM: .libvirt image installation for KVM (GNU/Linux only; preferably tested on a Debian host and another common host used by users running the latest or near-latest KVM version, such as Arch, Fedora, or NixOS.. etc).
OVA
[edit]VirtualBox: .ova image installation for VirtualBox (priority for Windows users running the latest Windows version and the latest VirtualBox version; then GNU/Linux users running the latest VirtualBox version, e.g. from the VirtualBox repository on Debian or similar case).
Qubes
[edit]- Qubes: From Qubes repository for both KS and Whonix.
ISO
[edit].iso installation should be tested on all supported hypervisors, as it can produce unexpected results e.g a completely black screen when booting the .iso for installation inside Qubes HVM due to a resolution issue [1].
Before and During Installation
[edit]- Download URLs are all functional with proper speed
- Validate checksum and signature verification (including the website instructions for verification).
- Follow the website instructions for the installation (if updated and available to the current version being tested) to check whether everything works the same or needs updating to report later.
- Booting time is normal.
- Legacy VS UEFI boot for the OS.
- Secure Boot and TPM (if supported/used).
For ISO specific
[edit]- No errors, failures, warnings, or unexpected behavior during ISO boot; if any occur, verify whether they are normal/expected.
- Boot into live mode.
- If live mode boots as expected, check:
- Internet connectivity (Static/Manual and DHCP, Wire/Wireless).
- Browser functionality.
- OS can read external flash/hard drives.
- Sound/audio, mic and camera are working.
- Any other expected tools are installed and functional (text editor, office suite, etc).
- If live mode boots as expected, check:
- Boot into installation mode.
- If installation mode boots as expected, check:
- Changing language (all expected options are displayed and functional).
- Changing region/location (all expected options are displayed and functional).
- Changing/adding keyboard layouts (all expected options are displayed and functional).
- Changing number and date format (all expected options are displayed and functional).
- Partitioning is functional (automatic/default and manual).
- Encryption is functional (note: also installing without encryption).
- Other options, if available, such as file system types e.g ext4, Btrfs..etc.
- If installation mode boots as expected, check:
- Check that the installation process is smooth (expected time, errors..).
For hypervisor-specific images
[edit]- The default/expected settings for the virtual machine inside the hypervisor are correct after import.
Post Installation
[edit]- Encryption is functional (.iso specific).
- No errors, failures, warnings, or unexpected behavior during ISO boot; if any occur, verify whether they are normal/expected.
- Make sure to:
- Boot into every available boot menu option (live, user/admin, persistent, etc).
- Test any keys indicated for specific functionality (e.g GRUB options).
- Confirm all expected packages/apps are present and functional.
- The update and upgrade process works (with the expected repositories).
- No unusual messages are recorded in
journalctlfrom boot time until the OS finishes loading. - All project changes pushed to the OS are functional (e.g X app should not display Y message when opened).
- OS can read external flash/hard drives.
- Taking snapshots and rolling back to a previous one is functional (if supported).
- Check that the default open ports match the firewall rules and listening services.
- Sound/audio, mic and camera are working.
- Internet connectivity (Static/Manual and DHCP, Wire/Wireless).
For hypervisor specific
[edit]- Make sure to check (according to the wiki instructions):
- Shared folder is functional.
- GPU passthrough is functional.
Footnotes
[edit]
We believe security software like Kicksecure needs to remain Open Source and independent. Would you help sustain and grow the project? Learn more about our 13 year success story and maybe DONATE!