Derivative-Maker - Debian based Linux Bootable Image Builder

From Kicksecure
< Dev
Jump to navigation Jump to search
Design Previous page: Security Reviews and Feedback Index page: Design Next page: Build Configuration Derivative-Maker - Debian based Linux Bootable Image Builder

Derivative-Makerarchive.org iconarchive.today icon is capable of building Debian Derivatives like Kicksecure and Whonix.

Introduction[edit]

A Linux distribution is - if done professionally - is built by a build script. This means no clicks by mouse and no running of manual commands is permissible.

For example, it would be very much inappropriate to:

  • go to extensions.gnome.orgarchive.org iconarchive.today icon and press the 'Install' button.
  • click File Manager -> Settings -> changing any settings.

This is also a requirement for:

  • consistent builds (custom rebuilds by third-parties, users, other developers resulting in an image with the same properties);
  • maintenance (it would be tedious to re-create new builds very time by manually installing Debian, then running commands to modify it and clicking)
  • It is also a prerequisite for reproducible builds.

See also:

Existing Derivative-Maker Features[edit]

Essential Derivative-Maker Features[edit]

  • Automated Dependencies: Install all necessary build dependencies on the host system automatically.
  • Build Kicksecure VM Images: Create virtual machine images for Kicksecure.
  • Build Whonix VM Images:
    • Whonix-Gateway: Generate Whonix-Gateway virtual machine images.
    • Whonix-Workstation: Generate Whonix-Workstation virtual machine images.
    • Unified Images: Construct a single OVA that includes both Whonix-Gateway and Whonix-Workstation.
    • Whonix Windows Installer: Build an installer for Whonix on Windows systems.
  • Boot Support:
  • Package Downloads: Fetch newer packages from third-party repositories such as The Tor Project APT repository and the VirtualBox APT repository.
  • Tor Browser Integration: Include Tor Browser by default in Whonix-Workstation builds.
  • APT Cache Support: Utilize an APT Cache to accelerate the build process.
  • VirtualBox Image Building: Use --target virtualbox to build VirtualBox OVA images.
    • Custom VirtualBox Settings: Adjust VirtualBox VM settings using commands like VBoxManage modifyvm "$VMNAME" --synthcpu on.
  • KVM Image Building: Employ --target qcow2 to construct KVM images.
    • Archive Creation: Generate xz archives.
    • Libvirt XML Integration: Include libvirt xml files in the xz archives.
  • User Account Setup: Establish a default login with username user and password changeme.
  • Strict Error Handling: Implement strict error handling with comprehensive exit code checks.
  • No Binary Base Boxes Required: Avoid the use of non-deterministic binary base boxes (vagrant VirtualBox images) to prevent depending on the base box being non-malicious.
  • Signature Verification: Ensure all digital software signatures are authenticated.
  • Future-Proofing: Lay the groundwork for creating deterministic images, preparing for when this becomes an achievable goal.
  • Unicode Scan: Scans the source code folder (which includes all packages to be built from source code) for unicode to mitigate Invisible Malicious Unicode Risks.

Non-Essential Derivative-Maker Features[edit]

  • Development Flexibility: Build steps can be executed manually to expedite the development process.
  • Build Customization: Integration of custom build steps is possible.
  • Environment Customization: Ability to create VM images without a desktop environment.
    • Kicksecure options: --flavor kicksecure-xfce, --flavor kicksecure-cli.
    • Whonix equivalents are also available.
  • Application Selection: Build VM images excluding default applications.

Undecided Priority Derivative-Maker Features[edit]

  • Automated Dependency Installation: Installs all required build dependencies on the host system.
  • Architecture Support: --arch parameter for specifying architecture (e.g., --arch amd64, --arch i386, potential arm64 support).
  • Kernel Options: --kernel and --headers parameters for kernel and headers customization (e.g., --kernel linux-image-amd64 --headers linux-headers-amd64).
  • Custom Repository Usage: Optionally (non-default) install derivative packages from a custom remote repository.
  • Interactive Error Handling: Error handler to repeat commands, open a shell, or ignore errors interactively.
  • Target Specification: --target root option for building with physical isolation in mind.
  • Raw Image Creation: --target raw for building raw disk images.
  • ISO Image Building: --target iso for creating ISO images.
  • Package Customization for Virtualization: Install specific packages for different virtualization platforms, like VirtualBox (virtualbox-guest-x11) and KVM (e.g., spice).
  • Combined VirtualBox/KVM Builds: --target virtualbox and --target qcow2 can be built at the same time. [1].
  • Dual Boot Compatibility: Build images supporting both legacy BIOS and EFI booting.
  • Secure Boot Feature: Support for Secure Boot during the boot process.
  • Build Security: Build from a local self-built apt repository instead of a binary remote repository for enhanced security.
  • Source Code Trust: During the build process no contents from kicksecure.com (or whonix.org) are used and no binaries created by the Kicksecure (or Whonix) project are used Builds from Source Code versus Builds including Binary Packages.
  • Verification and Signature: Image digital software signatures. Creation of hash sum verification and GPG signatures for digital software verification.
  • Remote Repository Exclusion: Build images that never had the derivative's remote/binary repository enabled for security.
  • Local Package Installation: Build and install all derivative packages during the derivative image build process.
  • Onion Source Building: Use onion apt sources for enhanced security during the build.
  • Build Stability: Protection mechanisms against unexpected build issues.
    • Uncommitted Changes Check: Option to break or continue the build when uncommitted changes are detected. This is useful to avoid temporary files and other unexpected changes to leak into the image to be build.
    • Tag Compliance: Enforces to build from a git tags during the build, which optionally can be disabled. This is useful to avoid users from accidentally building from arbitrary commit hashes (git head) and then wondering if the build process is broken or why the resulting image is not the version that the user intended to build.
  • Configuration Directory: --confdir /path/to/config/dir to specify a custom configuration directory.
  • Tor Browser Installation Control: --tb none|closed|open to manage Tor Browser installation behavior.
    • none: Do not install Tor Browser.
    • closed: Fail the build if Tor Browser cannot be installed.
    • open: Continue the build even if Tor Browser cannot be installed.
  • Virtual Machine Customization: Custom VM settings during build, which can be adjusted by the user post-build, such as:
    • --vmram 128 for RAM allocation.
    • --vram 12 for video memory allocation.
    • --vmsize 200G for virtual disk size.
  • Build Process Customization: Easy implementation for creating other image types (like raw images).
  • Build Cleanup Command: Command to remove temporary files and/or images post-build.
  • Build Step Skipping: Feature to optionally skip certain build steps.
  • User Freedom: All options and checks are optional as much as possible without requiring to derivative-maker source code modifications.

Repository Caching[edit]

apt-cacher-ng is used by default.

  • Why? To avoid developers re-downloading packages over and over again.
  • Why not make it optional? It's not easy to design the build script in a way that it does not use apt-cacher-ng by default but uses it when using some option. This is because use of apt-cacher-ng requires to use syntax http://HTTPS///archive.org iconarchive.today icon in APT sources used for the build process.

Repository Caching Debugging[edit]

1. AppArmor fix if using Kicksecure or Whonix.

sudo append-once /etc/apparmor.d/abstractions/base.d/kicksecure "/etc/resolv.conf.whonix r,"

sudo aa-enforce /etc/apparmor.d/usr.sbin.apt-cacher-ng

sudo systemctl restart apt-cacher-ng

Fixed in apparmor-profile-dist 3:9.5-1 and above.

2. If using Whonix-Workstation, disable stream isolation.

sudo dummy-dependency --yes --purge uwt

3.

Install package(s) apt-cacher-ng following these instructions

1 Platform specific notice.

2 Update the package lists and upgrade the system.

sudo apt update && sudo apt full-upgrade

3 Install the apt-cacher-ng package(s).

Using apt command line --no-install-recommends option is in most cases optional.

sudo apt install --no-install-recommends apt-cacher-ng

4 Platform specific notice.

  • Kicksecure: No special notice.
  • Kicksecure-Qubes: Shut down Template and restart App Qubes based on it as per Qubes Template Modification.

5 Done.

The procedure of installing package(s) apt-cacher-ng is complete.

4. apt-cacher-ng configuration

echo "AllowUserPorts: 0" | sudo tee -a /etc/apt-cacher-ng/acng.conf

5. apt-cacher-ng restart

sudo systemctl restart apt-cacher-ng

6. Backup /etc/apt/sources.list.d/debian.list.

If existing. Existing only in Kicksecure, Whonix.

Copy /etc/apt/sources.list.d/debian.list to user home folder.

cp /etc/apt/sources.list.d/debian.list ~/

7. Acquire apt-cacher-ng compatible Debian sources lists files.

From derivative-maker source code folder.

  • A) build_sources/debian_stable_current_clearnet.list , or
  • B) build_sources/debian_stable_current_onion.list ,

sudo cp ~/derivative-maker/build_sources/debian_stable_current_clearnet.list /etc/apt/sources.list.d/debian.list

8. APT update command.

sudo env REPO_PROXY="http://127.0.0.1:3142" http_proxy=${REPO_PROXY} https_proxy=${REPO_PROXY} ALL_PROXY=${REPO_PROXY} apt-get -o Acquire::http::Proxy=http://127.0.0.1:3142 -o Acquire::https::Proxy=http://127.0.0.1:3142 -o Acquire::tor::Proxy=http://127.0.0.1:3142 -o APT::Update::Error-Mode=any -o Acquire::Languages=none -o Acquire::IndexTargets::deb::Contents-deb::DefaultEnabled=false -o Apt::Install-Recommends=false -o Acquire::Retries=5 update

9. Check apt-cacher-ng configuration.

The following command outputs apt-cacher-ng configuration excluding comments and duplicate newlines.

sudo cat /etc/apt-cacher-ng/* | grep --invert-match '#\|^$'

Expected as of Debian bookworm: 

CacheDir: /var/cache/apt-cacher-ng
LogDir: /var/log/apt-cacher-ng
SupportDir: /usr/lib/apt-cacher-ng
Remap-secdeb: security.debian.org security.debian.org/debian-security deb.debian.org/debian-security /debian-security cdn-fastly.deb.debian.org/debian-security ; deb.debian.org/debian-security security.debian.org cdn-fastly.deb.debian.org/debian-security
ReportPage: acng-report.html
ExThreshold: 4
FollowIndexFileRemoval: 1
LocalDirs: acng-doc /usr/share/doc/apt-cacher-ng
AllowUserPorts: 0
http://ftp.debian.org/debian/
http://distfiles.gentoo.org/
http://kali.download/kali/
http://http.kali.org/kali/
http://archive.ubuntu.com/ubuntu/
PassThroughPattern: .*

10. Check apt-cacher-ng log.

sudo journalctl --boot --output cat -u apt-cacher-ng

11. apt-cacher-ng cache folder reset.

sudo systemctl stop apt-cacher-ng sudo mkdir --parents /var/cache/apt-cacher-ng sudo find /var/cache/apt-cacher-ng -mindepth 1 -print0 | xargs -0 sudo safe-rm --verbose -r -f sudo chown --recursive apt-cacher-ng:apt-cacher-ng /var/cache/apt-cacher-ng sudo systemctl restart apt-cacher-ng

This script has been added as help-steps/apt-cacher-ng-reset to git master: https://github.com/Kicksecure/derivative-maker/blob/master/help-steps/apt-cacher-ng-resetarchive.org iconarchive.today icon

Custom Packages[edit]

Untested.

Testers only! Warning: This is for testers-only!

For example, use the CLI version and

  • VM builds: For VM builds, see variable flavor_meta_packages_to_install.
  • ISO builds: In the future, we probably make that variable work for ISO builds too.

Use...

  • kicksecure-cli-vm, or
  • kicksecure-cli-host

Example:

flavor_meta_packages_to_install=" kicksecure-cli-vm pkg1 pkg2 pkg3 "

Prepend this variable in front of the derivative-maker build command or use add a Build Variables Change.

Footnotes[edit]

  1. ↑

Design Previous page: Security Reviews and Feedback Index page: Design Next page: Build Configuration

Notification image
Your support makes
all the difference!

We believe security software like Kicksecure needs to remain Open Source and independent. Would you help sustain and grow the project? Learn more about our 13 year success story and maybe DONATE!

Retrieved from "https://www.kicksecure.com/w/index.php?title=Dev/Derivative-Maker&oldid=96976"