API documentation

The following documentation is based on the source code of version 0.9.1 of the debuntu-tools package.

Available modules

debuntu_tools

The top level debuntu_tools module.

debuntu_tools.__version__ = '0.9.1'

The global version number of the debuntu-tools package (a string).

debuntu_tools.start_interactive_shell(context)

Start an interactive shell in the given execution context.

Parameters:context – An execution context created by executor.contexts.

Swallows return code 130 which can be caused by the operator typing Control-C followed by Control-D.

debuntu_tools.kernel_manager

Usage: debuntu-kernel-manager [OPTIONS] – [APT_OPTIONS]

Detect and remove old Linux kernel header, image and modules packages that can be safely removed to conserve disk space and speed up apt-get runs that install or remove kernels.

By default old packages are detected and reported on the command line but no changes are made. To actually remove old packages you need to use the -c, --clean or --remove option. Using the following command you can perform a dry run that shows you what will happen without actually doing it:

$ debuntu-kernel-manager --remove -- --dry-run

The debuntu-kernel-manager program is currently in alpha status, which means a first attempt at a usable program has been published but there are no guarantees about what it actually does. You have been warned :-).

Supported options:

Option Description
-c, --clean, --remove Remove Linux kernel header and/or image packages that are deemed to be safe to remove. The use of this option requires sudo access on the system in order to run the ‘apt-get remove’ command.
-f, --force When more than one Linux kernel meta package is installed the -c, --clean and --remove options will refuse to run apt-get and exit with an error instead. Use the -f or --force option to override this sanity check.
-p, --preserve-count=NUMBER Preserve the NUMBER newest versions of the kernel packages (defaults to 2).
-r, --remote-host=ALIAS Detect and remove old Linux kernel header and image packages on a remote host over SSH. The ALIAS argument gives the SSH alias that should be used to connect to the remote host.
-v, --verbose Increase verbosity (can be repeated).
-q, --quiet Decrease verbosity (can be repeated).
-h, --help Show this message and exit.
debuntu_tools.kernel_manager.REBOOT_REQUIRED_FILE = '/var/run/reboot-required'

The absolute pathname of the file that exists when a system reboot is required (a string).

debuntu_tools.kernel_manager.REBOOT_REQUIRED_PACKAGES_FILE = '/var/run/reboot-required.pkgs'

The absolute pathname of a file with details about why a system reboot is required (a string).

debuntu_tools.kernel_manager.main()

Command line interface for debuntu-kernel-manager.

class debuntu_tools.kernel_manager.KernelPackageManager(**kw)

Python API for automated Linux kernel image package cleanup on Debian based systems.

Here’s an overview of the KernelPackageManager class:

Superclass: PropertyManager
Public methods: cleanup_packages() and render_summary()
Properties: active_kernel_package, active_kernel_release, apt_options, cleanup_command, context, dry_run, force, installed_header_meta_packages, installed_header_packages, installed_image_meta_packages, installed_kernel_packages, installed_modules_packages, installed_package_groups, installed_packages, preserve_count, reboot_required, removable_header_packages, removable_kernel_packages, removable_modules_packages, removable_package_groups, removable_packages and running_newest_kernel

When you initialize a KernelPackageManager object you are required to provide values for the apt_options, context, force and preserve_count properties. You can set the values of the apt_options, context, force and preserve_count properties by passing keyword arguments to the class initializer.

apt_options

A list of strings with command line options to pass to apt-get.

Note

The apt_options property is a required_property. You are required to provide a value for this property by calling the constructor of the class that defines the property with a keyword argument named apt_options (unless a custom constructor is defined, in this case please refer to the documentation of that constructor). You can change the value of this property using normal attribute assignment syntax.

context

An execution context created by executor.contexts.

Note

The context property is a custom_property. You are required to provide a value for this property by calling the constructor of the class that defines the property with a keyword argument named context (unless a custom constructor is defined, in this case please refer to the documentation of that constructor). You can change the value of this property using normal attribute assignment syntax.

force

Whether to continue with removal despite warnings (a boolean, defaults to False).

Note

The force property is a required_property. You are required to provide a value for this property by calling the constructor of the class that defines the property with a keyword argument named force (unless a custom constructor is defined, in this case please refer to the documentation of that constructor). You can change the value of this property using normal attribute assignment syntax.

preserve_count

The number of kernel packages to preserve (an integer, defaults to 2).

Note

The preserve_count property is a required_property. You are required to provide a value for this property by calling the constructor of the class that defines the property with a keyword argument named preserve_count (unless a custom constructor is defined, in this case please refer to the documentation of that constructor). You can change the value of this property using normal attribute assignment syntax.

dry_run

True if cleanup_command performs a dry run, False otherwise.

installed_packages

A dictionary that maps package names (strings) to MaybeKernelPackage objects.

The value of this property is generated by parsing the output of the dpkg --list command.

Note

The installed_packages property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

installed_header_packages

A list of MaybeKernelPackage objects for the installed Linux kernel header packages.

Note

The installed_header_packages property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

installed_kernel_packages

A list of MaybeKernelPackage objects for the installed kernel images.

Note

The installed_kernel_packages property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

installed_modules_packages

A list of MaybeKernelPackage objects for the installed kernel modules packages.

Note

The installed_modules_packages property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

installed_header_meta_packages

A list of MaybeKernelPackage objects with installed meta packages for kernel headers.

Note

The installed_header_meta_packages property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

installed_image_meta_packages

A list of MaybeKernelPackage objects with installed meta packages for kernel images.

Note

The installed_image_meta_packages property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

installed_package_groups

A list of sets with MaybeKernelPackage objects for installed header and kernel packages.

Note

The installed_package_groups property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

active_kernel_release

The output of uname --kernel-release (a string).

Note

The active_kernel_release property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

active_kernel_package

The package name for the running kernel (a string).

Note

The active_kernel_package property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

reboot_required

True if REBOOT_REQUIRED_FILE exists, False otherwise.

Note

The reboot_required property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

removable_package_groups

A list of sets with MaybeKernelPackage objects considered to be removable.

Candidates for removal are selected from installed_package_groups, ignoring active_kernel_package and the newest preserve_count kernel images (minus one when active_kernel_package was ignored).

Note

The removable_package_groups property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

removable_header_packages

A list of MaybeKernelPackage objects for header packages considered to be removable.

Note

The removable_header_packages property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

removable_kernel_packages

A list of MaybeKernelPackage objects for kernel packages considered to be removable.

Note

The removable_kernel_packages property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

removable_modules_packages

A list of MaybeKernelPackage objects for modules packages considered to be removable.

Note

The removable_modules_packages property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

removable_packages

A list of MaybeKernelPackage objects for kernel related packages considered to be removable.

Note

The removable_packages property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

running_newest_kernel

True if the newest kernel is currently active, False otherwise.

Note

The running_newest_kernel property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

cleanup_command

A list of strings with the apt-get command to remove old packages.

Note

The cleanup_command property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

render_summary()

Render a summary of installed and removable kernel packages on the terminal.

cleanup_packages(**options)

Run apt-get to cleanup removable kernel related packages.

Parameters:options – Any keyword arguments are passed on to the execute() method of the context object.
Returns:True if a system reboot is required (to switch to the newest installed kernel image or because security updates have been installed), False otherwise.
Raises:CleanupError when multiple Linux kernel meta packages are installed and force is False.
class debuntu_tools.kernel_manager.MaybeKernelPackage(**kw)

Dumb container for entries parsed from dpkg --list output.

Here’s an overview of the MaybeKernelPackage class:

Superclass: PropertyManager
Properties: is_header_meta_package, is_header_package, is_image_meta_package, is_installed, is_kernel_package, is_modules_package, is_supported_package, kernel_type, name, status, tokenized_name, version and version_in_name

When you initialize a MaybeKernelPackage object you are required to provide values for the name, status and version properties. You can set the values of the name, status and version properties by passing keyword arguments to the class initializer.

name

The name of the package (a string).

Note

The name property is a key_property. You are required to provide a value for this property by calling the constructor of the class that defines the property with a keyword argument named name (unless a custom constructor is defined, in this case please refer to the documentation of that constructor). Once this property has been assigned a value you are not allowed to assign a new value to the property.

version

The version of the package (a Version object).

Note

The version property is a key_property. You are required to provide a value for this property by calling the constructor of the class that defines the property with a keyword argument named version (unless a custom constructor is defined, in this case please refer to the documentation of that constructor). Once this property has been assigned a value you are not allowed to assign a new value to the property.

status

The status of the package (a string of two characters, refer to the dpkg man pages for details).

Note

The status property is a required_property. You are required to provide a value for this property by calling the constructor of the class that defines the property with a keyword argument named status (unless a custom constructor is defined, in this case please refer to the documentation of that constructor). You can change the value of this property using normal attribute assignment syntax.

is_installed

True if the package is installed (or configuration files remain), False otherwise.

tokenized_name

The tokenized name of the package (a list of strings).

The value of this property is computed by calling tokenize_package_name() on name.

Note

The tokenized_name property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

is_header_meta_package

True if the package is a Linux kernel header meta package, False otherwise.

Note

The is_header_meta_package property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

is_image_meta_package

True if the package is a Linux kernel image meta package, False otherwise.

Note

The is_image_meta_package property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

is_header_package

True if the package is a specific version of the Linux kernel headers, False otherwise.

Note

The is_header_package property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

is_kernel_package

True if the package is a specific version of the Linux kernel image, False otherwise.

Note

The is_kernel_package property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

is_modules_package

True if the package contains Linux kernel modules, False otherwise.

Note

The is_modules_package property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

is_supported_package

True if the package concerns a Linux kernel image or modules or headers, False otherwise.

Note

The is_supported_package property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

version_in_name

The version encoded in the name of the package (a string or None).

Note

The version_in_name property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

kernel_type

The kernel type encoded in the name of the package (a string or None).

Note

The kernel_type property is a cached_property. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

debuntu_tools.kernel_manager.tokenize_package_name(package_name)

Split a (kernel) package name into tokens.

Parameters:package_name – The name of a package (a string).
Returns:A list of strings.

The tokenize_package_name() function uses a regular expression to tokenize package names by splitting on dashes while ignoring dashes with a digit on both sides (i.e. dashes that are part of a version number). For example:

>>> from debuntu_tools.kernel_manager import tokenize_package_name
>>> tokenize_package_name('linux-image-4.4.0-72-generic')
['linux', 'image', '4.4.0-72', 'generic']
debuntu_tools.kernel_manager.is_kernel_version(token)

Check whether a token in a package name looks like a kernel version.

Parameters:token – The token to check (a string).
Returns:True if the token looks like a kernel version, False otherwise.

The is_kernel_version() function uses a regular expression to check whether the given string looks like a kernel version of the form 4.4.0-72. The string is assumed to be a package name token returned by tokenize_package_name().

exception debuntu_tools.kernel_manager.CleanupError

Custom exception to detect known problems.

Raised by KernelPackageManager when multiple Linux kernel meta packages are installed but force is False.

debuntu_tools.nodejs_installer

Usage: debuntu-nodejs-installer [OPTIONS]

Install an up to date Node.js binary distribution on a Debian or Ubuntu system by configuring and using the NodeSource binary package repositories.

Due to the time it takes for new software releases to find their way into the Debian and Ubuntu ecosystems versus the speed with which the Node.js community is currently moving, the system packages that provide Node.js are hopelessly out of date. Fortunately the folks at NodeSource maintain Debian and Ubuntu package repositories that provide up to date Node.js binary distributions.

NodeSource makes installation scripts available and the suggested way to run these is to download and pipe them straight to a shell. That kind of rubs me the wrong way :-) but I’ve nevertheless had to set up NodeSource installations a dozen times now. One thing led to another and now there is this program.

Supported options:

Option Description
-i, --install Configure the system to use one of the NodeSource binary package repositories and install the ‘nodejs’ package from the repository.
-V, --version=NODEJS_VERSION

Set the version of Node.js to be installed. You can find a list of available versions on the following web page: https://github.com/nodesource/distributions/

Default: node_10.x (active LTS)
-s, --sources-file=FILENAME

Set the pathname of the ‘package resource list’ that will be added to the system during configuration of the NodeSource binary package repository.

Default: /etc/apt/sources.list.d/nodesource.list
-r, --remote-host=ALIAS Perform the requested action(s) on a remote host over SSH. The ALIAS argument gives the SSH alias that should be used to connect to the remote host.
-v, --verbose Increase verbosity (can be repeated).
-q, --quiet Decrease verbosity (can be repeated).
-h, --help Show this message and exit.
debuntu_tools.nodejs_installer.main()

Command line interface for debuntu-nodejs-installer.

class debuntu_tools.nodejs_installer.NodeInstaller(**kw)

Python API for the Node.js installer.

The logic in this installer is based on the manual installation instructions and setup scripts provided by NodeSource and my experience with system administration of Debian and Ubuntu systems.

The main method of this class is install() and the behavior of this method can be configured by setting the context, nodejs_version and sources_file properties by passing keyword arguments to the class initializer.

Here’s an overview of the NodeInstaller class:

Superclass: PropertyManager
Public methods: install(), install_https_transport(), install_package(), install_signing_key(), install_sources_file(), update_package_lists() and validate_system()
Properties: context, nodejs_version and sources_file

When you initialize a NodeInstaller object you are required to provide a value for the context property. You can set the values of the context, nodejs_version and sources_file properties by passing keyword arguments to the class initializer.

context

An execution context created using executor.contexts.

Note

The context property is a required_property. You are required to provide a value for this property by calling the constructor of the class that defines the property with a keyword argument named context (unless a custom constructor is defined, in this case please refer to the documentation of that constructor). You can change the value of this property using normal attribute assignment syntax.

nodejs_version

The Node.js version to install (a string, defaults to node_10.x).

Note

The nodejs_version property is a mutable_property. You can change the value of this property using normal attribute assignment syntax. To reset it to its default (computed) value you can use del or delattr().

sources_file

The absolute pathname of the ‘package resource list’ used to enable the NodeSource repositories (a string).

Defaults to /etc/apt/sources.list.d/nodesource.list.

Note

The sources_file property is a mutable_property. You can change the value of this property using normal attribute assignment syntax. To reset it to its default (computed) value you can use del or delattr().

install()

Enable the NodeSource repository and install the nodejs package.

Raises:UnsupportedSystemError when validate_system() fails.
validate_system()

Make sure the system is running a supported version of Debian or Ubuntu.

Raises:UnsupportedSystemError when validation fails.
install_signing_key(key_url='https://deb.nodesource.com/gpgkey/nodesource.gpg.key')

Install the signing key for the NodeSource repositories.

install_https_transport()

Install the apt-transport-https system package.

install_sources_file()

Install a ‘package resource list’ that points apt to the NodeSource repository.

update_package_lists()

Run apt-get update (with compensation if things break).

install_package()

Install the Node.js package on a Debian or Ubuntu system.

exception debuntu_tools.nodejs_installer.UnsupportedSystemError

Raised when an unsupported operating system is detected.

debuntu_tools.remote_reboot

Usage: reboot-remote-system [OPTIONS] [SSH_ALIAS]

Reboot a remote system and wait for the system to come back online. If the SSH alias matches a section in the ‘unlock-remote-system’ configuration, the root disk encryption of the remote system will be unlocked after it is rebooted.

Supported options:

Option Description
-s, --shell Start an interactive shell on the remote system after it has finished booting.
-v, --verbose Increase logging verbosity (can be repeated).
-q, --quiet Decrease logging verbosity (can be repeated).
-h, --help Show this message and exit.
debuntu_tools.remote_reboot.main()

Command line interface for reboot-remote-system.

debuntu_tools.remote_reboot.reboot_remote_system(context=None, name=None)

Reboot a remote Linux system (unattended).

Parameters:
  • context – A RemoteContext object (or None).
  • name – The name of the unlock-remote-system configuration section for the remote host (a string) or None.
Raises:

ValueError when the remote system appears to be using root disk encryption but there’s no unlock-remote-system configuration section available. The reasoning behind this is to err on the side of caution when we suspect we won’t be able to get the remote system back online.

This function reboots a remote Linux system, waits for the system to go down and then waits for it to come back up.

If the ssh_alias of the context matches a section in the unlock-remote-system configuration, the root disk encryption of the remote system will be unlocked after it is rebooted.

debuntu_tools.remote_reboot.get_uptime(context, multiplexed=False)

Get the uptime of a remote Linux system by reading /proc/uptime.

Parameters:context – An execution context created by executor.contexts.
Returns:The uptime of the remote system (as a float).

This function is used by reboot_remote_system() to wait until a remote Linux system has successfully rebooted.

The uptime check uses ssh -S none $REMOTE_HOST cat /proc/uptime where -S none opts out of OpenSSH connection multiplexing, because in my experience the master connection for multiplexed OpenSSH connections handles remote reboots rather ungracefully (it can take a minute or two before the client finally considers the master connection dead and successfully establishes a fresh connection to the rebooted remote host).

Any output on the standard error stream is silenced because 99% of the time this output will consist of SSH client connection errors due to the “retry until success” approach taken by reboot_remote_system().

debuntu_tools.remote_reboot.get_post_context(name)

Get an execution context for the post-boot environment of a remote host.

Parameters:name – The configuration section name or SSH alias of the remote host (a string).
Returns:A RemoteContext object.
debuntu_tools.remote_reboot.is_encrypted(context)

Detect whether a remote system is using root disk encryption.

Parameters:context – A RemoteContext object.
Returns:True if root disk encryption is being used, False otherwise.

debuntu_tools.remote_unlock

Usage: unlock-remote-system [OPTIONS] PRE_BOOT [POST_BOOT]

Boot a remote Linux system that’s waiting for the root disk encryption password to be entered into an interactive prompt by connecting to the remote system over the network using SSH and entering the password non-interactively. The remote Linux system needs to be configured in such a way that the pre-boot environment enables a static IP address and starts an SSH server like Dropbear.

The PRE_BOOT argument defines how to connect to the pre-boot environment:

  • Its value is assumed to be a host name, IP address or SSH alias.
  • It can optionally start with a username followed by an ‘@’ sign.
  • It can optionally end with a ‘:’ followed by a port number.

The default username is ‘root’ and the default port number 22. The optional POST_BOOT argument defines how to connect to the post-boot environment, this is useful when the pre and post-boot environments run SSH servers on different port numbers.

If the PRE_BOOT argument matches the name of a user defined configuration section the options in that section define how unlock-remote-system operates.

Supported options:

Option Description
-i, --identity-file=KEY_FILE Use the private key stored in KEY_FILE for SSH connections to the pre-boot environment. The post-boot environment is expected to use your default private key or have a suitable configuration in ~/.ssh/config.
-k, --known-hosts=HOSTS_FILE Use HOSTS_FILE as the “known hosts file” for SSH connections to the pre-boot environment. When this option is not given host key verification will be disabled to avoid conflicts between the host keys of the different SSH servers running in the pre-boot and post-boot environments.
-p, --password=NAME Get the password for the root disk encryption of the remote system from the local password store in ~/.password-store using the ‘pass’ program. The NAME argument gives the full name of the password.
-r, --remote-host=SSH_ALIAS Connect to the remote system through an SSH proxy.
-s, --shell Start an interactive shell on the remote system after it has finished booting.
-w, --watch Start monitoring the remote system and automatically unlock the root disk encryption when the remote system is rebooted. The monitoring continues indefinitely.
-a, --all Enable monitoring of all configured systems when combined with --watch.
-v, --verbose Increase logging verbosity (can be repeated).
-q, --quiet Decrease logging verbosity (can be repeated).
-h, --help Show this message and exit.
debuntu_tools.remote_unlock.EXPRESSION_PATTERN = <_sre.SRE_Pattern object>

A compiled regular expression pattern to parse connection profile expressions.

debuntu_tools.remote_unlock.HOST_KEYS_FILE = '~/.config/unlock-remote-system.d/known-hosts.ini'

The configuration file that’s used to store SSH host keys (a string).

debuntu_tools.remote_unlock.main()

Command line interface for unlock-remote-system.

debuntu_tools.remote_unlock.find_local_username()

Find the username of the current user on the local system.

debuntu_tools.remote_unlock.get_password_from_store(name, store=None)

Get the disk encryption password from the ‘pass’ program.

debuntu_tools.remote_unlock.prompt_for_password(hostname)

Prompt the operator to interactively enter the disk encryption password.

class debuntu_tools.remote_unlock.EncryptedSystem(**kw)

Python API for the unlock-remote-system program.

This class implements a Python API for remote unlocking of Linux systems with root disk encryption over SSH. The internals of this class differentiate between the pre-boot and post-boot environments:

  • The pre_boot and post_boot properties are ConnectionProfile objects that define how to connect to the SSH server in the pre-boot environment (usually this is Dropbear running in the initial ram disk) and the SSH server in the post-boot environment (usually this is OpenSSH).
  • The pre_context and post_context properties are RemoteContext objects that enable command execution in the pre-boot and post-boot environments. The values of these properties are created based on the corresponding connection profiles.
  • In addition to the command execution contexts for the pre-boot and post-boot environments there is the context property which provides a command execution context for commands outside of the remote system. This defaults to the local system on which unlock-remote-system is running but will be a remote system when ssh_proxy is set.

Here’s an overview of the EncryptedSystem class:

Superclass: PropertyManager
Special methods: __enter__() and __exit__()
Public methods: check_ssh_connection(), create_key_script(), find_process_id(), get_known_host_keys(), kill_emergency_shell(), kill_interactive_prompt(), offer_password(), parse_host_keys(), parse_server_header(), run_cryptroot_program(), scan_ssh_server(), store_host_keys(), test_ssh_connection(), unlock_system(), wait_for_post_boot(), wait_for_pre_boot(), watch_system(), write_file() and write_to_named_pipe()
Properties: boot_timeout, config, config_loader, config_section, connect_timeout, context, control_directory, cryptroot_config, cryptroot_program, have_cryptroot_config, have_cryptroot_program, have_named_pipe, interactive, key_script, known_hosts_file, named_pipe, password, post_boot, post_context, pre_boot, pre_context, retry_interval, scan_timeout, ssh_proxy and watch_interval

When you initialize a EncryptedSystem object you are required to provide values for the post_boot and pre_boot properties. You can set the values of the boot_timeout, config_loader, config_section, connect_timeout, control_directory, cryptroot_config, cryptroot_program, interactive, key_script, known_hosts_file, named_pipe, password, post_boot, pre_boot, retry_interval, scan_timeout, ssh_proxy and watch_interval properties by passing keyword arguments to the class initializer.

boot_timeout

The number of seconds to wait for the system to boot (a number, defaults to 5 minutes).

Note

The boot_timeout property is a custom_property. You can change the value of this property using normal attribute assignment syntax. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

config

A dictionary with configuration options.

See also config and config_section.

Note

The config property is a lazy_property. This property’s value is computed once (the first time it is accessed) and the result is cached.

config_loader

A ConfigLoader object.

See also config and config_section.

Note

The config_loader property is a custom_property. You can change the value of this property using normal attribute assignment syntax. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

config_section

The configuration section to use (a string or None).

When this option is set config can be loaded and store_host_keys() can persist SSH host keys.

See also config and config_loader.

Note

The config_section property is a mutable_property. You can change the value of this property using normal attribute assignment syntax. To reset it to its default (computed) value you can use del or delattr().

connect_timeout

How long to wait for the system to become reachable (an integer, defaults to 2 minutes).

Note

The connect_timeout property is a custom_property. You can change the value of this property using normal attribute assignment syntax. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

context

The command execution context from which the remote system is being controlled.

The computed value of this property is a command execution context created by executor.contexts. When ssh_proxy is set this will be a RemoteContext object, otherwise it will be a LocalContext object.

Note

The context property is a lazy_property. This property’s value is computed once (the first time it is accessed) and the result is cached.

control_directory

A temporary directory for SSH control sockets (a string).

Note

The control_directory property is a mutable_property. You can change the value of this property using normal attribute assignment syntax. To reset it to its default (computed) value you can use del or delattr().

cryptroot_config

The pathname of the ‘cryptroot’ configuration file (a string).

The value of this property sets the pathname of the ‘cryptroot’ configuration file in the pre-boot environment (the initial ram disk). It defaults to ‘/conf/conf.d/cryptroot’.

Note

The cryptroot_config property is a mutable_property. You can change the value of this property using normal attribute assignment syntax. To reset it to its default (computed) value you can use del or delattr().

cryptroot_program

The pathname of the ‘cryptroot’ program (a string).

The value of this property sets the pathname of the ‘cryptroot’ program in the pre-boot environment (the initial ram disk). It defaults to ‘/scripts/local-top/cryptroot’.

Note

The cryptroot_program property is a mutable_property. You can change the value of this property using normal attribute assignment syntax. To reset it to its default (computed) value you can use del or delattr().

have_cryptroot_config

True if cryptroot_config exists, False otherwise.

The existence of the /conf/conf.d/cryptroot configuration file is taken as confirmation that the remote system is currently in its pre-boot environment (initial ram disk).

have_cryptroot_program

True if cryptroot_program exists, False otherwise.

have_named_pipe

True if named_pipe exists, False otherwise.

The named pipe configured by the named_pipe property provides a simple and robust way to inject the root disk encryption pass phrase into the boot sequence. When the named pipe is available it will be used as the preferred method.

In my experience this works on Ubuntu 16.04 but it doesn’t work on Ubuntu 14.04 (because the named pipe doesn’t exist).

interactive

True to allow user interaction, False otherwise.

The value of interactive defaults to the return value of connected_to_terminal() when given sys.stdin.

Note

The interactive property is a custom_property. You can change the value of this property using normal attribute assignment syntax. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

key_script

The pathname of the generated key script (defaults to '/tmp/keyscript.sh').

Note

The key_script property is a mutable_property. You can change the value of this property using normal attribute assignment syntax. To reset it to its default (computed) value you can use del or delattr().

known_hosts_file

The filename of the “known hosts file” to use (a string or None).

Note

The known_hosts_file property is a mutable_property. You can change the value of this property using normal attribute assignment syntax. To reset it to its default (computed) value you can use del or delattr().

named_pipe

The pathname of the named pipe used by cryptsetup (a string).

The value of this property sets the pathname of the named pipe used by cryptsetup in the pre-boot environment (the initial ram disk). It defaults to ‘/lib/cryptsetup/passfifo’.

Note

The named_pipe property is a mutable_property. You can change the value of this property using normal attribute assignment syntax. To reset it to its default (computed) value you can use del or delattr().

password

The password that unlocks the root filesystem of the remote host (a string or None).

If the configuration section contains the option password-name then get_password_from_store() will be used to get the password by executing the pass program.

The optional configuration option password-store can be used to set the $PASSWORD_STORE_DIR environment variable.

Note

The password property is a custom_property. You can change the value of this property using normal attribute assignment syntax. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

post_boot

A connection profile for the post-boot environment (a ConnectionProfile object).

Note

The post_boot property is a custom_property. You are required to provide a value for this property by calling the constructor of the class that defines the property with a keyword argument named post_boot (unless a custom constructor is defined, in this case please refer to the documentation of that constructor). You can change the value of this property using normal attribute assignment syntax. This property’s value is computed once (the first time it is accessed) and the result is cached.

post_context

A command execution context for the post-boot environment.

The computed value of this property is a command execution context created by executor.contexts, more specifically it’s a RemoteContext object.

Note

The post_context property is a lazy_property. This property’s value is computed once (the first time it is accessed) and the result is cached.

pre_boot

A connection profile for the pre-boot environment (a ConnectionProfile object).

Note

The pre_boot property is a required_property. You are required to provide a value for this property by calling the constructor of the class that defines the property with a keyword argument named pre_boot (unless a custom constructor is defined, in this case please refer to the documentation of that constructor). You can change the value of this property using normal attribute assignment syntax.

pre_context

The command execution context inside the pre-boot environment.

The computed value of this property is a command execution context created by executor.contexts, more specifically it’s a RemoteContext object.

Note

The pre_context property is a lazy_property. This property’s value is computed once (the first time it is accessed) and the result is cached.

retry_interval

The time between connection attempts (an integer, defaults to 1 second).

Note

The retry_interval property is a custom_property. You can change the value of this property using normal attribute assignment syntax. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

scan_timeout

The timeout for ssh-keyscan (an integer, defaults to 5 seconds).

Note

The scan_timeout property is a custom_property. You can change the value of this property using normal attribute assignment syntax. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

ssh_proxy

The SSH alias of a proxy to the remote system (a string or None).

Note

The ssh_proxy property is a mutable_property. You can change the value of this property using normal attribute assignment syntax. To reset it to its default (computed) value you can use del or delattr().

watch_interval

The time between pre-boot environment checks (an integer, defaults to 60 seconds).

Note

The watch_interval property is a custom_property. You can change the value of this property using normal attribute assignment syntax. This property’s value is computed once (the first time it is accessed) and the result is cached. To clear the cached value you can use del or delattr().

__enter__()

Prepare control_directory.

__exit__(exc_type=None, exc_value=None, traceback=None)

Cleanup control_directory.

check_ssh_connection()

Verify SSH connectivity to the pre-boot environment.

create_key_script()

Create a key script in the pre-boot environment (initial ram disk).

This method creates a minimal key script (a shell script containing a single echo command) in the pre-boot environment and modifies the cryptroot_config file so that the key script is used to unlock the root disk.

find_process_id(program, **options)

Determine the process id of a program or script in the pre-boot environment.

Parameters:program – The name of a program or script (a string).
Returns:A process id (an integer) or None.
get_known_host_keys(option_name)

Get the configured SSH host keys.

Parameters:option_name – The name of the configuration option that holds known host keys (a string).
Returns:A set of strings.
kill_emergency_shell()

Kill the emergency shell process to resume the boot process.

kill_interactive_prompt()

Kill the process responsible for the interactive prompt in the pre-boot environment.

offer_password()

Make the root disk encryption pass phrase available to the remote host.

Raises:UnsupportedSystemError when have_named_pipe, have_cryptroot_config and have_cryptroot_program are all False.

If have_named_pipe is True and password is set, the internal method write_to_named_pipe() is used to write password to the named pipe. This is the preferred way to make the password available to the remote host because it’s simple and robust.

When the named pipe isn’t available but have_cryptroot_config is True the following internal methods are used instead:

This is more convoluted than the named pipe but it works :-).

parse_host_keys(cmd)

Find the SSH host keys in the output of ssh-keyscan.

Parameters:cmd – A RemoteCommand object.
Returns:A set of strings.
parse_server_header(cmd)

Find the SSH server header in the output of ssh-keyscan.

Parameters:cmd – A RemoteCommand object.
Returns:The SSH server header (a string).
scan_ssh_server(profile)

Get the SSH server header and host keys.

Parameters:profile – A ConnectionProfile object.
Returns:A ServerDetails object.

The scan_ssh_server() method runs ssh-keyscan in context and parses its output to get the SSH server header and host keys.

run_cryptroot_program(interactive=True)

Unlock the root disk encryption by running cryptroot_program.

store_host_keys(pre_server, post_server)

Store the SSH host keys in the configuration.

Parameters:
test_ssh_connection(profile, context)

Verify SSH connectivity to the pre-boot environment.

unlock_system(server=None)

Validate the pre-boot environment and unlock the root filesystem encryption.

Parameters:server – A ServerDetails object or None.

When the server argument isn’t given wait_for_pre_boot() is used to wait for the pre-boot environment to become available.

wait_for_post_boot(pre_server)

Wait for the post-boot environment to come online.

Parameters:pre_server – A ServerDetails object created by wait_for_pre_boot().
wait_for_pre_boot()

Wait for the pre-boot environment to become available.

Returns:

A ServerDetails object.
Raises:

The following exceptions can be raised:
watch_system()

Start monitoring the remote system for reboots.

When the remote system is rebooted, the root disk encryption will be unlocked automatically. The monitoring continues indefinitely.

write_file(filename, contents)

Write a file in the initial ram disk of the pre-boot environment.

Parameters:
  • filename – The pathname of the file to write (a string).
  • contents – The contents to write to the file (a string).

This method writes a file in the initial ram disk by running a remote ‘sh’ shell that redirects its standard input to the given filename.

write_to_named_pipe()

Write password to the named pipe configured by named_pipe.

class debuntu_tools.remote_unlock.ConnectionProfile(**kw)

SSH connection profiles.

Here’s an overview of the ConnectionProfile class:

Superclass: PropertyManager
Properties: expression, hostname, identity_file, port_number and username

When you initialize a ConnectionProfile object you are required to provide a value for the hostname property. You can set the values of the expression, hostname, identity_file, port_number and username properties by passing keyword arguments to the class initializer.

expression

The SSH connection profile encoded as a string.

Note

The expression property is a mutable_property. You can change the value of this property using normal attribute assignment syntax. To reset it to its default (computed) value you can use del or delattr().

hostname

The host name, IP address or SSH alias of the remote system (a string).

Note

The hostname property is a required_property. You are required to provide a value for this property by calling the constructor of the class that defines the property with a keyword argument named hostname (unless a custom constructor is defined, in this case please refer to the documentation of that constructor). You can change the value of this property using normal attribute assignment syntax.

port_number

The port number for SSH connections (an integer, defaults to 22).

Note

The port_number property is a mutable_property. You can change the value of this property using normal attribute assignment syntax. To reset it to its default (computed) value you can use del or delattr().

username

The username used to log in to the remote system (a string, defaults to ‘root’).

Note

The username property is a mutable_property. You can change the value of this property using normal attribute assignment syntax. To reset it to its default (computed) value you can use del or delattr().

identity_file

The pathname of the identity file used to connect to the remote system (a string or None).

Note

The identity_file property is a mutable_property. You can change the value of this property using normal attribute assignment syntax. To reset it to its default (computed) value you can use del or delattr().

class debuntu_tools.remote_unlock.ServerDetails(**kw)

Properties that can be used to uniquely identify SSH servers.

Here’s an overview of the ServerDetails class:

Superclass: PropertyManager
Public methods: match_header()
Properties: header and host_keys

When you initialize a ServerDetails object you are required to provide values for the header and host_keys properties. You can set the values of the header and host_keys properties by passing keyword arguments to the class initializer.

header

The SSH server header (a string).

This is the first line of output emitted by the SSH server when a client opens a TCP connection.

Note

The header property is a key_property. You are required to provide a value for this property by calling the constructor of the class that defines the property with a keyword argument named header (unless a custom constructor is defined, in this case please refer to the documentation of that constructor). Once this property has been assigned a value you are not allowed to assign a new value to the property.

host_keys

The SSH server host keys (a frozenset of strings).

Note

The host_keys property is a key_property. You are required to provide a value for this property by calling the constructor of the class that defines the property with a keyword argument named host_keys (unless a custom constructor is defined, in this case please refer to the documentation of that constructor). Once this property has been assigned a value you are not allowed to assign a new value to the property.

match_header(substring)

Check if header contains the given substring.

Parameters:substring – The substring to check (a string).
Returns:True if the header matches the substring, False otherwise.
exception debuntu_tools.remote_unlock.EncryptedSystemError

Base class for custom exceptions raised by debuntu_tools.remote_unlock.

exception debuntu_tools.remote_unlock.UnlockAbortedError

Raised when the operator changes their minds.

exception debuntu_tools.remote_unlock.SystemUnreachableError

Raised when connecting to the SSH server in the pre-boot environment fails.

exception debuntu_tools.remote_unlock.BootTimeoutError

Raised when the remote system doesn’t boot properly after unlocking.

exception debuntu_tools.remote_unlock.UnsupportedSystemError

Raised when the configuration file ‘/conf/conf.d/cryptroot’ is not found.

debuntu_tools.upgrade_system

Usage: upgrade-remote-system [OPTIONS] [SSH_ALIAS]

Upgrade the system packages on a remote Debian or Ubuntu system, reboot the system when this is required due to security updates or because the system isn’t yet running the newest kernel, remove old Linux kernel and header packages and optionally remove ‘auto-removable’ system packages.

If the given SSH alias matches a section in the ‘unlock-remote-system’ configuration, the root disk encryption of the remote system will be automatically unlocked when the system is rebooted.

Supported options:

Option Description
-s, --shell Start an interactive shell on the remote system afterwards.
-v, --verbose Increase logging verbosity (can be repeated).
-q, --quiet Decrease logging verbosity (can be repeated).
-h, --help Show this message and exit.
debuntu_tools.upgrade_system.main()

Command line interface for upgrade-remote-system.

debuntu_tools.upgrade_system.upgrade_remote_system(context, force_reboot=False)

Perform standard system maintenance tasks on a remote Debian or Ubuntu system.

Parameters:context – An execution context created by executor.contexts.

This function performs the following system maintenance tasks:

  1. The apt-get update command is run (using the Python API of the apt-mirror-updater program).
  2. The apt-get dist-upgrade command is run [1].
  3. The apt-get clean command is run.
  4. If the file /var/run/reboot-required exists (indicating that a reboot is required due to security updates) the remote system is rebooted using the Python API of the reboot-remote-system program, to enable automatic unlocking of remote root disk encryption.
  5. Old kernel packages are removed (using the Python API of the debuntu-kernel-manager program). If more than one meta package is installed a warning message is logged but no exception is raised.
  6. The apt-get autoremove --purge command is run to optionally [1] remove any ‘auto-removable’ system packages.
[1](1, 2) Because the apt-get option --yes is not used, the operator will be asked to confirm using an interactive confirmation prompt.