Difference between revisions of "ERIKA3 on the Xen hypervisor"

Latest revision as of 14:12, 30 September 2019

Introduction

Xen is an open-source type-1 or bare-metal hypervisor, that runs directly on the hardware and is responsible for handling CPU, memory, timers and interrupts.

The hypervisor is started by the bootloader and, once it is loaded, it starts the privileged domain Dom0 (short for "domain 0") containing the drivers of the system devices. Once Dom0 has started, one or more user domains, referred to as DomU, can be started and controlled in the Dom0. Xen supports both para-virtualized (PV) and fully hardware virtualized (HVM) user domains. Please refer to the following Xen installation procedure in order to install Xen hypervisor on x86-64 platform.

The following part of this tutorial will explain how to build and run ERIKA3 as Xen hardware-virtualized DomU on the x86-64 platform.

IMPORTANT: currently ERIKA on Xen is supported only on x86-64 platforms. In case you have an ARM64 platform, you can use the Jailhouse hypervisor.

Build ERIKA3 for Xen

Building ERIKA3 for Xen requires building an ERIKA3 image for bare-metal as described in the following tutorial: 'Building a bare-metal x86-64 image'.

The path of the generated ERIKA3 bare-metal image(e.g., erika3.iso) has to be included into the Xen configuration file. More in detail, such path, referred to as $ERIKA3_PATH, has to used to configure the boot disk configuration parameter as specified in the next section.

ERIKA3 Xen configuration

This section defines the configuration parameters that are used for running ERIKA3 as Xen DomU.

The ERIKA3 configuration file has to include the following basic parameters, such as domain name, memory requirements, virtualization mode, serial interfaces definition and so on:

• Name of domain (must be unique)

 name = "erika"

• Memory allocation (MB) for ERIKA3 (should be at least 2M)

 memory = 2048
 maxmem = 2048

• Boot disk (where $ERIKA3_PATH is the path of the ERIKA3 bare-metal image)

 ## Boot
 disk 	= [ 'file:$ERIKA3_PATH,hdc:cdrom,r' ]
 boot 	= "c"

• Virtualization mode

 type = "hvm"

• Enable Xen console (optional)

 serial = "pty"

• Domain exit behavior settings

 ## Behaviour                                                                    
 #Options: Default=None; Value='destroy|restart|preserve|rename-restart'
 on_poweroff = "preserve"
 on_reboot = "destroy"
 on_crash = "destroy"

Run ERIKA3 as Xen HVM DomU

The Xen command to start the ERIKA3 DomU is the following:

 # xl create erika3-xen.cfg

where erika3-xen.cfg is the ERIKA3 configuration file.

In case of console connection to the ERIKA3 DomU that is already running, the Xen command is the following:

 # xl console erika

where 'erika' is the domain name defined in the configuration file.

In order to connect to console directly when starting the ERIKA3 domU, launch the following command:

 # xl create -c erika3-xen.cfg

In order to poweroff the ERIKA3 DomU, the Xen command is the following:

 # xl destroy erika

where 'erika' is the domain name defined in the configuration file.

ERIKA3 CPUs configuration

CPU assignment
In order to tune ERIKA3 DomU, Xen configuration could include CPUs affinity in order to pin ERIKA on a given CPU. Note that current implementation of ERIKA3 on x86-64 is single-core. Thus, the number of virtual CPUs is 1. Such virtual CPU can be pinned to a given physical CPU. The ERIKA3 configuration file has to include the following items in order to set the CPUs affinity:

• Number of virtual CPUs to use (default is 1)>/li> vcpus = 1
• List of which CPUs this domain is allowed to use (in the example, it is the CPU with identifier 3)

 cpus = ['3']

CPU scheduler
Xen hypervisor can divide the physical cpus into distinct groups, called cpupools, so that each pool can have its entirely separate scheduler (e.g., null-scheduler). The "default pool" is named Pool-0 and physical CPUs can be removed from one cpupool and added to another. It follows the Xen basic operations to setup and modify a cpupool:

Show the cpupool list:

 # xl cpupool-list

Show the cpupool list with the corresponding physical CPUs:

 # xl cpupool-list -c

Setup a new empty pool using, for example, the null scheduler:

 # xl cpupool-create name="NullPool" sched="null"

Move a physical CPU from default pool to the new pool:

 # xl cpupool-cpu-remove Pool-0 3
 # xl cpupool-cpu-add NullPool 3

Domains are assigned to pools on creation, and can be moved from one pool to another.

In order to assign ERIKA3 to a given cpupool at DomU creation, the Xen configuration (i.e., erika3-xen.cfg) has to include the following item specifying the cpupool name:

 pool="NullPool"

CPU optimization: TSC emulation
To provide a "safe" TSC, i.e. to ensure both TSC monotonicity and a fixed rate, Xen provides rdtsc emulation. Note that rdtsc emulation is slower than the rdtsc instruction when executed natively. Thus, for environments where highest performance is a requirement, TSC emulation should be disabled.

TSC emulation can be specified in the DomU configuration as follows:

• tsc_mode="native": TSC emulation disabled
• tsc_mode="always_emulate": TSC emulation enabled (default)

ERIKA3 Xen Passthrough

PCI passthrough allows you to give control of physical devices to guests. In other words, PCI passthrough allows to assign a PCI device (NIC, disk controller, HBA, USB controller, firewire controller, soundcard, etc) to a domU, giving it full and direct access to the PCI device.

PCI device has to be "assignable" by using xl pci-assignable-add. For example, if you wanted to make the device at BDF 00:1f.6 available for guests, the Xen command is the following:

 # xl pci-assignable-add 00:1f.6

At this point, the device is ready to be assigned to a guest. You can verify this with the following command:

 # xl pci-assignable-list

The resulting list should contain the requested BDF.

In order to give control of physical devices to ERIKA3 domU, use the xl pci-attach commmand as follows:

 # xl pci-attach erika 00:1f.6
 # xl pci-list erika

In the same way, when destroying the domU, detach the PCI device so that it can be 'assignable' again:

 # xl destroy erika
 # xl pci-detach erika 00:1f.6

In order to give control of physical devices at the domU creation, Xen configuration should include the configuration for PCI passthrough. For example, the Xen configuration to allow the PCI passthrough for the Ethernet device with '00:1f.6' as BDF:

 ## PCI Passthrough
 pci_permissive=1
 pci = ['00:1f.6']