Integration with Forges
Now that you have installed CI-tron, you quite likely would like to expose your test machines (DUTs) on your projects’ GIT forges.
The forge integration has been designed with the following goals in mind:
Sane, but partially-or-fully overridable defaults: Allows the same job to run on every DUT in the farm, with the possibility to opt-in more control only when and where needed.
As seamless as possible: running on a CI-tron runner should be as similar to running on the default CI runners as possible;
The default boot configuration can be found in executor/server/src/valve_gfx_ci/executor/server/templates/boots_db.yml.j2.
executor/server/src/valve_gfx_ci/executor/server/templates/boots_db.yml.j2
#
# This file defines how to boot the various computers supported by CI-tron.
#
# Its goal is to provide sane boot defaults that enables running the same
# job on all the supported hardware, while also supporting jobs selectively
# opting-out of the defaults to get a more complete control over their boot
# configuration.
#
# To achieve this goal, we make use of the feature that enables splitting lists
# into categories. Categories that are not referenced by a job will be
# inherited automatically while uncategorized elements are evaluated first, so
# as to reduce the risks of user confusion. With that in mind, here are the
# rules on how to add support for new hardware:
#
# * Only use one of the following categories (no re-naming allowed):
#
# * 50-platform: Anything necessary to boot the machine before reaching the
# bootloader stage, or firmware that is expected by the platform and is
# not otherwise part of the linux-firmware project, such as the signed
# and non-redistributable firwmare files found on Qualcomm hardware.
#
# * 50-bootloader: Anything necessary to load the environment as specified by
# the `kernel`, `initramfs`, and `dtb` deployment options. Place in this
# category all the bootloader configuration files.
#
# * 50-kernel-modules: Anything necessary to make the default kernel boot to
# a usable environment (ie. kernel modules, firmware, ...). Nothing from
# this category should be included by default by a job that overrides the
# default kernel.
#
# * Make the default deployment boot into a boot2container environment,
# and be overridable without needing to set a category. In other words,
# set the default boot2container initramfs in the `uncategorised` section.
#
{% macro b2c_url(goarch) -%}
https://gitlab.freedesktop.org/gfx-ci/boot2container/-/releases/v0.9.16/downloads/initramfs.linux_{{ goarch | replace("386", "amd64") }}.cpio.xz
{%- endmacro %}
{% macro kernel_url(goarch) -%}
https://gitlab.freedesktop.org/gfx-ci/boot2container/-/releases/v0.9.16/downloads/linux-{{ goarch | replace("386", "x86_64") | replace("amd64", "x86_64") }}
{%- endmacro %}
{% macro dtb_url(goarch) -%}
https://gitlab.freedesktop.org/gfx-ci/boot2container/-/releases/v0.9.16/downloads/linux-{{ goarch | replace("386", "x86_64") | replace("amd64", "x86_64") }}.dtbs.cpio.xz
{%- endmacro %}
dhcp:
ipxe:
match:
user_class: iPXE
architecture:
- x86
- x86_64
- arm32
- arm64
defaults:
dhcp:
options:
bootfile: {{ job.http.url }}/boot/boot.ipxe
hostname: "dut-{{ dut.full_name }}"
storage:
http:
50-bootloader:
- path: /boot/boot.ipxe
data: |
#!ipxe
set semicolon:hex 3b
set ampersand:hex 26
set pipe:hex 7C
set cmdline {% if dhcp_request.firmware.name == "UEFI" %}initrd=initrd{% endif %} {{ job.deployment.kernel.cmdline }}
echo
echo Downloading the kernel
echo <ipxe> kernel {{ job.http.path_to("kernel") }}
echo <ipxe> cmdline ${cmdline}
kernel {{ job.http.path_to("kernel") }} ${cmdline}
echo
echo Downloading the initrd
echo <ipxe> initrd --name initrd {{ job.http.path_to("initramfs") }}
initrd --name initrd {{ job.http.path_to("initramfs") }}
echo Booting!
boot
kernel:
url: "{{ kernel_url(dhcp_request.architecture.goarch) }}"
initramfs:
url: "{{ b2c_url(dhcp_request.architecture.goarch) }}"
uboot:
match:
firmware: uboot
architecture:
- x86_64
- arm32
- arm64
- riscv64
defaults: &uboot-defaults
storage:
tftp:
50-bootloader:
- path: &uboot_script_path /boot.scr.uimg
format:
uboot:
architecture: {{ dhcp_request.architecture.value | default("unknown")}}
compression: none
os: linux
type: script
data: |
echo Loading the kernel
setenv bootargs '{{ job.deployment.kernel.cmdline | replace('\n', ' ') }}'
tftpboot ${kernel_addr_r} {{ job.tftp.path_to("kernel") }}
# The best compression ratio I found was 5.2x (XZ with x86_64), so let's go with 8x
# NOTE: We use a PoT kernel_comp_size to align the subsequent artifact as much as possible
setexpr kernel_comp_addr_r ${kernel_addr_r} + ${filesize}
setexpr kernel_comp_size ${filesize} * 8
echo Loading the compressed kernel at ${kernel_comp_addr_r}:${kernel_comp_size}
{% if job.deployment.dtb %}
setenv fdt_file {{ job.tftp.path_to("dtb") }}
{% endif %}
setexpr fdt_addr ${kernel_comp_addr_r} + ${kernel_comp_size}
echo Loading the fdt ${fdt_file} at ${fdt_addr}
if test -n "${fdt_file}" && tftpboot ${fdt_addr} ${fdt_file}
then
setexpr ramdisk_addr_r ${fdt_addr}} + ${filesize}
else
setenv ramdisk_addr_r ${fdt_addr}
setenv fdt_addr ${fdtcontroladdr}
fi
echo Loading the ramdisk at ${ramdisk_addr_r}
tftpboot ${ramdisk_addr_r} {{ job.tftp.path_to("initramfs") }}
setenv ramdisk_size ${filesize}
echo Booting!
booti ${kernel_addr_r} ${ramdisk_addr_r}:${ramdisk_size} ${fdt_addr}
bootz ${kernel_addr_r} ${ramdisk_addr_r}:${ramdisk_size} ${fdt_addr}
- path: /pxelinux.cfg/.*
data: |
default ci-tron
label ci-tron
kernel {{ job.tftp.path_to("kernel") }}
append {{ job.deployment.kernel.cmdline }}
initrd {{ job.tftp.path_to("initramfs") }}
{% if job.deployment.dtb %}
fdt {{ job.tftp.path_to("dtb") }}
{% endif %}
50-platform:
# Given that U-boot may potentially depend on the updated DTB
- path: /dtbs?/(.*)
url: "{{ dtb_url(dhcp_request.architecture.goarch) }}"
format:
archive:
match: boot/dtbs/\1
dhcp:
options:
bootfile: *uboot_script_path
hostname: "dut-{{ dut.full_name }}"
kernel:
url: "{{ kernel_url(dhcp_request.architecture.goarch) }}"
initramfs:
url: "{{ b2c_url(dhcp_request.architecture.goarch) }}"
raspberrypi_arm64_downstream:
match:
mac_address: &raspberrypi_mac_addrs
- b8:27:eb:.*
- 2c:cf:67:.*
- d8:3a:dd:.*
- dc:a6:32:.*
- e4:5f:01:.*
# NOTE: One can use the UUID field from the DHCP query to identify an RPi 4 and 5.
# Src: https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#DHCP_OPTION97
# uuid: "35695052-.*"
defaults:
storage:
tftp:
50-bootloader:
- path: /config.txt
data: |
[all]
arm_64bit=1
enable_uart=1
kernel={{ job.tftp.path_to("kernel") }}
initramfs {{ job.tftp.path_to("initramfs") }} followkernel
[pi5]
dtoverlay=vc4-kms-v3d-pi5
dtparam=pciex1_gen=3
dtoverlay=disable-bt
dtparam=uart0_console
[pi4]
dtoverlay=disable-bt
dtoverlay=vc4-kms-v3d-pi4
[pi3]
dtoverlay=disable-bt
dtoverlay=vc4-kms-v3d
# NOTE: File sizes should be limited to 32MB not to exceed the
# maximum TFTP transfer size using the default 512 bytes block size
[pi2]
dtoverlay=disable-bt
dtoverlay=vc4-kms-v3d
# NOTE: File sizes should be limited to 32MB not to exceed the
# maximum TFTP transfer size using the default 512 bytes block size
- path: /cmdline.txt
data: >-
modules_load=overlay,vc4,v3d,snd_soc_hdmi_codec,i2c-brcmstb,i2c-bcm2835,nbd {{ job.deployment.kernel.cmdline}}
50-platform:
# This allow serving all the binaries, DTBs, and DTB overlays required to boot the wanted kernel
- path: /(.*)
# NOTE: Zip files are more efficient for seek times
url: &rpi_firmware "https://github.com/raspberrypi/firmware/archive/b154632e320b87ea95c6ce8b59f96dbbe523ecf1.zip"
format:
archive:
match: firmware-.*/boot/\1
dhcp:
options:
hostname: "dut-{{ dut.full_name }}"
# Keep the spaces! Src: https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#dhcp-request-reply
43: "Raspberry Pi Boot "
kernel:
url: *rpi_firmware
format:
archive:
match: firmware-.*/boot/kernel8.img
initramfs:
categories:
50-kernel-modules:
- url: *rpi_firmware
format:
archive:
extension: "cpio"
keep:
- path: firmware-.*/(modules/.*-v8\+/(modules|kernel/(fs/overlayfs|sound|drivers/(video|gpu|media/cec/core|i2c/busses/i2c-.*|block/nbd.ko))).*)
rewrite: usr/lib/\1
uncategorised:
- url: "{{ b2c_url('arm64') }}"
bios_chainload_ipxe:
match:
firmware: bios
defaults:
storage:
tftp:
50-bootloader:
- path: &bios_ipxe_path /boot/ipxe.kpxe
url: "https://downloads.gfx-ci.steamos.cloud/ipxe-dut-client/2024-01-30_20-41-29-mupuf-i386-undionly.kpxe"
dhcp:
options:
bootfile: *bios_ipxe_path
hostname: "dut-{{ dut.full_name }}"
uefi_tftp_chainload_ipxe:
match:
firmware: uefi
protocol: tftp
architecture:
- x86_64
- arm32
- arm64
defaults:
storage:
tftp:
50-bootloader:
- path: &efi_ipxe_path /boot/ipxe.efi
url: "https://downloads.gfx-ci.steamos.cloud/ipxe-dut-client/2024-01-30_20-41-29-mupuf-{{ dhcp_request.architecture.ipxe_buildarch }}-snponly.efi"
dhcp:
options:
bootfile: *efi_ipxe_path
hostname: "dut-{{ dut.full_name }}"
uefi_http_chainload_ipxe:
match:
firmware: uefi
protocol: http
architecture:
- x86_64
- arm32
- arm64
defaults:
storage:
http:
50-bootloader:
- path: /boot/ipxe.efi
url: "https://downloads.gfx-ci.steamos.cloud/ipxe-dut-client/2024-01-30_20-41-29-mupuf-{{ dhcp_request.architecture.ipxe_buildarch }}-snponly.efi"
dhcp:
options:
bootfile: {{ job.http.url }}/boot/ipxe.efi
hostname: "dut-{{ dut.full_name }}"
# For platforms that are not supported by iPXE, let's assume that the bootloader
# is U-Boot and let it boot using the extlinux/pxelinux flow
uefi_tftp_using_uboot:
match:
firmware: uefi
protocol: tftp
architecture:
- riscv64
defaults: *uboot-defaults
fastboot:
hdk_8550:
match:
variables:
product: kalama
defaults:
fastboot:
header_version: 4
os_version: 14.0.0
os_patch_level: 2023-10
kernel:
url: https://fs.mupuf.org/hdk8650/linux-6.8-hdk8650-no-fw.gz
cmdline:
50-platform:
- pd_ignore_unused clk_ignore_unused
- nvme_core.default_ps_max_latency_us=0 # Disable NVME power management
- g_cdc.dev_addr={{ dut.mac_address }}
- androidboot.serialno={{ dut.id }}
initramfs:
categories:
50-platform:
- url: https://fs.mupuf.org/hdk8650/sm8550-hdk-firmware.cpio.xz
uncategorised:
- url: "{{ b2c_url("arm64") }}"
dtb:
url: "https://fs.mupuf.org/hdk8650/sm8650-hdk.dtb"
hdk_8650:
match:
variables:
product: pineapple
defaults:
kernel:
url: https://fs.mupuf.org/hdk8650/2025-01-23-msm-gpu-fault-fixes-msm-next-6dbabc6/linux-arm64
cmdline:
50-platform:
- pd_ignore_unused clk_ignore_unused
- nvme_core.default_ps_max_latency_us=0 # Disable NVME power management
- g_cdc.dev_addr={{ dut.mac_address }}
initramfs:
categories:
50-platform:
- url: https://fs.mupuf.org/hdk8650/sm8650-hdk-firmware.cpio.xz
uncategorised:
- url: "{{ b2c_url("arm64") }}"
dtb:
url: 'https://fs.mupuf.org/hdk8650/2025-01-23-msm-gpu-fault-fixes-msm-next-6dbabc6/linux-arm64.dtbs.cpio.xz'
format:
archive:
match: 'boot/dtbs/qcom/sm8650-hdk.dtb'
We currently only support GitLab, but support for more forges is planned and welcomed.
GitLab
To expose your DUTs on a GitLab instance, you will need to add define it in
MarsDB, in the gitlab:
element. Check out MarsDB’s YAML code
block for the details.
- .ci-tron-job-v1
Warning
The interface is not final and is not considered stable. Make sure to pin the commit from which you import this file from, along with explicitly setting
CI_TRON_JOB_TEMPLATE
.The low-level CI-tron integration GitLab CI job which you can use as the base for your CI jobs.
Warning
This job is more suited towards low-level bootloader/kernel/initrd testing. If you are looking for a more transparent way to run your GitLab jobs on bare-metal, please have a look at
.ci-tron-b2c-job-v1
instead.Including the job template
If your project is hosted on gitlab.freedesktop.org, use this:
# WARNING: it is strongly advised to use a commit hash instead of ``main`` variables: CI_TRON_TEMPLATE_PROJECT: &ci-tron-template-project gfx-ci/ci-tron CI_TRON_JOB_TEMPLATE_PROJECT_URL: https://gitlab.freedesktop.org/$CI_TRON_TEMPLATE_PROJECT CI_TRON_JOB_TEMPLATE_COMMIT: &ci-tron-template-commit main include: - project: *ci-tron-template-project ref: *ci-tron-template-commit file: '/.gitlab-ci/dut.yml'
Projects hosted elsewhere unfortunately have to specify the full URL in the
include
:# WARNING: it is strongly advised to use a commit hash instead of ``main`` # WARNING: be extra careful to always keep the variables and the include URL in sync! variables: CI_TRON_JOB_TEMPLATE_PROJECT_URL: https://gitlab.freedesktop.org/gfx-ci/ci-tron CI_TRON_JOB_TEMPLATE_COMMIT: main include: - remote: 'https://gitlab.freedesktop.org/gfx-ci/ci-tron/-/raw/main/.gitlab-ci/dut.yml'
Introductory example
stages: - integration testing variables: # WARNING: it is strongly advised to use a commit hash instead of ``main`` CI_TRON_JOB_TEMPLATE_PROJECT_URL: https://gitlab.freedesktop.org/gfx-ci/ci-tron CI_TRON_JOB_TEMPLATE_COMMIT: main include: - remote: 'https://gitlab.freedesktop.org/gfx-ci/ci-tron/-/raw/main/.gitlab-ci/dut.yml' # Print Hello World in the console log, then power off test: stage: integration testing extends: - .ci-tron-job-v1 tags: - cpu:arch:x86_64 - farm:$FARM_NAME variables: CI_TRON_PATTERN__SESSION_END__REGEX: '^.*It''s now safe to turn off your computer\r?$' CI_TRON_PATTERN__JOB_SUCCESS__REGEX: 'Execution is over, pipeline status: 0\r?$' CI_TRON_KERNEL__URL: https://gitlab.freedesktop.org/gfx-ci/boot2container/-/releases/v0.9.15.1/downloads/linux-x86_64 CI_TRON_INITRAMFS__B2C__URL: https://gitlab.freedesktop.org/gfx-ci/boot2container/-/releases/v0.9.15.1/downloads/initramfs.linux_amd64.cpio.xz CI_TRON_KERNEL_CMDLINE__SALAD: "SALAD.machine_id={{ dut.id }} console={{ dut.local_tty_device }},115200" CI_TRON_KERNEL_CMDLINE__B2C_NTP_PEER: "b2c.ntp_peer=ci-gateway" CI_TRON_KERNEL_CMDLINE__B2C_RUN: b2c.run="-t {{ fdo_proxy_registry }}/gfx-ci/ci-tron/executorctl:latest echo Hello world" CI_TRON_KERNEL_CMDLINE__B2C_POWEROFF: b2c.poweroff_delay=15
Timeouts
CI-tron exposes multiple sorts of timeouts:
OVERALL
: This limits the overall time the job can take to complete.BOOT_CYCLE
: This limits how long one boot -> test -> shutdown cycle can takeCONSOLE_ACTIVITY
: This limits the time between we receive 2 characters in the serial output.FIRST_CONSOLE_ACTIVITY
: This limits the time it can take to receive the first console log.
- CI_TRON_TIMEOUT__OVERALL__MINUTES
Number of minutes the job can take to complete, across reboots. Like the other timeouts below,
minutes
is a floating point number; eg.0.5
= 30 seconds.
- CI_TRON_TIMEOUT__BOOT_CYCLE__MINUTES
Number of minutes one boot -> test -> shutdown cycle can take.
- CI_TRON_TIMEOUT__BOOT_CYCLE__RETRIES
Number of times the BOOT_CYCLE timeout can trigger a reboot before causing the end of the job.
- CI_TRON_TIMEOUT__CONSOLE_ACTIVITY__MINUTES
Number of minutes the job can go without receiving any new console activity, after receiving the first byte.
- CI_TRON_TIMEOUT__CONSOLE_ACTIVITY__RETRIES
Number of times the CONSOLE_ACTIVITY timeout can trigger a reboot before causing the end of the job.
- CI_TRON_TIMEOUT__FIRST_CONSOLE_ACTIVITY__MINUTES
Number of minutes the job can wait for the first byte to arrive on the console.
- CI_TRON_TIMEOUT__FIRST_CONSOLE_ACTIVITY__RETRIES
Number of times the FIRST_CONSOLE_ACTIVITY timeout can trigger a reboot before causing the end of the job.
Watchdogs
Watchdogs are user-defined timeouts that can be used to enforce sections of execution make progress within the expected period.
In the following attributes, replace
SHORT_NAME
with whatever name you want to give to your watchdog.- CI_TRON_WATCHDOG__SHORT_NAME__MINUTES
Number of minutes that can happen between the
CI_TRON__BOOT_CYCLE__WD_START_REGEX
regular expression is matched and theCI_TRON__BOOT_CYCLE__WD_STOP_REGEX
one is matched. The timeout gets automatically reset to whenCI_TRON__BOOT_CYCLE__WD_RESET_REGEX
is matched.
- CI_TRON_WATCHDOG__SHORT_NAME__RETRIES
Number of times this timeout can be reset through a reboot before causing the end of the job.
Console Patterns
Console patterns are regular expressions that, when matched, influence the execution of the job.
- CI_TRON_PATTERN__SESSION_END__REGEX
This variable is required
A regular expression that, when matched, indicates that the job execution is over
- CI_TRON_PATTERN__JOB_SUCCESS__REGEX
This variable is required
A regular expression that, when matched, indicates that the job succeeded
- CI_TRON_PATTERN__JOB_WARNING__REGEX
A regular expression that, when matched, indicates that something went a little off during the execution of the job. The job can still complete, but the GitLab job will fail when matched, even if
CI_TRON_PATTERN__JOB_SUCCESS__REGEX
is matched.
- CI_TRON_PATTERN__JOB_REBOOT__REGEX
A regular expression that, when matched, indicates that the test machine requires a reboot. This will increase the BOOT_CYCLE timeout’s retry counter, which may abort the run if the current value exceeds
CI_TRON_TIMEOUT__BOOT_CYCLE__RETRIES
.
- CI_TRON_WATCHDOG__SHORT_NAME__START_REGEX
A regular expression that, when matched, starts the
SHORT_NAME
watchdog, as set byCI_TRON_WATCHDOG__SHORT_NAME__MINUTES
.
- CI_TRON_WATCHDOG__SHORT_NAME__RESET_REGEX
A regular expression that, when matched, resets the
SHORT_NAME
watchdog timeout to its original value.
- CI_TRON_WATCHDOG__SHORT_NAME__STOP_REGEX
A regular expression that, when matched, disables the
SHORT_NAME
watchdog untilCI_TRON_WATCHDOG__SHORT_NAME__START_REGEX
is matched again.
Container Image Store
CI-tron has the ability to download, cache, and share container images with test machines.
Warning
Due to the global/shared nature of the image store, you will need to provide the platform that should be used to pull the container as it may differ from the platform used to run CI-tron.
Additionally, since other jobs may pull containers during the execution of your job, you really should reference the image not by its label (which may be overwritten by a later pull), but by its image id:
{{ job.imagestore.public.short_name.image_id }}
.The information needed for the DUT to mount the image store can be found at
{{ imagestore.mount("public").nfs }}
. It contains the following fields:type
: The filesystem mount’s type (always NFS for now);src
: The filesystem mount’s source;opts
: The filesystem mount options;to_b2c_filesystem(name)
: A helper that converts these mount options into a Boot2container-compatible cmdline argument:b2c.filesystem={name},...
The following attributes can be used to list the containers that should be pulled, along with their pull options. Replace
SHORT_NAME
with whatever name you want to give to your container. This name can then be used to get the imageID of the container by using its lower-cased short name, like so{{ job.imagestore.public.short_name.image_id }}
.- CI_TRON_IMAGESTORE__PUBLIC__IMAGES__SHORT_NAME__NAME
This variable is required
The container image you want to pull, fully-qualified or with
docker://
as a prefix.
- CI_TRON_IMAGESTORE__PUBLIC__IMAGES__SHORT_NAME__PLATFORM
This variable is required
Typical values:
linux/amd64
,linux/arm64/v8
,linux/arm/v6
, orlinux/riscv64
The platform of the container that should be pulled. Useful for multi-architecture container manifests.
- CI_TRON_IMAGESTORE__PUBLIC__IMAGES__SHORT_NAME__TLS_VERIFY
Accepted values:
true
,false
A boolean to specify whether to require HTTPS and verify certificates when contacting the container registry during the pull. Defaults to
true
.
- CI_TRON_IMAGESTORE__PUBLIC__IMAGES__SHORT_NAME__PULL_POLICY
Default value:
relaxed_always
A string representing the pull policy to use. This may one of the following:
always
: Always try pulling the imagerelaxed_always
: Re-use a previous pull if it is less than 5 minutes old, otherwise try pulling the image againmissing
: Only pull the image if it is unknown
Network Block Devices
CI-tron allows you to create fresh Network Block Devices (NBD) for the job.
- CI_TRON_NBD__SHORT_NAME__SIZE
This variable is required Format: d+[kMGTP]i?B?
Instantiate an NBD server that exposes a network block device of the wanted size, in bytes. You may also use the convenience exponents
k
,M
,G
,T
, andP
. Follow that byB
if you want metric sizes, oriB
if you want a power-of-two size. If noB
is specified, we default is to use power-of-two sizes.For example, the following values
1048576
,1024kiB
,1M
, and1MiB
are equal, while1MB
<1MiB
.
- CI_TRON_NBD__SHORT_NAME__MAX_CONNECTIONS
default:
5
Maximum number of NBD connections that could be initiated by the client to this NBD server.
The information needed for the DUT to connect to the NBD server can be found at
{{ job.nbd.short_name }}
. It contains the following fields:name
: Name of the NBD resource;max_connections
: The content ofCI_TRON_NBD__SHORT_NAME__MAX_CONNECTIONS
;hostname
: Hostname of the NBD server exporting this resource;tcp_port
: TCP port of the NBD server exporting this resource;to_b2c_nbd(name)
: A helper that converts these parameters into a Boot2container-compatible cmdline argument:b2c.nbd={name},...
Collection Of Lists
CI-tron jobs have this unusual concept of a list split as a collection of lists. This feature is used throughout the job configuration to partially inherit or override sets of items from the default configuration.
The concept is simple: lists are splits in multiple categories, and a category is inherited from the defaults only if it is not mentioned at all. Additionally, there is an extra category called
uncategorised
which is the default category if the user choses not to name their list.When the final list is built, the categories are sorted by name then their content is copied in order. Finally, the content of
uncategorised
is added.Warning
Some users of collections of lists may decide to prioritize
uncategorised
elements. In the case of the HTTP and TFTP server, this is done to reduce the surprise factor and make it easier to override defaults paths without having to know about the category they may be inherited from.- LIST_NAME__SHORT_NAME__KEY
Add the element
KEY
to theuncategorised
part of theLIST_NAME
list. The keys are then sorted within this list based on theirSHORT_NAME
before their values being inserted in the list.
- LIST_NAME__RAW__SHORT_NAME
Rather than letting the template manage the elements in the
uncategorised
category of theLIST_NAME
list, this enables users to define the list themselves as a list of YAML fragments, ordered bySHORT_NAME
.This can be used a way to prevent inheritance of defaults by simply setting the value to
''
.Warning
Using
LIST_NAME__RAW__SHORT_NAME
completely overrides anyLIST_NAME__SHORT_NAME__KEY
that may be set.
- LIST_NAME__CATEGORY__CAT_NAME__SHORT_NAME__KEY
Add the element
KEY
to theCAT_NAME
category of theLIST_NAME
list. The keys are then sorted within this list based on theirSHORT_NAME
before their values being inserted in the list.
- LIST_NAME__CATEGORY__CAT_NAME__RAW__SHORT_NAME
Rather than letting the template manage the elements in the
CAT_NAME
category of theLIST_NAME
list, this enables users to define the list themselves as a list of YAML fragments, ordered bySHORT_NAME
.This can be used a way to prevent inheritance of defaults by simply setting the value to
''
.Warning
Using
LIST_NAME__CATEGORY__CAT_NAME__RAW__SHORT_NAME
completely overrides anyLIST_NAME__CATEGORY__CAT_NAME__SHORT_NAME__KEY
that may be set.
Default boot configuration
By default, CI-tron aims to provide a Boot2Container environment to its users. This means that if all you care about is userspace testing, you do not need to worry about bootloaders, kernels, device trees, or initramfses.
In the event you actually care about your boot process, you may configure your kernel using
CI_TRON_KERNEL
, kernel command line usingCI_TRON_KERNEL_CMDLINE
, DTBs usingCI_TRON_DTB
, or initramfses usingCI_TRON_INITRAMFS
.Additionally, the following categories may be inherited from the defaults in
CI_TRON_KERNEL_CMDLINE
,CI_TRON_INITRAMFS
,CI_TRON_HTTP_ARTIFACT
, orCI_TRON_TFTP_ARTIFACT
:50-platform: Anything necessary to boot the machine before reaching the bootloader stage, or firmware that is expected by the platform and is not otherwise part of the linux-firmware project, such as the signed and non-redistributable firwmare files found on Qualcomm hardware.
50-bootloader: Anything necessary to load the environment as specified by the
kernel
,initramfs
, anddtb
deployment options. Place in this category all the bootloader configuration files.50-kernel-modules: Anything necessary to make the default kernel boot to a usable environment (ie. kernel modules, firmware, …). Nothing from this category should be included by default by a job that overrides the default kernel.
Artifacts
CI-tron may serve artifacts to the test machine in various ways and various times.
Rather than duplicating this content throughout the documentation, let’s define it here first.
- ARTIFACT__PATH
This variable is only applicable to artifacts served via HTTP/TFTP
A regular expression that, when it matches the GET request’s path, will return the wanted artifact, as selected by
ARTIFACT__DATA
, orARTIFACT__URL
.If parts of the path are wrapped in parentheses, whatever content is matched may be referenced by
URL
,ARTIFACT__FORMAT__IDX__ARCHIVE__MATCH
,ARTIFACT__FORMAT__IDX__ARCHIVE__KEEP__SHORT_NAME__PATH
, andARTIFACT__FORMAT__IDX__ARCHIVE__KEEP__SHORT_NAME__REWRITE
.
- ARTIFACT__DATA
Back the content of the artifact with the specified string.
If the data to return is more complex than a small string, you may specify it in parts using
ARTIFACT__DATA__IDX
.Warning
This argument takes priority over
ARTIFACT__URL
if set.
- ARTIFACT__DATA__IDX
Back the content of the artifact by aggregating all the
DATA
variables, sorted byIDX
.IDX
does not have to be a number, it can be any string.If the data to return simple-enough, you may specify it using it in parts using
ARTIFACT__DATA
.Warning
Do not mix
ARTIFACT__DATA__IDX
andARTIFACT__DATA
as the result is unspecified.Warning
This argument takes priority over
ARTIFACT__URL
if set.
- ARTIFACT__URL
An HTTP(S) URL to the artifact you want to use.
Warning
This argument is ignored if
ARTIFACT__DATA
orARTIFACT__DATA__IDX
is set.Note
Parts of the URL may reference parts of the request URL, akin to an nginx url_rewrite.
Example: To answer any GET request matching
/linux-firmware/$path
with the corresponding file in the linux-firmware GIT repository, you could set the following variables:CI_TRON_HTTP_ARTIFACT__FW__PATH: /linux-firmware/(.*)
CI_TRON_HTTP_ARTIFACT__FW__URL: https://web.git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain/\1
U-Boot formatting
If you would like the artifact returned by the HTTP server to be a U-Boot uimage, you should at least specify
ARTIFACT__FORMAT__IDX__UBOOT__TYPE
, withIDX
the position in the formatting pipeline.- ARTIFACT__FORMAT__IDX__UBOOT__TYPE
This variable is required if wanting to serve a U-Boot image
Wrap the artifact returned by
ARTIFACT__PATH
as a u-boot image of the specified type. Supported values can be found at https://github.com/u-boot/u-boot/blob/master/include/image.h, without theIH_TYPE_
prefix and in lowercase.
- ARTIFACT__FORMAT__IDX__UBOOT__ARCHITECTURE
Supported values:
x86
,x86_64
,arm32
,arm64
(default),riscv32
,riscv64
Wrap the artifact returned by
ARTIFACT__PATH
as a u-boot image, indicating that the CPU architecture is the specified value.
- ARTIFACT__FORMAT__IDX__UBOOT__COMPRESSION
Supported values:
none
Wrap the artifact returned by
ARTIFACT__PATH
as a u-boot image, indicating that the compression type is the specified value.
- ARTIFACT__FORMAT__IDX__UBOOT__OS
Supported values:
linux
Wrap the artifact returned by
PATH
as a u-boot image, indicating that the Operating System is the specified value.
Archive formatting
If the artifact returned by the HTTP server is an archive, you may modify its content on the fly to, either:
Return one file from the archive by using
ARTIFACT__FORMAT__IDX__ARCHIVE__MATCH
Create a new archive which contains a subset of the files present in the original archive by using
ARTIFACT__FORMAT__IDX__ARCHIVE__KEEP__SHORT_NAME__PATH
. The returned files may be re-located to a different location by usingARTIFACT__FORMAT__IDX__ARCHIVE__KEEP__SHORT_NAME__REWRITE
.
In either case, the path of the files present in the original archive may reference parts of the request URL. This is useful when trying to find the corresponding file in an archive.
- ARTIFACT__FORMAT__IDX__ARCHIVE__EXTENSION
Supported values:
none
,cpio
,iso
,tar
,zip
.Name of the archive format that should be created as a result of this formatting operation. Leave the value unset, or set to
none
if you just want to extract a single file from the archive usingARTIFACT__FORMAT__IDX__ARCHIVE__MATCH
.
- ARTIFACT__FORMAT__IDX__ARCHIVE__COMPRESSION
Supported values:
none
,gz
,bz2
.Compression format that should be applied to the archive that should be created as a result of this formatting operation. Leave the value unset, or set it to
none
if you do not want to compress the archive.
- ARTIFACT__FORMAT__IDX__ARCHIVE__MATCH
Rather than creating a new archive, return the first file that matches the regular expression specified. Files are evaluated in the same order as the archive defines them.
Note
Parts of the specified path may reference parts of the request URL, akin to an nginx url_rewrite.
Example: Answer any TFTP read request matching /dtb/$path or /dtbs/$path with the file located at boot/dtbs/$path in the archive found at the specified URL, then wrap it in a U-Boot image:
CI_TRON_TFTP_ARTIFACT__DTB__PATH: /dtbs?/(.*)
CI_TRON_TFTP_ARTIFACT__DTB__URL: https://gitlab.freedesktop.org/gfx-ci/boot2container/-/releases/v0.9.15.1/downloads/linux-arm64.dtbs.cpio.xz
CI_TRON_TFTP_ARTIFACT__DTB__FORMAT__0__ARCHIVE__MATCH: boot/dtbs/\1
CI_TRON_TFTP_ARTIFACT__DTB__FORMAT__1__UBOOT__TYPE: flatdt
- ARTIFACT__FORMAT__IDX__ARCHIVE__KEEP__IDX__PATH
Allow filtering the content of the source archive to only contain the file paths that match the specified regular expression.
This value may be paired with
ARTIFACT__FORMAT__IDX__ARCHIVE__KEEP__IDX__REWRITE
in order to change the path of the files kept.Note
Parts of the specified path may reference parts of the request URL, akin to an nginx url_rewrite, similarly to
ARTIFACT__FORMAT__IDX__ARCHIVE__MATCH
.
- ARTIFACT__FORMAT__IDX__ARCHIVE__KEEP__IDX__REWRITE
Change the location of the files matched by
ARTIFACT__FORMAT__IDX__ARCHIVE__KEEP__IDX__PATH
using this substitution regular expression.Note
Parts of the specified path may reference parts of the request URL, akin to an nginx url_rewrite.
Example: Create a CPIO archive that contains all the Raspberry-Pi-firmware-provided modules for the arm64 kernel, but moved to
usr/lib/modules/$KERNELVERSION-v8+/
rather than the originalfirmware-$VERSION/modules/$KERNELVERSION-v8+/
and use it as an initramfs.CI_TRON_INITRAMFS__RPIFW__URL: https://github.com/raspberrypi/firmware/archive/b154632e320b87ea95c6ce8b59f96dbbe523ecf1.zip
CI_TRON_INITRAMFS__RPIFW__FORMAT__ARCHIVE__EXTENSION: cpio
CI_TRON_INITRAMFS__RPIFW__FORMAT__ARCHIVE__KEEP__0__PATH: firmware-.*/(modules/.*-v8+/.*)
CI_TRON_INITRAMFS__RPIFW__FORMAT__ARCHIVE__KEEP__0__REWRITE: usr/lib/1
HTTP Server
CI-tron allows you to instantiate an HTTP server which can expose read-only artifacts at any path wanted.
The URL to this web server can be referenced by using
{{ job.http.url }}
.You may also get the full URL to the test environment artifacts by using
{{ job.http.path_to("$target") }}
with $target being any of:kernel
initramfs
,initramfs.IDX
dtb
,dtb.IDX
- CI_TRON_HTTP_ARTIFACT
A collection of lists of artifacts. The uncategorised part of the list is evaluated first in lexicographic order of the keys, followed by every category (in lexicographic order), themselves sorted in lexicographic order.
Example
To answer any GET request matching
/linux-firmware/$path
with the corresponding file in the linux-firmware GIT repository, you could set the following variables:CI_TRON_HTTP_ARTIFACT__FW__PATH: /linux-firmware/(.*)
CI_TRON_HTTP_ARTIFACT__FW__URL: https://web.git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain/\1
TFTP Server
CI-tron allows you to instantiate a TFTP server which can expose read-only artifacts at any path wanted.
You may also get the full URL to the test environment artifacts by using
{{ job.tftp.path_to("$target") }}
with $target being any of:kernel
initramfs
,initramfs.IDX
dtb
,dtb.IDX
- CI_TRON_TFTP_ARTIFACT
A collection of lists of artifacts. The uncategorised part of the list is evaluated first in lexicographic order of the keys, followed by every category (in lexicographic order), themselves sorted in lexicographic order.
Example
Answer any TFTP read request matching /dtb/$path or /dtbs/$path with the file located at boot/dtbs/$path in the archive found at the specified URL, then wrap it in a U-Boot image:
CI_TRON_TFTP_ARTIFACT__DTB__PATH: /dtbs?/(.*)
CI_TRON_TFTP_ARTIFACT__DTB__URL: https://gitlab.freedesktop.org/gfx-ci/boot2container/-/releases/v0.9.15.1/downloads/linux-arm64.dtbs.cpio.xz
CI_TRON_TFTP_ARTIFACT__DTB__FORMAT__0__ARCHIVE__MATCH: boot/dtbs/\1
CI_TRON_TFTP_ARTIFACT__DTB__FORMAT__1__UBOOT__TYPE: flatdt
Kernel
As we saw in the Default boot configuration section, CI-tron provides a default kernel that is suitable for a Boot2container environment. You may however decide to override it using
CI_TRON_KERNEL
.You may also want to control the kernel command line. To do so, you will need to use
CI_TRON_KERNEL_CMDLINE
- CI_TRON_KERNEL
An artifact to override CI-tron’s default kernel with.
Example
CI_TRON_KERNEL__URL: https://gitlab.freedesktop.org/gfx-ci/boot2container/-/releases/v0.9.15.1/downloads/linux-arm64
- CI_TRON_KERNEL_CMDLINE
A collection of lists of strings that allow configuring the kernel command line in parts. The final command line will be the concatenation of all the fragments, ordered by their short name.
Example
Given the following variables set:
CI_TRON_KERNEL_CMDLINE__02_SECOND: world
CI_TRON_KERNEL_CMDLINE__01_FIRST: hello
The generated command line will be
hello world
.
- CI_TRON_INITRAMFS
A collection of lists of artifacts which will be provided as initramfses to the test machine.
These initramfses may be downloaded via HTTP or TFTP using:
{{ job.tftp.path_to("$target") }}
{{ job.http.path_to("$target") }}
With target being:
initramfs
: a concatenation of all the initramfses specified in the list;initramfs.N
: The N-th initramfs specified. Useful when the bootloader supports more than one initramfs and downloads them through TFTP which can have a size limit as low as 32MB.
- CI_TRON_DTB
A collection of lists of artifacts which may be used as device tree binaries for the bootloader / kernel to use. If more than one DTB is provided, the behavior will differ depending on the boot chain.
These DTBs may be downloaded via HTTP or TFTP using:
{{ job.tftp.path_to("$target") }}
{{ job.http.path_to("$target") }}
With target being:
dtb
: a concatenation of all the DTB specified in the list;dtb.N
: The N-th DTB specified.
Warning
Some boot methods only allow specifying one DTB. In this case, only the first DTB will be used.
User variables
If you wish to define variables that are related to job configuration but don’t want to accidentally influence the template generation, you may prefix them with
CI_TRON__
.Just like any other variable, they may be referenced using
{{ environ.CI_TRON__VARIABLE_NAME }}
.Executorctl parameters
CI-tron does not yet have true native support for GitLab. This job is an attempt to hide the fact that we need to call
executorctl
to queue the job. Here are variables that can be used to tune this behaviour.- CI_TRON_JOB_TEMPLATE
default:
${CI_TRON_JOB_TEMPLATE_PROJECT_URL}/-/raw/${CI_TRON_JOB_TEMPLATE_COMMIT}/.gitlab-ci/ci-tron-job-dut-v1.yml.j2
Path or URL to the job template that will be used to convert GitLab environment variables to the expected YAML job description CI-tron expects.
- CI_TRON_JOB_TEMPLATE_PROJECT_URL
This variable is required if CI_TRON_JOB_TEMPLATE is unset
URL (eg. https://gitlab.freedesktop.org/gfx-ci/ci-tron) of the project that is hosting the job template. Mandatory if you did not override
CI_TRON_JOB_TEMPLATE
, ignored if you did.
- CI_TRON_JOB_TEMPLATE_COMMIT
This variable is required if CI_TRON_JOB_TEMPLATE is unset
Commit-like of the job template that should be used. Mandatory if you did not override
CI_TRON_JOB_TEMPLATE
, ignored if you did.
- CI_TRON_JOB_SCRIPT_PATH
Path where the job script is saved; set this to a path in your job artifacts if you want to keep the job script.
- CI_TRON_JOB_SCRIPT_IGNORE_VARS_MATCHING
Regex of names of variables that contain values that should not be written down in the job script.
- CI_TRON_EXECUTORCTL_EXTRA_ARGS__SHORT_NAME
Extra parameters that will be passed to executorctl, in the same way a collection of lists would.
- .ci-tron-b2c-job-v1
The Boot2container-based job which enables a more seamless integration with GitLab CI.
This job is inheriting from
.ci-tron-job-v1
, and thus all the attributes it specifies are available. Please however avoid setting any variable calledCI_TRON__B2C_
outside of the ones documented under.Support matrix
Gitlab Feature
Support
Notes
⚠
Unsupported and will break in the future
✅
⚠
Unsupported and will break in the future
✅
✅
Not as efficient as using an HTTP artifact, can be improved
✅
❌
Use the
CI_TRON__B2C_IMAGE_UNDER_TEST
variable instead✅
Not as efficient as using an HTTP artifact, can be improved
❌
Unsupported. Use a general-purpose runner for this.
❌
Unsupported. Use a general-purpose runner for this.
✅
✅
❌
Unsupported, but may be supported in the future
⚠
Unsupported. May be emulated using
CI_TRON_HTTP_ARTIFACT__B2C_CFG__DATA__03_MY_SERVICE=b2c.run_service=...
❌
Use the
CI_TRON__B2C_EXEC_CMD
variable instead⚠
Use the
CI_TRON_TIMEOUT__OVERALL__MINUTES
variable to set the timeout, then ensuretimeout
is set to a value greater than it by at least 5 minutes.✅
Introductory example
Here is a simplistic example of how to integrate a CI-tron-provided DUT in your in GitLab CI pipelines:
.gitlab-ci.yml
:stages: - integration testing variables: # WARNING: it is strongly advised to use a commit hash instead of ``main`` CI_TRON_JOB_TEMPLATE_PROJECT_URL: https://gitlab.freedesktop.org/gfx-ci/ci-tron CI_TRON_JOB_TEMPLATE_COMMIT: main include: - '${CI_TRON_JOB_TEMPLATE_PROJECT_URL}/-/raw/${CI_TRON_JOB_TEMPLATE_COMMIT}/.gitlab-ci/dut.yml' pre-test: stage: integration testing variables: GIT_STRATEGY: none image: registry.freedesktop.org/gfx-ci/ci-tron/executorctl:latest script: - mkdir -p result - echo file1 > result/file1 - echo file2 > result/file2 artifacts: paths: - result # Concatenate the content of file1 and file2 then save it in file3. # Check out the generated artifacts to see file3. .test: stage: integration testing needs: - job: pre-test artifacts: true extends: - .ci-tron-job-v1 variables: GIT_STRATEGY: none CI_TRON__B2C_IMAGE_UNDER_TEST: 'registry.freedesktop.org/gfx-ci/ci-tron/machine-registration:latest' CI_TRON__B2C_EXEC_CMD: 'cat result/file1 result/file2 | tee result/file3' artifacts: paths: - result # Run .test on an amd64 DUT test-amd64: extends: - .test tags: - cpu:arch:x86_64 - farm:$FARM_NAME # Run .test on an arm64 DUT test-arm64: extends: - .test tags: - cpu:arch:aarch64 - farm:$FARM_NAME
Container images
Note
Test machines should access containers through proxy registries that de-duplicate image downloads and thus decrease boot time and increase boot reliability.
You may use the
registry.to_local_proxy()
function to convert an image name to one pointing towards its designated proxy, according to the rules configured in Registry config.yml’s format.Example:
{{ registry.to_local_proxy('${CI_TRON__B2C_CUSTOM_IMAGE}') }}
- CI_TRON__B2C_IMAGE_UNDER_TEST
This variable is required
The image you want to run, fully-qualified or with
docker://
as a prefix.
- CI_TRON__B2C_EXEC_CMD
This variable is required
The command or list of commands that should be executed, using the shell specified by
CI_TRON__B2C_EXEC_SHELL
.This variable may contain multiple lines, but should not contain double
"
, nor unescaped single quotes.
- CI_TRON__B2C_EXEC_SHELL
default:
sh
The shell that should be used to execute the
CI_TRON__B2C_EXEC_CMD
commands.
- CI_TRON__B2C_EXEC_SHELL_FLAGS
default:
-euc
Shell flags that should be set before executing the
CI_TRON__B2C_EXEC_CMD
in theCI_TRON__B2C_EXEC_SHELL
.
- CI_TRON__B2C_MACHINE_REGISTRATION_IMAGE
default:
registry.freedesktop.org/gfx-ci/ci-tron/machine-registration:latest
The container image to use to check that the test machine running the job really is the one we expected it to be.
- CI_TRON__B2C_MACHINE_REGISTRATION_CMD
default:
check
Machine registration command to execute. Could be set to
setup --tags TAG1,TAG2,...
instead to only check for the presence of the wanted tags and hiding un-wanted hardware.See also
Check out the Machine Registration Container for more information about the acceptable commands.
- CI_TRON__B2C_TELEGRAF_IMAGE
default:
registry.freedesktop.org/gfx-ci/ci-tron/telegraf:latest
The container image to use for monitoring of the test machine.
- CI_TRON__B2C_CACHE_DEVICE
default:
auto
The block device to use as a cache device. The supported values are specified under b2c.cache_device in B2C’s README.
See also
Please consider using
.ci-tron-b2c-diskless-v1
if you would like testing to be done in a diskless fashion.
- CI_TRON__B2C_POWEROFF_DELAY
default:
15
Number of seconds the test machine should remain on after the end of the tests, waiting for CI-tron to shut down the power.
- CI_TRON__B2C_SHARE_FOLDER_PATH
default:
./
Folder you would like to forward to the DUT, relative to the start working dir of the container image, i.e.
$CI_PROJECT_DIR
on GitLab instance.
- CI_TRON__B2C_SWAP_SIZE
default:
8G
Size of the swap partition/file that should be used for the job.
- CI_TRON__B2C_SWAP_SWAPPINESS
default:
60
Swappiness setting to use for the swap set by
CI_TRON__B2C_SWAP_SIZE
- .ci-tron-b2c-diskless-v1
If a job extends from the
.ci-tron-b2c-job-v1
job, it may also inherit from this one to make the DUT operate in a disk-less mode:Containers are pulled and stored on the gateway’s public image store which is shared between all the DUTs;
The DUT accesses the gateway’s image store through a read-only NFS share exported by the gateway, and by telling b2c’s podman to use this share as an additional image store;
Local storage on the DUT is handled through a Network Block Device exported by the gateway;
Swap on the DUT is handled through a separate Network Block Device exported by the gateway.
Because the containers are now pulled by the gateway instead of the DUT, it is important to specify the architecture / platform of the test device. You may do so by using
CI_TRON__B2C_DISKLESS_IMAGESTORE_PLATFORM
.Introductory example
Here is a simplistic example of how to integrate a CI-tron-provided DUT in your in GitLab CI pipelines:
.gitlab-ci.yml
:stages: - integration testing variables: # WARNING: it is strongly advised to use a commit hash instead of ``main`` CI_TRON_JOB_TEMPLATE_PROJECT_URL: https://gitlab.freedesktop.org/gfx-ci/ci-tron CI_TRON_JOB_TEMPLATE_COMMIT: main include: - '${CI_TRON_JOB_TEMPLATE_PROJECT_URL}/-/raw/${CI_TRON_JOB_TEMPLATE_COMMIT}/.gitlab-ci/dut.yml' test: stage: integration testing extends: - .ci-tron-b2c-job-v1 - .ci-tron-b2c-diskless-v1 variables: CI_TRON__B2C_DISKLESS_IMAGESTORE_PLATFORM: linux/amd64 # Same variables you would set in ``.ci-tron-b2c-job-v1``
- CI_TRON__B2C_DISKLESS_IMAGESTORE_PLATFORM
This variable is required
Typical values: linux/amd64, linux/arm64/v8, linux/arm/v6, linux/riscv64
The platform of the container that should be pulled and used for testing.
- CI_TRON_NBD__STORAGE__SIZE
default:
10G
Specifies the size of the Network Block Device that will be created on the gateway and used by the DUT as storage for any file created during the job.
- CI_TRON_NBD__STORAGE__MAX_CONNECTIONS
default:
5
Number of NBD connections that should be used to access the storage block device.
- CI_TRON_NBD__SWAP__MAX_CONNECTIONS
default:
5
Number of NBD connections that should be used to access the swap block device.
Note
The swap size can be controlled using
CI_TRON__B2C_SWAP_SIZE
.
- CI_TRON__B2C_IMAGESTORE_FS_TYPE
Supported values:
nfs
(default),cifs
Network filesystem that should be used to access the imagestore.