Raspberrypi 4B 运行 Jailhouse

[Jiaming]

Raspberrypi 4B 运行 Jailhouse

目录

• Raspberrypi 4B 运行 Jailhouse
• 1. 环境准备
• 2. git clone jailhouse source
• 3. 编译安装 Jailhouse
• 3.1 出现问题
• 4. 使能 Jailhouse
• 4.1 出现问题
• 5. 解决之前make & make install 时发生的 warning 问题
• 6. Jailhoue常用命令
• 7. Jailhouse运行状态
• 8. 创建 linux cell
• 9. 创建 inmate-demo 裸机程序
• 10. 替换内核
• 11. 创建 rootfs.cpio 文件

1. 环境准备

1. RaspberryPi 4B
2. TXD、RXD和GND串口连接
3. 使用Raspberry Pi Imager烧录系统Ubuntu 20.04 LTS server 64bit(RaspberryPi 3/4)至SD卡
4. 连接网线，启动树莓派
5. 刚开始进入时，需要重新设置密码
6. sudo apt-get install -y vim git make net-tools gcc
7. sudo apt-get install build-essential libncurses* libssl-dev bison flex

2. git clone jailhouse source

git clone -b v0.12 https://github.com/siemens/jailhouse.git # 尤其注意下载0.12版本，master版本可能不行！ 

3. 编译安装 Jailhouse

编译：

# cd jailhouse/
ubuntu@ubuntu:~/jailhouse$ make
  UPD     /home/ubuntu/jailhouse/hypervisor/include/generated/config.mk
  CC      /home/ubuntu/jailhouse/configs/arm64/amd-seattle-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/amd-seattle-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/amd-seattle-linux-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/amd-seattle-linux-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/amd-seattle.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/amd-seattle.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/espressobin-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/espressobin-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/espressobin-linux-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/espressobin-linux-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/espressobin.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/espressobin.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/foundation-v8-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/foundation-v8-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/foundation-v8-linux-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/foundation-v8-linux-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/foundation-v8.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/foundation-v8.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/hikey-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/hikey-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/hikey-linux-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/hikey-linux-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/hikey.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/hikey.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/imx8mq-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/imx8mq-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/imx8mq.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/imx8mq.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/jetson-tx1-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/jetson-tx1-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/jetson-tx1-linux-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/jetson-tx1-linux-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/jetson-tx1.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/jetson-tx1.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/jetson-tx2-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/jetson-tx2-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/jetson-tx2.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/jetson-tx2.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/k3-am654-idk-linux-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/k3-am654-idk-linux-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/k3-am654-idk.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/k3-am654-idk.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/k3-am654-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/k3-am654-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/k3-j721e-evm-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/k3-j721e-evm-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/k3-j721e-evm-linux-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/k3-j721e-evm-linux-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/k3-j721e-evm.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/k3-j721e-evm.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/macchiatobin-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/macchiatobin-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/macchiatobin-linux-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/macchiatobin-linux-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/macchiatobin.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/macchiatobin.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/miriac-sbc-ls1046a-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/miriac-sbc-ls1046a-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/miriac-sbc-ls1046a-linux-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/miriac-sbc-ls1046a-linux-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/miriac-sbc-ls1046a.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/miriac-sbc-ls1046a.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/qemu-arm64-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/qemu-arm64-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/qemu-arm64-linux-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/qemu-arm64-linux-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/qemu-arm64.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/qemu-arm64.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/rpi4-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/rpi4-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/rpi4-linux-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/rpi4-linux-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/rpi4.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/rpi4.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/ultra96-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/ultra96-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/ultra96-linux-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/ultra96-linux-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/ultra96.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/ultra96.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/zynqmp-zcu102-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/zynqmp-zcu102-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/zynqmp-zcu102-linux-demo-2.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/zynqmp-zcu102-linux-demo-2.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/zynqmp-zcu102-linux-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/zynqmp-zcu102-linux-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/zynqmp-zcu102.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/zynqmp-zcu102.cell
  DTC     /home/ubuntu/jailhouse/configs/arm64/dts/inmate-amd-seattle.dtb
  DTC     /home/ubuntu/jailhouse/configs/arm64/dts/inmate-espressobin.dtb
  DTC     /home/ubuntu/jailhouse/configs/arm64/dts/inmate-foundation-v8.dtb
  DTC     /home/ubuntu/jailhouse/configs/arm64/dts/inmate-hikey.dtb
  DTC     /home/ubuntu/jailhouse/configs/arm64/dts/inmate-jetson-tx1.dtb
  DTC     /home/ubuntu/jailhouse/configs/arm64/dts/inmate-k3-am654-idk.dtb
  DTC     /home/ubuntu/jailhouse/configs/arm64/dts/inmate-k3-j721e-evm.dtb
  DTC     /home/ubuntu/jailhouse/configs/arm64/dts/inmate-macchiatobin.dtb
  DTC     /home/ubuntu/jailhouse/configs/arm64/dts/inmate-miriac-sbc-ls1046a.dtb
  DTC     /home/ubuntu/jailhouse/configs/arm64/dts/inmate-qemu-arm64.dtb
  DTC     /home/ubuntu/jailhouse/configs/arm64/dts/inmate-rpi4.dtb
  DTC     /home/ubuntu/jailhouse/configs/arm64/dts/inmate-zynqmp-zcu102-2.dtb
  DTC     /home/ubuntu/jailhouse/configs/arm64/dts/inmate-zynqmp.dtb
  UPD     /home/ubuntu/jailhouse/hypervisor/include/generated/version.h
  CC [M]  /home/ubuntu/jailhouse/driver/cell.o
  CC [M]  /home/ubuntu/jailhouse/driver/main.o
  CC [M]  /home/ubuntu/jailhouse/driver/sysfs.o
  CC [M]  /home/ubuntu/jailhouse/driver/pci.o
  DTC     /home/ubuntu/jailhouse/driver/vpci_template.dtb
  DTB     /home/ubuntu/jailhouse/driver/vpci_template.dtb.S
  AS [M]  /home/ubuntu/jailhouse/driver/vpci_template.dtb.o
  LD [M]  /home/ubuntu/jailhouse/driver/jailhouse.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/asm-defines.s
  GEN     /home/ubuntu/jailhouse/hypervisor/arch/arm64/include/generated/asm/asm-defines.h
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/control.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/dbg-write.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/gic-v2.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/gic-v3.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/irqchip.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/ivshmem.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/lib.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/mmu_cell.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/paging.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/pci.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/psci.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/setup.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/smccc.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/uart-hscif.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/uart-imx.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/uart-mvebu.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/uart-pl011.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/uart-scifa.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/uart-xuartps.o
  AS      /home/ubuntu/jailhouse/hypervisor/arch/arm64/caches.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/control.o
  AS      /home/ubuntu/jailhouse/hypervisor/arch/arm64/entry.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/iommu.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/mmio.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/paging.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/setup.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/smmu-v3.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/ti-pvu.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/traps.o
  AR      /home/ubuntu/jailhouse/hypervisor/arch/arm64/lib.a
  LDS     /home/ubuntu/jailhouse/hypervisor/hypervisor.lds
  CC      /home/ubuntu/jailhouse/hypervisor/setup.o
  CC      /home/ubuntu/jailhouse/hypervisor/printk.o
  CC      /home/ubuntu/jailhouse/hypervisor/paging.o
  CC      /home/ubuntu/jailhouse/hypervisor/control.o
  CC      /home/ubuntu/jailhouse/hypervisor/lib.o
  CC      /home/ubuntu/jailhouse/hypervisor/mmio.o
  CC      /home/ubuntu/jailhouse/hypervisor/pci.o
  CC      /home/ubuntu/jailhouse/hypervisor/ivshmem.o
  CC      /home/ubuntu/jailhouse/hypervisor/uart.o
  CC      /home/ubuntu/jailhouse/hypervisor/uart-8250.o
  LD      /home/ubuntu/jailhouse/hypervisor/hypervisor.o
  OBJCOPY /home/ubuntu/jailhouse/hypervisor/jailhouse.bin
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/../alloc.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/../cmdline.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/../pci.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/../printk.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/../setup.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/../string.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/../uart-8250.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/gic-v2.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/gic-v3.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/gic.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/mem.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/pci.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/printk.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/setup.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/timing.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/uart-hscif.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/uart-imx.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/uart-mvebu.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/uart-pl011.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/uart-scifa.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/uart-xuartps.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/uart.o
  AS      /home/ubuntu/jailhouse/inmates/lib/arm64/header.o
  AR      /home/ubuntu/jailhouse/inmates/lib/arm64/lib.a
  LDS     /home/ubuntu/jailhouse/inmates/lib/arm64/inmate.lds
  CC      /home/ubuntu/jailhouse/inmates/demos/arm64/../arm/gic-demo.o
  LD      /home/ubuntu/jailhouse/inmates/demos/arm64/gic-demo-linked.o
  OBJCOPY /home/ubuntu/jailhouse/inmates/demos/arm64/gic-demo.bin
  CC      /home/ubuntu/jailhouse/inmates/demos/arm64/../arm/uart-demo.o
  LD      /home/ubuntu/jailhouse/inmates/demos/arm64/uart-demo-linked.o
  OBJCOPY /home/ubuntu/jailhouse/inmates/demos/arm64/uart-demo.bin
  CC      /home/ubuntu/jailhouse/inmates/demos/arm64/../ivshmem-demo.o
  LD      /home/ubuntu/jailhouse/inmates/demos/arm64/ivshmem-demo-linked.o
  OBJCOPY /home/ubuntu/jailhouse/inmates/demos/arm64/ivshmem-demo.bin
  CC      /home/ubuntu/jailhouse/inmates/tools/arm64/linux-loader.o
  LD      /home/ubuntu/jailhouse/inmates/tools/arm64/linux-loader-linked.o
  OBJCOPY /home/ubuntu/jailhouse/inmates/tools/arm64/linux-loader.bin
  GEN     /home/ubuntu/jailhouse/pyjailhouse/pci_defs.py
WARNING: Could not create the helper script to generate configurations on remote machines ("jailhouse-conf-collect"). You need Python and the Mako library for it.
  CC      /home/ubuntu/jailhouse/tools/jailhouse.o
  LD      /home/ubuntu/jailhouse/tools/jailhouse
  CC      /home/ubuntu/jailhouse/tools/ivshmem-demo.o
  LD      /home/ubuntu/jailhouse/tools/ivshmem-demo
  CC      /home/ubuntu/jailhouse/tools/jailhouse-gcov-extract.o
  LD      /home/ubuntu/jailhouse/tools/jailhouse-gcov-extract
  GEN     /home/ubuntu/jailhouse/tools/jailhouse.8
  GEN     /home/ubuntu/jailhouse/tools/jailhouse-cell.8
  GEN     /home/ubuntu/jailhouse/tools/jailhouse-enable.8
  Building modules, stage 2.
  MODPOST 1 modules
  CC [M]  /home/ubuntu/jailhouse/driver/jailhouse.mod.o
  LD [M]  /home/ubuntu/jailhouse/driver/jailhouse.ko

安装：

ubuntu@ubuntu:~/jailhouse$ sudo make install
WARNING: Could not create the helper script to generate configurations on remote machines ("jailhouse-conf-collect"). You need Python and the Mako library for it.
  Building modules, stage 2.
  MODPOST 1 modules
  INSTALL /home/ubuntu/jailhouse/driver/jailhouse.ko
At main.c:160:
- SSL error:02001002:system library:fopen:No such file or directory: ../crypto/bio/bss_file.c:69
- SSL error:2006D080:BIO routines:BIO_new_file:no such file: ../crypto/bio/bss_file.c:76
sign-file: certs/signing_key.pem: No such file or directory
  DEPMOD  5.4.0-1055-raspi
Warning: modules_install: missing 'System.map' file. Skipping depmod.
install -m 644 hypervisor/jailhouse*.bin /lib/firmware
install -m 644 inmates/tools/arm64/*.bin /usr/local/libexec/jailhouse
WARNING: Could not create the helper script to generate configurations on remote machines ("jailhouse-conf-collect"). You need Python and the Mako library for it.
WARNING: Could not install Python-based helpers. You need Python and pip in order to install them.
install jailhouse ivshmem-demo /usr/local/sbin
install -m 644 jailhouse-completion.bash /usr/share/bash-completion/completions/jailhouse
install -m 644 jailhouse.8 jailhouse-cell.8 jailhouse-enable.8 /usr/local/share/man/man8

3.1 出现问题

为了稳定，树莓派内核将使用 5.4.16(linux-jailhouse-rpi4)，而非树莓派原生内核！参考文末内核替换过程。
内核源码以及内核配置文件路径（来自 jailhouse-images 工程，参考添加bl31.bin文件小节）：/home/jiaming/Documents/jailhouse-images-v0.12/build/tmp/work/jailhouse-demo-arm64/linux-jailhouse-rpi/5.4.16-r0/linux-e569bd2d6d2d7b958973bb8c6e9db9cfc05c790b

这里以树莓派原生内核编译 Jailhouse 举例：

解决办法

1. 先查看当前正在使用的内核

2. 修改Makefile.lib文件

sudo vim /usr/src/linux-headers-5.4.0-1011-raspi/scripts/Makefile.lib
找到DTC_INCLUDE
下面添加
DTC_INCLUDE    += /usr/src/linux-headers-5.4.0-1011-raspi/include

3. 保存后在 Jailhouse 目录下，make，sudo make install。

4. 使能 Jailhouse

使用 ssh 登录，操作如下命令，在串口中会输出 jailhouse 运行信息。

ubuntu@ubuntu:~/jailhouse$ sudo insmod driver/jailhouse.ko # 如果 Invalid argument，则make clean，重新make即可，或者是启动了其它内核，修改对应Makefile.lib文件，make && sudo make install 即可
ubuntu@ubuntu:~/jailhouse$ sudo jailhouse enable configs/arm64/rpi4.cell # 如果发生ssh断连，参考4.1出现问题
ubuntu@ubuntu:~/jailhouse$ jailhouse cell list 
ID      Name                    State             Assigned CPUs           Failed CPUs             
0       Raspberry-Pi4           running           0-3                                             
ubuntu@ubuntu:~/jailhouse$ sudo jailhouse console 

Initializing Jailhouse hypervisor v0.12 (0-g92db71f2) on CPU 0
Code location: 0x0000ffffc0200800
Page pool usage after early setup: mem 39/994, remap 0/131072
Initializing processors:
 CPU 0... OK
 CPU 2... OK
 CPU 1... OK
 CPU 3... OK
Initializing unit: irqchip
Initializing unit: ARM SMMU v3
Initializing unit: PVU IOMMU
Initializing unit: PCI
Adding virtual PCI device 00:00.0 to cell "Raspberry-Pi4"
Adding virtual PCI device 00:01.0 to cell "Raspberry-Pi4"
Page pool usage after late setup: mem 61/994, remap 5/131072
Activating hypervisor
ubuntu@ubuntu:~/jailhouse$ sudo jailhouse cell create configs/arm64/rpi4-inmate-demo.cell 
ubuntu@ubuntu:~/jailhouse$ jailhouse cell list 
ID      Name                    State             Assigned CPUs           Failed CPUs             
0       Raspberry-Pi4           running           0,2-3                                           
1       inmate-demo             shut down         1                                               
ubuntu@ubuntu:~/jailhouse$ sudo jailhouse cell load inmate-demo inmates/demos/arm64/gic-demo.bin 
ubuntu@ubuntu:~/jailhouse$ sudo jailhouse cell start inmate-demo  

4.1 出现问题

解决办法

问题原因：jailhouse enable 阶段需要 offline 所有 CPU，但是实际情况并没有真正 offline，我们需要一个补丁来解决该问题。

1. 添加bl31.bin文件

# 关闭树莓派，拔出SD卡
git clone -b v0.12 https://github.com/siemens/jailhouse-images.git
cd jailhouse-images/
sudo ./build-images.sh # 选择raspberry pi4
# 得到bl31.bin文件
~/Documents/jailhouse-images-v0.12/jailhouse-images/build/tmp/work/jailhouse-demo-arm64/arm-trusted-firmware-rpi4/2.2-r0/arm-trusted-firmware-2.2/build/rpi4/release/bl31.bin
# 将该文件拷贝到boot分区中

2. 修改配置文件

修改cmdline.txt和config.txt文件，cmdline.txt 注意添加参数 mem=768M 即可。

修改 cmdline.txt 文件内容为：

elevator=deadline net.ifnames=0 console=serial0,115200 dwc_otg.lpm_enable=0 console=tty1 root=LABEL=writable rootfstype=ext4 rootwait fixrtc quiet splash mem=768M

修改 config.txt 文件内容为：

kernel=uboot_rpi_4.bin
device_tree_address=0x03000000
max_framebuffers=2
arm_boost=1
dtparam=audio=on
dtparam=i2c_arm=on
dtparam=spi=on
disable_overscan=1
arm_64bit=1
dtoverlay=dwc2
enable_uart=1
cmdline=cmdline.txt
enable_gic=1
armstub=bl31.bin

include syscfg.txt
include usercfg.txt

错误解决。

5. 解决之前make & make install 时发生的 warning 问题

产生warning的原因是没有安装Python脚本所需要的工具。

WARNING: Could not create the helper script to generate configurations on remote machines ("jailhouse-conf-collect"). You need Python and the Mako library for it.
...

# 安装python相关工具
sudo apt-get install python
sudo apt-get instal aptitude
sudo apt-get install python3-pip
sudo apt-get install python3-mako
aptitude search python
sudo apt-get install python-is-python3 # ubuntu18.04不支持该命令，可以手动实现，类似这样，将python指向python3，q@q-GA-MA785GMT-US2H:~/jailhouse$ ln -s /usr/bin/python3 /usr/bin/python

ubuntu@ubuntu:~/jailhouse$ make clean
  CLEAN   /home/ubuntu/jailhouse/configs
  CLEAN   /home/ubuntu/jailhouse/driver
  CLEAN   /home/ubuntu/jailhouse/hypervisor/arch/arm64
  CLEAN   /home/ubuntu/jailhouse/hypervisor
  CLEAN   /home/ubuntu/jailhouse/inmates/lib/arm64
  CLEAN   /home/ubuntu/jailhouse/inmates/demos/arm64
  CLEAN   /home/ubuntu/jailhouse/inmates/tools/arm64
  CLEAN   /home/ubuntu/jailhouse/tools
  CLEAN   /home/ubuntu/jailhouse
  CLEAN   /home/ubuntu/jailhouse/Module.symvers
ubuntu@ubuntu:~/jailhouse$ make
  CC      /home/ubuntu/jailhouse/configs/arm64/amd-seattle-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/amd-seattle-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/amd-seattle-linux-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/amd-seattle-linux-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/amd-seattle.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/amd-seattle.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/espressobin-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/espressobin-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/espressobin-linux-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/espressobin-linux-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/espressobin.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/espressobin.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/foundation-v8-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/foundation-v8-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/foundation-v8-linux-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/foundation-v8-linux-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/foundation-v8.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/foundation-v8.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/hikey-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/hikey-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/hikey-linux-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/hikey-linux-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/hikey.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/hikey.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/imx8mq-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/imx8mq-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/imx8mq.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/imx8mq.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/jetson-tx1-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/jetson-tx1-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/jetson-tx1-linux-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/jetson-tx1-linux-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/jetson-tx1.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/jetson-tx1.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/jetson-tx2-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/jetson-tx2-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/jetson-tx2.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/jetson-tx2.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/k3-am654-idk-linux-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/k3-am654-idk-linux-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/k3-am654-idk.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/k3-am654-idk.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/k3-am654-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/k3-am654-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/k3-j721e-evm-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/k3-j721e-evm-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/k3-j721e-evm-linux-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/k3-j721e-evm-linux-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/k3-j721e-evm.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/k3-j721e-evm.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/macchiatobin-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/macchiatobin-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/macchiatobin-linux-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/macchiatobin-linux-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/macchiatobin.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/macchiatobin.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/miriac-sbc-ls1046a-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/miriac-sbc-ls1046a-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/miriac-sbc-ls1046a-linux-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/miriac-sbc-ls1046a-linux-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/miriac-sbc-ls1046a.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/miriac-sbc-ls1046a.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/qemu-arm64-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/qemu-arm64-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/qemu-arm64-linux-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/qemu-arm64-linux-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/qemu-arm64.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/qemu-arm64.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/rpi4-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/rpi4-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/rpi4-linux-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/rpi4-linux-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/rpi4.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/rpi4.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/ultra96-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/ultra96-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/ultra96-linux-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/ultra96-linux-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/ultra96.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/ultra96.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/zynqmp-zcu102-inmate-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/zynqmp-zcu102-inmate-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/zynqmp-zcu102-linux-demo-2.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/zynqmp-zcu102-linux-demo-2.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/zynqmp-zcu102-linux-demo.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/zynqmp-zcu102-linux-demo.cell
  CC      /home/ubuntu/jailhouse/configs/arm64/zynqmp-zcu102.o
  OBJCOPY /home/ubuntu/jailhouse/configs/arm64/zynqmp-zcu102.cell
  DTC     /home/ubuntu/jailhouse/configs/arm64/dts/inmate-amd-seattle.dtb
  DTC     /home/ubuntu/jailhouse/configs/arm64/dts/inmate-espressobin.dtb
  DTC     /home/ubuntu/jailhouse/configs/arm64/dts/inmate-foundation-v8.dtb
  DTC     /home/ubuntu/jailhouse/configs/arm64/dts/inmate-hikey.dtb
  DTC     /home/ubuntu/jailhouse/configs/arm64/dts/inmate-jetson-tx1.dtb
  DTC     /home/ubuntu/jailhouse/configs/arm64/dts/inmate-k3-am654-idk.dtb
  DTC     /home/ubuntu/jailhouse/configs/arm64/dts/inmate-k3-j721e-evm.dtb
  DTC     /home/ubuntu/jailhouse/configs/arm64/dts/inmate-macchiatobin.dtb
  DTC     /home/ubuntu/jailhouse/configs/arm64/dts/inmate-miriac-sbc-ls1046a.dtb
  DTC     /home/ubuntu/jailhouse/configs/arm64/dts/inmate-qemu-arm64.dtb
  DTC     /home/ubuntu/jailhouse/configs/arm64/dts/inmate-rpi4.dtb
  DTC     /home/ubuntu/jailhouse/configs/arm64/dts/inmate-zynqmp-zcu102-2.dtb
  DTC     /home/ubuntu/jailhouse/configs/arm64/dts/inmate-zynqmp.dtb
  CC [M]  /home/ubuntu/jailhouse/driver/cell.o
  CC [M]  /home/ubuntu/jailhouse/driver/main.o
  CC [M]  /home/ubuntu/jailhouse/driver/sysfs.o
  CC [M]  /home/ubuntu/jailhouse/driver/pci.o
  DTC     /home/ubuntu/jailhouse/driver/vpci_template.dtb
  DTB     /home/ubuntu/jailhouse/driver/vpci_template.dtb.S
  AS [M]  /home/ubuntu/jailhouse/driver/vpci_template.dtb.o
  LD [M]  /home/ubuntu/jailhouse/driver/jailhouse.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/asm-defines.s
  GEN     /home/ubuntu/jailhouse/hypervisor/arch/arm64/include/generated/asm/asm-defines.h
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/control.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/dbg-write.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/gic-v2.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/gic-v3.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/irqchip.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/ivshmem.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/lib.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/mmu_cell.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/paging.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/pci.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/psci.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/setup.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/smccc.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/uart-hscif.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/uart-imx.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/uart-mvebu.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/uart-pl011.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/uart-scifa.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/../arm-common/uart-xuartps.o
  AS      /home/ubuntu/jailhouse/hypervisor/arch/arm64/caches.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/control.o
  AS      /home/ubuntu/jailhouse/hypervisor/arch/arm64/entry.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/iommu.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/mmio.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/paging.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/setup.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/smmu-v3.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/ti-pvu.o
  CC      /home/ubuntu/jailhouse/hypervisor/arch/arm64/traps.o
  AR      /home/ubuntu/jailhouse/hypervisor/arch/arm64/lib.a
  LDS     /home/ubuntu/jailhouse/hypervisor/hypervisor.lds
  CC      /home/ubuntu/jailhouse/hypervisor/setup.o
  CC      /home/ubuntu/jailhouse/hypervisor/printk.o
  CC      /home/ubuntu/jailhouse/hypervisor/paging.o
  CC      /home/ubuntu/jailhouse/hypervisor/control.o
  CC      /home/ubuntu/jailhouse/hypervisor/lib.o
  CC      /home/ubuntu/jailhouse/hypervisor/mmio.o
  CC      /home/ubuntu/jailhouse/hypervisor/pci.o
  CC      /home/ubuntu/jailhouse/hypervisor/ivshmem.o
  CC      /home/ubuntu/jailhouse/hypervisor/uart.o
  CC      /home/ubuntu/jailhouse/hypervisor/uart-8250.o
  LD      /home/ubuntu/jailhouse/hypervisor/hypervisor.o
  OBJCOPY /home/ubuntu/jailhouse/hypervisor/jailhouse.bin
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/../alloc.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/../cmdline.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/../pci.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/../printk.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/../setup.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/../string.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/../uart-8250.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/gic-v2.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/gic-v3.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/gic.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/mem.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/pci.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/printk.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/setup.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/timing.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/uart-hscif.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/uart-imx.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/uart-mvebu.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/uart-pl011.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/uart-scifa.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/uart-xuartps.o
  CC      /home/ubuntu/jailhouse/inmates/lib/arm64/../arm-common/uart.o
  AS      /home/ubuntu/jailhouse/inmates/lib/arm64/header.o
  AR      /home/ubuntu/jailhouse/inmates/lib/arm64/lib.a
  LDS     /home/ubuntu/jailhouse/inmates/lib/arm64/inmate.lds
  CC      /home/ubuntu/jailhouse/inmates/demos/arm64/../arm/gic-demo.o
  LD      /home/ubuntu/jailhouse/inmates/demos/arm64/gic-demo-linked.o
  OBJCOPY /home/ubuntu/jailhouse/inmates/demos/arm64/gic-demo.bin
  CC      /home/ubuntu/jailhouse/inmates/demos/arm64/../arm/uart-demo.o
  LD      /home/ubuntu/jailhouse/inmates/demos/arm64/uart-demo-linked.o
  OBJCOPY /home/ubuntu/jailhouse/inmates/demos/arm64/uart-demo.bin
  CC      /home/ubuntu/jailhouse/inmates/demos/arm64/../ivshmem-demo.o
  LD      /home/ubuntu/jailhouse/inmates/demos/arm64/ivshmem-demo-linked.o
  OBJCOPY /home/ubuntu/jailhouse/inmates/demos/arm64/ivshmem-demo.bin
  CC      /home/ubuntu/jailhouse/inmates/tools/arm64/linux-loader.o
  LD      /home/ubuntu/jailhouse/inmates/tools/arm64/linux-loader-linked.o
  OBJCOPY /home/ubuntu/jailhouse/inmates/tools/arm64/linux-loader.bin
  GEN     /home/ubuntu/jailhouse/pyjailhouse/pci_defs.py
  CC      /home/ubuntu/jailhouse/tools/jailhouse.o
  LD      /home/ubuntu/jailhouse/tools/jailhouse
  CC      /home/ubuntu/jailhouse/tools/ivshmem-demo.o
  LD      /home/ubuntu/jailhouse/tools/ivshmem-demo
  GEN     /home/ubuntu/jailhouse/tools/jailhouse-config-collect
  CC      /home/ubuntu/jailhouse/tools/jailhouse-gcov-extract.o
  LD      /home/ubuntu/jailhouse/tools/jailhouse-gcov-extract
  GEN     /home/ubuntu/jailhouse/tools/jailhouse.8
  GEN     /home/ubuntu/jailhouse/tools/jailhouse-cell.8
  GEN     /home/ubuntu/jailhouse/tools/jailhouse-enable.8
  Building modules, stage 2.
  MODPOST 1 modules
  CC [M]  /home/ubuntu/jailhouse/driver/jailhouse.mod.o
  LD [M]  /home/ubuntu/jailhouse/driver/jailhouse.ko

ubuntu@ubuntu:~/jailhouse$ sudo make install
  Building modules, stage 2.
  MODPOST 1 modules
  INSTALL /home/ubuntu/jailhouse/driver/jailhouse.ko
At main.c:160:
- SSL error:02001002:system library:fopen:No such file or directory: ../crypto/bio/bss_file.c:69
- SSL error:2006D080:BIO routines:BIO_new_file:no such file: ../crypto/bio/bss_file.c:76
sign-file: certs/signing_key.pem: No such file or directory
  DEPMOD  5.4.0-1055-raspi
Warning: modules_install: missing 'System.map' file. Skipping depmod.
install -m 644 hypervisor/jailhouse*.bin /lib/firmware
install -m 644 inmates/tools/arm64/*.bin /usr/local/libexec/jailhouse
install jailhouse ivshmem-demo /usr/local/sbin
install jailhouse-config-collect jailhouse-cell-linux jailhouse-cell-stats jailhouse-config-create jailhouse-hardware-check /usr/local/libexec/jailhouse
install -d -m 755 /usr/local/share/jailhouse
install -m 644 jailhouse-config-collect.tmpl root-cell-config.c.tmpl /usr/local/share/jailhouse
install -m 644 jailhouse-completion.bash /usr/share/bash-completion/completions/jailhouse
install -m 644 jailhouse.8 jailhouse-cell.8 jailhouse-enable.8 /usr/local/share/man/man8
python -m pip install --upgrade --force-reinstall  .
Processing /home/ubuntu/jailhouse
Building wheels for collected packages: pyjailhouse
  Building wheel for pyjailhouse (setup.py) ... done
  Created wheel for pyjailhouse: filename=pyjailhouse-0.12-py3-none-any.whl size=20250 sha256=28b06a6acbeb94c03d2028e0899507dad9b7e51df451f860e5484fc0e8c38fcb
  Stored in directory: /tmp/pip-ephem-wheel-cache-n55wj2h2/wheels/c9/70/5f/41e4b42a3693d675b8bd9bcdcb021956f8d467789b3be58740
Successfully built pyjailhouse
Installing collected packages: pyjailhouse
Successfully installed pyjailhouse-0.12

可以对比一下和第3部分的输出异同。

6. Jailhoue常用命令

7. Jailhouse运行状态

8. 创建 linux cell

ubuntu@ubuntu:~$ jailhouse cell linux -h
usage: jailhouse cell linux [-h] [--dtb DTB] [--initrd FILE] [--cmdline "STRING"] [--write-params FILE]
                            [--arch ARCH] [--kernel-decomp-factor N]
                            CELLCONFIG KERNEL

Boot Linux in a non-root cell.

positional arguments:
  CELLCONFIG            cell configuration file
  KERNEL                image of the kernel to be booted

optional arguments:
  -h, --help            show this help message and exit
  --dtb DTB, -d DTB     device tree for the kernel [arm/arm64 only]
  --initrd FILE, -i FILE
                        initrd/initramfs for the kernel
  --cmdline "STRING", -c "STRING"
                        kernel command line
  --write-params FILE, -w FILE
                        only parse cell configuration, write out parameters into the specified file and print
                        required jailhouse cell commands to boot Linux to the console
  --arch ARCH, -a ARCH  target architecture
  --kernel-decomp-factor N, -k N
                        decompression factor of the kernel image, used to reserve space between the kernel and the
                        initramfs

在树莓派4B的实际操作中，jailhouse cell linux ... 命令，对于 -i FILE 参数中 FILE 可以不指定，亦可指定 /boot/firmware/initrd.img，亦可自定义根文件系统（.cpio）。

举例：

sudo jailhouse cell linux configs/arm64/rpi4-linux-demo.cell /boot/vmlinuz-5.4.16 -d configs/arm64/dts/inmate-rpi4.dtb -i /home/ubuntu/rootfs.cpio -c "console=ttyS0,115200 ip=192.168.19.2"

9. 创建 inmate-demo 裸机程序

ubuntu@ubuntu:~/jailhouse$ sudo jailhouse cell create configs/arm64/rpi4-inmate-demo.cell 
ubuntu@ubuntu:~/jailhouse$ sudo jailhouse cell load inmate-demo inmates/demos/arm64/gic-demo.bin 
ubuntu@ubuntu:~/jailhouse$ sudo jailhouse cell start inmate-demo 
ubuntu@ubuntu:~/jailhouse$ jailhouse cell list 
ID      Name                    State             Assigned CPUs           Failed CPUs             
0       Raspberry-Pi4           running           0,2-3                                           
1       inmate-demo             shut down         1                                     

当同时启动了 linux cell 后，串口会复用。

10. 替换内核

sudo cp /boot/vmlinuz-5.4.16 /boot/firmware/vmlinuz
sudo cp /boot/initrd.img-5.4.16 /boot/firmware/initrd.img
sudo cp /boot/dtbs/5.4.16/bcm2711-rpi-4-b.dtb /boot/firmware/bcm2711-rpi-4-b.dtb

11. 创建 rootfs.cpio 文件

1. 参考链接：使用QEMU（x86）模拟运行ARM64架构并进行内核调试的使用 busybox小节构建 busybox 文件系统。
2. rootfs 中删除 linuxrc 文件，ln -s bin/busybox init
3. 执行命令打包：find -print0 | cpio -0 -oH newc | gzip -9 > ../rootfs.cpio

[进击的大白菜]

@Jiaming 我在jailhouse-image中运行了build-images.sh了，也选择了9:Rasberry Pi 4，但是通过之后没有生成bl31.bin文件，只是多了一个kas-docker的文件

[Jiaming]

@进击的大白菜 参考小节：添加bl31.bin文件，编译安装完成后，源码路径参考：/home/jiaming/Documents/jailhouse-images-v0.12/build/tmp/work/jailhouse-demo-arm64/linux-jailhouse-rpi/5.4.16-r0/linux-e569bd2d6d2d7b958973bb8c6e9db9cfc05c790b

[进击的大白菜]

@Carlota_chen 你好，请问我可以在哪里获得 5.4.16(linux-jailhouse-rpi4)的源码

[Carlota_chen]

补充：
（1）环境准备第五步串口连接，通过串口查询ip地址便于SSH远程连接，第一次连接 用户名和密码都是ubuntu
（2）进行环境准备第六步之前需要执行update命令(sudo apt-get update)
（3） 解决之前make & make install时发生的warning问题，#安装python相关工具第二行sudo apt-get instal aptitude少了一个“l”
（4）替换内核之前要加一个编译内核的步骤，需要3中内核路径中的内核文件和内核配置文件拷贝到树莓派的home目录下，编译依次执行下列命令：make clean，make menuconfig，make，make modules，sudo make modules_install，sudo make install
（5）sudo su - root切换到root用户，输入exit退出
（6）使用 uname -a 查询内核镜像版本
（7）创建 rootfs.cpio 文件，在本地虚拟机上进行，交叉编译
（8）更换完内核并重启后，需要重新编译Jailhouse，并且将cmdline.txt中的mem参数重新添加，使用free -m命令查看内存空间大小

[进击的大白菜]

https://s2.loli.net/2023/05/25/e2tPaBTqYdypVMH.png

[Jiaming]

@进击的大白菜 是的，在对应的 cell 配置文件（configs）中可以进行设置。

[进击的大白菜]

请问博主，如果我没有理解错的话，jailhouse是可以为虚拟机分配cpu的，是吗？

[进击的大白菜]

@Jiaming 博主我在./build_image.sh结束之后，jailhouse-image目录下都没有build这个文件夹，是不是我实际拉取没有成功？

[Jiaming]

@进击的大白菜 linux 源码路径参考这个：/home/jiaming/Documents/jailhouse-images-v0.12/build/tmp/work/jailhouse-demo-arm64/linux-jailhouse-rpi/5.4.16-r0/linux-e569bd2d6d2d7b958973bb8c6e9db9cfc05c790b

[Jiaming]

@进击的大白菜 并不是在工程目录下含有 linux 源码文件，这个是构建工具的配置文件，需要执行 build-images 脚本，拉取 linux 源码成功后，才可以看到。

[进击的大白菜]

博主你指的jailhouse-image中的这个源码吗？

[进击的大白菜]

@Jiaming 好的，感谢感谢

[Jiaming]

@进击的大白菜 添加社区管理员微信详聊吧

[进击的大白菜]

很抱歉打扰博主，麻烦你看一下，我执行了./build_image.sh 并且选择了树莓派，但是执行完成之后，在哪里都没有找到拉取成功的源码，很头疼

[ARM-IoT]

我编译的5.4.16 内核起不来，咋回事呢，能把详细的编译5.4.16内核的操作步骤发下吗？