dhand-galois/lmco_cloudgfe.md Secret

## lmco_cloudgfe.md

      
    Raw
  

              lmco_cloudgfe.md
            
          
    Minimal CloudGFE Setup - LMCO Edition

This readme will guide you through bringing up an F1 instance with a CloudGFE processor (Rocket P2 - Hard enabled) running Linux and FreeBSD.
This assumes you are familiar with AWS.
Start F1 Instance

Launch a new f1.2xlarge instance with FPGA Developer AMI - 1.6.0. It must be this specific AMI, otherwise you will have to compile your own kernel modules. It's recently become more difficult to search for this instance. If it does not appear under the Community AMI section using the name above, try searching for ami-02b792770bf83b668.
Once the instance launches, login and add your AWS keys, either via copy/pasting the environmental variables or using aws configure.
Access

The quick-start files are located on Amazon S3. The CloudGFE AFI is also access-controlled.
You'll need to create an github issue or send both your AWS ID and canonical user ID to dhand@galois.com for access to both. The canonical ID can quickly be found using aws s3api list-buckets.
Pull Minimal CloudGFE Package

Once you are granted access, download the setup package from S3:
cd ~
aws s3 cp s3://firesim-845509001885/minimal_cloudgfe_lmco.tgz .
tar xzvf minimal_cloudgfe_lmco.tgz

Initial Setup

This setup installs the FPGA SDK and necessary kernel modules. It will build the full SDK the first time it runs.
Subsequent runs should only setup the kernel modules. Thus, you can run this script once every time you reboot the F1
instance. It does not need to be run between simulations.
cd ~/minimal_cloudgfe_lmco
./setup.sh

Boot Linux

The ./run_sim.sh script will handle configuring networking, bringing up the switch software, programming the FPGA, and starting the simulation.
It takes three arguments:
./run_sim.sh <blockimage> <dwarf> <elf>


blockimage - the image file presented via the block device
dwarf - currently unknown purpose. Give it elf-dwarf as an argument for now
elf - ELF binary to be loaded into memory and executed

To boot linux:
cd ~/minimal_cloudgfe_lmco/sim
./run_sim.sh linux-uniform0-br-base.img linux-uniform0-br-base-bin-dwarf linux-uniform0-br-base-bin

The script launches 3 screen sessions:

fsim0 - UART console and simulator output
switch0 - Software switch log
bootcheck - Checks for successful Linux boot to bring up networking interface

The fsim0 screen will be attached automatically. You can exit it while keeping the sim running using Ctrl-a followed by d, or C-a d in screen terms.
Once Linux boots, the login is root and password firesim. You can also SSH into the target OS:
TERM=Linux ssh root@172.16.0.2

Running poweroff -f within the target OS will automatically stop the simulator cleanly. If it becomes stuck or unresponsive, you can also use the ./kill_sim.sh script.
Run Smoke Tests

The Linux image is pre-packaged with the provided smoke tests. You should be able to run them and see this output:
Welcome to Buildroot
buildroot login: root
Password: 
# ls
smoke001  smoke002  smoke003  smoke004
# ./smoke001
# ./smoke002
[   29.109628] smoke002[223]: unhandled signal 4 code 0x1 at 0x0000000000010498 in smoke002[10000+1000]
[   29.110126] CPU: 0 PID: 223 Comm: smoke002 Tainted: G           O      5.3.0-00002-g6a95b016aba5 #20
[   29.110438] sepc: 0000000000010498 ra : 00000015555940f0 sp : 0000003fffb46c10
[   29.110696]  gp : 0000000000012800 tp : 0000001555573210 t0 : 0000001555577d58
[   29.110955]  t1 : 0000001555594050 t2 : 0000000000012360 s0 : 0000003fffb46c30
[   29.111209]  s1 : 00000000000cbe90 a0 : 0000000000000001 a1 : 0000003fffb46d88
[   29.111461]  a2 : 0000003fffb46d98 a3 : 0000000000000000 a4 : 000000000000000a
[   29.111720]  a5 : 000000000000000a a6 : 0000001555674d20 a7 : 2f1d1f1f4a444042
[   29.111969]  s2 : 0000000000000000 s3 : 0000000000000000 s4 : 0000000000000000
[   29.112223]  s5 : 00000000000cb8f0 s6 : 0000000000000000 s7 : ffffffffffffffff
[   29.112815]  s8 : 0000000000000000 s9 : 00000000000fe1a0 s10: 00000000000fe180
[   29.113071]  s11: 0000000000000000 t3 : 0000000000000000 t4 : 0000000000020050
[   29.113317]  t5 : 0000000000000003 t6 : 0000000000000040
[   29.113532] sstatus: 8000000200006020 sbadaddr: 0000003fffb46c18 scause: 0000000000000002
Illegal instruction
# ./smoke003
# ./smoke004
[   31.626150] smoke004[225]: unhandled signal 4 code 0x1 at 0x0000000000010498 in smoke004[10000+1000]
[   31.626620] CPU: 0 PID: 225 Comm: smoke004 Tainted: G           O      5.3.0-00002-g6a95b016aba5 #20
[   31.626930] sepc: 0000000000010498 ra : 00000015555940f0 sp : 0000003fffb3fc10
[   31.627187]  gp : 0000000000012800 tp : 0000001555573210 t0 : 0000001555577d58
[   31.627444]  t1 : 0000001555594050 t2 : 0000000000012360 s0 : 0000003fffb3fc30
[   31.627701]  s1 : 00000000000cbe90 a0 : 0000000000000001 a1 : 0000003fffb3fd88
[   31.627956]  a2 : 0000003fffb3fd98 a3 : 0000000000000000 a4 : 000000000000000a
[   31.628212]  a5 : 000000000000000a a6 : 0000001555674d20 a7 : 2f1b1f1f4a444042
[   31.628802]  s2 : 0000000000000000 s3 : 0000000000000000 s4 : 0000000000000000
[   31.629061]  s5 : 00000000000fda40 s6 : 0000000000000000 s7 : ffffffffffffffff
[   31.629317]  s8 : 0000000000000000 s9 : 00000000000cb8f0 s10: 00000000001063d0
[   31.629570]  s11: 0000000000000000 t3 : 0000000000000000 t4 : 0000000000020050
[   31.629814]  t5 : 0000000000000003 t6 : 0000000000000040
[   31.630027] sstatus: 8000000200006020 sbadaddr: 0000003fffb3fc18 scause: 0000000000000002
Illegal instruction
# 

The kernel dumps are printed directly to UART but not over SSH. To see them on SSH, you can run dmesg.
Run an unmodified Rocket

You can also load an unmodified Rocket AFI to compare the results. Edit run_sim.sh and edit the line near the top to disable the LMCO AFI:
#!/bin/bash

# Set this to zero if you'd like to try running a regular, standard Rocket config
LMCO_ENABLED=0 # <-- Change this value

Load Linux again as described earlier. You'll now see:
Welcome to Buildroot
buildroot login: root
Password: 
# ./smoke001 
# ./smoke002 
# ./smoke003
# ./smoke004
# 

Boot FreeBSD

Read the Linux section above first - a lot of the same information applies. The current FreeBSD build does not include Ethernet or Block device drivers, so SSH will not work. freebsd.img is also just an empty file. The filesystem is stored within the ELF.
./run_sim.sh freebsd.img freebsd-bin-dwarf freebsd-bin

Networking Notes


The target OS has full internet access by default, but it is NAT'd behind the host OS.
This Linux image starts a simple Dropbear SSH server and Apache HTTP server. On the host OS you can run:

curl http://172.16.0.2
<html><body><h1>Hello!</h1>

This webpage is hosted by apache, running on Linux, running on a FireSim simulation
of RISC-V RocketChip on the FPGA of an EC2 F1 instance.

</body></html>

Known Issues


If the tap0 interface is already up when loading the simulator, Linux will get stuck when starting networking for an unknown reason. Keeping the interface down until fully booted fixes this issue.
Packets sent via SSH are generating a Invalid checksum message in the UART output. These don't seem to affect the actual connection very much. And using other networking tools, like wget operate normally without any messages.