automated-mips-verilog.md

Automated Verilog MIPS Testing

This documents our humble attempt at automating the testing of a MIPS project in Verilog using Perl.

We had two test scripts, general_test.pl and predefined_test.pl.

general_test.pl is made so that you can easily initialize some registers and pass a set of instructions and check the values of any number of registers after the execution; it was used for evaluating and grading the project.

predefined_test.pl is a set of tests that don't need initializing registers, nor checking the values of random registers; it depends on $0 being 0 always, and on $at being initialized with 1, and after the execution it checks only the values of $s0 and $s1; it was used for checking that we haven't messed anything up yet.

What we did

1. Make the instruction memory read the instructions from a hexadecimal file:

reg [31:0] I_Memory [0:255];  // 2 power 8 instructions of length 32-bits
initial $readmemh("instructions.hex", I_Memory);

2. In the case of general_test, which requires initializing the register file, make it read from a hexfile:

reg signed [31:0] data [31:0];
initial $readmemh("registers.hex", data);

And in the case of predefined_test, initialize $at to 1.

3. Once xxxs enter the register file, terminate the execution with $finish, but before terminating:

In case of general_test, dump the registers to a hexfile, using:

    $writememh("registers.hex", data);

And in case of predefined_test, print $s0 and $s1 in a specific format:

    $display("T: %d %d", data[16], data[17]);

4. Put your instructions, followed by 4 nops (because of the previous point), in the suitable place in predefined_test.pl or general_test.pl.

5. Modify and run general_test.pl and/or predefined_test.pl as needed.

Now we have the assembler, the testing script, and code inside the processor that helps us to test it. all we need is to start testing!

The two testing scripts are attached; the assembler can be found here.

general-test.pl

#!/usr/bin/env perl
use strict; use warnings;

my %regdata = map { $_ => 0 } (0..31);
my @reg = qw(zero at v0 v1 a0 a1 a2 a3 t0 t1 t2 t3 t4 t5 t6 t7
               s0 s1 s2 s3 s4 s5 s6 s7 t8 t9 k0 k1 gp sp fp ra);
my %reg = map { $_ => $_, $reg[$_] => $_ } (0..31);

# put your test after this line

# `x => 1` is equiv to `'x', 1` in perl ;)
set(t0 => 8);
set(t1 => 9);
set(t2 => 10);
set(t5 => 40);

run('
    sw $t5, 40($0)
    add $s0, $t0, $t0
    add $s0, $s0, $s0
    lw $s1, 8($s0)
    add $s2, $s1, $s1
    beq $s2, $s1, L
    add $s2, $s2, $t0
L:  add $s2, $s2, $t0
    nop
    nop
    nop
    nop
');

chk(s1 => 40);
chk(s2 => 96);

# test ends here

sub set{
    my ($r, $v) = @_;
    unless (defined $reg{$r}) { warn "ERROR: unknown register $r\n"; exit; }
    $regdata{ $reg{$r} } = $v;
}

sub run {
    my ($instructions) = @_;
    # Registers
    my $regfile = 'registers.hex';
    open REGFILE, ">$regfile" or die $!;
    for my $r (0..31) {
        printf { *REGFILE } "%08x\n", $regdata{$r};
    }
    close REGFILE;
    # Instructions and Assembling
    my $asmfile = ".test.asm";
    my $hexfile = 'instructions.hex';
    # edit the following variable so it matches all the needed files
    my $mipsfiles = '*.v instruction_memory-sim.V RegFile-test.V';
    open ASMFILE, ">$asmfile" or die $!;
    print { *ASMFILE } $instructions;
    close ASMFILE;
    system('sh', '-c', "perl mas.pl '$asmfile' >$hexfile");
    # Running
    my $result = `sh -c 'iverilog $mipsfiles && ./a.out &>/dev/null'`;
    # Cleaning
    unlink $asmfile;
    unlink $hexfile;
    unlink 'a.out';
    # Reading Registers
    open REGFILE, "<$regfile" or die $!;
    my $r = 0;
    while ($r < 32) {
        my $in = <REGFILE>;
        next if ($in =~ m|^//|);
        chomp($in); # remove the newline
        $regdata{$r} = $in;
        ++$r;
    }
    close REGFILE;
    unlink($regfile);
}

sub prn {
    my ($r) = @_;
    unless (defined $reg{$r}) { warn "ERROR: unknown register $r\n"; exit; }
    printf $r . ":\t" . $regdata{ $reg{$r} } . "\n";
}

sub chk {
    my ($r, $v) = @_;
    unless (defined $reg{$r}) { warn "ERROR: unknown register $r\n"; exit; }
    if ($v == hex($regdata{ $reg{$r} })) {
        print "\e[1;32m$r:\t$v\e[m\n";
    } else {
        print "\e[1;31m$r:\tResult: " . hex($regdata{ $reg{$r} })
            ."\tExpected: $v\e[m\n";
    }
}

predefined-test.pl

#!/usr/bin/env perl
# MIPS test suite

# We assemble several assembly tests into `instructions.hex` file, which is
# loaded by our Verilog MIPS implementation's simulation. We then run the
# simulation, which on finishing prints the values of $s0 and $s1.

# each function call is a string representing the test name, then a multiline
# string representing the instructions, then the expected values of $s0 and $s1.

# `$0` is always zero, and `$1` (`$at`) is initialized to one.

# Our MIPS ends the simulation once an instruction of `x`s enters the decoding
# stage. So we need to wait four cycles so that all instructions are complete;
# hence the four `nop` at the end of every test.

test("independant add 1", '
    add $s0, $0, $1
    add $s1, $0, $1
    nop
    nop
    nop
    nop
    ', 1, 1);

test("independant add 2", '
    add $s0, $1, $1
    add $s0, $s0, $s0
    add $s0, $s0, $s0
    add $s0, $s0, $s0
    add $s1, $s0, $s0
    nop
    nop
    nop
    nop
    ', 16, 32);

test("independant sub", '
    sub $s0, $0, $1
    sub $s1, $1, $0
    nop
    nop
    nop
    nop
    ', -1, 1);

test("independant sll", '
    sll $s0, $1, 1
    sll $s1, $1, 12
    nop
    nop
    nop
    nop
    ', 2, 4096);

test("independant and 1", '
    and $s0, $0, $1
    and $s1, $1, $0
    and $s2, $0, $1
    nop
    nop
    nop
    nop
    ', 0, 0);

test("independant and 2", '
    and $s0, $0, $0
    and $s1, $1, $1
    nop
    nop
    nop
    nop
    ', 0, 1);

test("independant or 1", '
    or $s0, $0, $1
    or $s1, $1, $0
    nop
    nop
    nop
    nop
    ', 1, 1);

test("independant or 2", '
    or $s0, $0, $0
    or $s1, $1, $1
    nop
    nop
    nop
    nop
    ', 0, 1);

test("dependant on branch taken", '
    add $s0, $1, $0
    add $s1, $1, $0
    beq $s0, $s1, L
    add $s0, $s1, $s0
L:  sub $s1, $s1, $s0
    nop
    nop
    nop
    nop
    ', 1, 0);

test("dependant on branch not taken", '
    add $s0, $1, $0
    add $s1, $1, $1
    beq $s0, $s1, L
    add $s0, $s1, $s0
L:  sub $s1, $s1, $s0
    nop
    nop
    nop
    nop
    ', 3, -1);

test("independant load and store", '
    add $s0, $1, $1
    add $s1, $0, $1
    sw $s0, 1($1)
    lw $s1, 1($1)
    nop
    nop
    nop
    nop
    ', 2, 2);

test("dependant sw then lw then sw then lw", '
    add $s0, $1, $1
    add $s1, $0, $1
    sw $s0, 1($1)
    lw $s1, 0($s0)
    sw $s1, 0($s1)
    lw $s0, 0($s1)
    add $s0, $s0, $1
    nop
    nop
    nop
    nop
    ', 3, 2);

test("forward A 1", '
    add $s0, $0, $1
    add $s1, $s0, $1
    nop
    nop
    nop
    nop
    ', 1, 2);

test("forward A 2", '
    add $t0, $0, $1
    add $s0, $t0, $1
    sub $s1, $t0, $1
    nop
    nop
    nop
    nop
    ', 2, 0);

test("forward B", '
    add $t0, $0, $1
    add $s0, $1, $t0
    sub $s1, $1, $t0
    nop
    nop
    nop
    nop
    ', 2, 0);

test("add then sw; and lw then add", '
    add $t0, $1, $1
    sw $t0, 4($0)
    lw $s0, 4($0)
    add $s1, $1, $s0
    nop
    nop
    nop
    nop
    ', 2, 3);

test("add then lw then add", '
    add $t0, $1, $1
    sw $t0, 4($0)
    add $t0, $1, $1
    lw $s0, 2($t0)
    add $s1, $1, $s0
    nop
    nop
    nop
    nop
    ', 2, 3);

test("add then sw then add then lw then add", '
    add $t0, $1, $1
    sw $t0, 4($0)
    add $t0, $1, $t0
    lw $s0, 1($t0)
    add $s1, $t0, $s0
    nop
    nop
    nop
    nop
    ', 2, 5);

test("beq taken then lw", '
    sw $1, 4($0)
    beq $1, $1, L
    lw $s0, 4($0)
L:  nop
    nop
    nop
    nop
    ', 'x', 'x');

test("beq not taken then lw", '
    sw $1, 4($0)
    beq $1, $0, L
    lw $s0, 4($0)
L:  nop
    nop
    nop
    nop
    ', 1, 'x');

test("beq taken then sw", '
    add $s0, $1, $1
    beq $1, $1, L
    sw $s0, 4($0)
    lw $s1, 4($0)
L:  nop
    nop
    nop
    nop
    ', 2, 'x');

test("beq not taken then sw", '
    add $s0, $1, $1
    beq $1, $0, L
    sw $s0, 4($0)
    lw $s1, 4($0)
L:  nop
    nop
    nop
    nop
    ', 2, 2);

sub test {
    # read function arguments
    my ($name, $instructions, $expected_s0, $expected_s1) = @_;
    my $asmfile = ".$name.test.asm";
    my $hexfile = 'instructions.hex';
    # write asm to a file
    my $mipsfile = '*.v *-sim.V'; # change this to match all needed files
    open ASMFILE, ">$asmfile" or die $!;
    print { *ASMFILE } $instructions;
    close ASMFILE;
    # assemble
    system('sh', '-c', "perl mas.pl '$asmfile' >$hexfile");
    # compile, simulate, and run
    my $result = `sh -c 'iverilog $mipsfile && ./a.out | grep ^T'`;
    # check the result; here be dragons
    chomp $result;
    my ($result_s0, $result_s1) = $result =~ /T:\s*([-0-9Xx]+)\s*([-0-9Xx]+)\s*/;
    my $res_eq_exp = (($result_s0 eq $expected_s0) and ($result_s1 eq $expected_s1));
    print(($res_eq_exp ? "\e[1;32m" : "\e[1;31m") . "$name\e[m\n");
    unless ($res_eq_exp) {
        # print "$result\n";
        print "RESULTED\tEXPECTED\n--------\t--------\n";
        print "$result_s0\t\t$expected_s0\n$result_s1\t\t$expected_s1\n";
        print "--------\t--------\n";
    }
    #print " \e[34$result\e[m\n\e[35m$expected\e[m\n";
    # clean
    unlink $asmfile;
    unlink $hexfile;
    unlink 'a.out';
}