Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
StratixV second PCIe HIP

Short walkthrough on how to enable second, hidden PCIe HIP on 5SGSMD5K1F40C1 (5SGSKF40I3LNAC)

  1. Create a file named preload_me.c with following content:
#include <stdio.h>
#include <dlfcn.h>
#include <stdlib.h>
#include <string.h>
#include <stdbool.h>

void *lib_ptr = NULL;
bool (*is_global_id_enabled_next)(void *db, unsigned id);

void do_init() {
	if (lib_ptr != NULL)
		return;
	lib_ptr = dlopen("libddb_dev.so", RTLD_LOCAL | RTLD_LAZY);
	if (lib_ptr == NULL) {
		fprintf(stderr, "failed to open libddb_dev.so\n");
		exit(1);
	}
	is_global_id_enabled_next = dlsym(lib_ptr, "_ZNK12DEV_DIE_INFO20is_global_id_enabledEj");
	if (is_global_id_enabled_next == NULL) {
		fprintf(stderr, "failed to dlsym _ZNK12DEV_DIE_INFO20is_global_id_enabledEj\n");
		exit(1);
	}
}

bool _ZNK12DEV_DIE_INFO20is_global_id_enabledEj(void *db, unsigned id) {
	do_init();
	if (id == 1174964) // global id of forbidden PCIe block
		return 1;
	bool next_result = is_global_id_enabled_next(db, id);
	if (!next_result) // something else disabled, print it for interest's sake
		fprintf(stderr, "disabled %u\n", id);
	return next_result;
}
  1. Compile it to a .so file:
gcc -shared -o preload_me.so -fPIC preload_me.c
  1. Launch Quartus with LD_PRELOAD to enable second HIP:
LD_PRELOAD=`pwd`/preload_me.so /path_to_quartus/intelFPGA/version/quartus/bin/quartus
  1. Instantiate a second PCIe HIP in the design; it'll be placed in the right spot based on the reference clock I/O contstraint
@alexforencich
Copy link

alexforencich commented Oct 17, 2021

The tools seem to accept 5SGSKF40I3LNAC as a valid part number (either by setting this via the command line or by editing the qsf). However, this does not seem to be a complete solution, as both the GUI and the fitter report that the selected device is 5SGSMD5K2F40I3L (which matches the presentation from Microsoft), but the fitter fails with more than one PCIe core in the design. I wonder if there is some other directive that can be added to the QSF to enable the additional PCIe core, without preloading anything.

@Nic30
Copy link

Nic30 commented Mar 26, 2022

Is it somehow possible to edit .ddb files and create a part in Quartus database?
(It seems to me that the device info must be in these files because they are only files in device support package which I am not able to read. And judging by filenames they must contain routing info. Which is just what we need.)

If I understand correctly these files are loaded by libddb_* libraries from Quartus. Content is encrypted or compressed.
I have seen someone stealing contents of encrypted IP cores because it was somehow possible to dump them from Quartus after they were loaded to virtual filesystem of Quartus. Also there probably were many people which were using Catapult cards, it may be possible that someone can provide Quartus device files. Do you think that it may be possible that the device was once supported by Quartus? I checked supported devices in Quartus 9-21 but this custom device was not mentioned.

Also @wirebond how did you find out that 1174964 is a correct ID? I mean I do have only user knowledge of radare2 and other tools but I was not able to find out on my own. I was just wondering if there is some trick to it.

@wirebond
Copy link
Author

wirebond commented May 11, 2022

@alexforencich, @Nic30, sorry for the long answer, haven't been paying attention to my notifs on this account.

The tools seem to accept 5SGSKF40I3LNAC as a valid part number ...

Indeed, 5SGSKF40I3LNAC is present in the internal device DB in Quartus (with 'hidden' trait, which explains it being missing in all GUI dialogs), but we couldn't figure out how to enable the second PCIe HIP without LD_PRELOAD yet.

Do you think that it may be possible that the device was once supported by Quartus?

Wdym? Catapult FPGA device (both as 'special' 5SGSKF40I3LNAC and 'standard' 5SGSMD5K1F40C1 SKUs) is supported in a wide range of Quartus versions, including the latest Standard 21.1.

Also @wirebond how did you find out that 1174964 is a correct ID?

I'm not nearly smart enough to do it myself. : )
Awesome folks from project Mistral (they work on an opensource Intel FPGA tooling) figured out how to enable this feature for us.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment