This python script fixes symlinks in idm files on IRIX

fix_symlinks2.py

You need to pre-process the idb file and create a file which has all the symlinks from the filesytem.
In this scenario we have a hierarchy of files on our sgi. You could just tarball them up and move them to a new system and extract. But in this case we want to use inst to install, and therefore gendist to create a tardist.

So there are two directories involved with a tardist, gendist called them source base and target base.

For this case, the source base (sbase) is /usr/sgug_raw_selfhoster and the target base is /usr/sgug.

This means the files to use to create the tardist exist in /usr/sgug_raw_selfhoster and when inst installs them I want them to go into /usr/sgug. Be aware of those paths in the example below and change as necessary.

1. Run Software Packager - point at the top of your SBASE. Add all the files into one sw.base for ease of use. Set the target up to be usr/sgug (no leading /). Save the IDB and SPEC files somewhere you can edit them. The rest will be done by hand. Quit Software Packager.

This sample made a tardist with over 11,000 files in it. I ran Software Packager on a Fuel w/ 4G ram and 900mhz CPU and it takes a bit of time. If you have a slower system be prepared for SP to take some time to move between stages. I didn't change or add any defaults to the files beyond tagging them and setting the target base.

Do NOT name the package anything fancy, do not use any dashes in the name.

Sample:
$ cat rse-b5-self-hoster.spec
product rse
    id "R S E -- SGUG"
    image sw
        id "RSE Bootstrap Software"
        version 1.5
        order 9999
        subsys base default
            id "Base Software"
            replaces self
            exp rse.sw.base
        endsubsys
    endimage
endproduct

You were warned. Use a single name for the exp!


In this sample the IDB file is named: rse-b5-self-hoster.idb
A file called fs_link_input_file.txt is created and used to map symlinks

2. Sort the idb file properly:
$ sort -k 5 rse-b5-self-hoster.idb > rse-b5-self-hoster.idb_sorted
$ cp rse-b5-self-hoster.idb rse-b5-self-hoster.idb_orig
$ cp rse-b5-self-hoster.idb_sorted rse-b5-self-hoster.idb

3. Generate symlink input file. You need this as input to the python script to show it where the symlinks link to
$ find /usr/sgug_raw_selfhoster -type l -ls | awk '{print $11 " " $13}' > fs_link_input_file.txt

4. Fix this file using sed, as we want the paths in it to match our TBASE (not SBASE)
$ sed 's/sgug_raw_selfhoster/sgug/g' fs_link_input_file.txt > fs_link_input_file2.txt

The file should look like this:

$ cat fs_link_input_file2.txt
...
/usr/sgug/lib32/libdicl-0.1.so.0 libdicl-0.1.so.0.0.0
/usr/sgug/lib32/libffi.so.6 libffi.so.6.0.4
/usr/sgug/lib32/libbz2.so.1 libbz2.so.1.0.8
/usr/sgug/lib32/libbz2.so.1.0 libbz2.so.1.0.8
/usr/sgug/lib32/liblzma.so.5 liblzma.so.5.2.4
/usr/sgug/lib32/libgcc_s.so libgcc_s.so.1
/usr/sgug/lib32/libgcc_s.so.1 libgcc_s-9-20190812.so.1
/usr/sgug/lib32/libz.so.1 libz.so.1.2.11
...

It should have the symlink a space and the file it's linked to.


5. Run the python script (below)
$ python ./fix_symlinks2.py > rse-b5-self-hoster.idb_fixed

6. Create the tardist
$ /usr/sbin/gendist -sbase /usr/sgug_raw_selfhoster -idb /usr/people/dillera/rse-b5-self-hoster.idb_fixed -spec /usr/people/dillera/rse-b5-self-hoster.spec -dist /usr/people/dillera/dist -verbose -nostrip

Use -dryrun to see details if there are errors.

7. Make a tar file
~ $ cd dist/
~/dist $ tar cvf ../rse.tardist *
rse
rse.idb
rse.sw

Now you have a tardist named "rese.tardist" and you can distribute it and install it via inst.



Python script:



#!/usr/bin/env python
# Read in prepared list of symlinks from fs and
# a IDM file; write out new IDM file with symlinks
# pointing to their source as stdout

# v0.1

LINK = "l"

###########################################################
# read in the specially prepared symlink file
# generate with 
# find /usr/sgug_raw_selfhoster -type l -ls | awk '{print $11 " " $13}'
# then fix links with
# sed 's/sgug_raw_selfhoster/sgug/g' 

# put each line into a dict with the value as the STREE path
# and the key as the DTREE target
# we will use the actual idm file STREE path to match the key
# and then take the value, pre-prend the right path for DTREE
# and write it out
# by default Software Packager just points symlinks at themselves!

symlinks = {}
with open("fs_link_input_file2.txt") as f:
    for line in f:
       (key, val) = line.split()
       symlinks[(key)] = val

#for item in symlinks:
#    print("Key : {} , Value : {}".format(item,symlinks[item]))

###########################################################
# read in the idb file to be corrected.
# the fixed file will be written out on stdout

with open("rse-b5-self-hoster_bad_links.idb") as file_in:
    idb = []
    for line in file_in:
        idb.append(line.strip())

###########################################################
# main loop to fix the idm file, line by line looking for symlink lines

try:
    # go over every line from the idm file, looking for symlink lines to fix
    for line in idb:
        splits = line.split()                   # split each line up
        if splits[0] == "l":                    # this line from IDB is a link
            thisLink = "/" + splits[4]          # correct missing slash from the symlinks file
            # go over each symlink item, I could not find a better way to get the key based on thisLink
            for key, value in symlinks.items() :
                if  (key == thisLink):          # matched key (path for symlink)
                    print( splits[0] + " " + splits[1] + " " +  splits[2] +  " " + splits[3] + " " + splits[4] + " " + splits[5] + " " + splits[6] + " symval(" + value + ")" )

                    #path = thisLink.split("/")  # split the /usr/sgug/XXX path up
                    #target_root = path[3]       # take the element after sgug
                    # print out the new idb line, fixing the symlink to point to the actual file not itself, fix that path with target_root
                    #print( splits[0] + " " + splits[1] + " " +  splits[2] +  " " + splits[3] + " " + splits[4] + " " + target_root + "/" + value + " " + splits[6] ) 
        # if this is not a symlink line then just spit it out as is
        else:
            print( splits[0] + " " + splits[1] + " " +  splits[2] +  " " + splits[3] + " " + splits[4] + " " + splits[5] + " " + splits[6] )
except:
    print("error")
    pass