Reading RPMs

README.md

Something in RPM changed between RHEL5 and RHEL6.

I can read RPMs for CentOS5 and earlier, but in EL6, the signature header is longer than expected by exactly 4 bytes. I have also observed some normal header (non signature) sections being longer than expected by 16 bytes. It is quite confusing.

If you have insight, please let me know here.

header-verify.rb

HEADER_MAGIC = "\x8e\xad\xe8\x01\x00\x00\x00\x00"
TAG_ENTRY_SIZE = 16 # tag id, type, offset, count == 16 bytes

def read_header(io)
  # RPM 'header' section looks like:
  #
  # MAGIC (8 bytes) index_count (4 bytes), data_length (4 bytes )
  #
  # * index_count is the number of 'tags' in this header.
  # * data_length is a blob containing all the values for the tags
  #
  # Header has a header of 'magic' + index_count (4 bytes) + data_length (4 bytes)
  p "start of header" => io.pos
  data = io.read(16).unpack("a8NN")

  # TODO(sissel): @index_count is really a count, rename?
  @magic, @index_count, @data_length = data
  if @magic != HEADER_MAGIC
    puts "Magic value in header was wrong. Expected #{HEADER_MAGIC.inspect}, but got #{@magic.inspect}"
    return
  end
  
  @index_size = @index_count * TAG_ENTRY_SIZE
  tag_data = io.read(@index_size)
  data = io.read(@data_length)
  p "end of header" => io.pos

  (0 ... @index_count).each do |i|
    offset = i * TAG_ENTRY_SIZE
    entry_data = tag_data[i * TAG_ENTRY_SIZE, TAG_ENTRY_SIZE]
    tag, tag_type, offset, count  = entry_data.unpack("NNNN")
    if block_given?
      yield :tag => tag, :type => tag_type, :offset => offset, :count => count
    end
    #entry << data
  end # each index
end

if ARGV.length != 1
  puts "Usage: #{$0} blah.rpm"
  exit 1
end

rpm = File.new(ARGV[0])

# Read the 'lead' - it's mostly an ignored part of the rpm file.
lead = rpm.read(96)
magic, major, minor, type, archnum, name, osnum, signature_type, reserved = lead.unpack("A4CCnnA66nnA16")

puts "RPM file version #{major}.#{minor} (#{signature_type == 5 ? "signed" : "unsigned"})"

if signature_type == 5
  # Read a header for the rpm signature. This has the same format as a normal
  # rpm header
  puts "Checking signature"
  read_header(rpm) do |tag|
    # print each tag in this header
    p tag
  end
end

# Read the rpm header
puts "Checking header"
read_header(rpm)

output-el5

% ruby header-verify.rb ../sample-rpms/redhat-lsb-4.0-2.1.4.el5.i386.rpm
RPM file version 3.0 (signed)
Checking signature
{"start of header"=>96}
{"end of header"=>440}
{:tag=>62, :type=>7, :offset=>200, :count=>16}
{:tag=>267, :type=>7, :offset=>0, :count=>65}
{:tag=>269, :type=>6, :offset=>65, :count=>1}
{:tag=>1000, :type=>4, :offset=>108, :count=>1}
{:tag=>1004, :type=>7, :offset=>112, :count=>16}
{:tag=>1005, :type=>7, :offset=>128, :count=>65}
{:tag=>1007, :type=>4, :offset=>196, :count=>1}
Checking header
{"start of header"=>440}
{"end of header"=>16055}

output-el6

% ruby header-verify.rb ../sample-rpms/redhat-lsb-4.0-3.el6.centos.x86_64.rpm
RPM file version 3.0 (signed)
Checking signature
{"start of header"=>96}
{"end of header"=>1380}
{:tag=>62, :type=>7, :offset=>1140, :count=>16}
{:tag=>268, :type=>7, :offset=>0, :count=>536}
{:tag=>269, :type=>6, :offset=>536, :count=>1}
{:tag=>1000, :type=>4, :offset=>580, :count=>1}
{:tag=>1002, :type=>7, :offset=>584, :count=>536}
{:tag=>1004, :type=>7, :offset=>1120, :count=>16}
{:tag=>1007, :type=>4, :offset=>1136, :count=>1}
Checking header
{"start of header"=>1380}
Magic value in header was wrong. Expected "\x8E\xAD\xE8\x01\x00\x00\x00\x00", but got "\x00\x00\x00\x00\x8E\xAD\xE8\x01"

Magic value in header was wrong. Expected "\x8E\xAD\xE8\x01\x00\x00\x00\x00", but got "\x00\x00\x00\x00\x8E\xAD\xE8\x01"

Reading the 2nd header section failed because the signature header is actually longer by 4 bytes than I calculate. I cannot figure out where this extra 4 bytes comes from. All 4 bytes are zeros as seen above.

I've read lib/header.c and bounced around other parts of the rpm source and I can't figure out what I am doing wrong or how librpm is handling these two different rpm files separately.

The code I was reviewing was was here: http://rpm.org/releases/rpm-4.9.x/rpm-4.9.1.2.tar.bz2

If I test a bunch of RPMs, here is the result:

OK: ../sample-rpms/redhat-lsb-3.0-8.EL.x86_64.rpm
OK: ../sample-rpms/redhat-lsb-3.1-19.fc8.x86_64.rpm
OK: ../sample-rpms/redhat-lsb-3.2-6.fc10.1.i386.rpm
OK: ../sample-rpms/redhat-lsb-4.0-2.1.4.el5.i386.rpm
FAIL: ../sample-rpms/redhat-lsb-4.0-2.fc13.x86_64.rpm
FAIL: ../sample-rpms/redhat-lsb-4.0-3.el6.centos.x86_64.rpm
FAIL: ../sample-rpms/redhat-lsb-4.0-4.fc11.i586.rpm
FAIL: ../sample-rpms/redhat-lsb-4.0-4.fc12.i686.rpm
FAIL: ../sample-rpms/redhat-lsb-4.0-7.fc16.x86_64.rpm

Looks like the change happened between Fedora 10 and Fedora 11, and CentOS 5 and CentOS6.

Victory! The rpm mailing list was kind and pointed out that the 'security' header is padded up to an 8-byte boundary based on the length of the security header.

Victory!

% for i in ../sample-rpms/*.rpm; do ruby printrpm.rb $i > /dev/null; file /tmp/rpm.payload | sed -e "s,^,$i ,"; done
../sample-rpms/redhat-lsb-3.0-8.EL.x86_64.rpm /tmp/rpm.payload: gzip compressed data, from Unix
../sample-rpms/redhat-lsb-3.1-19.fc8.x86_64.rpm /tmp/rpm.payload: gzip compressed data, from Unix
../sample-rpms/redhat-lsb-3.2-6.fc10.1.i386.rpm /tmp/rpm.payload: gzip compressed data, from Unix
../sample-rpms/redhat-lsb-4.0-2.1.4.el5.i386.rpm /tmp/rpm.payload: gzip compressed data, from Unix
../sample-rpms/redhat-lsb-4.0-2.fc13.x86_64.rpm /tmp/rpm.payload: XZ compressed data
../sample-rpms/redhat-lsb-4.0-3.el6.centos.x86_64.rpm /tmp/rpm.payload: XZ compressed data
../sample-rpms/redhat-lsb-4.0-4.fc11.i586.rpm /tmp/rpm.payload: gzip compressed data, from Unix
../sample-rpms/redhat-lsb-4.0-4.fc12.i686.rpm /tmp/rpm.payload: XZ compressed data
../sample-rpms/redhat-lsb-4.0-7.fc16.x86_64.rpm /tmp/rpm.payload: XZ compressed data