Skip to content

Instantly share code, notes, and snippets.

@dlangille dlangille/process_vuxml.pl
Last active Nov 26, 2017

Embed
What would you like to do?
invoked as: /usr/local/bin/perl ./process_vuxml.pl < ${VULNFILE} ^C ${LOGFILE}
#!/usr/local/bin/perl
#
# $Id: process_vuxml.pl,v 1.6 2013-01-16 15:37:57 dan Exp $
#
# Copyright (c) 2001-2012 DVL Software
#
# much of this file is based on contributions from Matthew Seamon
#
# @{#} $Id: process_vuxml.pl,v 1.6 2013-01-16 15:37:57 dan Exp $
#
# Split up the vuln.xml file into sections for individual
# vulnerabilities. Save into files using the vid guid field as name.
# Calculate SHA256 checksum for the XML snippet and write out to an
# index file.
#use 5.10.1;
use strict;
use warnings;
use Digest::SHA qw(sha256_hex);
use autodie qw(:default);
use IO::File;
use committer_opt_in;
use database;
use vuxml;
use vuxml_parsing;
use vuxml_mark_commits;
#use feature qw(switch);
$0 =~ s@.*/@@;
# Reads vuln.xml on stdin
my $start = time;
MAIN:
{
my %vulns;
my @vulns;
my $fh;
my $p;
# slurp vuln.xml whole.
local $/;
@vulns = split /\n+(?=\s+<vuln vid="([^"]+)")/, <>;
# Discard the boilerplate at the top of the file.
shift(@vulns);
# Discard the boilerplate at the end of the file.
$vulns[-1] =~ s@\n</vuxml>.*\Z@@s;
%vulns = @vulns;
my $dbh;
$dbh = FreshPorts::Database::GetDBHandle();
if ($dbh->{Active}) {
my $vuxml = FreshPorts::vuxml->new( $dbh );
eval {
for my $v ( sort keys %vulns ) {
# Make sure xml snippet is terminated with a newline
$vulns{$v} =~ s/\n*\Z/\n/s;
# print $vulns{$v};
my $csum = sha256_hex( $vulns{$v} );
# fetch the checksum from the database
my $checksum = $vuxml->FetchChecksumByVID($v);
my $updateRequired = 1;
if (defined($checksum))
{
if ($csum eq $checksum && 1)
{
$updateRequired = 0;
}
print "$v = '$csum' '$checksum'\n";
}
else
{
print "$v = '$csum' not found\n";
}
if ($updateRequired)
{
print "update is required\n";
print "WHAT WE HAVE IS: '" . $vulns{$v} . "'\n";
$fh = IO::File->new();
if ($fh->open(\$vulns{$v}, '<')) {
$p = FreshPorts::vuxml_parsing->new(Stream => $fh,
DBHandle => $dbh,
UpdateInPlace => 1);
$p->parse_xml($csum);
if ($p->database_updated())
{
print "yes, the database was updated\n";
}
else
{
print "no, the database was NOT updated\n";
next;
}
$fh->close;
} else {
print "open failed\n";
}
# process $vulns{$v} via vuxml_processing
print 'invoking vuxml_mark_commits with ' . $v . "\n";
my $CommitMarker = FreshPorts::vuxml_mark_commits->new(DBHandle => $dbh,
vid => $v);
print 'invoking ProcessEachRangeRecord'. "\n";
my $i = $CommitMarker->ProcessEachRangeRecord();
print 'invoking ClearCachedEntries' . "\n";
$CommitMarker->ClearCachedEntries($v);
}
}
};
print 'finished with eval()' . "\n";
# if something went wrong in the eval, abort and don't do a commit
if ($@) {
print "We've got a problem.";
print "$0: $@\n";
FreshPorts::CommitterOptIn::RecordErrorDetails("error processing vuxml", $0);
die "$0: $@\n";
}
print "committing\n";
$dbh->commit();
$dbh->disconnect();
}
}
system();
my $end = time();
print "Total time: " . ($end - $start) . " seconds\n";
#
# That's All Folks!
#
print 'process_vuxml.pl finishes' . "\n";
@dlangille

This comment has been minimized.

Copy link
Owner Author

dlangille commented Nov 25, 2017

Fails on line 95... not sure why. Ideas?

@dlangille

This comment has been minimized.

Copy link
Owner Author

dlangille commented Nov 25, 2017

This is processing a vuxml file by splitting each vid into a variable, doing a checksum on it to detect changes, so we process only changed/new vid.

@dlangille

This comment has been minimized.

Copy link
Owner Author

dlangille commented Nov 26, 2017

Here is an example:

update is required
WHAT WE HAVE IS: '  <vuln vid="1f8dea68-3436-11d9-952f-000c6e8f12ef">
    <cancelled superseded="9be819c6-4633-11d9-a9e7-0001020eed82"/>
  </vuln>
'
open failed
@dlangille

This comment has been minimized.

Copy link
Owner Author

dlangille commented Nov 26, 2017

ahhh: IO::Handle: bad open mode: SCALAR(0x80757e570) at ./process_vuxml.pl line 96

@dlangille

This comment has been minimized.

Copy link
Owner Author

dlangille commented Nov 26, 2017

After updating the IF to if ($fh->open($vulns{$v}, '<')) {
I get: We've got a problem.process_vuxml.pl:
no element found at line 1, column 0, byte 0 at /usr/local/lib/perl5/site_perl/mach/5.24/XML/Parser.pm line 187.

Idea: leading spaces are messing it up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.