Howto remove undesirable datasets from RRD

README.md

Overview

Sometimes arises the wish to delete some broken datasets from RRD files. This is not possible out-of-the-box by design. But there are the rrdtool dump/restore commands. The interchange format is XML. So the idea is to modify the XML before restoring it.

There are two little deficiencies with the XML. It's not documented and some useful information is provided by XML comments instead of be part of the XML itself. This makes XSLT processing difficult.

Here are some XSLT transformations, that are trying to handle the deletion of some datasets and producing a new working RRD file.

Usage

rrdtool dump $rrd.corrupt |\
  xsltproc rrd.dump.clean.xslt - |\
  xsltproc rrd.dump.modify.xslt - |\
  xsltproc rrd.dump.reorder.xslt - |\
  xmllint --format - |\
rrdtool restore - $rrd.new

Remarks

• rrd.dump.clean.xslt

  • pulls time information from XML comments to attributes, notably for <row/>
  • removes all XML comments

• rrd.dump.modify.xslt

  • sets lastupdate time and removes all RRA datasets beyond
  • sets lastupdate values to NaN because the real value is lost
  • deletes <cdp_prep/> because its not documented (seems rrdrestore 1.4.7. doesn't care about that)

• rrd.dump.reorder.xslt

  • moves the removed datasets to head of RRA

rrd.dump.clean.xslt

<?xml version="1.0"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output method="xml" indent="yes" encoding="iso-8859-1"/>

  <!-- see www.xmlplease.com/xsltidentity for further help -->
  <xsl:template match="@*|node()">
    <xsl:copy>
      <xsl:apply-templates select="@*|node()"/>
    </xsl:copy>
  </xsl:template>

  <xsl:template match="comment()"/>

  <xsl:template match="lastupdate">
    <xsl:copy>
      <xsl:attribute name="date">
        <xsl:value-of select="normalize-space(following-sibling::comment()[1])"/>
      </xsl:attribute>
      <xsl:apply-templates select="@*|node()"/>
    </xsl:copy>
  </xsl:template>

  <xsl:template match="rra/database/row">
    <xsl:copy>
      <xsl:attribute name="date">
        <xsl:value-of select="normalize-space(substring-before(preceding-sibling::comment()[1], ' / '))"/>
      </xsl:attribute>
      <xsl:attribute name="serial">
        <xsl:value-of select="normalize-space(substring-after(preceding-sibling::comment()[1], ' / '))"/>
      </xsl:attribute>
      <xsl:apply-templates select="@*|node()"/>
    </xsl:copy>
  </xsl:template>

  <xsl:template match="rra/pdp_per_row">
    <xsl:copy>
      <xsl:attribute name="interval">
        <xsl:value-of select="normalize-space(substring-before(following-sibling::comment()[1], 'seconds'))"/>
      </xsl:attribute>
      <xsl:apply-templates select="@*|node()"/>
    </xsl:copy>
  </xsl:template>
</xsl:stylesheet>

rrd.dump.modify.xslt

<?xml version="1.0"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output method="xml" indent="yes" encoding="iso-8859-1"/>

  <!-- see www.xmlplease.com/xsltidentity for further help -->
  <xsl:template match="@*|node()">
    <xsl:copy>
      <xsl:apply-templates select="@*|node()"/>
    </xsl:copy>
  </xsl:template>

  <xsl:template match="cdp_prep"/>

  <xsl:template match="lastupdate">
    <xsl:copy>
      <xsl:apply-templates select="@*"/>
      <xsl:text>1435968000</xsl:text>
    </xsl:copy>
  </xsl:template>

  <xsl:template match="ds/last_ds">
    <xsl:copy>
      <xsl:apply-templates select="@*"/>
      <xsl:text>NaN</xsl:text>
    </xsl:copy>
  </xsl:template>
  <xsl:template match="ds/value">
    <xsl:copy>
      <xsl:apply-templates select="@*"/>
      <xsl:text>NaN</xsl:text>
    </xsl:copy>
  </xsl:template>

  <xsl:template match="rra/database/row">
  <xsl:choose>
    <xsl:when test="@serial &lt; 1435968000">
      <xsl:copy>
        <xsl:apply-templates select="@*|node()"/>
      </xsl:copy>
    </xsl:when>
    <xsl:otherwise>
      <removed date="{@date}" serial="{@serial}"/>
    </xsl:otherwise>
  </xsl:choose>
  </xsl:template>
</xsl:stylesheet>

rrd.dump.reorder.xslt

<?xml version="1.0"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output method="xml" indent="yes" encoding="iso-8859-1"/>

  <!-- see www.xmlplease.com/xsltidentity for further help -->
  <xsl:template match="@*|node()">
    <xsl:copy>
      <xsl:apply-templates select="@*|node()"/>
    </xsl:copy>
  </xsl:template>

  <xsl:template match="removed"/>

  <xsl:template match="database">
    <database>
      <xsl:for-each select="removed">
        <row><v>NaN</v><v>NaN</v></row>
      </xsl:for-each>
      <xsl:for-each select="row">
        <xsl:copy>
          <xsl:apply-templates select="@*|node()"/>
        </xsl:copy>
      </xsl:for-each>
    </database>
  </xsl:template>
</xsl:stylesheet>