Log of last 20 commits to OWASP sanitizer as of 20 Apr 2019

git.log

$ git log -n 20 --format=full --stat

commit 659ab22922de7597793971fc90a5313e41f6538f
Author: Mike Samuel <mikesamuel@gmail.com>
Commit: Mike Samuel <mikesamuel@gmail.com>

    add test for XSS from twitter

 src/test/java/org/owasp/html/HtmlPolicyBuilderTest.java | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

commit e4eff4f8e995ad13c1f33b477443b544e8afa97b
Author: Mike Samuel <mikesamuel@gmail.com>
Commit: Mike Samuel <mikesamuel@gmail.com>

    documented

 src/main/java/org/owasp/html/HtmlChangeListener.java | 9 +++++++++
 src/main/java/org/owasp/html/HtmlChangeReporter.java | 2 +-
 2 files changed, 10 insertions(+), 1 deletion(-)

commit 778d5c26b88ff07812021376f18f1fbe9bc5a381
Author: Mike Samuel <mikesamuel@gmail.com>
Commit: Mike Samuel <mikesamuel@gmail.com>

    test previously untested global policy merging code in PolicyFactory

 .../java/org/owasp/html/PolicyFactoryTest.java     | 223 +++++++++++++++++++++
 1 file changed, 223 insertions(+)

commit c983b9c0e1d685d832890df187f564dc4b15af04
Author: Mike Samuel <mikesamuel@gmail.com>
Commit: Mike Samuel <mikesamuel@gmail.com>

    Fixed bug: srcset element joining missing space.
    
    The string
    
        /url1, /url2, /url3
    
    have two invalid URLs since the comma attaches to the URL.
    
    When there is no metadata this should be rendered
    
        /url1 , /url2 , /url3
    
    In practice this doesn't matter for non-malicious code
    since the only URL that has no metadata should be the last,
    fallback URL; and there's no attack scenario where appending
    a comma to the end of an attacker controlled URL helps the
    attacker.
    
    This commit uses " , " to separate URL&metadata pairs in
    srcset attributes.
    
    It also adds tests for corner cases of the srcset parser and
    valid floating point number parser.

 .../java/org/owasp/html/SrcsetAttributePolicy.java |  2 +-
 src/main/java/org/owasp/html/Strings.java          | 19 +++++---
 .../java/org/owasp/html/HtmlPolicyBuilderTest.java | 18 +++++---
 src/test/java/org/owasp/html/StringsTest.java      | 50 ++++++++++++++++++++++
 4 files changed, 76 insertions(+), 13 deletions(-)

commit 3ec5c201a334f924e340a260e11c4a49f31b2406
Author: Mike Samuel <mikesamuel@gmail.com>
Commit: Mike Samuel <mikesamuel@gmail.com>

    Bumped dev version

 aggregate/pom.xml  | 4 ++--
 empiricism/pom.xml | 4 ++--
 html-types/pom.xml | 4 ++--
 parent/pom.xml     | 2 +-
 pom.xml            | 2 +-
 5 files changed, 8 insertions(+), 8 deletions(-)

commit b7b4f42eed467bc4bcc9745662853e1c614b5d4f
Author: Mike Samuel <mikesamuel@gmail.com>
Commit: Mike Samuel <mikesamuel@gmail.com>

    Release candidate 20190325.1

 README.md               |  6 +++---
 aggregate/pom.xml       |  4 ++--
 change_log.md           |  2 +-
 docs/getting_started.md | 10 +++++-----
 docs/maven.md           |  2 +-
 empiricism/pom.xml      |  4 ++--
 html-types/pom.xml      |  4 ++--
 parent/pom.xml          |  2 +-
 pom.xml                 |  2 +-
 9 files changed, 18 insertions(+), 18 deletions(-)

commit 72abb6c9f018f4d5096bfe32cc2893e79a3a35b2
Author: Mike Samuel <mikesamuel@gmail.com>
Commit: Mike Samuel <mikesamuel@gmail.com>

    Delete lib/**
    
    Prior to moving to `mvn` for building, the lib directory collected
    dependencies.
    
    Fixes #160

 lib/commons-codec-1.4/LICENSE.txt                  |  202 --
 lib/commons-codec-1.4/NOTICE.txt                   |   14 -
 lib/commons-codec-1.4/RELEASE-NOTES.txt            |   57 -
 lib/commons-codec-1.4/commons-codec-1.4.jar        |  Bin 58160 -> 0 bytes
 lib/guava-libraries/COPYING                        |  202 --
 lib/guava-libraries/README                         |   46 -
 lib/guava-libraries/VERSION                        |    5 -
 lib/guava-libraries/guava-src.jar                  |  Bin 943695 -> 0 bytes
 lib/guava-libraries/guava.jar                      |  Bin 1648200 -> 0 bytes
 lib/htmlparser-1.3/LICENSE.txt                     |   96 -
 lib/htmlparser-1.3/README.txt                      |    5 -
 lib/htmlparser-1.3/doc/README                      |   15 -
 .../doc/named-character-references.html            |    4 -
 lib/htmlparser-1.3/doc/tokenization.txt            | 1147 ----------
 lib/htmlparser-1.3/doc/tree-construction.txt       | 2201 --------------------
 .../htmlparser-1.3-with-transitions.jar            |  Bin 430989 -> 0 bytes
 lib/htmlparser-1.3/htmlparser-1.3.jar              |  Bin 430979 -> 0 bytes
 lib/jsr305/COPYING                                 |   11 -
 lib/jsr305/jsr305.jar                              |  Bin 33036 -> 0 bytes
 lib/junit/README.html                              |  672 ------
 lib/junit/VERSION                                  |    1 -
 lib/junit/cpl-v10.html                             |  125 --
 lib/junit/junit-dep.jar                            |  Bin 217975 -> 0 bytes
 lib/junit/junit-src.jar                            |  Bin 130755 -> 0 bytes
 lib/junit/junit.jar                                |  Bin 237047 -> 0 bytes
 25 files changed, 4803 deletions(-)

commit 2ca428433034a349761eff03e84bbd9b763b4dcc
Author: Mike Samuel <mikesamuel@gmail.com>
Commit: Mike Samuel <mikesamuel@gmail.com>

    Properly parse srcset attributes
    
    Fixes issue #112
    
    Previously, extra URL handling for `src`, `href` and related attributes
    was naively applied to `srcset`.
    
    This failed safe but mangled content.
    
    I consolidated (in a previous commit) the code that ensures that URL and `style`
    attributes have extra checks done: the global URL and style policies
    respectively.
    
    This commit adds special handling for srcset which uses the global URL policy
    to vet each URL policy and leaves the metadata portion alone.
    
    Commas in the middle of URLs in a `srcset` value will be escaped to `%2c`.

 change_log.md                                      |   2 +
 .../java/org/owasp/html/HtmlPolicyBuilder.java     |  10 +-
 .../java/org/owasp/html/SrcsetAttributePolicy.java | 134 +++++++++++++++++++++
 src/main/java/org/owasp/html/Strings.java          |  81 +++++++++++++
 .../java/org/owasp/html/HtmlPolicyBuilderTest.java |  76 ++++++++++++
 5 files changed, 302 insertions(+), 1 deletion(-)

commit 6557bfb86363fd9ad77ce96b9c6a7f432e63dea0
Author: Mike Samuel <mikesamuel@gmail.com>
Commit: Mike Samuel <mikesamuel@gmail.com>

    cleanup uses of deprecated Guava APIs

 src/main/java/org/owasp/html/AutoCloseableHtmlStreamRenderer.java | 6 +++---
 src/main/java/org/owasp/html/Handler.java                         | 8 +++++---
 src/main/java/org/owasp/html/examples/EbayPolicyExample.java      | 6 +++---
 src/main/java/org/owasp/html/examples/SlashdotPolicyExample.java  | 4 ++--
 src/test/java/org/owasp/html/Benchmark.java                       | 2 +-
 src/test/java/org/owasp/html/ExamplesTest.java                    | 8 +++++---
 src/test/java/org/owasp/html/HtmlSanitizerFuzzerTest.java         | 7 +++++--
 7 files changed, 24 insertions(+), 17 deletions(-)

commit 3be8d576f718c9a582a232d00af2ebfe92da2289
Author: Mike Samuel <mikesamuel@gmail.com>
Commit: Mike Samuel <mikesamuel@gmail.com>

    consolidate special attribute handling code in preparation for srcset support

 .../java/org/owasp/html/HtmlPolicyBuilder.java     | 152 +++++++++++++--------
 1 file changed, 95 insertions(+), 57 deletions(-)

commit eaa97f11cbb5e098e278b40e6c76836f7c729d8a
Author: Mike Samuel <mikesamuel@gmail.com>
Commit: Mike Samuel <mikesamuel@gmail.com>

    note on guava version change

 change_log.md | 5 +++++
 1 file changed, 5 insertions(+)

commit 2ae734d840b9a3f8a9ed43c83ea2901e22a26a5f
Author: Mike Samuel <mikesamuel@gmail.com>
Commit: Mike Samuel <mikesamuel@gmail.com>

    remove single digit major version assumption

 scripts/build_for_travis.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

commit 5c4da34c1725cc18fface815877f9689e870cc86
Author: Mike Samuel <mikesamuel@gmail.com>
Commit: Mike Samuel <mikesamuel@gmail.com>

    use mvn install in script/build_for_travis.sh instead of using the Travis mvn default

 .travis.yml | 1 +
 1 file changed, 1 insertion(+)

commit e1790de136c3a8564188fa70a8273c975a3295ad
Author: Mike Samuel <mikesamuel@gmail.com>
Commit: Mike Samuel <mikesamuel@gmail.com>

    Debug Travis CI build
    
    Make sure javadoc builds on JDK11.
    Add some trace to figure out why local jdk7 build differs from TRAVIS.
    
    Pin a modern protobuf version since there was an out of bounds decoder CVE
    in trusted-types/pom.xml.

 parent/pom.xml              | 13 +++++++++++++
 scripts/build_for_travis.sh |  3 ++-
 2 files changed, 15 insertions(+), 1 deletion(-)

commit 4b35ba040a52e11cd719acaa9ac1f101f9fac32d
Author: Mike Samuel <mikesamuel@gmail.com>
Commit: Mike Samuel <mikesamuel@gmail.com>

    Travis deprecated oraclejdk10.  Use openjdk10 instead

 .travis.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

commit 712b400aca9efbab77973f04ba600a1cfa36e595
Author: Mike Samuel <mikesamuel@gmail.com>
Commit: Mike Samuel <mikesamuel@gmail.com>

    Fix CI build scripts that pick flags appropriate to each JDK

 .travis.yml                 |  6 ++----
 scripts/build_for_travis.sh | 26 +++++++++++++++++++++-----
 2 files changed, 23 insertions(+), 9 deletions(-)

commit a50feb294871f00b4f72b74ea44f850d38df7fb9
Author: Mike Samuel <mikesamuel@gmail.com>
Commit: Mike Samuel <mikesamuel@gmail.com>

    document explicit plugin dependency

 parent/pom.xml | 1 +
 1 file changed, 1 insertion(+)

commit af88a5cf415a24def22f3438f76753b0ed03a42d
Author: Mike Samuel <mikesamuel@gmail.com>
Commit: Mike Samuel <mikesamuel@gmail.com>

    run dependency audit pre-release and in CI

 RELEASE-checklist.sh        | 1 +
 scripts/build_for_travis.sh | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)

commit a1dd006133971008ec8b3a6654e54f1547d482a5
Author: Mike Samuel <mikesamuel@gmail.com>
Commit: Mike Samuel <mikesamuel@gmail.com>

    Update guava.version to 27.1
    
    https://github.com/OWASP/java-html-sanitizer/issues/162 identifies a
    problem with depending on older guava versions.
    
    https://ossindex.sonatype.org/vuln/24585a7f-eb6b-4d8d-a2a9-a6f16cc7c1d0
    affects Guava < 24.1.
    
    This does not directly affect the sanitizer but could allow denial of
    service of clients.
    
    Clients that require older JDKs should use `-Dguava.version=` with `mvn`
    to specify a version of their choice or add to their POM's `<properties>`
    element
    
    ```xml
      <guava.version>123.1</guava.version>
    ```

 parent/pom.xml              | 12 ++++++++++--
 scripts/build_for_travis.sh |  4 ++--
 2 files changed, 12 insertions(+), 4 deletions(-)

commit 36687a426e65a0ac024928ce825d4da0f8beae14
Author: Mike Samuel <mikesamuel@gmail.com>
Commit: Mike Samuel <mikesamuel@gmail.com>

    run travis w/ Java versions 10,11

 .travis.yml | 2 ++
 1 file changed, 2 insertions(+)

results.txt

touchSourceFiles=5
touchTestFiles=5
touchEither=7
pct touch test files=71.4285714285714%

script-to-count.pl

use strict;

my $touchSourceFiles = 0;
my $touchTestFiles = 0;
my $touchEither = 0;

my $currentCommitTouchesSourceFile = 0;
my $currentCommitTouchesTestFile = 0;

sub finishProcessingCommit() {
  if ($currentCommitTouchesSourceFile || $currentCommitTouchesTestFile) {
      ++$touchSourceFiles if $currentCommitTouchesSourceFile;
      ++$touchTestFiles if $currentCommitTouchesTestFile;
      ++$touchEither;
  }
  $currentCommitTouchesSourceFile = 0;
  $currentCommitTouchesTestFile = 0;
}

while (<>) {
  if (m/^commit/) {
      finishProcessingCommit();
  }
  if (m/^ \S*Test[.]java/) {
      $currentCommitTouchesTestFile = 1;
  } elsif (m/^ \S*[.]java/) {
      $currentCommitTouchesSourceFile = 1;
  }
}
finishProcessingCommit();

print "touchSourceFiles=$touchSourceFiles\n";
print "touchTestFiles=$touchTestFiles\n";
print "touchEither=$touchEither\n";
print "pct touch test files=" . (($touchTestFiles / $touchEither) * 100) . "%\n";