Skip to content

Instantly share code, notes, and snippets.

@alpmestan

alpmestan/validate.md Secret

Last active March 12, 2018 17:07
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save alpmestan/c371840968f086c8dc5b56af8325f0a9 to your computer and use it in GitHub Desktop.
Save alpmestan/c371840968f086c8dc5b56af8325f0a9 to your computer and use it in GitHub Desktop.
Download ZIP
breaking down the `./validate --slow` failures
Raw
validate.md

Summary

SUMMARY for test run started at Wed Mar  7 20:54:57 2018 CET
 1:11:08 spent to go through
    6280 total tests, which gave rise to
   23392 test cases, of which
    4250 were skipped

     136 had missing libraries
   18651 expected passes
     287 expected failures

       0 caused framework failures
       0 caused framework warnings
       0 unexpected passes
      39 unexpected failures
      29 unexpected stat failures

Unexpected failures:
   /tmp/ghctest-n4fi8zlk/test   spaces/./codeGen/should_run/T5129.run               T5129 [bad exit code] (hpc)
   /tmp/ghctest-n4fi8zlk/test   spaces/./codeGen/should_run/T5129.run               T5129 [bad exit code] (optasm)
   /tmp/ghctest-n4fi8zlk/test   spaces/./codeGen/should_run/T5129.run               T5129 [bad exit code] (threaded2)
   /tmp/ghctest-n4fi8zlk/test   spaces/./codeGen/should_run/T5129.run               T5129 [bad exit code] (dyn)
   /tmp/ghctest-n4fi8zlk/test   spaces/./codeGen/should_run/T5129.run               T5129 [bad exit code] (optllvm)
   /tmp/ghctest-n4fi8zlk/test   spaces/./deSugar/should_run/dsrun014.run            dsrun014 [bad stderr] (hpc)
   /tmp/ghctest-n4fi8zlk/test   spaces/./deSugar/should_run/dsrun014.run            dsrun014 [bad stderr] (optasm)
   /tmp/ghctest-n4fi8zlk/test   spaces/./deSugar/should_run/dsrun014.run            dsrun014 [bad stderr] (threaded2)
   /tmp/ghctest-n4fi8zlk/test   spaces/./deSugar/should_run/dsrun014.run            dsrun014 [bad stderr] (dyn)
   /tmp/ghctest-n4fi8zlk/test   spaces/./deSugar/should_run/dsrun014.run            dsrun014 [bad stderr] (optllvm)
   /tmp/ghctest-n4fi8zlk/test   spaces/./driver/T4114c.run                          T4114c [bad exit code] (ghci)
   /tmp/ghctest-n4fi8zlk/test   spaces/./driver/T4114d.run                          T4114d [bad exit code] (ghci)
   /tmp/ghctest-n4fi8zlk/test   spaces/./ffi/should_fail/UnsafeReenter.run          UnsafeReenter [bad exit code] (threaded1)
   /tmp/ghctest-n4fi8zlk/test   spaces/./ffi/should_fail/UnsafeReenter.run          UnsafeReenter [bad exit code] (threaded2)
   /tmp/ghctest-n4fi8zlk/test   spaces/./numeric/should_compile/T8542.run           T8542 [stderr mismatch] (hpc)
   /tmp/ghctest-n4fi8zlk/test   spaces/./patsyn/should_run/T14228.run               T14228 [bad stderr] (ghci)
   /tmp/ghctest-n4fi8zlk/test   spaces/./rts/T2783.run                              T2783 [bad exit code] (threaded1)
   /tmp/ghctest-n4fi8zlk/test   spaces/./rts/stack004.run                           stack004 [bad stdout] (ghci)
   /tmp/ghctest-n4fi8zlk/test   spaces/./rts/flags/T12870a.run                      T12870a [bad stderr] (ghci)
   /tmp/ghctest-n4fi8zlk/test   spaces/./rts/flags/T12870b.run                      T12870b [bad exit code] (ghci)
   /tmp/ghctest-n4fi8zlk/test   spaces/./rts/flags/T12870c.run                      T12870c [bad exit code] (ghci)
   /tmp/ghctest-n4fi8zlk/test   spaces/./rts/flags/T12870d.run                      T12870d [bad stderr] (ghci)
   /tmp/ghctest-n4fi8zlk/test   spaces/./rts/flags/T12870e.run                      T12870e [bad stderr] (ghci)
   /tmp/ghctest-n4fi8zlk/test   spaces/./rts/flags/T12870f.run                      T12870f [bad stderr] (ghci)
   /tmp/ghctest-n4fi8zlk/test   spaces/./rts/flags/T12870e.run                      T12870e [bad stdout] (threaded2)
   /tmp/ghctest-n4fi8zlk/test   spaces/./rts/flags/T12870f.run                      T12870f [bad stdout] (threaded2)
   /tmp/ghctest-n4fi8zlk/test   spaces/./rts/flags/T12870g.run                      T12870g [bad stderr] (ghci)
   /tmp/ghctest-n4fi8zlk/test   spaces/./rts/flags/T12870h.run                      T12870h [bad stderr] (ghci)
   /tmp/ghctest-n4fi8zlk/test   spaces/./simplCore/should_compile/T13543.run        T13543 [stderr mismatch] (hpc)
   /tmp/ghctest-n4fi8zlk/test   spaces/./simplCore/should_compile/T13543.run        T13543 [stderr mismatch] (optasm)
   /tmp/ghctest-n4fi8zlk/test   spaces/./simplCore/should_compile/T13543.run        T13543 [stderr mismatch] (optllvm)
   /tmp/ghctest-n4fi8zlk/test   spaces/./th/T13366.run                              T13366 [bad stderr] (ghci)
   /tmp/ghctest-n4fi8zlk/test   spaces/./typecheck/should_compile/T13032.run        T13032 [stderr mismatch] (hpc)
   /tmp/ghctest-n4fi8zlk/test   spaces/./typecheck/should_run/StrictPats.run        StrictPats [bad stderr] (ghci)
   /tmp/ghctest-n4fi8zlk/test   spaces/./typecheck/should_run/T12809.run            T12809 [bad stderr] (ghci)
   /tmp/ghctest-n4fi8zlk/test   spaces/./typecheck/should_run/EtaExpandLevPoly.run  EtaExpandLevPoly [bad stderr] (ghci)
   /tmp/ghctest-n4fi8zlk/test   spaces/../../libraries/base/tests/T8089.run         T8089 [bad exit code] (ghci)
   /tmp/ghctest-n4fi8zlk/test   spaces/../../libraries/base/tests/T8089.run         T8089 [bad exit code] (threaded1)
   /tmp/ghctest-n4fi8zlk/test   spaces/../../libraries/base/tests/T8089.run         T8089 [bad exit code] (threaded2)

Unexpected stat failures:
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/compiler/parsing001.run         parsing001 [stat not good enough] (normal)
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/compiler/T13379.run             T13379 [stat not good enough] (hpc)
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/compiler/T12707.run             T12707 [stat not good enough] (hpc)
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/compiler/T13379.run             T13379 [stat not good enough] (optllvm)
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/compiler/T12707.run             T12707 [stat not good enough] (optasm)
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/compiler/T12707.run             T12707 [stat not good enough] (optllvm)
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/compiler/ManyConstructors.run   ManyConstructors [stat not good enough] (hpc)
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/compiler/ManyAlternatives.run   ManyAlternatives [stat not good enough] (hpc)
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/compiler/ManyAlternatives.run   ManyAlternatives [stat not good enough] (optasm)
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/compiler/ManyAlternatives.run   ManyAlternatives [stat not good enough] (optllvm)
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/compiler/ManyConstructors.run   ManyConstructors [stat not good enough] (optasm)
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/compiler/ManyConstructors.run   ManyConstructors [stat not good enough] (optllvm)
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/compiler/MultiLayerModules.run  MultiLayerModules [stat not good enough] (hpc)
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/compiler/T13719.run             T13719 [stat not good enough] (hpc)
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/compiler/T13719.run             T13719 [stat not good enough] (optasm)
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/compiler/MultiLayerModules.run  MultiLayerModules [stat not good enough] (optllvm)
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/compiler/T13701.run             T13701 [stat not good enough] (hpc)
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/compiler/T14697.run             T14697 [stat not good enough] (hpc)
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/compiler/T14697.run             T14697 [stat not good enough] (optasm)
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/compiler/T13719.run             T13719 [stat not good enough] (optllvm)
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/compiler/T14697.run             T14697 [stat not good enough] (optllvm)
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/compiler/T13701.run             T13701 [stat not good enough] (optasm)
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/compiler/T9630.run              T9630 [stat not good enough] (hpc)
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/compiler/T13701.run             T13701 [stat not good enough] (optllvm)
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/space_leaks/space_leak_001.run  space_leak_001 [stat too good] (hpc)
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/space_leaks/space_leak_001.run  space_leak_001 [stat too good] (optasm)
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/space_leaks/T4334.run           T4334 [stat not good enough] (threaded2)
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/space_leaks/space_leak_001.run  space_leak_001 [stat too good] (dyn)
   /tmp/ghctest-n4fi8zlk/test   spaces/./perf/space_leaks/space_leak_001.run  space_leak_001 [stat too good] (optllvm)

I did install the libraries listed on this page but I nonetheless am missing some still, apparently. All the unexpected passes/failures are examined below.

Unexpected passes:

  • # Simon says in #11330: The "simplifier ticks exhausted" error is expected;
# see Section 7 of the paper:
# http://research.microsoft.com/en-us/um/people/simonpj/papers/haskell-dynamic/
test('dynamic-paper', expect_fail_for(['optasm', 'optllvm']), compile, [''])

    /tmp/ghctest-c6940g1m/test   spaces/./dependent/should_compile/dynamic-paper.run  dynamic-paper [unexpected] (optasm)
/tmp/ghctest-c6940g1m/test   spaces/./dependent/should_compile/dynamic-paper.run  dynamic-paper [unexpected] (optllvm)

    Expecting too many ticks, but it just compiles here. With 8.2 too apparently. Probably just "fixed" ? I posted a comment on the trac ticket. Will push a diff to expect that test to pass.

  • test('read029', expect_broken_for(10181, ['optasm', 'optllvm']), compile, [''])

    /tmp/ghctest-c6940g1m/test   spaces/./parser/should_compile/read029.run           read029 [unexpected] (optasm)
/tmp/ghctest-c6940g1m/test   spaces/./parser/should_compile/read029.run           read029 [unexpected] (optllvm)

    Just "fixed" as well ? Even though the ticket this test links to doesn't really show any relevant update. But it builds fine with the past 4 major releases of GHC on my machine. Since not all the tests from that ticket show an unexpected result, maybe there isn't anything more to do on this. Will tweak this so that we expect this test to pass.

  • test('rn041', expect_broken_for(10181, ['optasm', 'optllvm']), compile, ['']

    /tmp/ghctest-c6940g1m/test   spaces/./rename/should_compile/rn041.run             rn041 [unexpected] (optasm)
/tmp/ghctest-c6940g1m/test   spaces/./rename/should_compile/rn041.run             rn041 [unexpected] (optllvm)

    As you can see, this test refers to the same ticket as the parser test right above. It builds fine with HEAD, 8.2.2, 8.0.2, 7.10.3. Will tweak this so that we expect this test to pass.

  • test('T7411', [expect_broken_for(7411, ['optasm', 'optllvm',
                                   'hpc', 'threaded2', 'dyn']),
          exit_code(1)], compile_and_run, [''])

    /tmp/ghctest-c6940g1m/test   spaces/./simplCore/should_fail/T7411.run             T7411 [unexpected] (hpc)

    Should be safe to remove 'hpc' from the expected-broken list, but there isn't much more to do I suppose?

  • test('tc165', expect_broken_for(10181, ['optasm', 'optllvm']), compile, [''])

    /tmp/ghctest-c6940g1m/test   spaces/./typecheck/should_compile/tc165.run          tc165 [unexpected] (optasm)
/tmp/ghctest-c6940g1m/test   spaces/./typecheck/should_compile/tc165.run          tc165 [unexpected] (optllvm)

    Related to #10181 again. We could turn this into normal.

Unexpected failures

  • test('T5129', normal, compile_and_run, [''])

    /tmp/ghctest-c6940g1m/test   spaces/./codeGen/should_run/T5129.run               T5129 [bad exit code] (hpc)
/tmp/ghctest-c6940g1m/test   spaces/./codeGen/should_run/T5129.run               T5129 [bad exit code] (optasm)
/tmp/ghctest-c6940g1m/test   spaces/./codeGen/should_run/T5129.run               T5129 [bad exit code] (threaded2)
/tmp/ghctest-c6940g1m/test   spaces/./codeGen/should_run/T5129.run               T5129 [bad exit code] (dyn)
/tmp/ghctest-c6940g1m/test   spaces/./codeGen/should_run/T5129.run               T5129 [bad exit code] (optllvm)

    I tried building/running this one with the inplace GHC built by ./validate, and the result basically depends on the optimisation level:

    alp@vm:~/ghc$ inplace/bin/ghc-stage2 -O2 -fforce-recomp testsuite/tests/codeGen/should_run/T5129.hs 
[1 of 1] Compiling Main             ( testsuite/tests/codeGen/should_run/T5129.hs, testsuite/tests/codeGen/should_run/T5129.o )
Linking testsuite/tests/codeGen/should_run/T5129 ...
alp@vm:~/ghc$ testsuite/tests/codeGen/should_run/T5129
T5129: HUnitFailure "must throw when given a negative number"
alp@vm:~/ghc$ inplace/bin/ghc-stage2 -O -fforce-recomp testsuite/tests/codeGen/should_run/T5129.hs 
[1 of 1] Compiling Main             ( testsuite/tests/codeGen/should_run/T5129.hs, testsuite/tests/codeGen/should_run/T5129.o )
Linking testsuite/tests/codeGen/should_run/T5129 ...
alp@vm:~/ghc$ testsuite/tests/codeGen/should_run/T5129
T5129: HUnitFailure "must throw when given a negative number"
alp@vm:~/ghc$ inplace/bin/ghc-stage2 -O0 -fforce-recomp testsuite/tests/codeGen/should_run/T5129.hs 
[1 of 1] Compiling Main             ( testsuite/tests/codeGen/should_run/T5129.hs, testsuite/tests/codeGen/should_run/T5129.o )
Linking testsuite/tests/codeGen/should_run/T5129 ...I'
alp@vm:~/ghc$ testsuite/tests/codeGen/should_run/T5129

    I then looked into the precise commands used to build and run this with say the optasm way:

    =====> T5129(optasm) 1 of 1 [0, 1, 0]
cd "/tmp/ghctest-9qwxobkn/test   spaces/./codeGen/should_run/T5129.run" &&  "/home/alp/ghc/bindisttest/install   dir/bin/ghc" -o T5129 T5129.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -dno-debug-output  -O -fasm 
cd "/tmp/ghctest-9qwxobkn/test   spaces/./codeGen/should_run/T5129.run" && ./T5129  
Wrong exit code for T5129(optasm)(expected 0 , actual 1 )
Stderr ( T5129 ):
T5129: HUnitFailure "must throw when given a negative number"
*** unexpected failure for T5129(optasm)

    So it looks like we broke the behaviour this test relied on. When? I tried to find out, would appear to be somewhere on the way to 8.2.1.

    $ nix-shell -p haskell.compiler.ghc822 --run 'ghc-8.2.2 -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -dno-debug-output  -O -fasm -fforce-recomp ~/WT/ghc/testsuite/tests/codeGen/should_run/T5129.hs'
[1 of 1] Compiling Main             ( /home/alp/WT/ghc/testsuite/tests/codeGen/should_run/T5129.hs, /home/alp/WT/ghc/testsuite/tests/codeGen/should_run/T5129.o )
Linking /home/alp/WT/ghc/testsuite/tests/codeGen/should_run/T5129 ...
$ /home/alp/WT/ghc/testsuite/tests/codeGen/should_run/T5129
T5129: HUnitFailure "must throw when given a negative number"
$ nix-shell -p haskell.compiler.ghc821 --run 'ghc-8.2.1 -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -dno-debug-output  -O -fasm -fforce-recomp ~/WT/ghc/testsuite/tests/codeGen/should_run/T5129.hs && /home/alp/WT/ghc/testsuite/tests/codeGen/should_run/T5129'
[1 of 1] Compiling Main             ( /home/alp/WT/ghc/testsuite/tests/codeGen/should_run/T5129.hs, /home/alp/WT/ghc/testsuite/tests/codeGen/should_run/T5129.o )
Linking /home/alp/WT/ghc/testsuite/tests/codeGen/should_run/T5129 ...
T5129: HUnitFailure "must throw when given a negative number"
$ nix-shell -p haskell.compiler.ghc802 --run 'ghc-8.0.2 -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -dno-debug-output  -O -fasm -fforce-recomp ~/WT/ghc/testsuite/tests/codeGen/should_run/T5129.hs && /home/alp/WT/ghc/testsuite/tests/codeGen/should_run/T5129'
[1 of 1] Compiling Main             ( /home/alp/WT/ghc/testsuite/tests/codeGen/should_run/T5129.hs, /home/alp/WT/ghc/testsuite/tests/codeGen/should_run/T5129.o )
Linking /home/alp/WT/ghc/testsuite/tests/codeGen/should_run/T5129 ...
$

    I reduced the flags further:

    $ nix-shell -p haskell.compiler.ghc821 --run 'ghc-8.2.1 -O -fforce-recomp ~/WT/ghc/testsuite/tests/codeGen/should_run/T5129.hs && /home/alp/WT/ghc/testsuite/tests/codeGen/should_run/T5129'
[1 of 1] Compiling Main             ( /home/alp/WT/ghc/testsuite/tests/codeGen/should_run/T5129.hs, /home/alp/WT/ghc/testsuite/tests/codeGen/should_run/T5129.o )
Linking /home/alp/WT/ghc/testsuite/tests/codeGen/should_run/T5129 ...
T5129: HUnitFailure "must throw when given a negative number"

$ nix-shell -p haskell.compiler.ghc802 --run 'ghc-8.0.2 -O -fforce-recomp ~/WT/ghc/testsuite/tests/codeGen/should_run/T5129.hs && /home/alp/WT/ghc/testsuite/tests/codeGen/should_run/T5129'
[1 of 1] Compiling Main             ( /home/alp/WT/ghc/testsuite/tests/codeGen/should_run/T5129.hs, /home/alp/WT/ghc/testsuite/tests/codeGen/should_run/T5129.o )
Linking /home/alp/WT/ghc/testsuite/tests/codeGen/should_run/T5129 ...

$

    So somewhere between 8.0.2 and 8.2.1 the behaviour of the optimiser with -O changed. I'll mark this test as expected broken for those ways for now. I also left a little comment on the relevant ticket.

  • test('dsrun014', normal, compile_and_run, ['-fobject-code'])

    /tmp/ghctest-n4fi8zlk/test   spaces/./deSugar/should_run/dsrun014.run            dsrun014 [bad stderr] (hpc)
/tmp/ghctest-n4fi8zlk/test   spaces/./deSugar/should_run/dsrun014.run            dsrun014 [bad stderr] (optasm)
/tmp/ghctest-n4fi8zlk/test   spaces/./deSugar/should_run/dsrun014.run            dsrun014 [bad stderr] (threaded2)
/tmp/ghctest-n4fi8zlk/test   spaces/./deSugar/should_run/dsrun014.run            dsrun014 [bad stderr] (dyn)
/tmp/ghctest-n4fi8zlk/test   spaces/./deSugar/should_run/dsrun014.run            dsrun014 [bad stderr] (optllvm)

    So this one is a quite simple program:

    {-# LANGUAGE UnboxedTuples #-}

module Main where

import Debug.Trace

{-# NOINLINE f #-}
f :: a -> b -> (# a,b #)
f x y = x `seq` y `seq` (# x,y #)

g :: Int -> Int -> Int
g v w = case f v w of
          (# a,b #) -> a+b

main = print (g (trace "one" 1) (trace "two" 2))
-- The args should be evaluated in the right order!

    where all the 5 ways you can see above (but not, say, the ghci way for example!) end up generating the trace:

    two
one

    instead of the expected

    one
two

    I couldn't find a ticket attached to this test, so I created one here. Until we fix it, I'll make the testsuite expect this test to be broken for those 5 ways.

  • test( 'T4114c', fobject_code, compile_and_run, ['-no-keep-o-files'])
test( 'T4114d', fobject_code, compile_and_run,
     ['-hisuf .myhi -osuf .myo -no-keep-o-files'])

    /tmp/ghctest-n4fi8zlk/test   spaces/./driver/T4114c.run                          T4114c [bad exit code] (ghci)
/tmp/ghctest-n4fi8zlk/test   spaces/./driver/T4114d.run                          T4114d [bad exit code] (ghci)

    These two simple tests here check that -no-keep-o-files is honored, and that happens not to be the case with the ghci way:

    =====> T4114c(ghci) 1 of 2 [0, 0, 0]
cd "./driver/T4114c.run" && "/home/alp/ghc/inplace/test   spaces/ghc-stage2" T4114c.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -dno-debug-output -fobject-code --interactive -v0 -ignore-dot-ghci -fno-ghci-history +RTS -I0.1 -RTS -no-keep-o-files< T4114c.genscript
Wrong exit code for T4114c(ghci) (expected 0 , actual 1 )
Stderr ( T4114c ):
T4114c: error: intermediate 'T4114cSub.o' exists
CallStack (from HasCallStack):
  error, called at ./T4114cSub.hs:9:8 in main:T4114cSub
*** unexpected failure for T4114c(ghci)
=====> T4114d(ghci) 2 of 2 [0, 1, 0]
cd "./driver/T4114d.run" && "/home/alp/ghc/inplace/test   spaces/ghc-stage2" T4114d.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -dno-debug-output -fobject-code --interactive -v0 -ignore-dot-ghci -fno-ghci-history +RTS -I0.1 -RTS -hisuf .myhi -osuf .myo -no-keep-o-files< T4114d.genscript
Wrong exit code for T4114d(ghci) (expected 0 , actual 1 )
Stderr ( T4114d ):
T4114d: error: intermediate 'T4114dSub.myo' exists
CallStack (from HasCallStack):
  error, called at ./T4114dSub.hs:9:8 in main:T4114dSub
*** unexpected failure for T4114d(ghci)

    I left a comment on the relevant ticket. Looks like an actual regression, Until we figure this out, I'll mark those tests as broken.

  • test('UnsafeReenter', [omit_ways(['ghciext', 'ghci']), exit_code(1)], compile_and_run, ['-v0 UnsafeReenterC.c'])

    /tmp/ghctest-n4fi8zlk/test   spaces/./ffi/should_fail/UnsafeReenter.run          UnsafeReenter [bad exit code] (threaded1)
/tmp/ghctest-n4fi8zlk/test   spaces/./ffi/should_fail/UnsafeReenter.run          UnsafeReenter [bad exit code] (threaded2)

    This one is a quite simple program:

    {-# LANGUAGE ForeignFunctionInterface #-}

-- | Test that unsafe FFI calls crash the RTS if they attempt to re-enter
-- Haskell-land
module Main where

import Foreign

foreign import ccall "wrapper" wrap_f :: IO () -> IO (FunPtr (IO ()))
foreign import ccall unsafe hello :: FunPtr (IO ()) -> IO ()

f :: IO ()
f = putStrLn "Back in Haskell"

main :: IO ()
main = do
    putStrLn "In Haskell"
    wrap_f f >>= hello
    putStrLn "Finished"

    This just hangs with the threaded1 and threaded2 ways, instead of erroring out with:

    UnsafeReenter: schedule: re-entered unsafely.
   Perhaps a 'foreign import unsafe' should be 'safe'?

    as "expected". I created a ticket (there wasn't any for this test) here and will expect this test to be broken for those 2 ways.

  • test('T8542', normal, compile, [''])

    numeric/should_compile/T8542.run  T8542 [stderr mismatch] (hpc)

    I left a comment on the appropriate ticket to discuss this seemingly buggy behaviour. I'll push a patch expecting the test to be broken for the hpc.

  • test('T14228', normal, compile_and_run, [''])

    /tmp/ghctest-n4fi8zlk/test   spaces/./patsyn/should_run/T14228.run               T14228 [bad stderr] (ghci)

    This program expects an empty stderr but we get an error when loading this test in ghci:

    =====> T14228(ghci) 1 of 1 [0, 0, 0]
cd "./patsyn/should_run/T14228.run" && "/home/alp/ghc/inplace/test   spaces/ghc-stage2" T14228.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -dno-debug-output  --interactive -v0 -ignore-dot-ghci -fno-ghci-history +RTS -I0.1 -RTS < T14228.genscript
Actual stderr output differs from expected:
diff -uw "/dev/null" "./patsyn/should_run/T14228.run/T14228.run.stderr.normalised"
--- /dev/null	2018-03-07 13:03:27.344000000 +0100
+++ ./patsyn/should_run/T14228.run/T14228.run.stderr.normalised	2018-03-08 11:16:59.919965000 +0100
@@ -0,0 +1,4 @@
+
+T14228:6:30:
+    Not in scope: ‘Main.main’
+    No module named ‘Main’ is imported.
*** unexpected failure for T14228(ghci)

    The module is:

    {-# LANGUAGE UnboxedSums #-}
{-# LANGUAGE PatternSynonyms #-}
module Main where

type Maybe' t = (# t | () #)

pattern Just' :: a -> Maybe' a
pattern Just' x = (# x | #)

pattern Nothing' :: Maybe' a
pattern Nothing' = (# | () #)

foo x = case x of
  Nothing' -> putStrLn "nothing"
  Just' _ -> putStrLn "just"

main = do
  putStrLn "Nothing'"
  foo Nothing'

  putStrLn "Just'"
  foo (Just' "hello")

    The problem turns out to be that ghci doesn't manage to load the module (for a good reason) and therefore cannot find Main nor Main.main afterwards. Let's simply skip the ghci way for this test.

  • # Blackhole-detection test.
# Skip GHCi due to #2786
test('T2783', [ omit_ways(['ghci']), exit_code(1) ], compile_and_run, [''])

    /tmp/ghctest-n4fi8zlk/test   spaces/./rts/T2783.run                              T2783 [bad exit code] (threaded1)

    The program in question (below) is expected to error out with T2783: <<loop>>, but what I actually get with the threaded1 way is an assertion failure:

    main = print $ do x <- [ 0 .. 5 ] ; let { y = 5 - y } ; return y

    =====> T2783(threaded1) 1 of 1 [0, 0, 0]
cd "./rts/T2783.run" &&  "/home/alp/ghc/inplace/test   spaces/ghc-stage2" -o T2783 T2783.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -dno-debug-output  -threaded -debug 
cd "./rts/T2783.run" && ./T2783  
Wrong exit code for T2783(threaded1)(expected 1 , actual 134 )
Stderr ( T2783 ):
T2783: internal error: ASSERTION FAILED: file rts/ThreadPaused.c, line 314

    (GHC version 8.5.20180306 for x86_64_unknown_linux)
    Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug
Aborted (core dumped)
*** unexpected failure for T2783(threaded1)

    The code in question is in the threadPaused function:

    // We should never have made it here in the event of blackholes that
// we already own; they should have been marked when we blackholed
// them and consequently we should have stopped our stack walk
// above.
ASSERT(!((bh_info == &stg_BLACKHOLE_info)
         && (((StgInd*)bh)->indirectee == (StgClosure*)tso)));

    This appears to be quite a real problem. I commented on the relevant ticket. We can expect this test to be broken for the threaded1 way in the meantime.

  • # Test that +RTS -K0 (e.g. no stack limit) parses correctly
test('stack004', [extra_run_opts('+RTS -K0 -RTS') ], compile_and_run, [''])

    /tmp/ghctest-n4fi8zlk/test   spaces/./rts/stack004.run                           stack004 [bad stdout] (ghci)

    The program in question, and the output:

    module Main where

import Control.Monad
import GHC.RTS.Flags

-- Ensure that +RTS -K0 is parsed
main :: IO ()
main = do
    flags <- getGCFlags
    unless (maxStkSize flags == 0) $ putStrLn "uh oh"
    =====> stack004(ghci) 1 of 1 [0, 0, 0]
cd "./rts/stack004.run" && "/home/alp/ghc/inplace/test   spaces/ghc-stage2" stack004.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -dno-debug-output  --interactive -v0 -ignore-dot-ghci -fno-ghci-history +RTS -I0.1 -RTS < stack004.genscript
Actual stdout output differs from expected:
diff -uw "/dev/null" "./rts/stack004.run/stack004.run.stdout.normalised"
--- /dev/null	2018-03-07 13:03:27.344000000 +0100
+++ ./rts/stack004.run/stack004.run.stdout.normalised	2018-03-08 11:53:55.339965000 +0100
@@ -0,0 +1 @@
+uh oh
*** unexpected failure for stack004(ghci)

    Looks like a genuine failure. Apparently the testsuite driver doesn't respect extra_run_opts for the ghci way. Will mark the test as broken for ghci. Ticket here.

  • #Standard handling of RTS arguments
test('T12870a',
    [extra_run_opts('+RTS -G2 -RTS arg1 --RTS +RTS arg2'), extra_files(['T12870.hs'])],
    multimod_compile_and_run,
    ['T12870', '-rtsopts -main-is T12870'])

test('T12870b',
    [extra_run_opts('+RTS -G2 -RTS arg1 --RTS +RTS arg2'), extra_files(['T12870.hs']),
        exit_code(1), ignore_stderr],
    multimod_compile_and_run,
    ['T12870', '-rtsopts=none -main-is T12870'])

test('T12870c',
    [extra_run_opts('+RTS -G2 -RTS arg1 --RTS +RTS arg2'), extra_files(['T12870.hs']),
    exit_code(1)],
    multimod_compile_and_run,
    ['T12870', '-rtsopts=some -main-is T12870'])

test('T12870d',
    [extra_run_opts('+RTS -G2 -RTS arg1 --RTS +RTS arg2'), extra_files(['T12870.hs'])],
    multimod_compile_and_run,
    ['T12870', '-main-is T12870'])

#RTS options should be passed along to the program
test('T12870e',
    [extra_run_opts('+RTS -G2 -RTS arg1 --RTS +RTS arg2'), extra_files(['T12870.hs'])],
    multimod_compile_and_run,
    ['T12870', '-rtsopts=ignore -main-is T12870'])
test('T12870f',
    [extra_run_opts('+RTS -G2 -RTS arg1 --RTS +RTS arg2'), extra_files(['T12870.hs'])],
    multimod_compile_and_run,
    ['T12870', '-rtsopts=ignoreAll -main-is T12870'])

#Check handling of env variables
test('T12870g',
    [extra_files(['T12870g.hs']), cmd_prefix('GHCRTS=-G7 '), extra_files(['T12870g.hs'])],
    multimod_compile_and_run,
    ['T12870g', '-rtsopts -main-is T12870g -with-rtsopts="-G3"'])

test('T12870h',
    [extra_files(['T12870g.hs']), cmd_prefix('GHCRTS=-G7 '), extra_files(['T12870g.hs'])],
    multimod_compile_and_run,
    ['T12870g', '-rtsopts=ignoreAll -main-is T12870g -with-rtsopts="-G3"'])

    /tmp/ghctest-n4fi8zlk/test   spaces/./rts/flags/T12870a.run                      T12870a [bad stderr] (ghci)
/tmp/ghctest-n4fi8zlk/test   spaces/./rts/flags/T12870b.run                      T12870b [bad exit code] (ghci)
/tmp/ghctest-n4fi8zlk/test   spaces/./rts/flags/T12870c.run                      T12870c [bad exit code] (ghci)
/tmp/ghctest-n4fi8zlk/test   spaces/./rts/flags/T12870d.run                      T12870d [bad stderr] (ghci)
/tmp/ghctest-n4fi8zlk/test   spaces/./rts/flags/T12870e.run                      T12870e [bad stderr] (ghci)
/tmp/ghctest-n4fi8zlk/test   spaces/./rts/flags/T12870f.run                      T12870f [bad stderr] (ghci)
/tmp/ghctest-n4fi8zlk/test   spaces/./rts/flags/T12870e.run                      T12870e [bad stdout] (threaded2)
/tmp/ghctest-n4fi8zlk/test   spaces/./rts/flags/T12870f.run                      T12870f [bad stdout] (threaded2)
/tmp/ghctest-n4fi8zlk/test   spaces/./rts/flags/T12870g.run                      T12870g [bad stderr] (ghci)
/tmp/ghctest-n4fi8zlk/test   spaces/./rts/flags/T12870h.run                      T12870h [bad stderr] (ghci)

    So on the one hand, this whole T12870* series of tests fail in ghci mode, and in addition to that the e and f variants fail with threaded2. As you can see above, all the variants but g and h are different calls to the following program:

    module T12870 where

import System.Environment

main :: IO ()
main = getArgs >>= putStr . show

    While g and `h are calls to:

    module T12870g where

import GHC.RTS.Flags (getGCFlags, generations)

main :: IO ()
main = do
    gcFlags <- getGCFlags
    putStr . show $ generations gcFlags

    So it seems like ghci breaks all of our expectations here, all the tests fail with this error message:

    Actual stderr output differs from expected:
diff -uw "/dev/null" "./rts/flags/T12870h.run/T12870h.run.stderr.normalised"
--- /dev/null	2018-03-07 13:03:27.344000000 +0100
+++ ./rts/flags/T12870h.run/T12870h.run.stderr.normalised	2018-03-08 13:34:45.003965000 +0100
@@ -0,0 +1,4 @@
+
+T12870h:6:30:
 +    Not in scope: ‘Main.main’
 +    No module named ‘Main’ is imported.
*** unexpected failure for T12870h(ghci)

    Which is another manifestation of ghci not managing to load a module but erroring out with a confusing message.

    The threaded2 failures are:

    =====> T12870e(threaded2) 1 of 2 [0, 0, 0]
cd "./rts/flags/T12870e.run" &&  "/home/alp/ghc/inplace/test   spaces/ghc-stage2" --make -o T12870e T12870 -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -dno-debug-output  -O -threaded -eventlog -rtsopts=ignore -main-is T12870
cd "./rts/flags/T12870e.run" && ./T12870e +RTS -N2 -ls -RTS +RTS -G2 -RTS arg1 --RTS +RTS arg2
Actual stdout output differs from expected:
diff -uw "./rts/flags/T12870e.run/T12870e.stdout.normalised" "./rts/flags/T12870e.run/T12870e.run.stdout.normalised"
--- ./rts/flags/T12870e.run/T12870e.stdout.normalised	2018-03-08 13:37:41.099965000 +0100
+++ ./rts/flags/T12870e.run/T12870e.run.stdout.normalised	2018-03-08 13:37:41.099965000 +0100
@@ -1 +1 @@
-["+RTS","-G2","-RTS","arg1","--RTS","+RTS","arg2"]
+["+RTS","-N2","-ls","-RTS","+RTS","-G2","-RTS","arg1","--RTS","+RTS","arg2"]
*** unexpected failure for T12870e(threaded2)
=====> T12870f(threaded2) 2 of 2 [0, 1, 0]
cd "./rts/flags/T12870f.run" &&  "/home/alp/ghc/inplace/test   spaces/ghc-stage2" --make -o T12870f T12870 -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -dno-debug-output  -O -threaded -eventlog -rtsopts=ignoreAll -main-is T12870
cd "./rts/flags/T12870f.run" && ./T12870f +RTS -N2 -ls -RTS +RTS -G2 -RTS arg1 --RTS +RTS arg2
Actual stdout output differs from expected:
diff -uw "./rts/flags/T12870f.run/T12870f.stdout.normalised" "./rts/flags/T12870f.run/T12870f.run.stdout.normalised"
--- ./rts/flags/T12870f.run/T12870f.stdout.normalised	2018-03-08 13:37:41.671965000 +0100
+++ ./rts/flags/T12870f.run/T12870f.run.stdout.normalised	2018-03-08 13:37:41.671965000 +0100
@@ -1 +1 @@
-["+RTS","-G2","-RTS","arg1","--RTS","+RTS","arg2"]
+["+RTS","-N2","-ls","-RTS","+RTS","-G2","-RTS","arg1","--RTS","+RTS","arg2"]
*** unexpected failure for T12870f(threaded2)

    I'm not sure what to think about those failures. "+RTS","-N2","-ls","+RTS" comes from the options that we always insert when running programs the threaded2 way, but these vary from one way to another, so we cannot expect all the ways to give us the same output. I'm also not sure what -rtsopts=ignoreAll is supposed to accomplish exactly here, I don't see anything being ignored, only some more options being given to the program.

    I reported a summary to the relevant ticket. This prompted Andreas to go ahead and fix those issues, the diff has already been accepted.

  • test('T13543', normal, compile, ['-ddump-str-signatures'])

    /tmp/ghctest-n4fi8zlk/test   spaces/./simplCore/should_compile/T13543.run        T13543 [stderr mismatch] (hpc)
/tmp/ghctest-n4fi8zlk/test   spaces/./simplCore/should_compile/T13543.run        T13543 [stderr mismatch] (optasm)
/tmp/ghctest-n4fi8zlk/test   spaces/./simplCore/should_compile/T13543.run        T13543 [stderr mismatch] (optllvm)

    Ben pointed out in the relevant ticket that we probably only want to run this with optasm, so I'll proceed with that and ask for Simon's approval before pushing that patch.

  • test('T13366', normal, compile_and_run, ['-lstdc++ -v0'])

    /tmp/ghctest-n4fi8zlk/test   spaces/./th/T13366.run                              T13366 [bad stderr] (ghci)

    This failure is about ghci not managing to resolve a C++ symbol from some code generated by TH with addForeignFile LangCxx. I reported this on the appropriate ticket. Marking the test as broken with the ghci way.

  • test('T13032', normal, compile, [''])

    /tmp/ghctest-n4fi8zlk/test   spaces/./typecheck/should_compile/T13032.run        T13032 [stderr mismatch] (hpc)

    =====> T13032(hpc) 1 of 1 [0, 0, 0]
cd "./typecheck/should_compile/T13032.run" &&  "/home/alp/ghc/inplace/test   spaces/ghc-stage2" -c T13032.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -dno-debug-output -fno-warn-incomplete-patterns -O -fhpc 
Actual stderr output differs from expected:
diff -uw "./typecheck/should_compile/T13032.run/T13032.stderr.normalised" "./typecheck/should_compile/T13032.run/T13032.comp.stderr.normalised"
--- ./typecheck/should_compile/T13032.run/T13032.stderr.normalised	2018-03-08 15:01:30.059965000 +0100
+++ ./typecheck/should_compile/T13032.run/T13032.comp.stderr.normalised	2018-03-08 15:01:30.059965000 +0100
@@ -7,7 +7,7 @@
f :: forall a b. ((a :: *) ~ (b :: *)) => a -> b -> Bool
[LclIdX]
f = / (@ a) (@ b) _ [Occ=Dead] _ [Occ=Dead] _ [Occ=Dead] ->
-      GHC.Types.True
+      hpc<T13032,1> hpc<T13032,0> GHC.Types.True

-- RHS size: {terms: 5, types: 0, coercions: 0, joins: 0/0}
T13032.$trModule :: GHC.Types.Module
*** unexpected failure for T13032(hpc)

    Let's just do omit_ways(['hpc']), as comparison of Core code is always going to be fragile with the hpc way, given that it seems to add annotations of sorts.

  • test('StrictPats', normal, compile_and_run, [''])

    /tmp/ghctest-n4fi8zlk/test   spaces/./typecheck/should_run/StrictPats.run        StrictPats [bad stderr] (ghci)

    This is another occurence of ghci not being able to load the module (because unboxed tuples). We can therefore just skip the ghci way for this one.

  • test('T12809', normal, compile_and_run, [''])

    /tmp/ghctest-n4fi8zlk/test   spaces/./typecheck/should_run/T12809.run            T12809 [bad stderr] (ghci)

    Same as above. Will skip the ghci way.

  • test('EtaExpandLevPoly', normal, compile_and_run, [''])

    /tmp/ghctest-n4fi8zlk/test   spaces/./typecheck/should_run/EtaExpandLevPoly.run  EtaExpandLevPoly [bad stderr] (ghci)

    Same as above. Skipping ghci.

  • # Test no runtime crash. Report success and kill with `timeout` (exit code 99)
# after a few seconds. From https://phabricator.haskell.org/D1075:
#
#   "I used a fairly conservative timeout. IF there is a regression it will
#   crash as soon as the timeout's C call is done. The tricky bit is
#   guesstimating how much time it needs to run to guarantee it's reached the
#   C call.
#   Probably something like 1s is already enough, but I don't know enough to
#   make an educated guess how long it needs to be guaranteed to reach the C
#   call."
test('T8089',
     [exit_code(99), run_timeout_multiplier(0.01)],
     compile_and_run, [''])

    /tmp/ghctest-n4fi8zlk/test   spaces/../../libraries/base/tests/T8089.run         T8089 [bad exit code] (ghci)
/tmp/ghctest-n4fi8zlk/test   spaces/../../libraries/base/tests/T8089.run         T8089 [bad exit code] (threaded1)
/tmp/ghctest-n4fi8zlk/test   spaces/../../libraries/base/tests/T8089.run         T8089 [bad exit code] (threaded2)

    All 3 fail because of the same problem: we expect exit code 99, get 0. I'm not sure about what's going on, I commented on the ticket.

    =====> T8089(ghci) 1 of 1 [0, 0, 0]
cd "../../libraries/base/tests/T8089.run" && "/home/alp/ghc/inplace/test   spaces/ghc-stage2" T8089.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts -fno-warn-missed-specialisations -fshow-warning-groups -fdiagnostics-color=never -fno-diagnostics-show-caret -dno-debug-output  --interactive -v0 -ignore-dot-ghci -fno-ghci-history +RTS -I0.1 -RTS < T8089.genscript
Wrong exit code for T8089(ghci) (expected 99 , actual 0 )
*** unexpected failure for T8089(ghci)

Unexpected stat failures

Not done yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment