Skip to content

Instantly share code, notes, and snippets.

@AlainODea
Last active January 11, 2016 03:58
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save AlainODea/98141991849093285c52 to your computer and use it in GitHub Desktop.
Save AlainODea/98141991849093285c52 to your computer and use it in GitHub Desktop.
GHC 7.10.3 Test Failures on SmartOS
/* DO NOT EDIT: This file is automatically generated by Cabal */
/* package rts-1.0 */
#define VERSION_rts "1.0"
#define MIN_VERSION_rts(major1,major2,minor) (\
(major1) < 1 || \
(major1) == 1 && (major2) < 0 || \
(major1) == 1 && (major2) == 0 && (minor) <= 0)
/* package ghc-prim-0.4.0.0 */
#define VERSION_ghc_prim "0.4.0.0"
#define MIN_VERSION_ghc_prim(major1,major2,minor) (\
(major1) < 0 || \
(major1) == 0 && (major2) < 4 || \
(major1) == 0 && (major2) == 4 && (minor) <= 0)
/* package integer-gmp-1.0.0.0 */
#define VERSION_integer_gmp "1.0.0.0"
#define MIN_VERSION_integer_gmp(major1,major2,minor) (\
(major1) < 1 || \
(major1) == 1 && (major2) < 0 || \
(major1) == 1 && (major2) == 0 && (minor) <= 0)
/* tool cpphs-1.19.3 */
#define TOOL_VERSION_cpphs "1.19.3"
#define MIN_TOOL_VERSION_cpphs(major1,major2,minor) (\
(major1) < 1 || \
(major1) == 1 && (major2) < 19 || \
(major1) == 1 && (major2) == 19 && (minor) <= 3)
/* tool gcc-4.8 */
#define TOOL_VERSION_gcc "4.8"
#define MIN_TOOL_VERSION_gcc(major1,major2,minor) (\
(major1) < 4 || \
(major1) == 4 && (major2) < 8 || \
(major1) == 4 && (major2) == 8 && (minor) <= 0)
/* tool ghc-7.10.3 */
#define TOOL_VERSION_ghc "7.10.3"
#define MIN_TOOL_VERSION_ghc(major1,major2,minor) (\
(major1) < 7 || \
(major1) == 7 && (major2) < 10 || \
(major1) == 7 && (major2) == 10 && (minor) <= 3)
/* tool ghc-pkg-7.10.3 */
#define TOOL_VERSION_ghc_pkg "7.10.3"
#define MIN_TOOL_VERSION_ghc_pkg(major1,major2,minor) (\
(major1) < 7 || \
(major1) == 7 && (major2) < 10 || \
(major1) == 7 && (major2) == 10 && (minor) <= 3)
/* tool haddock-2.14.3 */
#define TOOL_VERSION_haddock "2.14.3"
#define MIN_TOOL_VERSION_haddock(major1,major2,minor) (\
(major1) < 2 || \
(major1) == 2 && (major2) < 14 || \
(major1) == 2 && (major2) == 14 && (minor) <= 3)
/* tool hpc-0.67 */
#define TOOL_VERSION_hpc "0.67"
#define MIN_TOOL_VERSION_hpc(major1,major2,minor) (\
(major1) < 0 || \
(major1) == 0 && (major2) < 67 || \
(major1) == 0 && (major2) == 67 && (minor) <= 0)
/* tool hsc2hs-0.67 */
#define TOOL_VERSION_hsc2hs "0.67"
#define MIN_TOOL_VERSION_hsc2hs(major1,major2,minor) (\
(major1) < 0 || \
(major1) == 0 && (major2) < 67 || \
(major1) == 0 && (major2) == 67 && (minor) <= 0)
/* tool pkg-config-0.26 */
#define TOOL_VERSION_pkg_config "0.26"
#define MIN_TOOL_VERSION_pkg_config(major1,major2,minor) (\
(major1) < 0 || \
(major1) == 0 && (major2) < 26 || \
(major1) == 0 && (major2) == 26 && (minor) <= 0)
/* tool strip-2.24 */
#define TOOL_VERSION_strip "2.24"
#define MIN_TOOL_VERSION_strip(major1,major2,minor) (\
(major1) < 2 || \
(major1) == 2 && (major2) < 24 || \
(major1) == 2 && (major2) == 24 && (minor) <= 0)
#define CURRENT_PACKAGE_KEY "base_HQfYBxpPvuw8OunzQu6JGM"
Unexpected results from:
TEST="print020 process010 posix005 T2464 T9878 T9878b T9032 T7145b recomp011 T7354 processGroup002 T2276_ghci T5321FD T5030 T4801 T6048 T5631 T5837 T5642 T9020 T3064 parsing001 T1969 T9675 T5321Fun T783 T3294 T9961"
OVERALL SUMMARY for test run started at Wed Dec 30 19:16:15 2015 UTC
0:12:19 spent to go through
4491 total tests, which gave rise to
14487 test cases, of which
10351 were skipped
55 had missing libraries
3989 expected passes
64 expected failures
6 caused framework failures
0 unexpected passes
12 unexpected failures
16 unexpected stat failures
Unexpected failures:
../../libraries/process/tests process010 [bad stdout] (normal)
../../libraries/unix/tests processGroup002 [bad exit code] (normal)
../../libraries/unix/tests/libposix posix005 [bad stdout] (normal)
driver T2464 [stderr mismatch] (normal)
driver/recomp011 recomp011 [bad stdout] (normal)
ffi/should_run T2276_ghci [bad exit code] (ghci)
ghci.debugger/scripts print020 [bad stdout] (ghci)
ghci/scripts T9878 [bad stderr] (ghci)
ghci/scripts T9878b [bad stderr] (ghci)
indexed-types/should_fail T7354 [stderr mismatch] (normal)
rename/should_compile T7145b [stderr mismatch] (normal)
rename/should_fail T9032 [bad stderr] (normal)
Unexpected stat failures:
perf/compiler T1969 [stat not good enough] (normal)
perf/compiler T3064 [stat not good enough] (normal)
perf/compiler T3294 [stat not good enough] (normal)
perf/compiler T4801 [stat not good enough] (normal)
perf/compiler T5030 [stat not good enough] (normal)
perf/compiler T5321FD [stat not good enough] (normal)
perf/compiler T5321Fun [stat not good enough] (normal)
perf/compiler T5631 [stat not good enough] (normal)
perf/compiler T5642 [stat not good enough] (normal)
perf/compiler T5837 [stat not good enough] (normal)
perf/compiler T6048 [stat not good enough] (optasm)
perf/compiler T783 [stat not good enough] (normal)
perf/compiler T9020 [stat not good enough] (optasm)
perf/compiler T9675 [stat not good enough] (optasm)
perf/compiler T9961 [stat not good enough] (normal)
perf/compiler parsing001 [stat not good enough] (normal)
@AlainODea
Copy link
Author

   driver                               T2464 [stderr mismatch] (normal)

Specific output for that test is:

=====> T2464(normal) 3803 of 4491 [0, 0, 6] 
cd ./driver &&  "/home/ghcbld/ghc-7.10.3/inplace/bin/ghc-stage2" -c T2464.hs -fforce-recomp -dcore-lint -dcmm-lint -dno-debug-output -no-user-package-db -rtsopts -fno-warn-tabs -fno-ghci-history   > T2464.comp.stderr 2>&1
Actual stderr output differs from expected:
--- ./driver/T2464.stderr       2015-12-30 02:19:25.749348616 +0000
+++ ./driver/T2464.comp.stderr  2015-12-31 04:29:28.239753744 +0000
@@ -1,3 +1,3 @@

-T2464.hs:3:16:
-    Warning: -fffi is deprecated: use -XForeignFunctionInterface or pragma {-# LANGUAGE ForeignFunctionInterface #-} instead
+/tmp/ghc72341_0/ghc_1.hscpp:7:16: Warning:
+    -fffi is deprecated: use -XForeignFunctionInterface or pragma {-# LANGUAGE ForeignFunctionInterface #-} instead
*** unexpected failure for T2464(normal)

The kernel of the problem here is that the deprecation warning is showing autogenerated filename and position information (/tmp/ghc72341_0/ghc_1.hscpp:7:16) rather than the source filename and position information (T2464.hs:3:16).

This suggests that the bug is in the construction of the Errors and Warnings Monad:
https://github.com/ghc/ghc/blob/c2fab84239299176a8e72aa26ae894019d677bd9/compiler/main/CmdLineParser.hs#L121

The behaviour of EwM is correct on SmartOS for a naked expression at top level parse error like the following:

broken.hs:

nonsense

Here's an example:

$ ghc broken.hs
[1 of 1] Compiling Main             ( broken.hs, broken.o )

broken.hs:1:1:
    Parse error: naked expression at top level
    Perhaps you intended to use TemplateHaskell

Type signatures without accompanying bindings:

nonsense :: a -> a

Have correct failure output:

$ ghc broken.hs 
[1 of 1] Compiling Main             ( broken.hs, broken.o )

broken.hs:1:1:
    The type signature for ‘nonsense’ lacks an accompanying binding

Unterminated C pre-processor ifdefs:

{-# LANGUAGE CPP #-}

#ifdef
#endif

Also have correct failure output:

$ ghc broken.hs 

broken.hs:3:0:  error: no macro name given in #ifdef directive

Files using deprecated functionality like T5281.hs yield correct warning output:

$ ghc T5281.hs
[1 of 2] Compiling T5281A           ( T5281A.hs, T5281A.o )
[2 of 2] Compiling T5281            ( T5281.hs, T5281.o )

T5281.hs:6:5: Warning:
    In the use of ‘deprec’ (imported from T5281A):
    Deprecated: "This is deprecated"

So the EwH monad handling isn't completely broken, it's broken for deprecated language pragma definitions, but not for other deprecations, parse errors, or C pre-processor issues.

@AlainODea
Copy link
Author

setArg is responsible for establishing the location of an EwH. As a result it seems likely that the problem is in the data flow to this function.

This function is called from processArgs.

And I'm realizing now this is all about GHC argument deprecation and not deprecation within Haskell language constructs at all so this is the wrong avenue to pursue. The issue is elsewhere.

@AlainODea
Copy link
Author

The deprecation relevant to the test failure is defined as follows:

  flagSpec' "ffi"                             LangExt.ForeignFunctionInterface
    (deprecatedForExtension "ForeignFunctionInterface"),

SOURCE: https://github.com/ghc/ghc/blob/bec5350da09fde51231004b1b7d33641ac51800e/compiler/main/DynFlags.hs#L3029-L3030

That uses deprecatedForExtension which is defined as follows:

deprecatedForExtension :: String -> TurnOnFlag -> DynP ()
deprecatedForExtension lang turn_on
    = deprecate ("use -X"  ++ flag ++
                 " or pragma {-# LANGUAGE " ++ flag ++ " #-} instead")
    where
      flag | turn_on    = lang
           | otherwise = "No"++lang

SOURCE: https://github.com/ghc/ghc/blob/bec5350da09fde51231004b1b7d33641ac51800e/compiler/main/DynFlags.hs#L2815-L2821

That uses deprecate which is in CmdLineParser.hs as follows:

deprecate :: Monad m => String -> EwM m ()
deprecate s = do
    arg <- getArg
    addWarn (arg ++ " is deprecated: " ++ s)

SOURCE: https://github.com/ghc/ghc/blob/bb7f2e33197e667eb694bd1243f125c722a0a868/compiler/main/CmdLineParser.hs#L123-L126

So I may have been on track originally anyway. This error has to do with fLangFlags* and is an interesting one because it arises from GHC options embedded directly in the source:

{-# OPTIONS_GHC -fffi #-} -- deprecation warning

It should be able to identify the file location, but for some reason identifies a temp file instead of the source.

@AlainODea
Copy link
Author

Interesting discovery in the compiler comments:

          -- at various points, for example when we read the OPTIONS_GHC
          -- pragmas in the Cpp phase.

SOURCE: https://github.com/ghc/ghc/blob/6746549772c5cc0ac66c0fce562f297f4d4b80a2/compiler/main/PipelineMonad.hs#L64-L65

So OPTIONS_GHC gets handled by the Cpp phase of the compiler.

Remembering the failure mode is a temp filename ending with .hscpp

/tmp/ghc72341_0/ghc_1.hscpp

And that is marked as intermediate only here:
https://github.com/ghc/ghc/blob/8e735fd0f88454b74fbb866a59b608925a2b3e48/compiler/main/DriverPhases.hs#L272

So we may have a problem with the handling of the intermediate nature of compiler phase inputs.

@AlainODea
Copy link
Author

The Cpp phase is in DrivePipeline:
https://github.com/ghc/ghc/blob/8e735fd0f88454b74fbb866a59b608925a2b3e48/compiler/main/DriverPipeline.hs#L839-L875

As is the HsPp phase that also handles flags like this:

Most notably the calls to handleFlagWarnings:
https://github.com/ghc/ghc/blob/8e735fd0f88454b74fbb866a59b608925a2b3e48/compiler/main/DriverPipeline.hs#L854
https://github.com/ghc/ghc/blob/8e735fd0f88454b74fbb866a59b608925a2b3e48/compiler/main/DriverPipeline.hs#L870
https://github.com/ghc/ghc/blob/8e735fd0f88454b74fbb866a59b608925a2b3e48/compiler/main/DriverPipeline.hs#L903

handleFlagWarnings is defined in compiler/main/HscTypes.hs and has this type signature:

handleFlagWarnings :: DynFlags -> [Located String] -> IO ()

SOURCE: https://github.com/ghc/ghc/blob/c8c44fd91b509b9eb644c826497ed5268e89363a/compiler/main/HscTypes.hs#L326-L334

Given that the output is /tmp/ghc72341_0/ghc_1.hscpp instead of the expected H2464.hs that suggests that the warns being handed to those handleFlagWarnings invocations aren't being set correctly.

This code calling parseDynamicFilePragma is responsible for setting the value of warns:

            (dflags1, unhandled_flags, warns)
                <- liftIO $ parseDynamicFilePragma dflags src_opts

SOURCES:

  1. DriverPipeline.hs:846
  2. DriverPipeline.hs:866
  3. DriverPipeline.hs:899-900

@AlainODea
Copy link
Author

parseDynamicFilePragma is defined in compiler/main/DynFlags.hs:
https://github.com/ghc/ghc/blob/bec5350da09fde51231004b1b7d33641ac51800e/compiler/main/DynFlags.hs#L2018-L2025

The args ([Located String]) are then handed to processArgs in CmdLineParser.hs. This is going in circles at this point and the [Located String] is not manipulated by the process so I need to find the source of the data rather than following its propagation.

In both the Cpp and HsPp phases args originates from src_opts by invoking getOptionsFromFile as follows:

src_opts <- liftIO $ getOptionsFromFile dflags output_fn

SOURCES:

  1. https://github.com/ghc/ghc/blob/8e735fd0f88454b74fbb866a59b608925a2b3e48/compiler/main/DriverPipeline.hs#L865
  2. https://github.com/ghc/ghc/blob/8e735fd0f88454b74fbb866a59b608925a2b3e48/compiler/main/DriverPipeline.hs#L898

getOptionsFromFile is defined in compiler/main/HeaderInfo.hs:
https://github.com/ghc/ghc/blob/c1e25536d67fba33ad6ddae5556115340d99000a/compiler/main/HeaderInfo.hs#L137-L159

Notably that is called with output_fn and HeaderInfo uses mkRealSrcLoc:
https://github.com/ghc/ghc/blob/c1e25536d67fba33ad6ddae5556115340d99000a/compiler/main/HeaderInfo.hs#L170

My reading of mkRealSrcLoc is as a tag notifying the consumer that they have an intermediate filename and need to convert it back to a real source name in order to display it correctly.

@AlainODea
Copy link
Author

This is the most suspicious line:

src_opts <- liftIO $ getOptionsFromFile dflags0 output_fn

SOURCE: https://github.com/ghc/ghc/blob/8e735fd0f88454b74fbb866a59b608925a2b3e48/compiler/main/DriverPipeline.hs#L865

Given the behavior of getOptionsFromFile that would guarantee (in my mind) that the warning will show the intermediate filename instead of the true source filename. If that is correct, why does it work on Ubuntu and fail on SmartOS? One possibility is that this branch doesn't get executed for this test on Ubuntu. but that seems illogical based on the condition for the alternate branch:

not (xopt LangExt.Cpp dflags1)

Very oddly if I comment out this the CPP language pragma:

--{-# LANGUAGE CPP #-}
{-# OPTIONS_GHC -DTEST #-}
{-# OPTIONS_GHC -fffi #-} -- deprecation warning
#ifdef TEST
{-# LANGUAGE EmptyDataDecls #-}
#endif

module Test2464 where
data T

I get the correct filename in the warning, but it (obviously) then can't handle the other C pre-processor code in the file:

$ ghc T2464.hs

T2464.hs:3:16: Warning:
    -fffi is deprecated: use -XForeignFunctionInterface or pragma {-# LANGUAGE ForeignFunctionInterface #-} instead

T2464.hs:4:2: lexical error at character 'i'

This isn't new in 7.10.3 either. It's reproducible on 7.8.4 on SmartOS as well.

This suggests that the problem may lie in xopt or the parsing of DynFlags. That doesn't make sense either though, because the branch it follows on SmartOS is perfectly logical and expected. Somehow GHC on Ubuntu corrects for this filename and yields the original source filename.

@AlainODea
Copy link
Author

GCC Specs shows some interesting differences:

gcc -dumpspecs

On Ubuntu the Spec for cpp is:

*cpp:
%{posix:-D_POSIX_SOURCE} %{pthread:-D_REENTRANT}

On SmartOS the Spec for cpp is:

*cpp:
%{,assembler-with-cpp:-P} %(cpp_subtarget)

GCC's Spec Files documentation talks about overriding such specs with specis files using the -specs= command-line switch.

I think this means that you could specify "gcc -specs=/path/to/overridecpp.spec" as --with-hs-cpp when building GHC.

Here's an example:

Create overridecpp.spec:

*cpp:
%{cpp_subtarget}

Create T7145b.hs:

{-# LANGUAGE CPP #-}
module T7145b ( A.Applicative(pure) ) where

import qualified Control.Applicative as A

pure :: ()
pure = ()

Run GCC without the spec to reproduce undesired behavior:

tempdir=$(mktemp --tmpdir --directory ghcXXXXX_0)
tempfile="$tempdir/ghc_1.hscpp"
touch $tempfile
gcc -E -undef -traditional  '-D__GLASGOW_HASKELL__=709' '-Dsolaris2_BUILD_OS=1' '-Di386_BUILD_ARCH=1' '-Dsolaris2_HOST_OS=1' '-Di386_HOST_ARCH=1' -x assembler-with-cpp T7145b.hs -o $tempfile

And you get the input as output in /tmp/ghc_XXXXX_0/ghc_1.hscpp :

{-# LANGUAGE CPP #-}
module T7145b ( A.Applicative(pure) ) where

import qualified Control.Applicative as A

pure :: ()
pure = ()

As a result the [Located String] cannot be reversed to source locations and we get the ugly output referencing names and lines of temp files.

It gets interesting if we provide the specs file (overridecpp.spec) to the invocation of gcc's C preprocessor:

gcc -E -undef -traditional  '-D__GLASGOW_HASKELL__=709' '-Dsolaris2_BUILD_OS=1' '-Di386_BUILD_ARCH=1' '-Dsolaris2_HOST_OS=1' '-Di386_HOST_ARCH=1' -x assembler-with-cpp T7145b.hs -o $tempfile -specs=overridecpp.spec

Now we get the desired output in /tmp/ghc_XXXXX_0/ghc_1.hscpp:

#1 "T7145b.hs"
#1 "<command-line>"
#1 "T7145b.hs"
{-# LANGUAGE CPP #-}
module T7145b ( A.Applicative(pure) ) where

import qualified Control.Applicative as A

pure :: ()
pure = ()

This isn't exactly what Ubuntu produces, but it may be sufficient for GHC.

@AlainODea
Copy link
Author

No dice on ./configure --with-hs-cpp="gcc -specs=/home/ghcbld/overridecpp.spec". It prevents GHC from building.

ghc-stage1: could not execute: /opt/local/bin/gcc -specs /home/ghcbld/overridecpp.spec

Next step. Try a wrapper script /opt/local/bin/ghc-cpp:

#!/bin/bash
gcc -specs=/home/ghcbld/overridecpp.spec $@

@AlainODea
Copy link
Author

The wrapper script with GCC specs override works. T2464 passes with it in place.

@AlainODea
Copy link
Author

Unexpected results from:
TEST="processGroup002 process010 posix005 T2276_ghci outofmem T5321FD T5030 T4801 T6048 T5631 T5837 T5642 T9020 T3064 parsing001 T1969 T9675 T5321Fun T783 T3294 T9961"

OVERALL SUMMARY for test run started at Fri Jan  1 21:03:11 2016 UTC
 0:12:17 spent to go through
    4491 total tests, which gave rise to
   14487 test cases, of which
   10351 were skipped

      55 had missing libraries
    3996 expected passes
      64 expected failures

       6 caused framework failures
       0 unexpected passes
       5 unexpected failures
      16 unexpected stat failures

Unexpected failures:
   ../../libraries/process/tests        process010 [bad stdout] (normal)
   ../../libraries/unix/tests           processGroup002 [bad exit code] (normal)
   ../../libraries/unix/tests/libposix  posix005 [bad stdout] (normal)
   ffi/should_run                       T2276_ghci [bad exit code] (ghci)
   rts                                  outofmem [bad stderr] (normal)

Unexpected stat failures:
   perf/compiler  T1969 [stat not good enough] (normal)
   perf/compiler  T3064 [stat not good enough] (normal)
   perf/compiler  T3294 [stat not good enough] (normal)
   perf/compiler  T4801 [stat not good enough] (normal)
   perf/compiler  T5030 [stat not good enough] (normal)
   perf/compiler  T5321FD [stat not good enough] (normal)
   perf/compiler  T5321Fun [stat not good enough] (normal)
   perf/compiler  T5631 [stat not good enough] (normal)
   perf/compiler  T5642 [stat not good enough] (normal)
   perf/compiler  T5837 [stat not good enough] (normal)
   perf/compiler  T6048 [stat not good enough] (optasm)
   perf/compiler  T783 [stat not good enough] (normal)
   perf/compiler  T9020 [stat not good enough] (optasm)
   perf/compiler  T9675 [stat not good enough] (optasm)
   perf/compiler  T9961 [stat not good enough] (normal)
   perf/compiler  parsing001 [stat not good enough] (normal)

Woohoo!

We're down to 5 failures. Fixing C preprocessing was a very fruitful investment!

@AlainODea
Copy link
Author

Next test I'll tackle is process010.

It fails as follows:

=====> process010(normal) 4434 of 4491 [0, 0, 6] 
cd ../../libraries/process/tests &&  "/home/ghcbld/ghc-7.10.3/inplace/bin/ghc-stage2" -o process010 proce
ss010.hs -fforce-recomp -dcore-lint -dcmm-lint -dno-debug-output -no-user-package-db -rtsopts -fno-warn-t
abs -fno-ghci-history   > process010.comp.stderr 2>&1
cd ../../libraries/process/tests && ./process010    </dev/null > process010.run.stdout 2> process010.run.
stderr
Actual stdout output differs from expected:
--- ../../libraries/process/tests/process010.stdout-i386-unknown-solaris2       2015-12-30 02:19:57.65550
8483 +0000
+++ ../../libraries/process/tests/process010.run.stdout 2016-01-02 17:49:15.438338983 +0000
@@ -1,4 +1,4 @@
 ExitSuccess
-ExitFailure 255
+ExitFailure 1
 Exc: /non/existent: rawSystem: runInteractiveProcess: exec: does not exist (No such file or directory)
 Done
*** unexpected failure for process010(normal)

The exit code is 255 instead of the expected 1.

On further investigation the exit code is expected to be 1 on all platforms except those that appear to be solaris2. Why Illumos/SmartOS differs from Solaris here is unknown. Karel's builder also passes this test as is so I don't want to introduce a change that will work for Illumos, but break Solaris.

@AlainODea
Copy link
Author

T2276_ghci also fails:

=====> T2276_ghci(ghci) 4133 of 4491 [0, 0, 6] 
cd ./ffi/should_run && $MAKE -s --no-print-directory T2276_ghci_setup
cd ./ffi/should_run && "/home/ghcbld/ghc-7.10.3/inplace/bin/ghc-stage2" T2276_ghci.hs -fforce-recomp -dcore-lint -dcmm-lint -dno-debug-output -no-user-package-db -rtsopts -fno-warn-tabs -fno-ghci-history  --interactive -v0 -ignore-dot-ghci +RTS -I0.1 -RTS -fobject-code T2276_ghci_c.o < T2276_ghci.genscript  > T2276_ghci.interp.stdout 2> T2276_ghci.interp.stderr
Wrong exit code (expected 0 , actual 1 )
Stdout:

Stderr:
Too late for parseStaticFlags: call it before runGhc or runGhcT

*** unexpected failure for T2276_ghci(ghci)

Based on the stderr output it looks like a state machine failure.

@AlainODea
Copy link
Author

posix005 is another failure:

=====> posix005(normal) 4461 of 4491 [0, 0, 6] 
cd ../../libraries/unix/tests/libposix &&  "/home/ghcbld/ghc-7.10.3/inplace/bin/ghc-stage2" -o posix005 p
osix005.hs -fforce-recomp -dcore-lint -dcmm-lint -dno-debug-output -no-user-package-db -rtsopts -fno-warn
-tabs -fno-ghci-history   > posix005.comp.stderr 2>&1
cd ../../libraries/unix/tests/libposix && ./posix005    </dev/null > posix005.run.stdout 2> posix005.run.
stderr
Actual stdout output differs from expected:
--- ../../libraries/unix/tests/libposix/posix005.stdout 2015-12-30 02:20:04.255718893 +0000
+++ ../../libraries/unix/tests/libposix/posix005.run.stdout     2016-01-02 18:40:19.094454408 +0000
@@ -1,7 +1,7 @@
 vt100
-[("one","1"),("two","2")]
-[("one","1"),("two","2"),("foo","bar")]
-[("one","1"),("two","2"),("foo","baz")]
-[("one","1"),("two","2"),("foo","baz"),("fu","bar")]
-[("one","1"),("two","2"),("fu","bar")]
+[("two","2"),("one","1")]
+[("foo","bar"),("two","2"),("one","1")]
+[("foo","baz"),("two","2"),("one","1")]
+[("fu","bar"),("foo","baz"),("two","2"),("one","1")]
+[("fu","bar"),("two","2"),("one","1")]
 []
*** unexpected failure for posix005(normal)

This test is asserting the order of environment variables is the order in which they are declared. I'll need to check the standard to see if that is a legit expectation or just a lucky coincidence.

The failure output is identical on Karel's Solaris 11 builder.

@AlainODea
Copy link
Author

processGroup002 is another failure:

=====> processGroup002(normal) 4484 of 4491 [0, 0, 6] 
cd ../../libraries/unix/tests &&  "/home/ghcbld/ghc-7.10.3/inplace/bin/ghc-stage2" -o processGroup002 processGroup002.hs -fforce-recomp -dcore-lint -dcmm-lint -dno-debug-output -no-user-package-db -rtsopts -fno-warn-tabs -fno-ghci-history  -package unix > processGroup002.comp.stderr 2>&1
cd ../../libraries/unix/tests && ./processGroup002    </dev/null > processGroup002.run.stdout 2> processGroup002.run.stderr
Wrong exit code (expected 0 , actual 1 )
Stdout:
Testing joinProcessGroup: 
Stderr:
processGroup002: joinProcessGroup: permission denied (Not owner)

This fails identically on Karel's Solaris 11 builder so it seems to be a common issue with process permissions on Illumos and Solaris.

@AlainODea
Copy link
Author

cpphs compiles with GHC 7.10.3 installed on SmartOS.

T7145b.hs:

{-# LANGUAGE CPP #-}
module T7145b ( A.Applicative(pure) ) where

import qualified Control.Applicative as A

pure :: ()
pure = ()

Run cpphs on it with the --cpp switch so it knows to emulate/ignore gcc CLI args:

tempdir=$(mktemp --tmpdir --directory ghcXXXXX_0)
tempfile="$tempdir/ghc_1.hscpp"
touch $tempfile
cpphs --cpp -E -undef -traditional  '-D__GLASGOW_HASKELL__=709' '-Dsolaris2_BUILD_OS=1' '-Di386_BUILD_ARCH=1' '-Dsolaris2_HOST_OS=1' '-Di386_HOST_ARCH=1' -x assembler-with-cpp T7145b.hs -o $tempfile

$tempfile now contains:

#line 1 "T7145b.hs"
{-# LANGUAGE CPP #-}
module T7145b ( A.Applicative(pure) ) where

import qualified Control.Applicative as A

pure :: ()
pure = ()

In a nicely circular manner I can now use cpphs as the hs-cpp when building GHC.

cp ~/.cabal/bin/cpphs /opt/local/bin/cpphs
cd ~/ghc-7.10.3/
./configure --with-hs-cpp=/opt/local/bin/cpphs --with-hs-cpp-flags="--cpp"

In a similar manner GCC could be hacked into mostly working as follows (assuming overridecpp.spec is present as I've previously described):

./configure --with-hs-cpp=/opt/local/bin/gcc --with-hs-cpp-flags="-specs=overridecpp.spec"

Triggering a make now with the cpphs configuration. I'll run another one tomorrow with the GCC workaround.

@AlainODea
Copy link
Author

Build with cpphs fails in final phase with this error:

cpphs: macro MIN_VERSION_integer_gmp expected 3 arguments, but was given 0
libraries/base/ghc.mk:4: recipe for target 'libraries/base/dist-install/build/.depend-v-p-dyn.haskell' failed
make[1]: *** [libraries/base/dist-install/build/.depend-v-p-dyn.haskell] Error 1
Makefile:71: recipe for target 'all' failed
make: *** [all] Error 2

MIN_VERSION_integer_gmp is used in at least three files in the GHC source:

I cannot reproduce the cpphs error with Bits.hs:

cpphs --cpp -E -undef -traditional  '-D__GLASGOW_HASKELL__=709' '-Dsolaris2_BUILD_OS=1' '-Di386_BUILD_ARCH=1' '-Dsolaris2_HOST_OS=1' '-Di386_HOST_ARCH=1' -x assembler-with-cpp libraries/base/Data/Bits.hs -o $tempfile -Iincludes

Using pargs.d to debug I get this:

  7   9131         exec_common:exec-success     6964669646:     /opt/local/bin/cpphs --cpp -DOPTIMISE_INTEGER_GCD_LCM -include libraries/base/d
argv[0]: /opt/local/bin/cpphs
argv[1]: --cpp
argv[2]: -DOPTIMISE_INTEGER_GCD_LCM
argv[3]: -include
argv[4]: libraries/base/dist-install/build/autogen/cabal_macros.h
argv[5]: -I
argv[6]: libraries/base/dist-install/build
argv[7]: -I
argv[8]: libraries/base/dist-install/build
argv[9]: -I
argv[10]: libraries/base/dist-install/build/autogen
argv[11]: -I
argv[12]: libraries/base/include
argv[13]: -I
argv[14]: /root/ghc-7.10.3/libraries/integer-gmp2/include
argv[15]: -I
argv[16]: /root/ghc-7.10.3/rts/dist/build
argv[17]: -I
argv[18]: /root/ghc-7.10.3/includes
argv[19]: -I
argv[20]: /root/ghc-7.10.3/includes/dist-derivedconstants/header
argv[21]: -D__GLASGOW_HASKELL__=710
argv[22]: -include
argv[23]: /root/ghc-7.10.3/includes/ghcversion.h
argv[24]: -Dsolaris2_BUILD_OS=1
argv[25]: -Di386_BUILD_ARCH=1
argv[26]: -Dsolaris2_HOST_OS=1
argv[27]: -Di386_HOST_ARCH=1
argv[28]: -D__GLASGOW_HASKELL_TH__=NO
argv[29]: -x
argv[30]: assembler-with-cpp
argv[31]: libraries/base/GHC/Natural.hs
argv[32]: -o
argv[33]: /tmp/ghc69539_0/ghc_37.hscpp

cpphs: macro MIN_VERSION_integer_gmp expected 3 arguments, but was given 0
libraries/base/ghc.mk:4: recipe for target 'libraries/base/dist-install/build/.depend-v-p-dyn.haskell' failed
make[1]: *** [libraries/base/dist-install/build/.depend-v-p-dyn.haskell] Error 1
Makefile:71: recipe for target 'all' failed
make: *** [all] Error 2
dtrace: pid 69206 has exited

@AlainODea
Copy link
Author

GHC 7.10.3's build fails when cpphs is the configured hs-cpp. It runs into an error when preprocessing libraries/base/GHC/Natural.hs.

Prep on either SmartOS or Ubuntu:

tar xf ghc-7.10.3-src.tar.bz2
cd ghc-7.10.3/
./configure --with-hs-cpp=$(which cpphs) --with-hs-cpp-flags="--cpp"
make -j8

On SmartOS:

tempdir=$(mktemp --tmpdir --directory ghcXXXXX_0)
tempfile="$tempdir/ghc_1.hscpp"
touch $tempfile
/opt/local/bin/cpphs --cpp -DOPTIMISE_INTEGER_GCD_LCM -include libraries/base/dist-install/build/autogen/cabal_macros.h -I libraries/base/dist-install/build -I libraries/base/dist-install/build -I libraries/base/dist-install/build/autogen -I libraries/base/include -I /root/ghc-7.10.3/libraries/integer-gmp2/include -I /root/ghc-7.10.3/rts/dist/build -I /root/ghc-7.10.3/includes -I /root/ghc-7.10.3/includes/dist-derivedconstants/header -D__GLASGOW_HASKELL__=710 -include /root/ghc-7.10.3/includes/ghcversion.h -Dsolaris2_BUILD_OS=1 -Di386_BUILD_ARCH=1 -Dsolaris2_HOST_OS=1 -Di386_HOST_ARCH=1 -D__GLASGOW_HASKELL_TH__=NO -x assembler-with-cpp libraries/base/GHC/Natural.hs -o $tempfile

On Ubuntu:

tempdir=$(mktemp --tmpdir --directory ghcXXXXX_0)
tempfile="$tempdir/ghc_1.hscpp"
touch $tempfile
~/.cabal/bin/cpphs --cpp -DOPTIMISE_INTEGER_GCD_LCM -include libraries/base/dist-install/build/autogen/cabal_macros.h -I libraries/base/dist-install/build -I libraries/base/dist-install/build -I libraries/base/dist-install/build/autogen -I libraries/base/include -I /home/alain_odea/Documents/haskell/ghc-7.10.3/libraries/integer-gmp2/include -I /home/alain_odea/Documents/haskell/ghc-7.10.3/rts/dist/build -I /home/alain_odea/Documents/haskell/ghc-7.10.3/includes -I /home/alain_odea/Documents/haskell/ghc-7.10.3/includes/dist-derivedconstants/header -D__GLASGOW_HASKELL__=710 -include /home/alain_odea/Documents/haskell/ghc-7.10.3/includes/ghcversion.h -Dlinux_BUILD_OS=1 -Di386_BUILD_ARCH=1 -Dlinux_HOST_OS=1 -Di386_HOST_ARCH=1 -D__GLASGOW_HASKELL_TH__=NO -x assembler-with-cpp libraries/base/GHC/Natural.hs -o $tempfile

Both result in this error:

cpphs: macro MIN_VERSION_integer_gmp expected 3 arguments, but was given 0

In either case removing "-include libraries/base/dist-install/build/autogen/cabal_macros.h" from the cpphs arguments makes the error go away.

@AlainODea
Copy link
Author

cabal-macros.h is now in this gist. Using cpphs on it alone has no error messages.

cpphs --cpp libraries/base/dist-install/build/autogen/cabal_macros.h

@AlainODea
Copy link
Author

outofmem passes when run individually.

First pass at using specs failed:

--with-hs-cpp-flag="-specs=/opt/local/etc/ghc/overridecpp.spec"

It doesn't configure the C preprocessor properly and cannot compile GHC 7.10.3 successfully.

Using --with-hs-cpp-flags appears to remove the other arguments GHC needs for GCC's preprocessor so I'm trying this:

--with-hs-cpp-flag="-specs=/opt/local/etc/ghc/overridecpp.spec -E -undef -traditional -x assembler-with-cpp"

@AlainODea
Copy link
Author

Using this as overridecpp.spec:

*cpp:
%{posix:-D_POSIX_SOURCE} %{pthread:-D_REENTRANT}

@AlainODea
Copy link
Author

Configuring GHC with modified hs-cpp-flags works to fix the tests:

./configure --with-hs-cpp-flags="-specs=/opt/local/etc/ghc/overridecpp.spec -E -undef -traditional -x assembler-with-cpp"

overridecpp.spec:

*cpp:
%{posix:-D_POSIX_SOURCE} %{pthread:-D_REENTRANT}

The challenge now is how to integrate this extra file and new default hs-cpp-flags cleanly into the GHC build and install. I'll spin that by the ghc-devs mailing list.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment