Skip to content

Instantly share code, notes, and snippets.

@Yarith

Yarith/compile-ghc-mod-5.9.0-with-stack.md

Last active December 8, 2018 21:26
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Yarith/9600cb2df80fb6b2fb257c1f8e6f6f97 to your computer and use it in GitHub Desktop.
Save Yarith/9600cb2df80fb6b2fb257c1f8e6f6f97 to your computer and use it in GitHub Desktop.
Download ZIP
Installation and configuration of the Haskell IDE Engine (HIE) together with the Elm Compiler Source for the Haskell Language Server extension in Visual Studio Code. Additional information about the Installation of the ghc-mod executable from source.
Raw
compile-ghc-mod-5.9.0-with-stack.md

Compile ghc-mod on OpenSUSE Tumbleweed (2018-12-04)

Notes about the compilation of ghc-mod 5.9.0 for ghc 8.2.2.

Working yaml file for the master branch at changeset 96c2207

Used commit: changeset 96c2207 Maybe the newer commit with changeset e5b7daf which is used by the Haskell IDE Engine in my today installation maybe work too. Have not tested it standalone.

Working stack.yaml file:

resolver: lts-11.22
packages:
- location: .
- location: ./core
extra-deps:
- monad-journal-0.7.2
- either-4.4.1.1
- free-4.12.4
- hlint-2.0.8
- haskell-src-exts-1.19.1
- extra-1.5.3
- cabal-helper-0.8.1.2@sha256:da0698b950fd28fef469c1092cda5f0e71ac56204961afd9d800fff2cc908607
- cabal-plan-0.4.0.0@sha256:0bb2b5194d7f757437a55b98d8c35312d362f5912268668dc2faa2ca977b3b98
- pretty-show-1.8.2@sha256:e16e0174bf4ad3dae49fea69b1f65ca9d8d4b61b3d80c78fbe3320f2181e92e9
flags: {}
extra-package-dbs: []

After saving the stack.yaml just execute stack install.

Troubleshooting

How to solve ld -lgmp error?

Install package gmp-devel

How to solve ld -ltinfo error?

Install package ncurses-devel

zlib package does not compile

Install package zlib-devel

Configuring zlib-0.6.2...
Cabal-simple_mPHDZzAJ_2.0.1.0_ghc-8.2.2: Missing dependency on a foreign
library:
* Missing (or bad) header file: zlib.h
* Missing C library: z

Multi-way if-expressions need MultiWayIf turned on

Add to the ghc-mod-core.cabal under the section Default-Extensions the MultiWayIf entry.

Raw
elm-compiler-ghc-mod-can-not-provide-info-for-all-modules.md

elm compiler - what is preventing us to inspect the code with ghc-mod?

Should you need help for setting up the elm compiler directory or the Haskell IDE Engine i have another file that discribes the installation: Configure elm compiler for use with Haskell IDE Engine

This document explains and shows a way to fight the error by modify some .hs files:

EXCEPTION: browse:
    Error: bytecode compiler can't handle unboxed tuples and sums.
    Possibly due to foreign import/export decls in source.
    Workaround: use -fobject-code, or compile this module to .o separately.

If you checkout the Elm Compiler and open it with Visual Studio Code with configured Haskell IDE Engine you will notice really fast, that for some modules are no information available. This document tracks my journey to answer: Why it does not work and what we can change? For me it is enough that we can at least inspect our modules again. This solution changes the code in this way that the ghc-mod never enters the Interpreter Mode which gets activated if there are some language extensions used.

Find the module which prevents inspecting all modules

There are some files in the elm/compiler repository that can not be inspected with ghc-mod because of the unpacked tuple error. To find out which module is causing it you can use this bash command:

ghc-mod debug | while read f
do
    regex="([a-zA-Z0-9.]+) \(.*\.hs\)"
    if [[ $f =~ $regex ]]
    then
        module="${BASH_REMATCH[1]}"
        echo "*** Module: $module ***"
        ghc-mod browse $module
    fi
done > project-modules-symbols.txt 2>&1

This will try to get all symbols for every module that is declared in the project. After it is done you can look for errors. I know that there will be done a second run, but i dont know how i can add a distinct for the module names.

Tracing back the source

The Bump module is the first module that could not be loaded in our list. Lets start the investigation on this module.

  1. Lets look for the Bump. It depends on the other 4 modules. We look up if they could be loaded and found out that the modules Elm.Bump and Elm.Project also could not be loaded.
  2. Lets look deeper in the Elm.Bump module. It depends on 12 other modules. Here is Elm.Project also used, which we already know that it could not be loaded. Every other module could be loaded.
  3. Lets look deeper in the Elm.Project module. It depends on 15 other modules. Here only the module Generate.Output could not be loaded.
  4. Lets look deeper in the Generate.Output module. It depends on 22 other modules. All of them could be loaded.
  5. Lets look if all modules which could not be loaded depends on Generate.Output. After checking all modules manually, we find out that every other depends on the module Generate.Output or a module which depends on it.

It seems so that the Generate.Output has something which ghc-mod does not like with our current configuration.

Lets inspect why our found source can not be loaded

We found out that the module Generate.Output itself can not be loaded by ghc-mod. The tests are done with ghc-mod -v -g "-O0" -g "-fobject-code" browse followed by the module name.

Lets delete some code from the module until the error disappears.

  1. We comment everything out and leave only the the type Mode and the export of the type. This should always work. After a test with ghc-mod browse the error is gone.
  2. Comment all imports in. The error appeared again. So there must be a dependency which is causing this.
  3. Look for the imports that is causing the error. The import of one module Generate.Functions alone is enough to cause the switch to the Interpreter Mode, but we get still results without an error. The module Generate.Html is causing the Interpreter Mode too without an error. All other modules are not causing the Interpreter Mode and if they are with the other two, there happens no error.
  4. Which module is not able to run within Interpreter Mode? Tested by letting Generate.Functions imported and trying out the other imports. So the result is these modules are not working with Interpreter Mode: Elm.Compiler, Elm.Compiler.Module, Elm.Compiler.Objects, Elm.Project.Json, Elm.Project.Summary, File.Args, File.Crawl, File.IO, Reporting.Exit, Reporting.Exit.Make, Reporting.Task, Stuff.Paths and Terminal.Args. These are quite many.

So we found out: We have 2 modules that are causing the Interpreter Mode and 13 modules that are not working with the Interpreter Mode. So what is the Interpreter Mode? Why is it neccessary and why is it causing problems? So we know from the info output of ghc-mod, that it is normal that it is enabled when the language extensions TemplateHaskell, QuasiQuotes, PatternSynonyms are used. QuasiQuotes for example is used for the modules Generate.Functions, Generate.Html, Develop.Generate.Help. QuasiQuotes is used to extend the syntax of haskell programs, here it adds a simpler form for multiline raw strings and using of quotes without the need to escape them. TemplateHaskell is used only for the runIO function.

Get rid of the language extensions that changes the behavior of ghc-mod

Translate the QuasiQuote multiline strings to regular multiline strings

Haskell has already multiline strings, but they require for every line at the end the new line sequence and the escape of the whitespace characters. Also we need to escape every backslash and double quote. Because in the code there are only a few places i go manually through the code and replace it. At this stage it is only a test if it is enough to solve our problem with ghc-mod.

Steps for the modification:

  1. Remove the QuasiQuote language extensions
  2. Remove the Text.RawString.QQ module from the imports
  3. Remove the surrounding [r||] from a multiline text and use double quotes instead.
  4. Escape all double quotes and backslashes in the content.
  5. Add after every line \n\ except the last and before every new line \. For empty lines add \\n\.
  6. Repeat starting at 3. until no [r||] block are left.

After the modification i run the ghc-mod browse Generate.Functions command. And it runs through without error! Lets rerun the bash script above for the generation of the file project-modules-symbols.txt. After it is done we can see how many modules are now inspectable with ghc-mod.

There are only 3 modules which can not be loaded by ghc-mod. These are: Main, Develop, Develop.StaticFiles. In this order they are depend on the next one. While only Develop.StaticFiles contains the last remaining language extension which causes the switch to the Interpreter Mode in ghc-mod.

Remove the need of TemplateHaskell in Develop.StaticFiles

At this point we already know that the runIO from the module Language.Haskell.TH requires the language extension TemplateHaskell. Now we need to figure out how we can change the code so the runIO function is not required anymore. At this point i have no clou if and how it is possible.

Next step is reading the docs, maybe there is a standard equivalent already documented like it is for the do notation. So, after reading the docs it is clear, that the execution of the part between $( and ) is executed on compile time and constructs an expression which contains the resulting data. In Develop.StaticFiles they are all ByteStrings. The function bsToExp constructs a Q Exp from an input of type ByteString which must be converted to the build time constant ByteString with the $() operation.

We need an alternative approach for the module Data.FileEmbed. These are basically resources which i know from other languages. It seems so the compiler has no standard syntax to embed a file as a ByteString on compile time. I think for this problem is no alternative solution available. So i will declare an embed with the IO ByteString result of the loader functions. In my use case i will only analyze the application but never compile it directly for execution.

Changing the file

First remove the TemplateHaskell macro line and then remove the imports of the modules Data.FileEmbed and Language.Haskell.TH.

I added following function to the Develop.StaticFiles module:

-- TEMPLATE HASKELL ALWAYS EMPTY MOCK
embed :: IO BS.ByteString -> BS.ByteString
embed _ =
  ""

And replaced all usages of the template syntax $() with the newly created embed function. The calls to the Develop.StaticFiles.Build are still there so we can find it with Find all references from the IDE.

-- ELM


elm :: BS.ByteString
elm =
  embed(Build.compile)



-- CSS


css :: BS.ByteString
css =
  embed(Build.readAsset "styles.css")



-- FONTS


codeFont :: BS.ByteString
codeFont =
  embed(Build.readAsset "source-code-pro.ttf")


sansFont :: BS.ByteString
sansFont =
  embed(Build.readAsset "source-sans-pro.ttf")



-- IMAGES


favicon :: BS.ByteString
favicon =
  embed(Build.readAsset "favicon.ico")


waiting :: BS.ByteString
waiting =
  embed(Build.readAsset "waiting.gif")

This change has basically removed the resources for the elm reactor and index.html creation. If you do not need both you could even use the compiled executable.

Lets test if we can now get the module symbols with the ghc-mod browse Develop.StaticFiles command. It worked! Test if Develop and Main are working too. Alright, everything worked!

Convert the files to ByteString literal so the application works as it should

You can do this step if you need to start the Elm Compiler with the reactor. But for my case it is not neccessary so i will stop at this point my journey and continue to inspect the source with full HIE support.

🎉 🎉 🎉

Raw
prepare-elm-compiler-with-haskell-ide-engine-for-vscode.md

Configure elm compiler for use with Haskell IDE Engine

This document describes the steps to get a working copy of the elm compiler source code which can be inspected via the Haskell IDE Engine.

My setup used the ghc-8.2.2. The latest lts-resolver for this version was: lts-11.22. This stack resolver is used for all compilations.

Install the Haskell IDE Engine 4.0.1 with stack for ghc-8.2.2

  1. Clone the repository git clone https://github.com/haskell/haskell-ide-engine.git -b 0.4.0.1 --recursive This gets the version with the tag 0.4.0.1 on the commit df6da63.
  2. Install the Haskell IDE Engine via stack install --stack-yaml stack-8.2.2.yaml --resolver lts-11.22.
  3. Prepare the hoogle database via stack --stack-yaml stack-8.2.2.yaml --resolver lts-11.22 exec hoogle generate If you are not doing this step you will probably get following warning later in vscode: No hoogle db found. Check the README for instructions to generate one

Install and configure the Haskell Language Server extension

Install and Configure the vscode extension Haskell Language Server to use the ~/.local/bin directory if you have not added it to the $PATH environment variable.

Setup the elm directory

  1. Clone the repository git clone https://github.com/elm/compiler.git.
    At the time of writing the current commit was a936e95.
  2. Create a stack.yaml file with stack init and then open the stack.yaml file with an editor and change the resolver to lts-11.22.
  3. Execute stack build
  4. Maybe it requires you to add some extra-deps just add them as suggested and run stack build again. At some point i removed them again and it does still work.

Help! Not for all modules are information available

Maybe you will encounter soon the error:

EXCEPTION: browse:
    Error: bytecode compiler can't handle unboxed tuples and sums.
    Possibly due to foreign import/export decls in source.
    Workaround: use -fobject-code, or compile this module to .o separately.

I tried hours after hours to change everything around ghc-mod to get rid of the error. Nothing seemed to help. Today i have looked which module is causing the errors and finally was able to open the elm compiler project with full type information in every module. I have took a slightly detailed document with notes about my process and saved it in an own file: elm compiler - what is preventing us to inspect the code with ghc-mod?

Optional: Install the ghc-mod cli to access some data without the Haskell IDE Engine

If you want to compile the matching ghc-mod-5.9.0 for ghc-8.2.2 i have already a file about it: Notes about compiling ghc-mod 5.9.0 with stack

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment