HIEFile GSOC progress report.md

Links

Description of .hie files: https://ghc.haskell.org/trac/ghc/wiki/HIEFiles

Modifications to GHC: https://github.com/ghc/ghc/compare/master...wz1000:hiefile

Modifications to Haddock: https://github.com/haskell/haddock/compare/ghc-head...wz1000:hiefile2

Hyperlinked source built using .hie files: https://drive.google.com/open?id=1mmwH91zrAJtzaXbt0OlFxo3qBsllj7ss

Instructions to use/build

$ git clone https://github.org/wz1000/ghc
$ git checkout hiefile
$ cd ghc/utils/haddock
$ git remote add wz1000 https://github.com/wz1000/haddock
$ git fetch wz1000
$ git checkout wz1000/hiefile2

Now add the following lines to ghc/mk/build.mk

GhcLibHcOpts += -fenable-ide-info -fvalidate-hie
GhcStage2HcOpts += -fenable-ide-info -fvalidate-hie
HADDOCK_DOCS = YES
EXTRA_HADDOCK_OPTS += --hyperlinked-source

Finally, in the ghc root directory

$ ./boot && ./configure && make -jN

On a system with 8G or less memory, the haddock phase will most probably require 1-2G swap or memory compression with zram/zswap/... in order to complete.

After this step, hyperlinked source should be available in compiler/stage2/doc and libraries/*/dist-install/doc

You should also see .hie files in compiler/stage2/build and libraries/*/dist-install/build

I've included a link in the above section to hyperlinked source built against ghc:hiefile on 2018-08-12

Tasks completed over summer

• Collecting informations from GHC for inclusion into .hie files
• Modifications to GHC so that SrcSpans become more accurate/representative of actual source
• Serialisation of .hie files, including a compact method to store redundant type information
• Modifying GHC to produce and keep track of .hie files when compiling with a new -fenable-ide-info flag
• Rewrite of haddocks --hyperlinked-source feature to use .hie files(exclusively, no other information required)
• Testing and dogfooding the .hie files and generated hyperlinked source over the ghc:head tree and fixing edge cases
• A few tests/sanity checks for .hie files now built into ghc(enabled via -fvalidate-hie) with a few test cases added to the testsuite

Pending work

• Merge/integrate with sjacobis work on Hi Haddock
• Fix haddock html tests
• Many possible enhancements for hyperlinked-source
• • Prettyprint hover types as hyperlinked html
• • Haddock documentation on hover
• • Searching for references of symbols
• • Exposing the scope information collected in .hie files
• Implement more applications for .hie files
• • Package db wide references tracker
• • Overhaul ghci :set +c commands
• • Integrate into haskell-ide-engine

Do you think we could start by merging the GHC side portion first (without integrating with Hi Haddock)? The way I see it, here's a timeline on how things can proceed:

• GHC-side generation of HIE files is merged into GHC
• Today's Haddock is adapted to make use of those files for hyperlinked sources
• Then we rebase the ongoing Hi Haddock work on top of HIE file changes

In particular, it seems like we should be able to debug the Hyperlinker tests completely separately from Hi Haddock.

Based on the links you've pasted above, the next concrete step would be to open a Diff on Phabricator for your GHC commits. You could then also concurrently open a PR to Haddock (it will make a fine motivating example for folks reviewing the GHC-side changes). Having a diff and a PR open is going to make it harder for this code to bitrot and be forgotten.

Feel free to reach out to me by email at alec.theriault@gmail.com or on IRC.