Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Upgrade to elm 0.19

This gist started Wednesday 29th August. We have until Friday 7th September to upgrade to Elm 0.19. This is a bunch of notes which I'm keeping track of to eventually turn into an article later on. Hope it helps your upgrade.

Day 1

Pre upgrade

428 Elm files.

github.com/AlDanial/cloc v 1.72  T=3.22 s (133.0 files/s, 23274.7 lines/s)
-------------------------------------------------------------------------------------------------------
File                                                                blank        comment           code
-------------------------------------------------------------------------------------------------------
./src/Types/Api/Enums.elm                                            2507             74           4030
./src/Membership/Forms.elm                                            344              3           1414
./src/Lookup.elm                                                      652             36           1375
./src/View/Components.elm                                             343             33            952
./src/Main.elm                                                        208             19            788
./src/Join/Panel.elm                                                  172             28            779
./src/Letter.elm                                                      121              5            648
./src/Person/Forms.elm                                                150              6            570
./src/Types/Api/ClaimHistoryDetail.elm                                 85              2            517
./src/Membership.elm                                                   78              5            483

...

-------------------------------------------------------------------------------------------------------
SUM:                                                                16643           1406          56859
-------------------------------------------------------------------------------------------------------

Compiling from scratch:

➜  frontend git:(integration) time elm-make ./src/Main.elm --output=./src/static/elm.js
Success! Compiled 589 modules.
Successfully generated ./src/static/elm.js
elm-make ./src/Main.elm --output=./src/static/elm.js  1705.72s user 742.49s system 467% cpu 8:43.84 total

Compile from scratch using RTS options

➜  frontend git:(integration) time elm-make +RTS -A128M -H128M -n8m -RTS src/Main.elm --output=./src/static/elm.js
Success! Compiled 589 modules.
Successfully generated ./src/static/elm.js
elm-make +RTS -A128M -H128M -n8m -RTS src/Main.elm   410.30s user 123.24s system 202% cpu 4:24.05 total

Compiling the infamous Lookup.elm

➜  frontend git:(integration) touch src/Lookup.elm
➜  frontend git:(integration) time elm-make +RTS -A128M -H128M -n8m -RTS src/Main.elm --output=./src/static/elm.js
Success! Compiled 178 modules.
Successfully generated ./src/static/elm.js
elm-make +RTS -A128M -H128M -n8m -RTS src/Main.elm   361.66s user 109.20s system 223% cpu 3:30.46 total

Compiling a typical page

➜  frontend git:(integration) touch src/Membership/Forms/TerminateRebate.elm
➜  frontend git:(integration) time elm-make +RTS -A128M -H128M -n8m -RTS src/Main.elm --output=./src/static/elm.js
Success! Compiled 4 modules.
Successfully generated ./src/static/elm.js
elm-make +RTS -A128M -H128M -n8m -RTS src/Main.elm   52.68s user 25.45s system 327% cpu 23.848 total

And lastly, compiling a popular Alfred.Logic module

➜  frontend git:(integration) touch src/Alfred/Logic.elm
➜  frontend git:(integration) time elm-make +RTS -A128M -H128M -n8m -RTS src/Main.elm --output=./src/static/elm.js
Success! Compiled 195 modules.
Successfully generated ./src/static/elm.js
elm-make +RTS -A128M -H128M -n8m -RTS src/Main.elm   363.02s user 131.70s system 217% cpu 3:47.29 total

Forgot about Main.elm:

➜  frontend git:(integration) ✗ time elm-make ./src/Main.elm --output=./src/static/elm.js
Success! Compiled 1 module.
Successfully generated ./src/static/elm.js
elm-make ./src/Main.elm --output=./src/static/elm.js  45.18s user 19.94s system 443% cpu 14.690 total
➜

Final dependencies prior to upgrade:

    "NoRedInk/elm-decode-pipeline": "3.0.0 <= v < 4.0.0",
    "billstclair/elm-localstorage": "4.0.2 <= v < 5.0.0",
    "cuducos/elm-format-number": "5.0.2 <= v < 6.0.0",
    "debois/elm-dom": "1.2.3 <= v < 2.0.0",
    "eeue56/elm-all-dict": "2.0.1 <= v < 3.0.0",
    "elm-community/elm-time": "1.0.14 <= v < 2.0.0",
    "elm-community/list-extra": "4.0.0 <= v < 5.0.0",
    "elm-community/maybe-extra": "4.0.0 <= v < 5.0.0",
    "elm-community/result-extra": "2.2.0 <= v < 3.0.0",
    "elm-community/string-extra": "1.4.0 <= v < 2.0.0",
    "elm-lang/core": "5.0.0 <= v < 6.0.0",
    "elm-lang/dom": "1.1.1 <= v < 2.0.0",
    "elm-lang/html": "2.0.0 <= v < 3.0.0",
    "elm-lang/http": "1.0.0 <= v < 2.0.0",
    "elm-lang/keyboard": "1.0.1 <= v < 2.0.0",
    "elm-lang/navigation": "2.0.1 <= v < 3.0.0",
    "elm-lang/svg": "2.0.0 <= v < 3.0.0",
    "elm-lang/virtual-dom": "2.0.4 <= v < 3.0.0",
    "evancz/url-parser": "2.0.1 <= v < 3.0.0",
    "lukewestby/elm-http-builder": "5.0.0 <= v < 6.0.0",
    "mgold/elm-random-pcg": "4.0.2 <= v < 5.0.0",
    "terezka/elm-plot": "5.1.0 <= v < 6.0.0",
    "truqu/elm-base64": "2.0.3 <= v < 3.0.0"

Other stuff

  • 77 flips
  • 266 shadowed variables

elm-upgrade

SUCCESS! Your project's dependencies and code have been upgraded.
However, your project may not yet compile due to API changes in your
dependencies.

See <https://github.com/elm/compiler/blob/master/upgrade-docs/0.19.md>
and the documentation for your dependencies for more information.

WARNING! 6 of your dependencies have not yet been upgraded to
support Elm 0.19.
  - https://github.com/eeue56/elm-all-dict
  - https://github.com/elm-community/elm-time
  - https://github.com/elm-community/maybe-extra
  - https://github.com/elm-community/result-extra
  - https://github.com/elm-lang/navigation
  - https://github.com/terezka/elm-plot

Compile issues

  • broken imports Dom, UrlParser, fixed links Browser.Dom Url.Parser
  • x.Extras community packages not yet updated - imported the functions locally
  • moved elm-all-dict locally

Day 2

Compile issues ( cont )

  • using justinmimbs/date as a dropin replacement for Time.DateTime, BEING CAREFUL NOT TO RENAME ALL DateTime TYPES JUST YET
  • make assumptions when converting functions like toPercent : comparable -> String about the most popular callers ie toPercent : Float -> String then fix the broken ones passing in Int ( minority ), Use Debug.toString sparingly a valid use case is when we really want a string value |> Debug.toString |> Alfred.removeSurroundingQuotes
  • hum..... :\ "record update type error" hope there's not too much of these ones, 2 so far
mapValue : (a -> b) -> Option a -> Option b
mapValue f ({ value } as model) =
    { visible = model.visible
    , description = model.description
    , value = f value
    }
  • 2136 occurances of DateTime... O_O, luckly 1785 is in Types.Api, only 241 to go through manually
  • creating our own DateTime type to restrict any future api changes to Date or Time, type DateTime = DateTime Date Posix
  • DateTime issue caused by own misuse of DateTime when we just meant Date, rewrite of Alfred.Dates, massive change to over 200+ files, backend Types.Api generated files included
  • bending over backwards to convert iso -> ISO8601 ( epoch ) -> Posix and back.
  • exposing (..) replaced with explicit functions really annoying when the api is constantly changing

Day 3

  • fix all regexes
  • JDP.decode -> JD.succeed in 100+ generated files, love the autogenerated files
  • recurd update type error +1
  • getting ISO8601.toTime (https://package.elm-lang.org/packages/jweir/elm-iso8601/latest/ISO8601) changed from Time -> Float to Time -> Posix so I don't have to do the ISO conversion in the middle with an artificial failure state (Result err ok) (jweir/elm-iso8601#8)
  • Json.Encode.list now takes in the value conversion a -> Value so we don't have to List.map beforehand! eg:
List.map EnumValue.encoder |> JE.list

-- becomes

|> JE.list EnumValue.encoder
  • type error msgs are a bit confusing, here is some annotations
-- TYPE MISMATCH ------------------------------ ../frontend/src/Model/Person.elm

Something is off with the body of the `getCoverStartDate` definition:

18|>    Debug.log "Person cover start date:"
19|>        (person.coverInfoDetails
20|>            |> List.sortWith sorter
21|>            |> List.reverse
22|>            |> List.head
23|>            |> Maybe.map .startDate

The body is:

    Maybe DateTime  -- This is the _inferred_ type, irrespective of what it's supposed to be. 


But the type annotation on `getCoverStartDate` says it should be:

    Maybe Date      -- This is the function signature, but it's also the type you end up with starting with ln 19 to 23

Naturally, when you start from ln 19 and work out what the type is, it's the correct type, Maybe Date, and the return type is also correct, but the problem is in the usage of the return value. The usage is Maybe DateTime and so the compiler says that the body is Maybe DateTime.

  • going to suggest a maintainable way to do record type updates, otherwise the intention of the function gets lost:
-- 0.19
map : (a -> b) -> Column subject a -> Column subject b
map mapper column =
    { -- updated
      toBody = column.toBody >> Html.map mapper
    , toHeader = column.toHeader >> Html.map mapper

    -- unchanged
    , title = column.title
    , alignment = column.alignment
    , getRawData = column.getRawData
    , more = column.more
    }

before:

-- 0.18
map : (a -> b) -> Column subject a -> Column subject b
map mapper column =
    { column
        | toHeader = column.toHeader >> Html.map mapper
        , toBody = column.toBody >> Html.map mapper
    }
  • recurd update type error +2, +1

  • back to playing with Date, Time, DateTime... by far the biggest time spent so far

  • holy shit the compiler is FAST, 57 modules just blinked by

  • hello! blow away elm-stuff, compile

elm make ../frontend/src/View/Components.elm
elm: not enough bytes
CallStack (from HasCallStack):
  error, called at src/Data/Binary.hs:212:21 in binary-0.8.6.0-1SWOkUSZj1B1k0FQ5fyFqI:Data.Binary

Process finished with exit code 1
  • fixing view Components that make use of comparable to display strings, pick the most popular between Int/Float and fix the rest
  • looking for the positive side:
map : (a -> b) -> Grid subject a -> Grid subject b
map mapper grid =
    { columns = List.map (Column.map mapper) grid.columns
    , rowAttributes = grid.rowAttributes >> List.map (HA.map mapper)
    , rowClicked = Maybe.map (\rowClicked -> rowClicked >> mapper) grid.rowClicked
    , toParentMsg = grid.toParentMsg >> mapper

    -- unchanged
    , showMore = grid.showMore
    , showPager = grid.showPager
    , showMoreRowFilters = grid.showMoreRowFilters
    , pageSize = grid.pageSize
    , pagePosition = grid.pagePosition
    , maxHeight = grid.maxHeight
    , comparer = grid.comparer
    , toUniqueId = grid.toUniqueId
    , selection = grid.selection
    , multiSelect = grid.multiSelect
    , sorter = grid.sorter
    , sortGrid = grid.sortGrid
    }
  • going through all Alfreds, all Components fixing as we go now that the lower level helpers are compiling
  • updated backend Elm codegen to generate correct new types, since we don't do a View Model, as in we map the generated type straight to the UI Components through the pages, it means most pages will require minimal changes
  • upgrade has opened up some opportunities for refactor, DateTime and Enums, we're past the point of no return, but this is a massive change which I'd have unhappily put off for another few months
  • end of day 3: 419 files changed, 14708 insertions(+), 9929 deletions(-)

Day 4

  • Realised today the outdated plugin I'd been using for webstorm has been updated to 0.19! One month ago... https://github.com/klazuka/intellij-elm/releases/tag/v1.3.0
  • Side quest: Update all enums to tagged union type.
  • file by file compile errors now, 100+ done, 100+ to go
  • 0.19's error reporting uses colors which is great unless your editor doesn't support colors in which case you get a very general message. elm/error-message-catalog#269
  • Url.parseHash -> Url.Parser.fragment
  • Url.Parser.custom now returns a Maybe a instead of a Result String a, adjust accordingly
custom : String -> (String -> Maybe a) -> Parser (a -> b) b

-- using this opportunity to advertise for Alfred.
-- When it was a `Result String a`, we used this:

|> Result.fromMaybe ("Url segment is not a membership form: " ++ segment)

-- but now it's a Maybe type, it's simply changed to.
-- This is because `|> Debug.log ...` will always log, no matter what the result is

|> Alfred.sayIfNothing ("Url segment is not a membership form: " ++ segment)

sayIfNothing : String -> a -> Maybe b -> Maybe b
sayIfNothing something printable data =
    case data of
        Nothing ->
            Debug.log something printable |> (\_ -> data)

        Just _ ->
            data

Day 5

  • if you are certain you have no 'TYPE MISMATCH' error but the compiler thinks there is, then rm -rf elm-stuff, R.I.P 1 hr of my life. (even after numerous recompiles, might still be editor, file changed or not related )
  • 200+ files changed, 200+ files to go... feeling the pressure now, 3 days left
  • 431 files changed, 15858 insertions(+), 11417 deletions(-)

Day 6

  • All compile errors and no success makes Joe a very sad boy. ( a.k.a in the darkest part of the tunnel atm )
  • by this time, 100+ of errs later, most of the errs are so similar alot of it is guesswork, 99% of time it's right. Gives me ideas about automating more of our helpers... about 50 files to go...
  • let's compare TYPE MISMATCH errors, 185 lines for one error:
-- TYPE MISMATCH ------------------------------------- src/Membership/Claims.elm

Something is off with the body of the `addClaimHistoryDetailRows` definition:

356|>    Grid.makeColumn "Claim #" (.claimId >> String.fromInt)
...
380|>        >> Grid.makeHiddenColumn "Adj. Flag" (.adjustmentFlag >> Lookup.flag lookup)
...

The body is:

    Grid
        { accommodationLevel : Enums.AccommodationLevel
        , adjustmentFlag : Enums.Flag
        , ...
        }
        Msg
    -> Grid
           { accommodationLevel : Enums.AccommodationLevel
           , adjustmentFlag : Enums.Flag
           , ...
           }
           Msg

But the type annotation on `addClaimHistoryDetailRows` says it should be:

    Grid ClaimHistoryDetail Msg -> Grid ClaimHistoryDetail Msg
  • Elm now gets confused if a imported and exposed Type tag conflicts with a locally declared Type tag
  • self shadowing function:
updateGrid : ...
updateGrid msg receipts model =
    let

        updateGrid =
            ...
    in
       updateGrid
  • Finally! BOSS LEVEL: Fairly standard errors, having survived through 100s of these, this should be a piece of cake ;)
➜  frontend git:(upgrade/0.19) ./make.sh 
Attempting to run elm with RTS flags
Detected errors in 1 module.                                         
-- SHADOWING ---------------------------------------------------- ./src/Main.elm

-- NAMING ERROR ------------------------------------------------- ./src/Main.elm

I cannot find a `Url.Parser.parse` variable:

376|                 |> Url.Parser.parse Routes.parser

-- NAMING ERROR ------------------------------------------------- ./src/Main.elm

I cannot find a `Navigation.Location` type:

231| init : Types.Flags.JSFlags -> Navigation.Location -> ( Model, Cmd Msg )

-- NAMING ERROR ------------------------------------------------- ./src/Main.elm

I cannot find a `Navigation.programWithFlags` variable:

54|     Navigation.programWithFlags UrlChange

-- NAMING ERROR ------------------------------------------------- ./src/Main.elm

I cannot find a `Time.DateTime.addMinutes` variable:

351|                                 |> Time.DateTime.addMinutes lookup.flags.utcOffset

  • Quick and dirty nav guide for those migrating to Browser.application:
-- init, before and after
init : Types.Flags.JSFlags -> Navigation.Location -> ( Model, Cmd Msg )
init : Types.Flags.JSFlags -> Url -> Browser.Navigation.Key -> ( Model, Cmd Msg )

-- main before
main : Program Types.Flags.JSFlags Model Msg
main =
    Navigation.programWithFlags UrlChange
        { view = view
        , init = init
        , update = update
        , subscriptions = subscriptions
        }
        
-- main after ( add stubs for onUrl... )
main : Program Types.Flags.JSFlags Model Msg
main =
    Browser.application
        { view = view
        , init = init
        , update = update
        , subscriptions = subscriptions
        , onUrlRequest = Debug.log "UrlRequest" >> always NoOp
        , onUrlChange = Debug.log "UrlChange" >> always NoOp
        }

-- add a field for Navigation.Key
type alias Model = {
    ...
    , key : Navigation.Key
    }
    
-- handle Browser.UrlRequest everywhere Navigation.Location is mentioned
type Msg
    = ...
    -- | UrlChange Navigation.Location -- 
    | UrlChange Browser.UrlRequest    
    
-- view, return a record instead of Html
view : Model -> Html Msg
view : Model -> Browser.Document Msg
view model = 
  { title = "My app"
  , body = [ originalView model ]
  }  

  • Day 6, 9:53pm, Main.elm compiles.
  • STATS STATS STATS, in the same order as above
➜  frontend git:(upgrade/0.19) cloc --include-lang=Elm --by-file ./src
     785 text files.
     782 unique files.                                          
     355 files ignored.

github.com/AlDanial/cloc v 1.74  T=1.31 s (331.5 files/s, 59394.4 lines/s)
-------------------------------------------------------------------------------------------------------
File                                                                blank        comment           code
-------------------------------------------------------------------------------------------------------
./src/Types/Api/Enums.elm                                            2619             77           4211
./src/Lookup.elm                                                      660             36           1639
./src/Membership/Forms.elm                                            346              3           1413
./src/View/Components.elm                                             359             40           1115
./src/Main.elm                                                        217             20            813
./src/Join/Panel.elm                                                  187             28            779
./src/Letter.elm                                                      129              5            646
./src/Person/Forms.elm                                                151              6            570
./src/Types/Api/ClaimHistoryDetail.elm                                 85              2            518
./src/Components/Grid.elm                                             145             45            484
./src/Membership.elm                                                   78              5            482
...
-------------------------------------------------------------------------------------------------------
SUM:                                                                17386           1630          58753
-------------------------------------------------------------------------------------------------------

Somewhere along the lines, we gained 1.5k loc

Total beans counted:

429 files changed, 16422 insertions(+), 12116 deletions(-)

Compiling from scratch:

➜  frontend git:(upgrade/0.19) rm -rf elm-stuff 
➜  frontend git:(upgrade/0.19) time elm make ./src/Main.elm --output=./src/static/elm.js
Dependencies loaded from local cache.
Dependencies ready!                
Success! Compiled 398 modules.                                       
elm make ./src/Main.elm --output=./src/static/elm.js  14.96s user 0.52s system 92% cpu 16.703 total

Compiling with RTS flags, looks like elm 19 has them exposed by default, wooohooo

➜  frontend git:(upgrade/0.19) time elm make +RTS -A128M -H128M -n8m -RTS src/Main.elm --output=./src/static/elm.js
Dependencies loaded from local cache.
Dependencies ready!                
Success! Compiled 398 modules.                                       
elm make +RTS -A128M -H128M -n8m -RTS src/Main.elm   8.62s user 0.46s system 88% cpu 10.288 total

The infamous Lookup.elm

➜  frontend git:(upgrade/0.19) touch src/Lookup.elm 
➜  frontend git:(upgrade/0.19) time elm make +RTS -A128M -H128M -n8m -RTS src/Main.elm --output=./src/static/elm.js
Success! Compiled 178 modules.                                       
elm make +RTS -A128M -H128M -n8m -RTS src/Main.elm   7.64s user 0.40s system 99% cpu 8.043 total

NO LONGER SO INFAMOUS!

Common page

➜  frontend git:(upgrade/0.19) touch src/Membership/Forms/TerminateRebate.elm
➜  frontend git:(upgrade/0.19) time elm make +RTS -A128M -H128M -n8m -RTS src/Main.elm --output=./src/static/elm.js
Success! Compiled 4 modules.                                         
elm make +RTS -A128M -H128M -n8m -RTS src/Main.elm   5.10s user 0.58s system 96% cpu 5.906 total

A popular helper, Logic.elm

➜  frontend git:(upgrade/0.19) touch src/Alfred/Logic.elm
➜  frontend git:(upgrade/0.19) time elm make +RTS -A128M -H128M -n8m -RTS src/Main.elm --output=./src/static/elm.js
Success! Compiled 195 modules.                                       
elm make +RTS -A128M -H128M -n8m -RTS src/Main.elm   8.54s user 0.42s system 98% cpu 9.109 total

and last but not least, Main.elm

➜  frontend git:(upgrade/0.19) touch src/Main.elm 
➜  frontend git:(upgrade/0.19) time elm make +RTS -A128M -H128M -n8m -RTS src/Main.elm --output=./src/static/elm.js
Success! Compiled 1 module.                                          
elm make +RTS -A128M -H128M -n8m -RTS src/Main.elm   3.74s user 0.33s system 99% cpu 4.065 total
➜

Day 7

  • tried to port a localStorage package, ended up rewriting it in a much simpler fashion, only what we needed
  • Time strikes again, diff packages, diff formats, many bugs
  • Routing change from fragment to html5 was quite easy: we had a function
toHash : Route -> String
toHash route =
    case route of
        Home ->
            "#"

        Login ->
            "#login"

        Logout ->
            "#logout"

it was just a matter of changing it to

toPath : Route -> String
toPath route =
    case route of
        Home ->
            "/"

        Login ->
            "/login"

        Logout ->
            "/logout"

Didn't end up changing the parser at all:

parser : Parser (Route -> a) a
parser =
    Url.Parser.oneOf
        [ Url.Parser.map Login top
        , Url.Parser.map Login (s "login")
        ...
        ]

-- Ahh richard, you've done it again: Turns out we have alot of problems with html5 and must take a step back to fragments, the change is trivial

-- Route.elm

{-| Lifted from: <https://github.com/rtfeldman/elm-spa-example/blob/master/src/Route.elm#L59-L65>
-}
parseUrlFragmentAsPath : Url.Url -> Maybe Route
parseUrlFragmentAsPath url =
    { url | path = Maybe.withDefault "" url.fragment, fragment = Nothing }
        |> Url.Parser.parse parser


-- Main.elm

UrlChange url ->
    url
        -- |> Url.Parser.parse Routes.parser  -- before
        |> Routes.parseUrlFragmentAsPath      -- after
                

Day 8

  • iframe works still, untouched

  • girish got froala editor to work... don't know how or why it's working but it does.

  • It's brilliant! So we used to do this:

editView : String -> String -> Html msg
editView id html =
    Html.div
        [ ...
        ]
        [ ...
        , Html.node "script"
            [ ...
            ]
            [ Html.text "froala.createFroalaElement()"
            ]
        ]

This says, when elm renders the WYSIWYG editor div, render a script node which will automatically call the initialiser for the element. However, in elm 0.19, script elements were removed so this stopped working.

But then came arrive.js, with MutationObservers which allows you to hook into dom tree addition/removals, so all that was needed was:

<body>
    <script ... elm.init >
    <script type="text/javascript">
        $(document).arrive("#froala-editor-wrapper", function(newElem) {
            froala.createFroalaElement()
        });
    </script>

This says, whenever a element with the froala-editor-wrapper id is introduced into the dom, run the create element.

Turns out this is exactly what we did before using a script element, we're just doing it now with mutation observers!

  • Running on CI, elm make --yes was removed so trying the work around yes | elm make ... elm/compiler#1752 (comment)

  • Turns out the --yes flag wasn't for installing packages? was a non issue for us

  • running tests: 78 passing (8m) 28 pending 18 failing

  • Total beans counted at merge

*.elm 429 files changed, 16463 insertions(+), 12167 deletions(-)

Types.Api 178 files changed, 9308 insertions(+), 7505 deletions(-)

local_packages 3 files changed, 1529 insertions(+)

Files we changed 248 files, +5626, -4662

~10%, of project (58857 loc)

@DanL
Copy link

DanL commented Jan 29, 2019

This was super helpful, thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment