Skip to content

Instantly share code, notes, and snippets.

@chrismccord
Last active October 14, 2024 09:32
Show Gist options
  • Save chrismccord/bb1f8b136f5a9e4abc0bfc07b832257e to your computer and use it in GitHub Desktop.
Save chrismccord/bb1f8b136f5a9e4abc0bfc07b832257e to your computer and use it in GitHub Desktop.
Phoenix 1.3.x to 1.4.0 Upgrade Guides

Phoenix 1.4 ships with exciting new features, most notably with HTTP2 support, improved development experience with faster compile times, new error pages, and local SSL certificate generation. Additionally, our channel layer internals receiveced an overhaul, provided better structure and extensibility. We also shipped a new and improved Presence javascript API, as well as Elixir formatter integration for our routing and test DSLs.

This release requires few user-facing changes and should be a fast upgrade for those on Phoenix 1.3.x.

Install the new phx.new project generator

The mix phx.new archive can now be installed via hex, for a simpler, versioned installation experience.

To grab the new archive, simply run:

$ mix archive.uninstall phx_new
$ mix archive.install hex phx_new 1.4.0

Update Phoenix and Cowboy deps

To get started, simply update your Phoenix dep in mix.exs:

{:phoenix, "~> 1.4.0"}

Next, replace your :cowboy dependency with :plug_cowboy:

{:plug_cowboy, "~> 2.0"}
{:plug, "~> 1.7"}

To upgrade to Cowboy 2 for HTTP2 support, use ~> 2.0 as above. To stay on cowboy 1, pass ~> 1.0.

Finally, remove your explicit :ecto dependency and update your :phoenix_ecto and :ecto_sql dependencies with the following versions:

  ...,
  {:ecto_sql, "~> 3.0"},
  {:phoenix_ecto, "~> 4.0"}

After running mix deps.get, then be sure to grab the latest npm pacakges with:

$ cd assets
$ npm install

Update your Jason configuration

Phoenix 1.4 uses Jason for json generation in favor of poison. Poison may still be used, but you must add :poison to your deps to continue using it on 1.4. To use Jason instead, :jason to your deps in mix.exs:

  ...,
  {:jason, "~> 1.0"},

Then add the following configuration in config/config.exs:

# Use Jason for JSON parsing in Phoenix
config :phoenix, :json_library, Jason

Update your UserSocket

Phoenix 1.4 deprecated the transport macro, in favor of providing transport information directly on the socket call in your endpoint. Make the following changes:

# app_web/channels/user_socket.ex
- transport :websocket, Phoenix.Transports.WebSocket
- transport :longpoll, Phoenix.Transports.LongPoll, [check_origin: ...]

# app_web/endpoint.ex
- socket "/socket", MyAppWeb.UserSocket
+ socket "/socket", MyAppWeb.UserSocket, 
+   websocket: true # or list of options
+   longpoll: [check_origin: ...]

Update your Presence javascript

A new, backwards compatible Presence JavaScript API has been introduced to both resolve race conditions as well as simplify the usage. Previously, multiple channel callbacks against "presence_state and "presence_diff" events were required on the client which dispatched to Presence.syncState and Presence.syncDiff functions. Now, the interface has been unified to a single onSync callback and the presence instance tracks its own channel callbacks and state. For example:

import {Socket, Presence} from "phoenix"

let renderUsers(presence){
-  someContainer.innerHTML = Presence.list(presence, (id, user) {
-    `<br/>${escape(user.name)}`
-  }.join("")
+  someContainer.innerHTML = presence.list((id, user) {
+    `<br/>${escape(user.name)}`   
+  }).join("")
}

let onJoin = (id, current, newPres) => {
  if(!current){
    console.log("user has entered for the first time", newPres)
  } else {
    console.log("user additional presence", newPres)
  }
}
 
let onLeave = (id, current, leftPres) => {
  if(current.metas.length === 0){
    console.log("user has left from all devices", leftPres)
  } else {
    console.log("user left from a device", leftPres)
  }
})

let channel = new socket.channel("...")
- let presence = {}
- channel.on("presence_state", state => {
-   presence = Presence.syncState(presence, state, onJoin, onLeave)
-   renderUsers(presence)
- })
- channel.on("presence_diff", diff => {
-   presence = Presence.syncDiff(presence, diff, onJoin, onLeave)
-   renderUsers(presence)
- })
+ let presence = new Presence(channel)

+ presence.onJoin(onJoin)
+ presence.onLeave(onLeave)
+ presence.onSync(() => renderUsers(presence))

Optional Updates

The above changes are the only ones necessary to be up and running with Phoenix 1.4. The remaining changes will bring you up to speed with new conventions, but are strictly optional.

Add formatter support

Phoenix 1.4 includes formatter integration for our routing and test DSLs. Create or ammend your .formatter.exs in the root of your project(s) with the following:

[
  import_deps: [:phoenix],
  inputs: ["*.{ex,exs}", "{config,lib,priv,test}/**/*.{ex,exs}"]
]

Add a Routes alias and update your router calls

A Routes alias has been added to app_web.ex for view and controller blocks in favor over the previously imported AppWeb.Router.Helpers.

The new Routes alias makes it clearer where page_path/page_url and friends exist and removes compile-time dependencies across your web stack. To use the latest conventions, make the following changes to app_web.ex:

- import AppWeb.Router.Helpers
+ alias AppWeb.Router.Helpers, as: Routes

Next, update any controllers, views, and templates calling your imported helpers or static_path|url, to use the new alias, for example:

- <%= link "show", to: user_path(@conn, :show, @user) %>
+ <%= link "show", to: Routes.user_path(@conn, :show, @user) %>

- <script type="text/javascript" src="<%= static_url(@conn, "/js/app.js") %>"></script>
+ <script type="text/javascript" src="<%= Routes.static_url(@conn, "/js/app.js") %>"></script>

Replace Brunch with webpack

The mix phx.new generator in 1.4 now uses webpack for asset generation instead of brunch. The development experience remains the same – javascript goes in assets/js, css goes in assets/css, static assets live in assets/static, so those not interested in JS tooling nuances can continue the same patterns while using webpack. Those in need of optimal js tooling can benefit from webpack's more sophisticated code bunding, with dead code elimination and more.

To proceed:

  • update assets/package.json to replace Brunch with webpack:
   "repository": {},
   "license": "MIT",
   "scripts": {
-    "deploy": "brunch build --production",
-    "watch": "brunch watch --stdin"
+    "deploy": "webpack --mode production",
+    "watch": "webpack --mode development --watch-stdin"
   },
   "dependencies": {
     "phoenix": "file:../deps/phoenix",
     "phoenix_html": "file:../deps/phoenix_html"
   },
   "devDependencies": {
-    "babel-brunch": "6.1.1",
-    "brunch": "2.10.9",
-    "clean-css-brunch": "2.10.0",
-    "uglify-js-brunch": "2.10.0"

+    "@babel/core": "^7.0.0",
+    "@babel/preset-env": "^7.0.0",
+    "babel-loader": "^8.0.0",
+    "copy-webpack-plugin": "^4.5.0",
+    "css-loader": "^0.28.10",
+    "mini-css-extract-plugin": "^0.4.0",
+    "optimize-css-assets-webpack-plugin": "^4.0.0",
+    "uglifyjs-webpack-plugin": "^1.2.4",
+    "webpack": "4.4.0",
+    "webpack-cli": "^2.0.10"
   }
 }
  • delete assets/brunch-config.js
  • create assets/.babelrc with the following contents:
{
    "presets": [
        "env"
    ]
}
  • create assets/webpack.config.js with the following contents:
const path = require('path');
const glob = require('glob');
const MiniCssExtractPlugin = require('mini-css-extract-plugin');
const UglifyJsPlugin = require('uglifyjs-webpack-plugin');
const OptimizeCSSAssetsPlugin = require('optimize-css-assets-webpack-plugin');
const CopyWebpackPlugin = require('copy-webpack-plugin');

module.exports = (env, options) => ({
  optimization: {
    minimizer: [
      new UglifyJsPlugin({ cache: true, parallel: true, sourceMap: false }),
      new OptimizeCSSAssetsPlugin({})
    ]
  },
  entry: {
      './js/app.js': ['./js/app.js'].concat(glob.sync('./vendor/**/*.js'))
  },
  output: {
    filename: 'app.js',
    path: path.resolve(__dirname, '../priv/static/js')
  },
  module: {
    rules: [
      {
        test: /\.js$/,
        exclude: /node_modules/,
        use: {
          loader: 'babel-loader'
        }
      },
      {
        test: /\.css$/,
        use: [MiniCssExtractPlugin.loader, 'css-loader']
      }
    ]
  },
  plugins: [
    new MiniCssExtractPlugin({ filename: '../css/app.css' }),
    new CopyWebpackPlugin([{ from: 'static/', to: '../' }])
  ]
});
  • Update config/dev.exs to use webpack instead of brunch:
-  watchers: [node: ["node_modules/brunch/bin/brunch", "watch", "--stdin",
+  watchers: [
+    node: [
+      "node_modules/webpack/bin/webpack.js",
+      "--mode",
+      "development",
+      "--watch-stdin",
+      cd: Path.expand("../assets", __DIR__)
+    ]

CSS changes

  • The main CSS bundle must now be imported from the app.js file. Add the following line to the top of assets/js/app.js:
import css from '../css/app.css';
  • If you are using the default css, replace assets/css/phoenix.css with the following:

https://raw.githubusercontent.com/phoenixframework/phoenix/89cdcfbaa041da1daba39e39b0828f6a28b6d52f/installer/templates/phx_assets/phoenix.css

@pushpankar
Copy link

I am getting

ERROR in ./js/app.js
Module build failed (from ./node_modules/babel-loader/lib/index.js):
Error: Cannot find module 'babel-preset-env' from '/Users/pushpankar/til/assets'
- Did you mean "@babel/env"?

@arlomba
Copy link

arlomba commented Oct 18, 2018

@pushpankar read my other comment, should fix your issue.

@chrismccord can you please update .babelrc?

@pushpankar
Copy link

pushpankar commented Oct 19, 2018

@Anticide When I use this one as .babelrc

{
  "presets": [
    "@babel/preset-env"
  ]
}

I get syntax error

ERROR in ./js/app.js
Module build failed (from ./node_modules/babel-loader/lib/index.js):
SyntaxError: /Users/pushpankar/til/assets/js/app.js: Unexpected token (59:2)

  57 | // console.log(client)
  58 | const App = () => (
> 59 |   <BrowserRouter>
     |   ^
  60 |     <ApolloProvider client={client}>
  61 |       <Main />
  62 |     </ApolloProvider>

I tried to add react to .babelrc like this

{
  "presets": [
    "@babel/preset-env",
    "react"
  ]
}

but then got
Error: Plugin/Preset files are not allowed to export objects, only functions. In /Users/pushpankar/til/assets/node_modules/babel-preset-react/lib/index.js
I guess issue is related to react/webpack not phoenix1.4

@arlomba
Copy link

arlomba commented Oct 19, 2018

You're using React, Babel doesn't understand JSX, so you need to install another preset that translates JSX to JS.

First install the dependency:

npm install --save-dev @babel/preset-react

Then add it to .babelrc:

{
  "presets": [
    "@babel/preset-env",
    "@babel/preset-react"
  ]
}

And that's it, more info: https://babeljs.io/docs/en/next/babel-preset-react

@pushpankar
Copy link

Thank you @Anticide. I used "@babel/react" instead of "react" and the problem was solved.

@phoffer
Copy link

phoffer commented Oct 20, 2018

The web socket example is missing a comma, just for people copy-pasting @chrismccord. Should be after websocket: true

# app_web/endpoint.ex
- socket "/socket", MyAppWeb.UserSocket
+ socket "/socket", MyAppWeb.UserSocket, 
+   websocket: true, # or list of options
+   longpoll: [check_origin: ...]

@behe
Copy link

behe commented Oct 24, 2018

When using brunch I get an error when trying to build phoenix.js for production:

LOGGY_STACKS=1 node_modules/brunch/bin/brunch build --production

21:20:22 - error: Optimizing of ../priv/static/js/app.js failed. L154:1366 SyntaxError: Unexpected token: name (f)

@raphlcx
Copy link

raphlcx commented Oct 25, 2018

Regarding plug:

{:plug_cowboy, "~> 2.0"}
{:plug, "~> 1.7"}

:plug_cowboy has an explicit dependency on :plug ~> 1.7, so I think it's safe to omit specifying plug in our mix.exs when upgrading from phoenix 1.3.x to 1.4.0.

@navinpeiris
Copy link

You can also see a full diff of changes between apps generated between 1.4.0 and earlier versions at: https://www.phoenixdiff.org/?source=1.3.4&target=1.4.0

@hykw
Copy link

hykw commented Nov 8, 2018

It also requires:

{:cowboy, "~> 2.5"}

and the following, before mix deps.get

  • mix deps.unlock cowlib
  • mix deps.unlock ranch

@zacksiri
Copy link

zacksiri commented Nov 8, 2018

It also requires:

{:cowboy, "~> 2.5"}

and the following, before mix deps.get

  • mix deps.unlock cowlib
  • mix deps.unlock ranch

Actually if you have plug_cowboy you don't need to add cowboy directly anymore.

@hykw
Copy link

hykw commented Nov 8, 2018

Actually if you have plug_cowboy you don't need to add cowboy directly anymore.

Ah, yes. But since this is an upgrade guide from 1.3, it is better to write either {:cowboy, "~> 2.5"} or remove {:cowboy, "~> 1.0"} then.

@humberaquino
Copy link

Used this guide to migrate from Phoenix 1.3.3 to 1.4.0 and everything worked great except for webpack. But after using "@babel/preset-env" instead of "env" within the .babelrc file, the bundle got created.

Thanks @Anticide for the fix and @chrismccord for this guide and the awesome release!

@rockwood
Copy link

If it helps anyone else this grep, xargs, sed invocation dealt with 99% of my route call changes (your mileage may vary, please ensure you have a version you can revert to if it goes wrong in your environment, etc.)

grep -Rl "\s[a-zA-Z_]\+_\(path\|url\)(@\?conn" lib | xargs sed -i -r -e "s/\s([a-zA-Z_]+)_(path|url)\((@)?conn/ Routes.\1_\2(\3conn/g"

For those on Mac OS, sed behaves a little differently. This worked for me:

grep -Rl "\s[a-zA-Z_]\+_\(path\|url\)(@\?conn" lib | xargs sed -i.bk -E "s/([a-zA-Z_]+)_(path|url)\((@)?conn/Routes.\1_\2(\3conn/g"

@TomBers
Copy link

TomBers commented Nov 12, 2018

Thank you @chrismccord for this excellent guide. Can I just echo, if the right fix for the .babelrc is to change "env" to "@babel/preset-env" can the main doc reflect this, I was scratching my head for a while.

@AlexGascon
Copy link

AlexGascon commented Nov 19, 2018

@chrismccord Thanks for the guide, it has been really useful! However, I found a problem related with the switch from Poison to Jason, not sure if it's only me or if it's a generalized issue. Despite exactly following the steps, I got the following error when trying to start the server:

== Compilation error in file lib/alexgascon_web/endpoint.ex ==
** (ArgumentError) invalid :json_decoder option. The module Poison is not loaded and could not be found

After analyzing the line that was failing, I found that the cause was the following:

  plug Plug.Parsers,
    parsers: [:urlencoded, :multipart, :json],
    pass: ["*/*"],
    json_decoder: Poison

It was solved just by changing the last line

- json_decoder: Poison
+ json_decoder: Jason

Fortunately it was an easy change, but maybe it will be helpful to mention it because it can be can confusing to encounter an error despite following the guide step by step.

@hlship
Copy link

hlship commented Nov 24, 2018

My upgrade worked fine, once I worked out a few typos, but one step is missing:

If you are still using the default Phoenix CSS (based on Bootstrap), then add the following to assets/css/app.css:

@import "./phoenix.css";

I suspect most people missed this because they are further along past the prototype stage. YMMV.

@nbw
Copy link

nbw commented Dec 10, 2018

I ran into:

==> ecto_sql
Compiling 23 files (.ex)

== Compilation error in file lib/ecto/migration/schema_migration.ex ==
** (CompileError) lib/ecto/migration/schema_migration.ex:5: module Ecto.Schema is not loaded and could not be found
    (elixir) expanding macro: Kernel.use/1
    lib/ecto/migration/schema_migration.ex:5: Ecto.Migration.SchemaMigration (module)
could not compile dependency :ecto_sql, "mix compile" failed. You can recompile this dependency with "mix deps.compile ecto_sql", update it with "mix deps.update ecto_sql" or clean it with "mix deps.clean ecto_sql"

I cleaned all my deps and ran mix deps.compile and that got it to work.

@dokicro
Copy link

dokicro commented Dec 11, 2018

I am getting this warning warning: retrieving the :adapter from config files for Gaby.Repo is deprecated.

Solved by editing Repo.ex

from

use Ecto.Repo,
otp_app: :gaby

to

use Ecto.Repo,
otp_app: :gaby
adapter: Ecto.Adapters.Postgres

@jhollinger
Copy link

jhollinger commented Dec 13, 2018

I'm unable to use query or body params after the upgrade. The following code results in (ArgumentError) cannot fetch key "username" from conn.params because they were not fetched. When I inspect params they're a %Plug.Conn.Unfetched{aspect: :params}. Can't find anything on Google.

def login(conn, params) do
  username = params["username"]
  ...
end

Solved

Somehow the following got deleted from my endpoints.ex file:

plug Plug.Parsers,
  parsers: [:urlencoded, :multipart, :json],
  pass: ["*/*"],
  json_decoder: Jason

@tonnenpinguin
Copy link

We just spend two hours trying to figure out why our websockets suddenly stopped working. Turns out one has to update the nginx config when upgrading cowboy as described in the last comments here: phoenixframework/phoenix#3165

@chrismccord could you add a note?

@drozzy
Copy link

drozzy commented Jan 5, 2019

There should be a comma after:

websocket: true # or list of options

@kevinstueber
Copy link

kevinstueber commented Feb 5, 2019

@chrismccord It might be worth noting that any calls to use Phoenix.Socket should be removed from your channel modules.

@oshanz
Copy link

oshanz commented Feb 25, 2019

nice work @chrismccord. hope to see this on the phoenix site. shall I create a pull request to phoenixframework/phoenix_site#42 using this gist?

@mazz
Copy link

mazz commented Apr 19, 2019

To avoid this error right after webpack is watching the files:

ERROR in ./js/app.js
Module build failed (from ./node_modules/babel-loader/lib/index.js):
Error: Cannot find module 'babel-preset-env' from '/Users/michael/src/phx/Phoenix-File-Upload/assets'
- Did you mean "@babel/env"?
    at Function.module.exports [as sync] (/Users/michael/src/phx/Phoenix-File-Upload/assets/node_modules/resolve/lib/sync.js:58:15)

I needed to make this my .babelrc file:

{
  "presets": [
    "@babel/preset-env"
  ]
}

@x-ji
Copy link

x-ji commented May 15, 2019

Note that Phoenix and Ecto are separate packages. Phoenix 1.4 can also work with Ecto 2, so it's not mandatory that one upgrades to Ecto 3 (or to phoenix_ecto 4) along with the upgrade to Phoenix 1.4

@yellow5
Copy link

yellow5 commented Jun 10, 2019

It may be worth updating the Webpack config entry config to this:

  entry: {
      './js/app.js': glob.sync('./vendor/**/*.js').concat(['./js/app.js'])
  },

This makes sure that ./js/app.js is loaded last and that any code is available for export to any library configured in the Webpack output section.

Details can be found here (1).

(1) phoenixframework/phoenix#3436

@JonRowe
Copy link

JonRowe commented Jul 18, 2019

After upgrading my socket tests started failing with:

`(ArgumentError) argument error` from `:ets.lookup(MyApp.Endpoint, :__phoenix_pubsub_server__)`

Anyone have any ideas?

@gaynetdinov
Copy link

If your Phoenix 1.3 app streams HTTP responses using Plug.Conn.chunk/2, you should NOT upgrade to Phoenix 1.4 with Cowboy 2. See elixir-plug/plug_cowboy#10.

@BobbyMcWho
Copy link

If your Phoenix 1.3 app streams HTTP responses using Plug.Conn.chunk/2, you should NOT upgrade to Phoenix 1.4 with Cowboy 2. See elixir-plug/plug_cowboy#10.

This is resolved now if anyone comes across this and doesn't want to read through that issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment