refack & tniessen about MSI

aboutMSI.irclog

[2017-07-13 09:24:50] <tniessen> Hi, are you there?
[2017-07-13 09:44:07] <refack> Yep
[2017-07-13 09:44:40] <refack> Now with irccloud I'm always there
[2017-07-13 09:46:00] <tniessen> Did we get a license?
[2017-07-13 09:46:33] <refack> not yet, but I subscribed...
[2017-07-13 09:47:45] <tniessen> Mhhh okay, seems like even their free option is okay, but only remains connected for 2 hours
[2017-07-13 09:48:01] <tniessen> Anyway, about the Windows installer... I might need to change the structure quite a bit
[2017-07-13 09:48:04] <refack> I was using mIRC with a license I bought some 15 years ago
[2017-07-13 09:48:30] <tniessen> I think it makes sense to put the npm node_modules folder into a truly global directory within Program Files
[2017-07-13 09:48:49] <refack> "optionaly"?
[2017-07-13 09:48:55] <tniessen> (btw, nodejs will be installed to C:\Program Files\nodejs\1.2.3 now)
[2017-07-13 09:49:10] <refack> all seems reasonalbe
[2017-07-13 09:49:15] <tniessen> I considered making it optional, but then it will clash with other versions
[2017-07-13 09:49:52] <refack> the global "node_modules"? there should be a "just for me" vs "all users"
[2017-07-13 09:50:33] <tniessen> Sure, the installer should let the user choose between "just for me" and "all users", but if he picks "all users", then the node_modules folder should be in Program Files as well, don't you think so?
[2017-07-13 09:50:43] <refack> Yes
[2017-07-13 09:51:13] <tniessen> Okay, so installing multiple versions should work and I am currently working on the npm folders
[2017-07-13 09:51:23] <refack> Maybe ProgramData? I'm not sure what the Microsoft guide lines are
[2017-07-13 09:51:35] <tniessen> I will look into that
[2017-07-13 09:51:59] <tniessen> Allowing both user-local and system-wide installation is not trivial, but AFAIK we don't support anything before Win7, that makes things simpler
[2017-07-13 09:52:07] <refack> Since it's downloadable code, some security considerations
[2017-07-13 09:52:19] <refack> YEs
[2017-07-13 09:52:24] <refack> no EOL platforms
[2017-07-13 09:52:32] <tniessen> If installed globally, only admins can install software for all users
[2017-07-13 09:52:38] <refack> yes
[2017-07-13 09:53:15] <tniessen> ProgramData: Unlike the Program Files folder, this folder can be used by applications to store data for standard users, because it does not require elevated permissions.
[2017-07-13 09:53:24] <tniessen> This does not sound like we should put node_modules there
[2017-07-13 09:53:50] <tniessen> Otherwise, every user could install arbitrary code
[2017-07-13 09:54:03] <tniessen> And put it into the PATH of others
[2017-07-13 09:54:03] <refack> I'll read more... since that will not allow non-admins to do `npm i -g`
[2017-07-13 09:54:29] <tniessen> It won't, but I think we don't want to permit arbitrary users to install packages globally?
[2017-07-13 09:54:54] <refack> sounds right, but let's think about it more
[2017-07-13 09:55:22] <tniessen> Non-admin users can still install either node or the modules locally, add them to his / her own PATH
[2017-07-13 09:55:29] <refack> if you set npm with `prefer globals` (where it installs -g and uses symlink
[2017-07-13 09:55:39] <refack> all `npm i` will fail
[2017-07-13 09:56:20] <tniessen> I didnt know about that option
[2017-07-13 09:56:44] <tniessen> It is possible to specify a prefix for node_modules for npm, so users could also use that option
[2017-07-13 09:57:21] <tniessen> According to npm, preferGlobal is deprecated and does not do anything anymore
[2017-07-13 09:58:15] <refack> npm config set link true
[2017-07-13 09:58:31] <refack> https://docs.npmjs.com/misc/config#link
[2017-07-13 09:59:26] <tniessen> but the same thing would fail on Linux right?
[2017-07-13 09:59:33] <refack> yep
[2017-07-13 09:59:34] <tniessen> I cannot do npm install -g on Linux without being root either
[2017-07-13 09:59:42] <refack> good
[2017-07-13 09:59:42] <tniessen> So this is kind of consistent
[2017-07-13 09:59:49] <refack> agreed
[2017-07-13 10:00:15] <tniessen> Okay, let's just assume for a moment that the general approach is valid, we can discuss this once I get to a PR
[2017-07-13 10:00:15] <refack> so %programfiles% it is
[2017-07-13 10:00:41] <refack> 👍
[2017-07-13 10:01:19] <tniessen> Within the node installation folder, there is stuff like node.exe, npm.cmd etc. For npm to work, we need a folder within the installation folder which can then contain node_modules. However, we cannot use the installation directory itself (which would be great)
[2017-07-13 10:01:42] <tniessen> Because packages could contain a binary called "node.exe" and would overwrite important files
[2017-07-13 10:02:00] <tniessen> So we need a subfolder within the installation dir
[2017-07-13 10:02:10] <refack> are you sure?
[2017-07-13 10:02:16] <refack> that's how I work
[2017-07-13 10:02:43] <tniessen> But npm will put the binary wrappers into that directory
[2017-07-13 10:02:45] <refack> it won't work simple because `npm i` runs node.exe so it's locked
[2017-07-13 10:03:13] <tniessen> Okay, then let's assume node.exe is locked, they can still overwrite nodevars.bat
[2017-07-13 10:03:20] <tniessen> or anything else in that directory
[2017-07-13 10:03:42] <tniessen> The problem is that npm installs both bash and cmd wrappers
[2017-07-13 10:03:54] <tniessen> And bash wrappers lack an extension, allowing packages to use arbitrary file names
[2017-07-13 10:03:55] <refack> AFAIK all you need is `node.exe` `npm.cmd` and `/etc/npmrc`
[2017-07-13 10:04:25] <refack> what is `nodevars.bat`?
[2017-07-13 10:04:46] <tniessen> Did you use the installer or zip?
[2017-07-13 10:05:15] <refack> neither 😄 I download only the binary
[2017-07-13 10:05:53] <refack> open an `npm` bug for that, we should not jump through hoops for that
[2017-07-13 10:06:15] <tniessen> Both the installer and the zip contain additional files: http://imgur.com/a/6JWg3
[2017-07-13 10:06:26] <refack> that does not sound like a "completely valid name for a binary shim"
[2017-07-13 10:07:08] <refack> that's malicious 
[2017-07-13 10:07:23] <tniessen> It is intentional, the installer contains specific code for those files
[2017-07-13 10:07:42] <tniessen> I assume man pages are for bash on Windows or something
[2017-07-13 10:07:55] <tniessen> that's our code, not npm's
[2017-07-13 10:08:04] <refack> no I mean writing a packadge.json with {bin : node.exe}
[2017-07-13 10:08:17] <refack> or {bin: nodevars.bat}
[2017-07-13 10:09:17] <tniessen> Yeah you might be right, I would still prefer to avoid any potentional there. Alternatively, we could create a folder npm within the installation dir and move npm.cmd, npm and node_modules there.
[2017-07-13 10:09:21] <tniessen> That should solve the problem
[2017-07-13 10:09:35] <tniessen> We would need to add two directories to the path, but the old installer did that anyway
[2017-07-13 10:10:59] <refack> I kind of like the "naive" approach... but it's not global, it's per version
[2017-07-13 10:11:41] <tniessen> I thought we wanted to keep node_modules separated per version
[2017-07-13 10:11:43] <refack> we could do
[2017-07-13 10:11:43] <refack> nodejs\
[2017-07-13 10:11:43] <refack>                 1.2.3\
[2017-07-13 10:11:43] <refack>                 node_modules\
[2017-07-13 10:11:43] <refack>                 4.5.6\
[2017-07-13 10:12:01] <refack> that's an open quastion...
[2017-07-13 10:12:08] <tniessen> That would add binaries into the folder nodejs\
[2017-07-13 10:12:17] <refack> yes
[2017-07-13 10:12:45] <refack> "binary shims"
[2017-07-13 10:12:48] <tniessen> If we go for that, we could as well provide a separate installer for NPM and put it directly into C:\Program Files\npm
[2017-07-13 10:13:00] <refack> ohhh that's nice
[2017-07-13 10:13:45] <tniessen> But let's say version A of a module is compatible with node X, but not node Y
[2017-07-13 10:13:46] <refack> and optionally have 
[2017-07-13 10:13:46] <refack> \npm\
[2017-07-13 10:13:46] <refack>              1.2.3\
[2017-07-13 10:13:46] <refack>              4.5.6\
[2017-07-13 10:14:02] <tniessen> And version B is compatible with Y but not X
[2017-07-13 10:14:11] <refack> what does nvm do?
[2017-07-13 10:14:12] <tniessen> Then there is no way for users to install the correct versions
[2017-07-13 10:14:22] <tniessen> I never used nvm for Windows, only on Linux
[2017-07-13 10:14:29] <refack> on linux
[2017-07-13 10:14:46] <tniessen> nvm installs everything user-local and just reconfigures your shell
[2017-07-13 10:14:54] <tniessen> it is strictly restricted to your session
[2017-07-13 10:15:07] <refack> we can loop in ljharb
[2017-07-13 10:15:43] <tniessen> nvm keeps node_modules per node version Afaik
[2017-07-13 10:15:59] <refack> https://github.com/creationix/nvm#migrating-global-packages-while-installing
[2017-07-13 10:16:36] <tniessen> node has always been very conservative about compatibility
[2017-07-13 10:16:47] <tniessen> that's why we use per-module node_modules folders
[2017-07-13 10:16:54] <tniessen> instead of a single global folder
[2017-07-13 10:17:10] <tniessen> nvm seems to use the right way by making node_modules per-version
[2017-07-13 10:17:45] <refack> Personaly I'd add that as an option, that how I work.  https://usercontent.irccloud-cdn.com/file/Tb6VW5bY/image.png
[2017-07-13 10:18:10] <refack> never had problems
[2017-07-13 10:18:33] <tniessen> sure, in practice everything should work fine because node is 99% backwards compatible, but not 100%
[2017-07-13 10:19:15] <tniessen> (I still think it is amazing that JAVA has been backwards-compatible for 20 years or something like that.)
[2017-07-13 10:19:19] <refack> yes, hence an installer option (if it's not too much work)
[2017-07-13 10:20:31] <refack> that's because java has a very small surface area, it's STDLIB is more libuv then node
[2017-07-13 10:21:14] <tniessen> But if a user decides to use a truly global npm installation for one version, and he installs a second node version, what should happen? Should he still be able to pick installation-local this time?
[2017-07-13 10:21:44] <tniessen> What if he picks global again? Should we then check whether the npm module in the global node_modules is outdated and replace it with the new one contained in the installer?
[2017-07-13 10:22:30] <refack> ..............
[2017-07-13 10:22:31] <refack> ...............
[2017-07-13 10:22:32] <refack> .............
[2017-07-13 10:22:36] <refack> I retract
[2017-07-13 10:22:52] <refack> put node_modules in the 1.2.3\ folder
[2017-07-13 10:23:17] <refack> and we'll think about "truely global" (a.k.a refack style) in the future
[2017-07-13 10:23:40] <refack> but
[2017-07-13 10:24:00] <refack> May have a `--reinstall-packages-from=node` option in the installer
[2017-07-13 10:24:17] <tniessen> The only alterative I can think of is to make npm installation optional and provide a separate installer for npm, so npm is really independent of node, if users really want that
[2017-07-13 10:24:28] <tniessen> "May have a `--reinstall-packages-from=node` option in the installer" thats a really good idea in that case
[2017-07-13 10:25:00] <refack> otherwise you get an empty node_modules
[2017-07-13 10:25:15] <tniessen> + our bundled npm package ?
[2017-07-13 10:25:31] <refack> yes...
[2017-07-13 10:25:42] <refack> "vanilla" not empty
[2017-07-13 10:26:32] <refack> I sort of like the idea of a separate installer for npm, makes separation of responsibility more clear
[2017-07-13 10:27:17] <refack> Question: how do I switch versions? New tool?
[2017-07-13 10:27:44] <tniessen> We would need to provide a tool, yes, maybe as a npm package :D
[2017-07-13 10:27:55] <tniessen> Currently, you can just select whether the new version should be added to the path
[2017-07-13 10:28:13] <refack> yes I was thinking about adopting an existing package manager
[2017-07-13 10:28:41] <refack> symlink node123.exe => \1.2.3\node.exe
[2017-07-13 10:29:02] <tniessen> That's actually a nice idea
[2017-07-13 10:29:17] <refack> so you have one "deafult" node.exe and all other are callable with version number
[2017-07-13 10:29:19] <refack> like python
[2017-07-13 10:29:33] <tniessen> Yes... Just need to decide where those symlinks should be
[2017-07-13 10:30:56] <refack> "C:\WINDOWS\system32" 😄
[2017-07-13 10:31:30] <tniessen> I know that java does (or did) that
[2017-07-13 10:31:43] <refack> going AFK for 5 minutes
[2017-07-13 10:31:46] <tniessen> Sure
[2017-07-13 10:33:24] <tniessen> That should probably not be a task for the installer, but rather for a separate tool
[2017-07-13 10:40:32] <refack> Ok. You can go to "installed programs" and run "change" on a different version to re-set PATH
[2017-07-13 10:40:47] <tniessen> Yep
[2017-07-13 10:41:05] <tniessen> I wonder what happens when I change it manually, I hope Windows won't run into problems
[2017-07-13 10:41:57] <refack> And we should recommend https://github.com/jasongin/nvs 
[2017-07-13 10:42:48] <refack> Don't think so windows does not re-validate installeres unless you explicitly click "fix"
[2017-07-13 10:43:27] <refack> I change PATH and registry values all the time. Windows doesn't care
[2017-07-13 10:43:31] <tniessen> So it will just ignore it if it cannot find it within the PATH
[2017-07-13 10:43:42] <tniessen> Okay, that sounds good
[2017-07-13 10:45:20] <refack> so:
[2017-07-13 10:45:20] <refack> 1. %PROGRAMFILES%\nodejs\1.2.3\node_modules
[2017-07-13 10:45:20] <refack> 2. open issue for `npm` against malicious bins
[2017-07-13 10:45:20] <refack> 3. switch versions through "installed programs"
<refack> Refael Ackermann I want to upload the log of this convo to the GitHub issue, any objections?
<tniessen> None from me, sorry for the late reply, was really busy today
<refack> Refael Ackermann NP