Skip to content

Instantly share code, notes, and snippets.

@easytarget
Last active August 23, 2023 07:16
Show Gist options
  • Star 5 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save easytarget/fc7a6dabf893d1b461c5e66573d0725a to your computer and use it in GitHub Desktop.
Save easytarget/fc7a6dabf893d1b461c5e66573d0725a to your computer and use it in GitHub Desktop.
LaserWeb development install on Raspian Bullseye and Buster.

Machine Prep

Note; this log was generated on a Pi4 4Gb running from a SSD. However, I also test on a Pi3 Model B+ and a Pi3 Model A.

For support please go to the MakerForums at: https://forum.makerforums.info/c/laserweb-cncweb/

These Instructions should work for all Pi 3 and 4 models running the Buster and Bullseye releases. Also see the wiki page about Pi installation for more info and notes on how to run LaserWeb4 as a service.

Machines running Bullseye

This is nice and easy; node 12 is part of the RPI OS release.

pi@lwserver:~ $ uname -a
Linux lwserver.easytarget.org 5.10.63-v8+ #1488 SMP PREEMPT Thu Nov 18 16:16:16 GMT 2021 aarch64 GNU/Linux

pi@lwserver:~ $ lsb_release -a
No LSB modules are available.
Distributor ID: Raspbian
Description:    Raspbian GNU/Linux 11 (bullseye)
Release:        11
Codename:       bullseye

pi@lwserver:~ $ sudo apt update
...
All packages are up to date.

Install build tools, NodeJS and NPM on Bullseye

Note: if you have previously followed instructions given here showing you how to install Node10 and hold to that version, you should unpin node with sudo apt-mark unhold nodejs and then remove the node repos with sudo rm /etc/apt/sources.list.d/nodesource.list

pi@lwserver:~ $ sudo apt install build-essential nodejs npm
... lots of output, on my test system 180'ish small packages are installed.. should result in success

pi@lwserver:~ $ node -v
v12.22.5

pi@lwserver:~ $ npm -v
7.5.2

Machines running Buster

Unfortunately, Node 12 is not part of the RPI OS release (Node 10 is the standard, but we are not compatible with that) and so we need to use node 12 from nodesource.org

pi@buster:~ $ uname -a
Linux buster.easytarget.org 5.10.63-v7+ #1496 SMP Wed Dec 1 15:58:11 GMT 2021 armv7l GNU/Linux

pi@buster:~ $ lsb_release -a
No LSB modules are available.
Distributor ID: Raspbian
Description:    Raspbian GNU/Linux 10 (buster)
Release:        10
Codename:       buster

pi@buster:~ $ sudo apt update
...
All packages are up to date.

Install build tools, NodeJS and NPM on Buster

We will ensure some required tools are in place, then run the setup script from nodesource.com, once that is done we can install nodejs and npm using apt in the same manner as for bullseye

pi@buster:~ $ sudo apt install curl dirmngr apt-transport-https lsb-release ca-certificates
... it's quite liklely that these are already installed

pi@buster:~ $ curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash -
... the script will run, and configure apt to use the node12 repositories 

pi@buster:~ $ sudo apt install build-essential nodejs
... should only install a few packages and result in success

pi@buster:~ $ node -v
v12.22.8

pi@buster:~ $ npm -v
6.14.15

we also need an extra Node package not included by npm6 by standard
pi@buster:~ $ npm install ip
+ ip@1.1.5
... and more info about npm package states

Download the repos from GitHub

These repos are somewhat large, the clone operations may be slow

pi@lwserver:~ $ ls
Desktop  Documents  Downloads  lwinst.txt  Music  Pictures  Public  sandify  Templates  Universal-G-Code-Sender  Videos

pi@lwserver:~ $ git clone https://github.com/LaserWeb/LaserWeb4.git
OR, if you have a GitHub account: git clone git@github.com:LaserWeb/LaserWeb4.git
Cloning into 'LaserWeb4'...

pi@lwserver:~ $ git clone https://github.com/LaserWeb/lw.comm-server.git
OR, if you have a GitHub account: git clone git@github.com:LaserWeb/lw.comm-server.git
Cloning into 'lw.comm-server'...

pi@lwserver:~ $ ls
Desktop    Downloads  lw.comm-server  lw-setup.txt  Pictures  sandify    Universal-G-Code-Sender
Documents  LaserWeb4  lwinst.txt      Music         Public    Templates  Videos

pi@lwserver:~ $ ls lw.comm-server/
app        create_env.bat  docker_entrypoint.sh  grblStrings.js          package.json       server.js
conf.env   dist            Dockerfile            LICENSE                 package-lock.json  version.txt
config.js  doc             firmwareFeatures.js   lw.comm-server.service  README.md

pi@lwserver:~ $ ls LaserWeb4/
dist        docs  jsdoc.json      LICENSE.md   package.json       README.md  webpack.config.js
Dockerfile  git   laserweb@0.4.0  logfile.txt  package-lock.json  src        win.shell.cmd

pi@lwserver:~ $ cd LaserWeb4/
pi@lwserver:~/LaserWeb4 $ git branch
* dev-es6
Here we are double-checking we are on 'dev-es6'; the current default branch. 

Now we must initilise and update the LW4 submodules
pi@lwserver:~/LaserWeb4 $ git submodule init
Submodule 'src/data/lw.machines' (https://github.com/LaserWeb/lw.machines) registered for path 'src/data/lw.machines'
Submodule 'src/data/lw.materials' (https://github.com/LaserWeb/lw.materials.git) registered for path 'src/data/lw.materials'

pi@lwserver:~/LaserWeb4 $ git submodule update
Cloning into '/home/pi/LaserWeb/LaserWeb4/src/data/lw.machines'...
Cloning into '/home/pi/LaserWeb/LaserWeb4/src/data/lw.materials'...
Submodule path 'src/data/lw.machines': checked out '685d9de193400a7bcf35d921eda21e4bedfbdc7b'
Submodule path 'src/data/lw.materials': checked out 'dce9f9ae104030e192a9716f095988dc33c0c0cd'

lw communications server

Install lw.comm.server requirements (node modules)

pi@lwserver:~ $ cd ~/lw.comm-server/

pi@lwserver:~/lw.comm-server $ npm install
... lots of output

On my Pi3B+ this takes: 3m23s, on my Pi4/4Gb+ssd this takes: 39s.
On a first run approximately 14Mb of packages will be downloaded and cached

Test lw.comm.server

The lw.comm-erver will bundle an older version of LaserWeb4 by defaut, but this is OK to test with

pi@lwserver:~/lw.comm-server $ node server
***************************************************************
        ---- LaserWeb Comm Server 4.1.000 ----        
***************************************************************
  Use  http://127.0.1.1:8000 to connect this server.

* Updates: 
  Remember to check the commit log on
  https://github.com/LaserWeb/lw.comm-server/commits/master
  regularly, to know about updates and fixes, and then when ready
  update accordingly by running git pull
 
* Support: 
  If you need help / support, come over to 
  https://forum.makerforums.info/c/laserweb-cncweb/78
***************************************************************

App connected! (id=0)
INFO: Requesting Server Config 
^C                                     <-- exit by pressing ctrl-c
pi@lwserver:~/lw.comm-server $

Update LaserWeb4

Not always needed; the version of LaserWeb4 bundled with lw.comm-server is very recent.

Build LaserWeb4

First we build laserweb4 and it's dependencies, then we create a new bundle for exporting to the comm-server

pi@lwserver:~/lw.comm-server $ cd ~/LaserWeb4/

pi@lwserver:~/LaserWeb4 $ npm install
... lots of output
On my Pi3B+ this takes: 11m13s, on my Pi4/4Gb+ssd this takes: 4m02s
On a first run approximately 130Mb of packages will be downloaded and cached

Generate a distribution (aka Bundle) the server build

The results end up in the dist folder of the repo

pi@lwserver:~/LaserWeb4 $ npm run bundle-dev
... lots of output
On my Pi3B+ this takes: 2m19.803s, on my Pi4/4Gb+ssd this takes: 1m07s
No additional data is downloaded for this

Copy bundle into lw.comms.server

This is done from within the lw.comm-server, it expects to find the new bundle in ../LaserWeb4

pi@lwserver:~/LaserWeb4 $ cd ../lw.comm-server/
pi@lwserver:~/lw.comm-server $ npm run update_frontend
... lots of output

Run the result

pi@lwserver:~/lw.comm-server $ node server

***************************************************************
        ---- LaserWeb Comm Server 4.1.000 ----        
***************************************************************
  Use  http://127.0.1.1:8000 to connect this server.
 
* Updates: 
  Remember to check the commit log on
  https://github.com/LaserWeb/lw.comm-server/commits/master
  regularly, to know about updates and fixes, and then when ready
  update accordingly by running git pull
 
* Support: 
  If you need help / support, come over to 
  https://forum.makerforums.info/c/laserweb-cncweb/78
***************************************************************

App connected! (id=0)
INFO: Requesting Server Config 

Use

Browse to http://lwserver:8000/ from the target workstation

Updating:

Pull the updates in using git

pi@lwserver:~ $ cd ~/lw.comm-server/
pi@lwserver:~/lw.comm-server $ git pull
Already up to date.

pi@lwserver:~/lw.comm-server $ cd ~/LaserWeb4/
pi@lwserver:~/LaserWeb4 $ git pull
Already up to date.

If any updates show up you can then re-run the Build steps again to obtain them.

Notes:

Lots of Warnings in builds

The lw.comm-server and LaserWeb4 npm builds generate a lot of warnings and depreciation notices. As of the time of this document (12 Aug 2020) none of these resulted in a failure, but they deo look quite ominous.

If you are a JS/Node developer looking for a challenge, this would be a good target for some assistance in the project.

Caveat for Pi3 model A (and Pi2's ?)

I have successfully followed these instructions on my 'real' target system; which is a Pi3+ model A; with only 512Mb of memory it struggled to complete the npm builds, but was successful in the end and serves LW4 very well.

The trick on low memory systems is that the npm actions fail due to low memory. But this is because npm uses a lot of memory while building, and doesnt free it up during the build. Repeatedly re-starting the build after such failures generally allows npm to eventually complete. The main LaserWeb4 build took four restarts, each time getting further through the build process until the final reatart was successful.

@cprezzi
Copy link

cprezzi commented Aug 13, 2020

Thank you very much for writing this down!

@SiegeMstr
Copy link

I am getting lots of errors, then eventually end with it not installing.
node server.js output>>

internal/modules/cjs/loader.js:905
throw err;
^

Error: Cannot find module 'serialport'
Require stack:

  • /home/pi/lw.comm-server/server.js
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:902:15)
    at Function.Module._load (internal/modules/cjs/loader.js:746:27)
    at Module.require (internal/modules/cjs/loader.js:974:19)
    at require (internal/modules/cjs/helpers.js:93:18)
    at Object. (/home/pi/lw.comm-server/server.js:29:20)
    at Module._compile (internal/modules/cjs/loader.js:1085:14)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
    at Module.load (internal/modules/cjs/loader.js:950:32)
    at Function.Module._load (internal/modules/cjs/loader.js:790:12)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:76:12) {
    code: 'MODULE_NOT_FOUND',
    requireStack: [ '/home/pi/lw.comm-server/server.js' ]
    }

@cprezzi
Copy link

cprezzi commented Oct 12, 2021

@SiegeMstr Did you run npm install in lw.comm-server folder?

@easytarget
Copy link
Author

Thankyou @cprezzi , that was just what I was going to suggest too.

  • it is always possible that these instructions break (with time and OS/Node updates), and I haven't been able to give LW/lw-comms the attention they deserve this year due to personal reasons, but I am still interested in any issues people find and plan to return to this once the current crises are over.

@easytarget
Copy link
Author

easytarget commented Nov 23, 2021

Looks like Nodejs 12 is available as standard in the Pi repos now for both Buster and Bullseye and this gets installed in preference to Node10 even after you have run the setup script.
As a tempory solution I added instructions on how to downgrade node
Ideal solution would be to get this working in Node12 (even better, Node16-LTS, but that's quite a big jump)
lw.comm-server now builds and runs under node12, instructions above updated!

@heili77
Copy link

heili77 commented Feb 14, 2022

I have issues at step test lw.comms.server

pi@raspberrypi:~/lw.comm-server $ node server

***************************************************************
        ---- LaserWeb Comm Server 4.1.000 ----
***************************************************************
  Use  http://192.168.188.34:8000 to connect to this server.

* Updates:
  Remember to check the commit log on
  https://github.com/LaserWeb/lw.comm-server/commits/master
  regularly, to know about updates and fixes, and then when ready
  update accordingly by running git pull

* Support:
  If you need help / support, come over to
  https://forum.makerforums.info/c/laserweb-cncweb/78
***************************************************************

Server binding to all local IP addresses on port: 8000
internal/modules/cjs/loader.js:818
  throw err;
  ^

Error: Cannot find module 'node-hid'
Require stack:
- /home/pi/lw.comm-server/server.js
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:815:15)
    at Function.Module._load (internal/modules/cjs/loader.js:667:27)
    at Module.require (internal/modules/cjs/loader.js:887:19)
    at require (internal/modules/cjs/helpers.js:74:18)
    at Object.exports.LWCommServer (/home/pi/lw.comm-server/server.js:168:13)
    at Object.<anonymous> (/home/pi/lw.comm-server/server.js:4141:13)
    at Module._compile (internal/modules/cjs/loader.js:999:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
    at Module.load (internal/modules/cjs/loader.js:863:32)
    at Function.Module._load (internal/modules/cjs/loader.js:708:14) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [ '/home/pi/lw.comm-server/server.js' ]
}

installing node-hid also does not work

pi@raspberrypi:~/lw.comm-server $ npm install node-hid --save
npm ERR! code 1
npm ERR! path /home/pi/lw.comm-server/node_modules/node-hid
npm ERR! command failed
npm ERR! command sh -c prebuild-install --runtime napi || node-gyp rebuild
npm ERR! prebuild-install WARN install No prebuilt binaries found (target=3 runtime=napi arch=arm64 libc= platform=linux)
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using node-gyp@7.1.2
npm ERR! gyp info using node@12.22.5 | linux | arm64
npm ERR! gyp info find Python using Python version 3.9.2 found at "/usr/bin/python3"
npm ERR! gyp info spawn /usr/bin/python3
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args   '/usr/share/nodejs/node-gyp/gyp/gyp_main.py',
npm ERR! gyp info spawn args   'binding.gyp',
npm ERR! gyp info spawn args   '-f',
npm ERR! gyp info spawn args   'make',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/home/pi/lw.comm-server/node_modules/node-hid/build/config.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/usr/share/nodejs/node-gyp/addon.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/usr/include/nodejs/common.gypi',
npm ERR! gyp info spawn args   '-Dlibrary=shared_library',
npm ERR! gyp info spawn args   '-Dvisibility=default',
npm ERR! gyp info spawn args   '-Dnode_root_dir=/usr/include/nodejs',
npm ERR! gyp info spawn args   '-Dnode_gyp_dir=/usr/share/nodejs/node-gyp',
npm ERR! gyp info spawn args   '-Dnode_lib_file=/usr/include/nodejs/<(target_arch)/node.lib',
npm ERR! gyp info spawn args   '-Dmodule_root_dir=/home/pi/lw.comm-server/node_modules/node-hid',
npm ERR! gyp info spawn args   '-Dnode_engine=v8',
npm ERR! gyp info spawn args   '--depth=.',
npm ERR! gyp info spawn args   '--no-parallel',
npm ERR! gyp info spawn args   '--generator-output',
npm ERR! gyp info spawn args   'build',
npm ERR! gyp info spawn args   '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! Package libusb-1.0 was not found in the pkg-config search path.
npm ERR! Perhaps you should add the directory containing `libusb-1.0.pc'
npm ERR! to the PKG_CONFIG_PATH environment variable
npm ERR! No package 'libusb-1.0' found
npm ERR! gyp: Call to 'pkg-config libusb-1.0 --cflags-only-I | sed s/-I//g' returned exit status 0 while in binding.gyp. while trying to load binding.gyp
npm ERR! gyp ERR! configure error
npm ERR! gyp ERR! stack Error: `gyp` failed with exit code: 1
npm ERR! gyp ERR! stack     at ChildProcess.onCpExit (/usr/share/nodejs/node-gyp/lib/configure.js:355:16)
npm ERR! gyp ERR! stack     at ChildProcess.emit (events.js:314:20)
npm ERR! gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:276:12)
npm ERR! gyp ERR! System Linux 5.10.92-v8+
npm ERR! gyp ERR! command "/usr/bin/node" "/usr/share/nodejs/node-gyp/bin/node-gyp.js" "rebuild"
npm ERR! gyp ERR! cwd /home/pi/lw.comm-server/node_modules/node-hid
npm ERR! gyp ERR! node -v v12.22.5
npm ERR! gyp ERR! node-gyp -v v7.1.2
npm ERR! gyp ERR! not ok

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/pi/.npm/_logs/2022-02-14T15_31_51_185Z-debug.log

@easytarget
Copy link
Author

easytarget commented Feb 14, 2022

humm.. two seperate errors I suspect.
@cprezzi just added some XHC HB04 MPG related commits; these make node-hid a requirement for lw.comm-server. But it is not yet in the package.json dependencies.

So what you did; npm install node-hid is correct; but it looks like that is failing due to missing build tools?

  • What architecture are you on (Raspbian version)?
  • Did you run apt install build-essential as part of the setup?

@easytarget
Copy link
Author

easytarget commented Feb 14, 2022

Aaah. I see you are using 64-bit raspbian.

When I install node-hid on either buster or bullseye (32-bit) a prebuilt module is downloaded and used.

However; it looks like there is no prebuild module for node-hid/arm64 in the repos; so your machine tries to build it locally and fails.

The operative error here is:
npm ERR! Package libusb-1.0 was not found in the pkg-config search path.

You probably need to apt install libusb-dev

  • but.. the npm install node-hid may still fail, there may be other -dev dependencies you are missing. Or there may be other build failures. You need to pursue those with the npm package maintainers and googling the erorors etc. it's pretty hard for me to pre-guess all the possible ways this can fail.

Later in the year I might set up a arm64 environment to test with, but I suspect you are paying an 'early adopters' price here.

@heili77
Copy link

heili77 commented Feb 14, 2022

Thanks for the quick analysis!
If I get the answer right, with 32bit os there should be all packages available. I will try this.

@heili77
Copy link

heili77 commented Feb 14, 2022

With 32bit os the same error shows up

@easytarget
Copy link
Author

Did you install libusb-dev?

The important part is:

npm ERR! prebuild-install WARN install No prebuilt binaries found (target=3 runtime=napi arch=arm64 libc= platform=linux)

This is saying that NPM does not have a prebuilt package for arm64 and will attempt to build one,
and you then get very conventional build errors as the compilation of node-hid fails due to a missing library (libusb-dev)

You will need to work out why npm is not installing node-hid properly on your system, we cant help with that in detail; every system and situation is different.

start at: https://github.com/node-hid/node-hid

For Reference: This is what happens on a Pi4 running 32-bit Raspbian Bullseye after I clear the cache and modules
Click to expand!
pi@marigold:~/LaserWeb-Upstream/lw.comm-server $ rm -rf ~/.npm/
pi@marigold:~/LaserWeb-Upstream/lw.comm-server $ rm -r node_modules/ package.json 

pi@marigold:~/LaserWeb-Upstream/lw.comm-server $ npm list node-hid
/home/pi/LaserWeb-Upstream/lw.comm-server
└── (empty)

pi@marigold:~/LaserWeb-Upstream/lw.comm-server $ npm install node-hid

added 64 packages, and audited 65 packages in 8s

6 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

pi@marigold:~/LaserWeb-Upstream/lw.comm-server $ npm list node-hid
lw.comm-server@ /home/pi/LaserWeb-Upstream/lw.comm-server
└── node-hid@2.1.1

pi@marigold:~/LaserWeb-Upstream/lw.comm-server $ npm --version
8.3.1

pi@marigold:~/LaserWeb-Upstream/lw.comm-server $ node --version
v16.14.0

pi@marigold:~/LaserWeb-Upstream/lw.comm-server $ npm show node-hid

node-hid@2.1.1 | (MIT OR X11) | deps: 3 | versions: 39
USB HID device access library
https://github.com/node-hid/node-hid#readme

bin: hid-showdevices

dist
.tarball: https://registry.npmjs.org/node-hid/-/node-hid-2.1.1.tgz
.shasum: f83c8aa0bb4e6758b5f7383542477da93f67359d
.integrity: sha512-Skzhqow7hyLZU93eIPthM9yjot9lszg9xrKxESleEs05V2NcbUptZc5HFqzjOkSmL0sFlZFr3kmvaYebx06wrw==
.unpackedSize: 662.5 kB

dependencies:
bindings: ^1.5.0         node-addon-api: ^3.0.2   prebuild-install: ^6.0.0 

maintainers:
- bminer <miner.blake@gmail.com>
- gorillatron <jornandretangen@gmail.com>
- hans <hans.huebner@gmail.com>
- natevw <natevw@yahoo.com>
- todbot <todbotdotcom@gmail.com>

dist-tags:
latest: 2.1.1  

published a year ago by todbot <todbotdotcom@gmail.com>
pi@marigold:~/LaserWeb-Upstream/lw.comm-server $ 

@cprezzi
Copy link

cprezzi commented Feb 15, 2022

I have added the missing dependency (node-hid) to lw.comm-server (master) package.json.
A quick test on MAC OS 10.15.7 with node 16.13.0 and npm 8.1.2 and on Windows 10 21H1 with node 14.19.0 / npm 6.14.16 or node 16.13.0 / npm 8.1.0 worked.
With node 16.14.0 / npm 8.3.1 on Windows I got an error on npm install. There seems to be a problem in npm 8.3.1.

For an update it might be needed to delete the node_modules folder before running npm install.

@pjwebster
Copy link

On low memory systems, such as Raspberry Pi's, you can stop the npm build from eating all the RAM by setting the max_old_space_size

On a RPi3 with 1G RAM I specified half the memory:

export NODE_OPTIONS=--max_old_space_size=500

Add that to the bottom of your .bashrc to make it permanent

@TechnicalJohn
Copy link

TechnicalJohn commented Dec 25, 2022

The operative error here is: npm ERR! Package libusb-1.0 was not found in the pkg-config search path.

You probably need to apt install libusb-dev

Just a note... I did a clean install of Raspbian OS and I needed to do this. Except specifically I needed:
sudo apt install libusb-1.0-0-dev

So you may want to add that to the "sudo apt install build-essential nodejs npm" line so it's sure to be installed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment