Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Building ROS Melodic with python3 support
# quick gist to accompany post on building ROS Melodic w/python3 support
# https://www.miguelalonsojr.com/blog/robotics/ros/python3/2019/08/20/ros-melodic-python-3-build.html
# Warning: don't try to run this as a script. It probably won't work.
# remove all things python (optional)
sudo apt-get remove python-*
# remove previous installations of ROS
sudo apt-get remove ros-*
sudo apt-get remove ros-melodic-*
sudo apt-get autoremove
# setup python3 dependencies
sudo apt update
sudo apt install -y python3 python3-dev python3-pip build-essential
sudo -H pip3 install rosdep rospkg rosinstall_generator rosinstall wstool vcstools catkin_tools catkin_pkg
# initializ catkin build environment
sudo rosdep init
rosdep update
cd ~
mkdir ros_catkin_ws
cd ros_catkin_ws
# TODO: get rqt_rviz rviz_plugin_tutorials librviz_tutorial building w/python3
catkin config --init -DCMAKE_BUILD_TYPE=Release --blacklist rqt_rviz rviz_plugin_tutorials librviz_tutorial --install
# setup ROS install
rosinstall_generator desktop_full --rosdistro melodic --deps --tar > melodic-desktop-full.rosinstall
wstool init -j8 src melodic-desktop-full.rosinstall
# if packages fail to download, run the following
wstool update -j4 -t src
# setup environment and install dependencies
export ROS_PYTHON_VERSION=3
pip3 install -U -f https://extras.wxpython.org/wxPython4/extras/linux/gtk3/ubuntu-18.04 wxPython
# create the following script in the ros_catkin_ws directory and call it install_skip
#!/bin/bash
#Check whether root
if [ $(whoami) != root ]; then
echo You must be root or use sudo to install packages.
return
fi
#Call apt-get for each package
for pkg in "$@"
do
echo "Installing $pkg"
sudo apt-get -my install $pkg >> install.log
done
# and chmod it
chmod +x install_skip
# sed magic
sudo ./install_skip `rosdep check --from-paths src --ignore-src | grep python | sed -e "s/^apt\t//g" | sed -z "s/\n/ /g" | sed -e "s/python/python3/g"`
rosdep install --from-paths src --ignore-src -y --skip-keys="`rosdep check --from-paths src --ignore-src | grep python | sed -e "s/^apt\t//g" | sed -z "s/\n/ /g"`"
find . -type f -exec sed -i 's/\/usr\/bin\/env[ ]*python/\/usr\/bin\/env python3/g' {} +
# build ROS
catkin build
@zzh112119

This comment has been minimized.

Copy link

@zzh112119 zzh112119 commented Oct 22, 2019

(Moved this from the blog for history preservation purpose)
Two errors were encountered during setting up with this tutorial:

  1. When catkin_make:
File "/usr/lib/python3.6/signal.py", line 47, in signal
TypeError: signal handler must be signal.SIG_IGN, signal.SIG_DFL, or a callable object

This will not affect launching roscore just as you posted before.

  1. When rosrun somepacakge somecode.py:
rosrun topic_demo pytalker.py
Traceback (most recent call last):
File "/root/catkin_ws/src/ROS-Academy-for-Beginners/topic_demo/scripts/pytalker.py", line 3, in <module>
import rospy
File "/root/ros_catkin_ws/install/lib/python3/dist-packages/rospy/__init__.py", line 47, in <module>
from std_msgs.msg import Header
File "/root/ros_catkin_ws/install/lib/python3/dist-packages/std_msgs/msg/__init__.py", line 1, in <module>
from ._Bool import *
File "/root/ros_catkin_ws/install/lib/python3/dist-packages/std_msgs/msg/_Bool.py", line 5, in <module>
import genpy
File "/root/ros_catkin_ws/install/lib/python3/dist-packages/genpy/__init__.py", line 34, in <module>
from . message import Message, SerializationError, DeserializationError, MessageException, struct_I
File "/root/ros_catkin_ws/install/lib/python3/dist-packages/genpy/message.py", line 44, in <module>
import yaml
File "/usr/lib/python3/dist-packages/yaml/__init__.py", line 284
class YAMLObject(metaclass=YAMLObjectMetaclass):
^
SyntaxError: invalid syntax

This is an interesting error.. Python seems to be complaining about the python3's metaclass syntax here. And with the traceback log, ROS is trying to run with python3 here but somehow ended up using python2 when the error happens.

Does anyone have any insight?

@drmaj

This comment has been minimized.

Copy link
Owner Author

@drmaj drmaj commented Oct 23, 2019

@zzh112119 Do you have another version of rosrun installed, perhaps the standard melodic install? You may want to check your paths to make sure which rosrun is being used if that's the case. Also, is the shebang in pytalker.py using python3? Also check genpy's message.py. It should be using python3 not python2. I recall seeing this issue in the past and it was related to the shebang.

@antietam76

This comment has been minimized.

Copy link

@antietam76 antietam76 commented Oct 24, 2019

Everything worked great until I tried to build it!

[build] Summary: 0 of 204 packages succeeded.                                  
[build]   Ignored:   33 packages were skipped or are blacklisted.              
[build]   Warnings:  None.                                                     
[build]   Abandoned: 203 packages were abandoned.                              
[build]   Failed:    1 packages failed.                                        
[build] Runtime: 2.7 seconds total.                                            
Exception ignored in: <bound method BaseEventLoop.__del__ of <_UnixSelectorEventLoop running=False closed=True debug=False>>
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/trollius/base_events.py", line 395, in __del__
  File "/usr/local/lib/python3.6/dist-packages/trollius/unix_events.py", line 65, in close
  File "/usr/local/lib/python3.6/dist-packages/trollius/unix_events.py", line 166, in remove_signal_handler
  File "/usr/lib/python3.6/signal.py", line 47, in signal
TypeError: signal handler must be signal.SIG_IGN, signal.SIG_DFL, or a callable object

What could have happened? this is on a fresh install of ubuntu.
Thanks!

@drmaj

This comment has been minimized.

Copy link
Owner Author

@drmaj drmaj commented Oct 24, 2019

@Antietam , please post the error received for the package that failed to build. Also, be sure to read the blog post https://www.miguelalonsojr.com/blog/robotics/ros/python3/2019/08/20/ros-melodic-python-3-build.html and follow the instructions there.

@drmaj

This comment has been minimized.

Copy link
Owner Author

@drmaj drmaj commented Oct 24, 2019

@zzh112119 Also, if you post the python script that you are trying to run, I can try to see what's going on.

@antietam76

This comment has been minimized.

Copy link

@antietam76 antietam76 commented Oct 24, 2019

@drmaj The link you shared is broken.

I re-fresh-installed ubuntu and have only followed your guide. I get to catkin build and it seems to fail on different packages everytime. the most common on it fails on it the dynamic_reconfigure

Errors     << dynamic_reconfigure:make /home/jacobksu/ros_catkin_ws/logs/dynamic_reconfigure/build.make.004.log
Traceback (most recent call last):
  File "/home/jacobksu/ros_catkin_ws/src/dynamic_reconfigure/cfg/Test.cfg", line 37, in <module>
    from dynamic_reconfigure.parameter_generator_catkin import *
  File "/home/jacobksu/ros_catkin_ws/devel/.private/dynamic_reconfigure/lib/python3/dist-packages/dynamic_reconfigure/__init__.py", line 35, in <module>
    exec(__fh.read())
  File "<string>", line 38, in <module>
  File "/home/jacobksu/ros_catkin_ws/install/lib/python3/dist-packages/roslib/__init__.py", line 50, in <module>
    from roslib.launcher import load_manifest
  File "/home/jacobksu/ros_catkin_ws/install/lib/python3/dist-packages/roslib/launcher.py", line 42, in <module>
    import rospkg
ImportError: No module named rospkg
make[2]: *** [/home/jacobksu/ros_catkin_ws/devel/.private/dynamic_reconfigure/include/dynamic_reconfigure/TestConfig.h] Error 1
make[1]: *** [CMakeFiles/dynamic_reconfigure_gencfg.dir/all] Error 2
make: *** [all] Error 2
cd /home/jacobksu/ros_catkin_ws/build/dynamic_reconfigure; catkin build --get-env dynamic_reconfigure | catkin env -si  /usr/bin/make --jobserver-fds=3,4 -j; cd -
...............................................................................
Failed     << dynamic_reconfigure:make                        [ Exited with code 2 ]

thanks again!

@zzh112119

This comment has been minimized.

Copy link

@zzh112119 zzh112119 commented Oct 24, 2019

@drmaj I was actually able to find out the problematic shebang in my own code..

And yes! ROS seems to be working properly with python3 env and code! (I installed ros_base instead of desktop_full version tho)

But I noticed that during the step sudo ./install_skip ......., python2.7, python2.7-minimal and libpython2.7:amd64 were installed and python was linked to python2.7 by default. And when I try to dpkg --purge the python2.7 out of my system, I found out there's a dependency tree that originated from libboost-all-dev. Do you know if there's a good way to totally get rid out python2 in the system during installation of ros?

@drmaj

This comment has been minimized.

Copy link
Owner Author

@drmaj drmaj commented Oct 25, 2019

@zzh112119 It seems like the only way is to a) build boost against python3 from scratch b) ignore the dependencies when installing boost, or c) use equivs to trick apt. Here's a more detailed description: https://askubuntu.com/questions/944035/installing-libboost-python-dev-for-python3-without-installing-python2-7 . Even then, any of these solutions may not work with my build steps. I haven't explored this since the python2 dependencies don't cause any trouble for me.

@muhyilmaz

This comment has been minimized.

Copy link

@muhyilmaz muhyilmaz commented Nov 1, 2019

When i use
$ rosdep update

I get this error :

"

komodo@komodo-desktop:~/ros_catkin_ws$ rosdep update
bash: /home/komodo/.local/bin/rosdep: /usr/bin/python: bad interpreter: No such file or directory

"

@drmaj

This comment has been minimized.

Copy link
Owner Author

@drmaj drmaj commented Nov 1, 2019

@MYComodo It seems like you have a different version of rosdep on your path. My rosdep with this build is under /usr/local/bin. Also, you can't just run rosdep. It will try to use the default melodic apt packages. You should use the following:

rosdep [COMMAND] --from-paths src --ignore-src -y --skip-keys="rosdep check --from-paths src --ignore-src | grep python | sed -e "s/^apt\t//g" | sed -z "s/\n/ /g""

where [COMMAND] is what you want to do with rosdep, e.g. install, update, check, etc. This will skip all of the python based package dependencies. You'll then have to satisfy those dependencies manually by cloning those ros packages into your workspace and building them along with your packages. It's the only way I've found to reliably use ros with python3. Hopefully, with the next ros release, this won't be an issue any longer, since python is reaching end of life in 2020.

@muhyilmaz

This comment has been minimized.

Copy link

@muhyilmaz muhyilmaz commented Nov 1, 2019

@MYComodo It seems like you have a different version of rosdep on your path. My rosdep with this build is under /usr/local/bin. Also, you can't just run rosdep. It will try to use the default melodic apt packages. You should use the following:

rosdep [COMMAND] --from-paths src --ignore-src -y --skip-keys="rosdep check --from-paths src --ignore-src | grep python | sed -e "s/^apt\t//g" | sed -z "s/\n/ /g""

where [COMMAND] is what you want to do with rosdep, e.g. install, update, check, etc. This will skip all of the python based package dependencies. You'll then have to satisfy those dependencies manually by cloning those ros packages into your workspace and building them along with your packages. It's the only way I've found to reliably use ros with python3. Hopefully, with the next ros release, this won't be an issue any longer, since python is reaching end of life in 2020.

i tried this code but it did not work. In my OS i am still using python3 and i have python2 too. This python3 can be the problem for this issue. Should i delete python3 too?

@drmaj

This comment has been minimized.

Copy link
Owner Author

@drmaj drmaj commented Nov 4, 2019

@MYComodo It seems like you have a different version of rosdep on your path. My rosdep with this build is under /usr/local/bin. Also, you can't just run rosdep. It will try to use the default melodic apt packages. You should use the following:
rosdep [COMMAND] --from-paths src --ignore-src -y --skip-keys="rosdep check --from-paths src --ignore-src | grep python | sed -e "s/^apt\t//g" | sed -z "s/\n/ /g""
where [COMMAND] is what you want to do with rosdep, e.g. install, update, check, etc. This will skip all of the python based package dependencies. You'll then have to satisfy those dependencies manually by cloning those ros packages into your workspace and building them along with your packages. It's the only way I've found to reliably use ros with python3. Hopefully, with the next ros release, this won't be an issue any longer, since python is reaching end of life in 2020.

i tried this code but it did not work. In my OS i am still using python3 and i have python2 too. This python3 can be the problem for this issue. Should i delete python3 too?

No, this technique requires a python3 installation. If anything, you should remove python2, not python3. the problem seems to be the version of rosdep that you have installed. You should remove any old setup for ros, remove python2, and then try again.

@kiselevsky

This comment has been minimized.

Copy link

@kiselevsky kiselevsky commented Nov 11, 2019

I'm trying to create a docker image using your instructions. And I have several problems:

  1. I can't use "SED magic" when I create image from Dockerfile due to file access issues:
sed: couldn't open temporary file ./sys/kernel/sedSaprdt: Read-only file system
sed: couldn't open temporary file ./sys/devices/pci0000:00/0000:00:05.0/ata17/host16/scsi_host/host16/sedJXIaWt: Read-only file system
sed: couldn't open temporary file ./sys/devices/system/clockevents/clockevent1/power/sed8nct7s: Read-only file system
sed: couldn't open temporary file ./sys/devices/virtual/block/nbd8/power/sedvpMyXO: Read-only file system
sed: couldn't open temporary file ./proc/sedLPWOHY: No such file or directory
sed: cannot rename ./etc/sedBGZwH9: Device or resource busy
  1. it seems that during the execution of the ./install_skip ... there is a duplication of 3 in the name of some packages, because of which their installation is skipped:
Installing python3-rospkg
Installing python33-lxml
E: Unable to locate package python33-lxml
Installing python33-yaml
E: Unable to locate package python33-yaml
Installing python33-mock
E: Unable to locate package python33-mock
Installing python33-catkin-pkg
E: Unable to locate package python33-catkin-pkg
Installing python33-rosdep
E: Unable to locate package python33-rosdep
Installing python33-coverage
E: Unable to locate package python33-coverage
Installing python33-rospkg
etc ...

But after checking the logs and manual installation it seems that everything is installed correctly... Is this a mistake?

  1. I managed to successfully build the ROS melodic in a running container, but roscore and other default pkgs work correctly only with default PYTHONPATH=/root/ros_catkin_ws/install/lib/python3/dist-packages/. Exporting the PYTHONPATH=/usr/lib/python3/dist-packages results in the following error:
Traceback (most recent call last):
  File "/root/ros_catkin_ws/install/bin/roscore", line 36, in <module>
    from rosmaster.master_api import NUM_WORKERS
ModuleNotFoundError: No module named 'rosmaster'
  1. But these were all solvable problems compared to the following. When trying to call catkin_make in my catkin_ws project directory, the following error occurs:
/root/ros_catkin_ws/install/share/catkin/cmake/em/order_packages.cmake.em:23: error: <class 'AssertionError'>: Unable to handle package.xml format version '3', please update catkin_pkg (e.g. on Ubuntu/Debian use: sudo apt-get update && sudo apt-get install --only-upgrade python-catkin-pkg)
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/em.py", line 3302, in <module>
    if __name__ == '__main__': main()
  File "/usr/lib/python3/dist-packages/em.py", line 3300, in main
    invoke(sys.argv[1:])
  File "/usr/lib/python3/dist-packages/em.py", line 3283, in invoke
    interpreter.wrap(interpreter.file, (file, name))
  File "/usr/lib/python3/dist-packages/em.py", line 2295, in wrap
    self.fail(e)
  File "/usr/lib/python3/dist-packages/em.py", line 2284, in wrap
    callable(*args)
  File "/usr/lib/python3/dist-packages/em.py", line 2359, in file
    self.safe(scanner, done, locals)
  File "/usr/lib/python3/dist-packages/em.py", line 2401, in safe
    self.parse(scanner, locals)
  File "/usr/lib/python3/dist-packages/em.py", line 2421, in parse
    token.run(self, locals)
  File "/usr/lib/python3/dist-packages/em.py", line 1425, in run
    interpreter.execute(self.code, locals)
  File "/usr/lib/python3/dist-packages/em.py", line 2595, in execute
    _exec(statements, self.globals, locals)
  File "<string>", line 17, in <module>
  File "/usr/lib/python3/dist-packages/catkin_pkg/topological_order.py", line 124, in topological_order
    for path, package in find_packages(space).items():
  File "/usr/lib/python3/dist-packages/catkin_pkg/packages.py", line 86, in find_packages
    packages = find_packages_allowing_duplicates(basepath, exclude_paths=exclude_paths, exclude_subspaces=exclude_subspaces, warnings=warnings)
  File "/usr/lib/python3/dist-packages/catkin_pkg/packages.py", line 146, in find_packages_allowing_duplicates
    xml, filename=filename, warnings=warnings)
  File "/usr/lib/python3/dist-packages/catkin_pkg/package.py", line 457, in parse_package_string
    assert pkg.package_format in [1, 2], "Unable to handle package.xml format version '%d', please update catkin_pkg (e.g. on Ubuntu/Debian use: sudo apt-get update && sudo apt-get install --only-upgrade python-catkin-pkg)" % pkg.package_format
AssertionError: Unable to handle package.xml format version '3', please update catkin_pkg (e.g. on Ubuntu/Debian use: sudo apt-get update && sudo apt-get install --only-upgrade python-catkin-pkg)
CMake Error at /root/ros_catkin_ws/install/share/catkin/cmake/safe_execute_process.cmake:11 (message):
  execute_process(/home/test_ws/build/catkin_generated/env_cached.sh
  "/usr/bin/python3" "/usr/lib/python3/dist-packages/em.py" "--raw-errors"
  "-F" "/home/test_ws/build/catkin_generated/order_packages.py" "-o"
  "/home/test_ws/build/catkin_generated/order_packages.cmake"
  "/root/ros_catkin_ws/install/share/catkin/cmake/em/order_packages.cmake.em")
  returned error code 1
Call Stack (most recent call first):
  /root/ros_catkin_ws/install/share/catkin/cmake/em_expand.cmake:25 (safe_execute_process)
  /root/ros_catkin_ws/install/share/catkin/cmake/catkin_workspace.cmake:35 (em_expand)
  CMakeLists.txt:67 (catkin_workspace)


-- Configuring incomplete, errors occurred!
See also "/home/test_ws/build/CMakeFiles/CMakeOutput.log".
See also "/home/test_ws/build/CMakeFiles/CMakeError.log".
Invoking "cmake" failed

All actions were performed in a container based on pure ubuntu:latest image. And I'm trying to build ROS-Comm (Bare Bones) This is my script:

    apt-get update && \
    apt-get install -y apt-utils && \
    apt-get install -y python3 python3-dev python3-pip build-essential && \
    pip3 install rosdep rospkg rosinstall_generator rosinstall wstool vcstools catkin_tools catkin_pkg && \
    rosdep init && \
    rosdep update && \
    cd ~ && \
    mkdir ros_catkin_ws && \
    cd ros_catkin_ws
    catkin config --init -DCMAKE_BUILD_TYPE=Release --blacklist rqt_rviz rviz_plugin_tutorials librviz_tutorial --install && \
    rosinstall_generator ros_comm --rosdistro melodic --deps --tar > melodic-ros_comm.rosinstall && \
    wstool init -j8 src melodic-ros_comm.rosinstall && \
    wstool update -j4 -t src && \
    export ROS_PYTHON_VERSION=3 && \
    pip3 install -U -f https://extras.wxpython.org/wxPython4/extras/linux/gtk3/ubuntu-18.04 wxPython && \
    echo '#!/bin/bash' >> install_skip && \
    echo 'for pkg in "$@"' >> install_skip && \
    echo 'do' >> install_skip && \
    echo '    echo "Installing $pkg"' >> install_skip && \
    echo '    DEBIAN_FRONTEND=noninteractive apt-get -my install $pkg >> install.log' >> install_skip && \
    echo 'done' >> install_skip && \
    chmod +x install_skip && \
    ./install_skip `rosdep check --from-paths src --ignore-src | grep python | sed -e "s/^apt\t//g" | sed -z "s/\n/ /g" | sed -e "s/python/python3/g"` && \
    rosdep install --from-paths src --ignore-src -y --skip-keys="`rosdep check --from-paths src --ignore-src | grep python | sed -e "s/^apt\t//g" | sed -z "s/\n/ /g"`" && \
    find . -type f -exec sed -i 's/\/usr\/bin\/env[ ]*python/\/usr\/bin\/env python3/g' {} + && \
    catkin build

I will be very grateful for help in this matter!

@erelson

This comment has been minimized.

Copy link

@erelson erelson commented Nov 18, 2019

I too ran into the python33 errors like in the previous comment. Without digging fulling into what strings are being replaced, perhaps tweaking sed -e "s/python/python3/g" with added dashes: sed -e "s/python-/python3-/g" makes sense?

@kiselevsky: I believe your issue (1) may be due to ~ in a docker likely pointing to / instead of /home/$username as it would be on a regular install.

@saifullah3396

This comment has been minimized.

Copy link

@saifullah3396 saifullah3396 commented Jan 3, 2020

@zzh112119 It seems like the only way is to a) build boost against python3 from scratch b) ignore the dependencies when installing boost, or c) use equivs to trick apt. Here's a more detailed description: https://askubuntu.com/questions/944035/installing-libboost-python-dev-for-python3-without-installing-python2-7 . Even then, any of these solutions may not work with my build steps. I haven't explored this since the python2 dependencies don't cause any trouble for me.

Installing boost with python3 from source works!

A few additional errors that I faced along with their solutions are given in following:
caktin_tools trollius error on build: trollius has a dependency on python-2 so installing caktin_tools from pip3 is still dependent on python3-trollius which has bugs of its own. The easiest solution for this was just to build latest catkin-tools from source which replaces trollius with asyncio.

AttributeError: 'bytes' object has no attribute 'read' runtime error on running roscore. I'm not sure why this happens, but I believe the previous version of shlex required bytestrings to be passed to its functions which now require simple strings. So I just removed the conversion from string to bytestring wherever shlex split() function was used raising this error. The files that I updated are following:

  1. ros_comm/roslaunch/src/roslaunch/loader.py. (Just remove the conversion of the command string to byte string)
  2. ros_comm/roslaunch/src/roslaunch/node_args.py.

One addition that I did while building the source files was that I didn't only replace the shebangs but used the 2to3 automated python script for converting all the source files to python3. There were a few files with wrong print syntaxes that were easily fixed with the script. The script can be called as follows.

#!/bin/bash
# File: convert.sh
find src/ -name '*.py' -type f -exec 2to3 -w -n {} \;
@anthonysimeonov

This comment has been minimized.

Copy link

@anthonysimeonov anthonysimeonov commented Feb 5, 2020

@kiselevsky

  1. But these were all solvable problems compared to the following. When trying to call catkin_make in my catkin_ws project directory, the following error occurs:
/root/ros_catkin_ws/install/share/catkin/cmake/em/order_packages.cmake.em:23: error: <class 'AssertionError'>: Unable to handle package.xml format version '3', please update catkin_pkg (e.g. on Ubuntu/Debian use: sudo apt-get update && sudo apt-get install --only-upgrade python-catkin-pkg)
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/em.py", line 3302, in <module>
    if __name__ == '__main__': main()
  File "/usr/lib/python3/dist-packages/em.py", line 3300, in main
    invoke(sys.argv[1:])
  File "/usr/lib/python3/dist-packages/em.py", line 3283, in invoke
    interpreter.wrap(interpreter.file, (file, name))
  File "/usr/lib/python3/dist-packages/em.py", line 2295, in wrap
    self.fail(e)
  File "/usr/lib/python3/dist-packages/em.py", line 2284, in wrap
    callable(*args)
  File "/usr/lib/python3/dist-packages/em.py", line 2359, in file
    self.safe(scanner, done, locals)
  File "/usr/lib/python3/dist-packages/em.py", line 2401, in safe
    self.parse(scanner, locals)
  File "/usr/lib/python3/dist-packages/em.py", line 2421, in parse
    token.run(self, locals)
  File "/usr/lib/python3/dist-packages/em.py", line 1425, in run
    interpreter.execute(self.code, locals)
  File "/usr/lib/python3/dist-packages/em.py", line 2595, in execute
    _exec(statements, self.globals, locals)
  File "<string>", line 17, in <module>
  File "/usr/lib/python3/dist-packages/catkin_pkg/topological_order.py", line 124, in topological_order
    for path, package in find_packages(space).items():
  File "/usr/lib/python3/dist-packages/catkin_pkg/packages.py", line 86, in find_packages
    packages = find_packages_allowing_duplicates(basepath, exclude_paths=exclude_paths, exclude_subspaces=exclude_subspaces, warnings=warnings)
  File "/usr/lib/python3/dist-packages/catkin_pkg/packages.py", line 146, in find_packages_allowing_duplicates
    xml, filename=filename, warnings=warnings)
  File "/usr/lib/python3/dist-packages/catkin_pkg/package.py", line 457, in parse_package_string
    assert pkg.package_format in [1, 2], "Unable to handle package.xml format version '%d', please update catkin_pkg (e.g. on Ubuntu/Debian use: sudo apt-get update && sudo apt-get install --only-upgrade python-catkin-pkg)" % pkg.package_format
AssertionError: Unable to handle package.xml format version '3', please update catkin_pkg (e.g. on Ubuntu/Debian use: sudo apt-get update && sudo apt-get install --only-upgrade python-catkin-pkg)
CMake Error at /root/ros_catkin_ws/install/share/catkin/cmake/safe_execute_process.cmake:11 (message):
  execute_process(/home/test_ws/build/catkin_generated/env_cached.sh
  "/usr/bin/python3" "/usr/lib/python3/dist-packages/em.py" "--raw-errors"
  "-F" "/home/test_ws/build/catkin_generated/order_packages.py" "-o"
  "/home/test_ws/build/catkin_generated/order_packages.cmake"
  "/root/ros_catkin_ws/install/share/catkin/cmake/em/order_packages.cmake.em")
  returned error code 1
Call Stack (most recent call first):
  /root/ros_catkin_ws/install/share/catkin/cmake/em_expand.cmake:25 (safe_execute_process)
  /root/ros_catkin_ws/install/share/catkin/cmake/catkin_workspace.cmake:35 (em_expand)
  CMakeLists.txt:67 (catkin_workspace)


-- Configuring incomplete, errors occurred!
See also "/home/test_ws/build/CMakeFiles/CMakeOutput.log".
See also "/home/test_ws/build/CMakeFiles/CMakeError.log".
Invoking "cmake" failed

Any updates here on a resolution? We're trying to do the same thing and ran into the same problem, except we're building with catkin build. Will appreciate any help!

@erelson

This comment has been minimized.

Copy link

@erelson erelson commented Feb 12, 2020

Unfortunatley I don't have a dockerfile handy, but I did get to the point where I had a better understanding of how to get a python3 package to build. The key thing that I did was to do it in docker: (1) started from the ubuntu:18.04 base image (2) figure out what dependencies were needed while (3) working towards building the base ros packages from source with the -DPYTHON_VERSION=3 flag, (4) then sourcing this python3-built workspace and building my workspace of interest, again with the -DPYTHON_VERSION=3 flag.

@Karthik-Ranga

This comment has been minimized.

Copy link

@Karthik-Ranga Karthik-Ranga commented Feb 12, 2020

I completed all the ROS build, when catkin_make was called, I got the below error.

  File "/home/karthik/ros_catkin_ws/install/bin/catkin_make", line 13, in <module>
    from catkin.terminal_color import disable_ANSI_colors, fmt
  File "/home/karthik/ros_catkin_ws/install/lib/python3/dist-packages/catkin/terminal_color.py", line 2, in <module>
    from catkin_pkg.terminal_color import *  # noqa
ModuleNotFoundError: No module named 'catkin_pkg.terminal_color'

has anyone solved this problem?

@jaypee92

This comment has been minimized.

Copy link

@jaypee92 jaypee92 commented Feb 14, 2020

build success.
but roscore does not start

setting /run_id to a19b38b4-4f0e-11ea-95bf-e8d8d16102f5
ERROR: cannot launch node of type [rosout/rosout]: can't locate node [rosout] in package [rosout]
RLException: failed to start core service [/rosout]
The traceback for the exception was written to the log file
[master] killing on exit

@saifullah3396

This comment has been minimized.

Copy link

@saifullah3396 saifullah3396 commented Feb 15, 2020

I know this goes against the thread but for anyone who wants to use ROS with python3 its better to just build your own project with python3 instead of building the whole ROS. You can build any project with -DPYTHON_VERSION=3 and ROS_PYTHON_VERSION=3. If you want to use anything that depends on python3, just rebuild that specific package with python3 as well. I tried this solution and it worked but I had to rebuild every project every time which was not worth it. Instead, when I just built my own project with python3 it worked completely well with original ros, meaning there are no conflicts running python3 and ros together since internally ros is already updated to work with python3 when you set the flag ROS_PYTHON_VERSION=3.

Cheers.

@Karthik-Ranga

This comment has been minimized.

Copy link

@Karthik-Ranga Karthik-Ranga commented Feb 18, 2020

I know this goes against the thread but for anyone who wants to use ROS with python3 its better to just build your own project with python3 instead of building the whole ROS. You can build any project with -DPYTHON_VERSION=3 and ROS_PYTHON_VERSION=3. If you want to use anything that depends on python3, just rebuild that specific package with python3 as well. I tried this solution and it worked but I had to rebuild every project every time which was not worth it. Instead, when I just built my own project with python3 it worked completely well with original ros, meaning there are no conflicts running python3 and ros together since internally ros is already updated to work with python3 when you set the flag ROS_PYTHON_VERSION=3.

Cheers.

Can you elaborate by what you mean? I am not sure of the process.

@saifullah3396

This comment has been minimized.

Copy link

@saifullah3396 saifullah3396 commented Feb 18, 2020

I know this goes against the thread but for anyone who wants to use ROS with python3 its better to just build your own project with python3 instead of building the whole ROS. You can build any project with -DPYTHON_VERSION=3 and ROS_PYTHON_VERSION=3. If you want to use anything that depends on python3, just rebuild that specific package with python3 as well. I tried this solution and it worked but I had to rebuild every project every time which was not worth it. Instead, when I just built my own project with python3 it worked completely well with original ros, meaning there are no conflicts running python3 and ros together since internally ros is already updated to work with python3 when you set the flag ROS_PYTHON_VERSION=3.
Cheers.

Can you elaborate by what you mean? I am not sure of the process.

I will just list down the steps (tested on ROS/melodic Ubuntu 18.04):

  1. Make a virtual env and install python3 within it. This is important since without the virtual env ubuntu (that comes with prebuilt python3 can cause conflicts with python2 used by ros).
  2. Activate the virtual env.
  3. Make a catkin workspace as usual for your custom project based on python3.
  4. Set the environment variable ROS_PYTHON_VERSION=3
  5. Build your project with catkin_make --cmake-args -DPYTHON_VERSION=3
  6. Call launch files, nodes, etc as usual.
@Karthik-Ranga

This comment has been minimized.

Copy link

@Karthik-Ranga Karthik-Ranga commented Feb 19, 2020

Thanks for your time. I did try the above steps, and I got the below error

Makefile:140: recipe for target 'all' failed
make: *** [all] Error 2
Invoking "make -j12 -l12" failed

I guess the package has lot of dependencies which could not be resolved. I tried tracing back to all the dependency but it is a huge mess.

@sharpe-developer

This comment has been minimized.

Copy link

@sharpe-developer sharpe-developer commented Feb 28, 2020

@drmaj Thank you for the detailed explanation.

The ROS team has started migrating towards python3 (for melodic on Ubuntu 18.04) which has caused an error in your sed command. The ROS team now has some instances of python3 within the install. Using sed to find the string python without using exact match causes sed to find instances of python which are in the string python3. Then when python is replaced with python3 the result is python33 which is not the desired result.

Changing the sed command to use exact match \<python\> fixes the issue.

In the sed magic section on your blog you state:

Be sure to only run this once. If you run this twice by mistake, you might end up with shebangs that have python33 instead of python3.

Using exact match also allows the command to be run multiple times without any problems. No need to warn users to "Be sure to only run this once."

The modified sed commands are shown below.

sudo ./install_skip `rosdep check --from-paths src --ignore-src | grep python | sed -e "s/^apt\t//g" | sed -z "s/\n/ /g" | sed -e "s/\<python\>/python3/g"`

find . -type f -exec sed -i 's/\/usr\/bin\/env[ ]*\<python\>/\/usr\/bin\/env python3/g' {} +
@yuanzhedong

This comment has been minimized.

Copy link

@yuanzhedong yuanzhedong commented Mar 11, 2020

@sharp-developer, thank you for the script, it helps!

By the way do you have a timeline about the ROS python2->python3 migration? I'm trying to use the TimeSynchronizer which works on python2 but somehow failed on python3.

@sharpe-developer

This comment has been minimized.

Copy link

@sharpe-developer sharpe-developer commented Mar 11, 2020

@yuanzhedong Glad I could help!

I ended up reverting back as I also ran into various issues. It's my understanding that the ROS Noetic will target only Python 3. It is scheduled to be released on May 23rd, 2020.

@AabedSolayman

This comment has been minimized.

Copy link

@AabedSolayman AabedSolayman commented Mar 11, 2020

@sharp-developer
Hi! your solution helped, but still I get the same error when I build.
"TypeError: signal handler must be signal.SIG_IGN, signal.SIG_DFL, or a callable object"
anyway after I used your solution 13 of 204 packages were succeded before that all the packages were abandoned or skipped

@nkueng

This comment has been minimized.

Copy link

@nkueng nkueng commented Mar 11, 2020

Noob here. Trying to run catkin config --init -DCMAKE_BUILD_TYPE=Release --blacklist rqt_rviz rviz_plugin_tutorials librviz_tutorial --install, I get the following error:

Traceback (most recent call last):
  File "/usr/local/bin/catkin", line 6, in <module>
    from catkin_tools.commands.catkin import main
  File "/usr/local/lib/python3.7/dist-packages/catkin_tools/commands/catkin.py", line 28, in <module>
    from catkin_tools.common import is_tty
  File "/usr/local/lib/python3.7/dist-packages/catkin_tools/common.py", line 23, in <module>
    import trollius as asyncio
  File "/usr/local/lib/python3.7/dist-packages/trollius/__init__.py", line 21, in <module>
    from .base_events import *
  File "/usr/local/lib/python3.7/dist-packages/trollius/base_events.py", line 42, in <module>
    from . import tasks
  File "/usr/local/lib/python3.7/dist-packages/trollius/tasks.py", line 565
    def async(coro_or_future, loop=None):
            ^
SyntaxError: invalid syntax

I am trying to follow the instructions on a Raspberry Pi running Raspbian Buster Lite and had previously installed Ros Melodic from source according to this tutorial. Before trying the instructions above, I simply deleted my existing ros_catkin_ws and catkin_ws.

Thank you very much for any help!

@sharpe-developer

This comment has been minimized.

Copy link

@sharpe-developer sharpe-developer commented Mar 12, 2020

@AabedSolayman @nkueng I was able to get through the entire process with the following changes (if my memory serves me correctly)

  • sed commands as described above
  • I skipped the first couple sections (removing Python and ROS) and started at the section # setup python3 dependencies

I started with a clean install of Ubuntu 18.04 (virtual machine)

That being said I ended up abandoning this build because I ran into more issues after finishing the build. Given the fact that Python 3 support will be available in just a few months in the next ROS release it wasn't worth the trouble.

@yev-d

This comment has been minimized.

Copy link

@yev-d yev-d commented Mar 31, 2020

I got the same errors as @antietam76, when performing catkin build. Are there any solutions to this?


Errors     << catkin:cmake /home/jek/ros_catkin_ws/logs/catkin/build.cmake.000.log                  
CMake Error at /home/jek/ros_catkin_ws/src/catkin/cmake/empy.cmake:29 (message):
  Unable to find either executable 'empy' or Python module 'em'...  try
  installing the package 'python-empy
Call Stack (most recent call first):
  cmake/all.cmake:164 (include)`
  CMakeLists.txt:8 (include)
cd /home/jek/ros_catkin_ws/build/catkin; catkin build --get-env catkin | catkin env -si  /usr/bin/cmake /home/jek/ros_catkin_ws/src/catkin --no-warn-unused-cli -DCATKIN_DEVEL_PREFIX=/home/jek/ros_catkin_ws/devel/.private/catkin -DCMAKE_INSTALL_PREFIX=/home/jek/ros_catkin_ws/install -DCMAKE_BUILD_TYPE=Release; cd -
....................................................................................................
Failed     << catkin:cmake                                    [ Exited with code 1 ]                
Failed    <<< catkin                                          [ 3.7 seconds ]                       
Abandoned <<< genmsg                                          [ Unrelated job failed ]

and so on for all 204 packages until the error summary:

[build] Summary: 0 of 204 packages succeeded.                                                       
[build]   Ignored:   33 packages were skipped or are blacklisted.                                   
[build]   Warnings:  None.                                                                          
[build]   Abandoned: 203 packages were abandoned.                                                   
[build]   Failed:    1 packages failed.                                                             
[build] Runtime: 6.7 seconds total.                                                                 
[build] Note: Workspace packages have changed, please re-source setup files to use them.
Exception ignored in: <bound method BaseEventLoop.__del__ of <_UnixSelectorEventLoop running=False closed=True debug=False>>
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/trollius/base_events.py", line 395, in __del__
  File "/usr/local/lib/python3.6/dist-packages/trollius/unix_events.py", line 65, in close
  File "/usr/local/lib/python3.6/dist-packages/trollius/unix_events.py", line 166, in remove_signal_handler
  File "/usr/lib/python3.6/signal.py", line 47, in signal
TypeError: signal handler must be signal.SIG_IGN, signal.SIG_DFL, or a callable object

So after installing python-empy and also catkin_pkg, the errors I get are a bit different. Since this time it doesn't ask to install anything, I'm at a loss.

Errors     << catkin:install /home/jek/ros_catkin_ws/logs/catkin/build.install.000.log               
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: setup.py --help [cmd1 cmd2 ...]
   or: setup.py --help-commands
   or: setup.py cmd --help

error: option --install-layout not recognized
CMake Error at catkin_generated/safe_execute_install.cmake:4 (message):
  
  execute_process(/home/jek/ros_catkin_ws/build/catkin/catkin_generated/python_distutils_install.sh)
  returned error code
Call Stack (most recent call first):
  cmake_install.cmake:171 (include)

make: *** [install] Error 1
cd /home/jek/ros_catkin_ws/build/catkin; catkin build --get-env catkin | catkin env -si  /usr/bin/make install; cd -
.....................................................................................................

I'm on Ubuntu 18.04.4 LTS
Python 3.7.3
ROS-Melodic

@metalglove

This comment has been minimized.

Copy link

@metalglove metalglove commented Apr 1, 2020

I got the same errors as @antietam76, when performing catkin build. Are there any solutions to this?

Errors     << catkin:cmake /home/jek/ros_catkin_ws/logs/catkin/build.cmake.000.log                  
CMake Error at /home/jek/ros_catkin_ws/src/catkin/cmake/empy.cmake:29 (message):
  Unable to find either executable 'empy' or Python module 'em'...  try
  installing the package 'python-empy
Call Stack (most recent call first):
  cmake/all.cmake:164 (include)`
  CMakeLists.txt:8 (include)
cd /home/jek/ros_catkin_ws/build/catkin; catkin build --get-env catkin | catkin env -si  /usr/bin/cmake /home/jek/ros_catkin_ws/src/catkin --no-warn-unused-cli -DCATKIN_DEVEL_PREFIX=/home/jek/ros_catkin_ws/devel/.private/catkin -DCMAKE_INSTALL_PREFIX=/home/jek/ros_catkin_ws/install -DCMAKE_BUILD_TYPE=Release; cd -
....................................................................................................
Failed     << catkin:cmake                                    [ Exited with code 1 ]                
Failed    <<< catkin                                          [ 3.7 seconds ]                       
Abandoned <<< genmsg                                          [ Unrelated job failed ]

and so on for all 204 packages until the error summary:

[build] Summary: 0 of 204 packages succeeded.                                                       
[build]   Ignored:   33 packages were skipped or are blacklisted.                                   
[build]   Warnings:  None.                                                                          
[build]   Abandoned: 203 packages were abandoned.                                                   
[build]   Failed:    1 packages failed.                                                             
[build] Runtime: 6.7 seconds total.                                                                 
[build] Note: Workspace packages have changed, please re-source setup files to use them.
Exception ignored in: <bound method BaseEventLoop.__del__ of <_UnixSelectorEventLoop running=False closed=True debug=False>>
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/trollius/base_events.py", line 395, in __del__
  File "/usr/local/lib/python3.6/dist-packages/trollius/unix_events.py", line 65, in close
  File "/usr/local/lib/python3.6/dist-packages/trollius/unix_events.py", line 166, in remove_signal_handler
  File "/usr/lib/python3.6/signal.py", line 47, in signal
TypeError: signal handler must be signal.SIG_IGN, signal.SIG_DFL, or a callable object

So after installing python-empy and also catkin_pkg, the errors I get are a bit different. Since this time it doesn't ask to install anything, I'm at a loss.

Errors     << catkin:install /home/jek/ros_catkin_ws/logs/catkin/build.install.000.log               
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: setup.py --help [cmd1 cmd2 ...]
   or: setup.py --help-commands
   or: setup.py cmd --help

error: option --install-layout not recognized
CMake Error at catkin_generated/safe_execute_install.cmake:4 (message):
  
  execute_process(/home/jek/ros_catkin_ws/build/catkin/catkin_generated/python_distutils_install.sh)
  returned error code
Call Stack (most recent call first):
  cmake_install.cmake:171 (include)

make: *** [install] Error 1
cd /home/jek/ros_catkin_ws/build/catkin; catkin build --get-env catkin | catkin env -si  /usr/bin/make install; cd -
.....................................................................................................

I'm on Ubuntu 18.04.4 LTS
Python 3.7.3
ROS-Melodic

I am at the same step. What I did here was removed all occurrences of "--install-layout=deb " in the "python_distutils_install.sh" files.
I managed to get 48 of the 200 packages to build, but the rest then fails on failed imports. Let me know if you managed to get past this point.
I think I did something like this:

find ./ -name 'python_distutils_install.sh' -exec sed -i 's/--install-layout=deb//g' {} \;
@sharpe-developer

This comment has been minimized.

Copy link

@sharpe-developer sharpe-developer commented Apr 1, 2020

@yev-d

So after installing python-empy and also catkin_pkg

Just a guess but are you installing the python3 versions? If using apt to install python-empy should be python3-empy. Maybe you are using pip3?

@metalglove

This comment has been minimized.

Copy link

@metalglove metalglove commented Apr 1, 2020

# ensure there are no ros packages
sudo apt-get remove --autoremove ros-*

# check for updates
sudo apt update

# ensure /etc/ros removal
sudo rm -rf /etc/ros/

# install the python3 libraries
sudo apt install -y python3 python3-dev python3-pip build-essential

# Remove python2
sudo apt purge -y python2.7-minimal

# link python -> python3
sudo ln -s /usr/bin/python3 /usr/bin/python

# Same for pip
sudo ln -s /usr/bin/pip3 /usr/bin/pip

# install the ros dependencies
sudo -H pip3 install --no-cache-dir --ignore-installed rosdep rospkg rosinstall_generator rosinstall wstool vcstools catkin_tools catkin_pkg

# initialize catkin build environment
sudo rosdep init && rosdep update

# create catkin workspace
mkdir -p ~/ros_catkin_ws/src && cd "$_/.."

# initialize catkin workspace (will show warning about Extending... ignore that)
catkin config --init -DCMAKE_BUILD_TYPE=Release --blacklist rqt_rviz rviz_plugin_tutorials librviz_tutorial --install

# generate a ros melodic install
rosinstall_generator desktop_full --rosdistro melodic --deps --tar > melodic-desktop-full.rosinstall

# initialize the install
wstool init -j8 src melodic-desktop-full.rosinstall

# setup environment and isntall dependencies
export ROS_PYTHON_VERSION=3

# install wxPython
pip3 install -U -f --no-cache-dir --ignore-installed https://extras.wxpython.org/wxPython4/extras/linux/gtk3/ubuntu-18.04 wxPython

# create install_skip file
printf '#/bin/bash\nif [ $(whoami) != root ]; then\n    echo You must be root or use sudo to install packages.\n    return\nfi\n\nfor pkg in "$@"\ndo\n    echo "Installing $pkg"\n    sudo apt-get -my install $pkg >> install.log\ndone' > install_skip

# make file executable
chmod +x install_skip

# install python 3 packages
sudo ./install_skip `rosdep check --from-paths src --ignore-src | grep python | sed -e "s/^apt\t//g" | sed -z "s/\n/ /g" | sed -e "s/\<python\>/python3/g"`

# skip python 2 packages
rosdep install --from-paths src --ignore-src -y --skip-keys="`rosdep check --from-paths src --ignore-src | grep python | sed -e "s/^apt\t//g" | sed -z "s/\n/ /g"`"

# rename all old python links to python3
find . -type f -exec sed -i 's/\/usr\/bin\/env[ ]*\<python\>/\/usr\/bin\/env python3/g' {} +

# build ros
catkin build

# export python path
export PYTHONPATH=/usr/lib/python3/dist-packages

# source setup
source ~/ros_catkin_ws/install/setup.bash

Running this worked for me.

@yev-d

This comment has been minimized.

Copy link

@yev-d yev-d commented Apr 1, 2020

@sharpe-developer, yes you were right, but when I installed python3-empy, and it didn't change anything.

@metalglove, this line gave this error:
~/ros_catkin_ws$ pip3 install -U -f --no-cache-dir --ignore-installed https://extras.wxpython.org/wxPython4/extras/linux/gtk3/ubuntu-18.04 wxPython

 Collecting https://extras.wxpython.org/wxPython4/extras/linux/gtk3/ubuntu-18.04
   Downloading https://extras.wxpython.org/wxPython4/extras/linux/gtk3/ubuntu-18.04
      \ 20kB 159kB/s
   Cannot unpack file /tmp/pip-zxfu_1fn-unpack/ubuntu-18.04 (downloaded from /tmp/pip-fz1jq3ce-build, content-type: text/html;charset=UTF-8); cannot detect archive format
 Cannot determine archive format of /tmp/pip-fz1jq3ce-build

Not sure what that was I continued. Everything was fine up to catkin build. It didn't, with the following:

 Errors     << catkin:check /home/jek/ros_catkin_ws/logs/catkin/build.check.001.log                  
 /home/jek/anaconda3/bin/python3: can't open file '/usr/bin/empy': [Errno 2] No such file or directory
 CMake Error at /home/jek/ros_catkin_ws/src/catkin/cmake/safe_execute_process.cmake:11 (message):
 
   execute_process(/home/jek/ros_catkin_ws/build/catkin/catkin_generated/env_cached.sh
   "/home/jek/anaconda3/bin/python3" "/usr/bin/empy" "--raw-errors" "-F"
   "/home/jek/ros_catkin_ws/build/catkin/catkin_generated/pkg.develspace.context.pc.py"
   "-o"
   "/home/jek/ros_catkin_ws/devel/.private/catkin/lib/pkgconfig/catkin.pc"
   "/home/jek/ros_catkin_ws/src/catkin/cmake/em/pkg.pc.em") returned error
   code 2
 Call Stack (most recent call first):
   cmake/em_expand.cmake:25 (safe_execute_process)
   cmake/catkin_package.cmake:327 (em_expand)
   cmake/catkin_package.cmake:102 (_catkin_package)
   CMakeLists.txt:11 (catkin_package)

 make: *** [cmake_check_build_system] Error 1
 cd /home/jek/ros_catkin_ws/build/catkin; catkin build --get-env catkin | catkin env -si  /usr/bin/make cmake_check_build_system; cd -

It says it can't find python3-empy, but it loks like it is installed and sits comfortably in usr/share/doc
Also, I'm not sure why it looks into conda environment when I execute all of this outside of conda.

@metalglove

This comment has been minimized.

Copy link

@metalglove metalglove commented Apr 1, 2020

@yev-d I could have tempered with that line last-minute... remove --no-cache-dir and --ignore-installed.

pip3 install -U -f https://extras.wxpython.org/wxPython4/extras/linux/gtk3/ubuntu-18.04 wxPython
@yev-d

This comment has been minimized.

Copy link

@yev-d yev-d commented Apr 1, 2020

@metalglove, that line worked, thanks! But the catkin build still fails to find empy. Do you have any thoughts on that?

@metalglove

This comment has been minimized.

Copy link

@metalglove metalglove commented Apr 1, 2020

@yev-d Did you run my complete script? what does the catkin build error show?

@yev-d

This comment has been minimized.

Copy link

@yev-d yev-d commented Apr 1, 2020

@metalglove, yup. Everything was fine except catkin build and this little thing.
When running this line:

 sudo ./install_skip `rosdep check --from-paths src --ignore-src | grep python | sed -e "s/^apt\t//g" | sed -z "s/\n/ /g" | sed -e "s/\<python\>/python3/g"

it doesn't find python3-wxtools, though other packages are ok:

 Installing python3-rospkg
 Installing python3-mock
 Installing python3-wxtools
 E: Unable to locate package python3-wxtools
 Installing python3-yaml
 Installing python3-matplotlib
 Installing python3-numpy
 Installing python3-psutil
 Installing python3-catkin-pkg
 Installing python3-opengl
 Installing python3-opencv
 Installing python3-cairo

Then as I come to catkin build command, this comes out:

 ----------------------------------------------------------------------------
 Profile:                     default
 Extending:                   None
 Workspace:                   /home/jek/ros_catkin_ws
 ----------------------------------------------------------------------------
 Build Space:        [exists] /home/jek/ros_catkin_ws/build
 Devel Space:        [exists] /home/jek/ros_catkin_ws/devel
 Install Space:      [exists] /home/jek/ros_catkin_ws/install
 Log Space:          [exists] /home/jek/ros_catkin_ws/logs
 Source Space:       [exists] /home/jek/ros_catkin_ws/src
 DESTDIR:            [unused] None
 ----------------------------------------------------------------------------
 Devel Space Layout:          linked
 Install Space Layout:        merged
 ----------------------------------------------------------------------------
 Additional CMake Args:       -DCMAKE_BUILD_TYPE=Release
 Additional Make Args:        None
 Additional catkin Make Args: None
 Internal Make Job Server:    True
 Cache Job Environments:      False
 ----------------------------------------------------------------------------
 Whitelisted Packages:        None
 Blacklisted Packages:        rqt_rviz rviz_plugin_tutorials librviz_tutorial
 ----------------------------------------------------------------------------
 
 
 ----------------------------------------------------------------------------
 WARNING: Your workspace is not extending any other result space, but it is
 set to use a `linked` devel space layout. This requires the `catkin` CMake
 package in your source space in order to be built.
 ----------------------------------------------------------------------------
 
 [build] Found '237' packages in 0.0 seconds.                                                        
 Starting  >>> catkin                                                                                
 ____________________________________________________________________________________________________
 Errors     << catkin:check /home/jek/ros_catkin_ws/logs/catkin/build.check.002.log                  
 /home/jek/anaconda3/bin/python3: can't open file '/usr/bin/empy': [Errno 2] No such file or directory
 CMake Error at /home/jek/ros_catkin_ws/src/catkin/cmake/safe_execute_process.cmake:11 (message):
 
   execute_process(/home/jek/ros_catkin_ws/build/catkin/catkin_generated/env_cached.sh
   "/home/jek/anaconda3/bin/python3" "/usr/bin/empy" "--raw-errors" "-F"
   "/home/jek/ros_catkin_ws/build/catkin/catkin_generated/pkg.develspace.context.pc.py"
   "-o"
   "/home/jek/ros_catkin_ws/devel/.private/catkin/lib/pkgconfig/catkin.pc"
   "/home/jek/ros_catkin_ws/src/catkin/cmake/em/pkg.pc.em") returned error
   code 2
 Call Stack (most recent call first):
   cmake/em_expand.cmake:25 (safe_execute_process)
   cmake/catkin_package.cmake:327 (em_expand)
   cmake/catkin_package.cmake:102 (_catkin_package)
   CMakeLists.txt:11 (catkin_package)
 
 
 make: *** [cmake_check_build_system] Error 1
 cd /home/jek/ros_catkin_ws/build/catkin; catkin build --get-env catkin | catkin env -si  /usr/bin/make cmake_check_build_system; cd -
 ....................................................................................................
 Failed     << catkin:check                                    [ Exited with code 2 ]                
 Failed    <<< catkin                                          [ 0.9 seconds ]                       
@metalglove

This comment has been minimized.

Copy link

@metalglove metalglove commented Apr 1, 2020

@yev-d Ah. "E: Unable to locate package python3-wxtools" makes sense, since we install it using the pip3 command above. And, the script changes all the python references to python3 hence it trying to install python3-wxtools. This can be ignored.

Try sudo apt install python-empy. This should work now.

I am currently not at my Ubuntu machine. Tomorrow I will spin up a new VM and run the complete script and check if I missed something in the script.

@yev-d

This comment has been minimized.

Copy link

@yev-d yev-d commented Apr 1, 2020

@metalglove, yeah, after installing python-empy (and not python3-empy), the error changes! Whoo-hoo!

 Errors     << catkin:install /home/jek/ros_catkin_ws/logs/catkin/build.install.002.log
 usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
    or: setup.py --help [cmd1 cmd2 ...]
    or: setup.py --help-commands
    or: setup.py cmd --help
 
 error: option --install-layout not recognized
 CMake Error at catkin_generated/safe_execute_install.cmake:4 (message):
   
   execute_process(/home/jek/ros_catkin_ws/build/catkin/catkin_generated/python_distutils_install.sh)
   returned error code
 Call Stack (most recent call first):
   cmake_install.cmake:171 (include)
 
 
 make: *** [install] Error 1
 cd /home/jek/ros_catkin_ws/build/catkin; catkin build --get-env catkin | catkin env -si  /usr/bin/make install; cd -
 ...............................................................................

Yeah, no worries, take your time. I'm gonna look for a support group for Emotionally-Defeated-by-ROS-Installation-Anonymous group.

@metalglove

This comment has been minimized.

Copy link

@metalglove metalglove commented Apr 1, 2020

Oh, I know how to fix that error!
cd ~/ros_catkin_ws
find ./ -name 'python_distutils_install.sh' -exec sed -i 's/--install-layout=deb//g' {} \;
Now it should work ;-)

--install-layout=deb needs to be removed from all the generated installs scripts.

@yev-d

This comment has been minimized.

Copy link

@yev-d yev-d commented Apr 1, 2020

@metalglove, nice! it solved THAT problem, and even started installing bunch of packages. But it failed miserably with 592 lines of errors for bunch of packages.
At this point, I'm going to try installing it on the new fresh Linux OS.

@yev-d

This comment has been minimized.

Copy link

@yev-d yev-d commented Apr 1, 2020

I've installed a new OS with Ubuntu 18.04.4, and I ran all the code that @metalglove commented above
The only line that I've changed is this one:

 # install wxPython
 pip3 install -U -f --no-cache-dir --ignore-installed https://extras.wxpython.org/wxPython4/extras/linux/gtk3/ubuntu-18.04 wxPython

To this one:

pip3 install -U -f https://extras.wxpython.org/wxPython4/extras/linux/gtk3/ubuntu-18.04 wxPython

catkin build installed all the packages, though with occasional warnings about missing "()" on "print". All 204 packages were installed, and 33 ignored.
I think when installing conda it does something that later on it is in conflict with the rest of the world.

So now that this is done, how can I make sure that it works? After so many failed attempts I have my doubts. Also, roscore doesn't work.

@metalglove

This comment has been minimized.

Copy link

@metalglove metalglove commented Apr 2, 2020

@yev-d conda uses its own python environment. Hence you will not be able to call roscore.
And, if you want to call python that uses the ros stuff do this:
make sure you are out of anaconda
conda deactivate
and source the ros setup:
source ~/ros_catkin_ws/install/setup.bash

you have to do this each time. You can add that last line to your .bashrc and comment the anaconda lines there.

@metalglove

This comment has been minimized.

Copy link

@metalglove metalglove commented Apr 2, 2020

The complete script for installing ROS with python3.

# ensure there are no ros packages
sudo apt-get remove --autoremove ros-*

# check for updates
sudo apt update

# ensure /etc/ros removal
sudo rm -rf /etc/ros/

# install the python3 libraries 
sudo apt install -y python3 python3-dev python3-pip build-essential

# Remove python2
sudo apt purge -y python2.7-minimal

# link python -> python3
sudo ln -s /usr/bin/python3 /usr/bin/python

# Same for pip
sudo ln -s /usr/bin/pip3 /usr/bin/pip

# install the ros dependencies
sudo -H pip3 install rosdep rospkg rosinstall_generator rosinstall wstool vcstools catkin_tools catkin_pkg

# initialize catkin build environment
sudo rosdep init && rosdep update

# create catkin workspace
mkdir -p ~/ros_catkin_ws/src && cd "$_/.."

# initialize catkin workspace (will show warning about Extending... ignore that)
catkin config --init -DCMAKE_BUILD_TYPE=Release --blacklist rqt_rviz rviz_plugin_tutorials librviz_tutorial --install

# generate a ros melodic install
rosinstall_generator desktop_full --rosdistro melodic --deps --tar > melodic-desktop-full.rosinstall

# initialize the install
wstool init -j8 src melodic-desktop-full.rosinstall

# setup environment and install dependencies
export ROS_PYTHON_VERSION=3

# install wxPython
pip install -U -f https://extras.wxpython.org/wxPython4/extras/linux/gtk3/ubuntu-18.04 wxPython

# create install_skip file
printf '#/bin/bash\nif [ $(whoami) != root ]; then\n    echo You must be root or use sudo to install packages.\n    return\nfi\n\nfor pkg in "$@"\ndo\n    echo "Installing $pkg"\n    sudo apt-get -my install $pkg >> install.log\ndone' > install_skip

# make file executable
chmod +x install_skip

# install python 3 packages
sudo ./install_skip `rosdep check --from-paths src --ignore-src | grep python | sed -e "s/^apt\t//g" | sed -z "s/\n/ /g" | sed -e "s/\<python\>/python3/g"`

# skip python 2 packages
rosdep install --from-paths src --ignore-src -y --skip-keys="`rosdep check --from-paths src --ignore-src | grep python | sed -e "s/^apt\t//g" | sed -z "s/\n/ /g"`"

# rename all old python links to python3
find . -type f -exec sed -i 's/\/usr\/bin\/env[ ]*\<python\>/\/usr\/bin\/env python3/g' {} +

# remove all depricated install-layout=deb arguments
find ./ -name 'python_distutils_install.sh' -exec sed -i 's/--install-layout=deb//g' {} \;

# install python-empy
sudo apt install -y python-empy

# build ros
catkin build

# export python path
export PYTHONPATH=/usr/lib/python3/dist-packages

# source setup
source ~/ros_catkin_ws/install/setup.bash
@yev-d

This comment has been minimized.

Copy link

@yev-d yev-d commented Apr 2, 2020

@metalglove, Just to clarify... so after all of this, should roscore work, or are there any other steps for it?
(P.s. if you will assume that you are talking to a noob, I think your answer will go a long way ;) )

@metalglove

This comment has been minimized.

Copy link

@metalglove metalglove commented Apr 2, 2020

@yev-d It should work. If everything installed fine. You should be able to open a new terminal and make sure that you have deactivated anaconda environment
conda deactivate
followed by a source of the ros setup
source ~/ros_catkin_ws/install/setup.bash
If you then run python and try to import rospy it should import just fine.
also, check if python -V shows python 3.
and roscore should run as well.

@praveenputhettu

This comment has been minimized.

Copy link

@praveenputhettu praveenputhettu commented May 4, 2020

Can i link ros-kinetic with python 3.5 using the above process

@kmfrick

This comment has been minimized.

Copy link

@kmfrick kmfrick commented May 8, 2020

I would add libgtk-3-dev to the list of dependencies. It's pre-installed on 99% of Ubuntu installations, but in some cases (i.e. embedded devices with no DE) it might not and the installation of wxPython would fail.

@juancamilog

This comment has been minimized.

Copy link

@juancamilog juancamilog commented May 11, 2020

@drmaj have you tried this with 20.04?

@kmfrick

This comment has been minimized.

Copy link

@kmfrick kmfrick commented May 11, 2020

@juancamilog Use Noetic in 20.04, it will be released in two weeks and supports Python 3 out of the box.

@dejongyeong

This comment has been minimized.

Copy link

@dejongyeong dejongyeong commented May 31, 2020

The complete script for installing ROS with python3.

# ensure there are no ros packages
sudo apt-get remove --autoremove ros-*

# check for updates
sudo apt update

# ensure /etc/ros removal
sudo rm -rf /etc/ros/

# install the python3 libraries 
sudo apt install -y python3 python3-dev python3-pip build-essential

# Remove python2
sudo apt purge -y python2.7-minimal

# link python -> python3
sudo ln -s /usr/bin/python3 /usr/bin/python

# Same for pip
sudo ln -s /usr/bin/pip3 /usr/bin/pip

# install the ros dependencies
sudo -H pip3 install rosdep rospkg rosinstall_generator rosinstall wstool vcstools catkin_tools catkin_pkg

# initialize catkin build environment
sudo rosdep init && rosdep update

# create catkin workspace
mkdir -p ~/ros_catkin_ws/src && cd "$_/.."

# initialize catkin workspace (will show warning about Extending... ignore that)
catkin config --init -DCMAKE_BUILD_TYPE=Release --blacklist rqt_rviz rviz_plugin_tutorials librviz_tutorial --install

# generate a ros melodic install
rosinstall_generator desktop_full --rosdistro melodic --deps --tar > melodic-desktop-full.rosinstall

# initialize the install
wstool init -j8 src melodic-desktop-full.rosinstall

# setup environment and install dependencies
export ROS_PYTHON_VERSION=3

# install wxPython
pip install -U -f https://extras.wxpython.org/wxPython4/extras/linux/gtk3/ubuntu-18.04 wxPython

# create install_skip file
printf '#/bin/bash\nif [ $(whoami) != root ]; then\n    echo You must be root or use sudo to install packages.\n    return\nfi\n\nfor pkg in "$@"\ndo\n    echo "Installing $pkg"\n    sudo apt-get -my install $pkg >> install.log\ndone' > install_skip

# make file executable
chmod +x install_skip

# install python 3 packages
sudo ./install_skip `rosdep check --from-paths src --ignore-src | grep python | sed -e "s/^apt\t//g" | sed -z "s/\n/ /g" | sed -e "s/\<python\>/python3/g"`

# skip python 2 packages
rosdep install --from-paths src --ignore-src -y --skip-keys="`rosdep check --from-paths src --ignore-src | grep python | sed -e "s/^apt\t//g" | sed -z "s/\n/ /g"`"

# rename all old python links to python3
find . -type f -exec sed -i 's/\/usr\/bin\/env[ ]*\<python\>/\/usr\/bin\/env python3/g' {} +

# remove all depricated install-layout=deb arguments
find ./ -name 'python_distutils_install.sh' -exec sed -i 's/--install-layout=deb//g' {} \;

# install python-empy
sudo apt install -y python-empy

# build ros
catkin build

# export python path
export PYTHONPATH=/usr/lib/python3/dist-packages

# source setup
source ~/ros_catkin_ws/install/setup.bash

Hi, I would like to know how could I catkin_make or catkin build certain package, e.g. video_stream_opencv in src folder? Thanks

@metalglove

This comment has been minimized.

Copy link

@metalglove metalglove commented May 31, 2020

The complete script for installing ROS with python3.

# ensure there are no ros packages
sudo apt-get remove --autoremove ros-*

# check for updates
sudo apt update

# ensure /etc/ros removal
sudo rm -rf /etc/ros/

# install the python3 libraries 
sudo apt install -y python3 python3-dev python3-pip build-essential

# Remove python2
sudo apt purge -y python2.7-minimal

# link python -> python3
sudo ln -s /usr/bin/python3 /usr/bin/python

# Same for pip
sudo ln -s /usr/bin/pip3 /usr/bin/pip

# install the ros dependencies
sudo -H pip3 install rosdep rospkg rosinstall_generator rosinstall wstool vcstools catkin_tools catkin_pkg

# initialize catkin build environment
sudo rosdep init && rosdep update

# create catkin workspace
mkdir -p ~/ros_catkin_ws/src && cd "$_/.."

# initialize catkin workspace (will show warning about Extending... ignore that)
catkin config --init -DCMAKE_BUILD_TYPE=Release --blacklist rqt_rviz rviz_plugin_tutorials librviz_tutorial --install

# generate a ros melodic install
rosinstall_generator desktop_full --rosdistro melodic --deps --tar > melodic-desktop-full.rosinstall

# initialize the install
wstool init -j8 src melodic-desktop-full.rosinstall

# setup environment and install dependencies
export ROS_PYTHON_VERSION=3

# install wxPython
pip install -U -f https://extras.wxpython.org/wxPython4/extras/linux/gtk3/ubuntu-18.04 wxPython

# create install_skip file
printf '#/bin/bash\nif [ $(whoami) != root ]; then\n    echo You must be root or use sudo to install packages.\n    return\nfi\n\nfor pkg in "$@"\ndo\n    echo "Installing $pkg"\n    sudo apt-get -my install $pkg >> install.log\ndone' > install_skip

# make file executable
chmod +x install_skip

# install python 3 packages
sudo ./install_skip `rosdep check --from-paths src --ignore-src | grep python | sed -e "s/^apt\t//g" | sed -z "s/\n/ /g" | sed -e "s/\<python\>/python3/g"`

# skip python 2 packages
rosdep install --from-paths src --ignore-src -y --skip-keys="`rosdep check --from-paths src --ignore-src | grep python | sed -e "s/^apt\t//g" | sed -z "s/\n/ /g"`"

# rename all old python links to python3
find . -type f -exec sed -i 's/\/usr\/bin\/env[ ]*\<python\>/\/usr\/bin\/env python3/g' {} +

# remove all depricated install-layout=deb arguments
find ./ -name 'python_distutils_install.sh' -exec sed -i 's/--install-layout=deb//g' {} \;

# install python-empy
sudo apt install -y python-empy

# build ros
catkin build

# export python path
export PYTHONPATH=/usr/lib/python3/dist-packages

# source setup
source ~/ros_catkin_ws/install/setup.bash

Hi, I would like to know how could I catkin_make or catkin build certain package, e.g. video_stream_opencv in src folder?

In the folder video_stream_opencv.
Catkin build —this.
Optionally —no-deps for no dependencies.

@dejongyeong

This comment has been minimized.

Copy link

@dejongyeong dejongyeong commented May 31, 2020

@metalglove Thanks!!

@dejongyeong

This comment has been minimized.

Copy link

@dejongyeong dejongyeong commented May 31, 2020

I would like to know should I ignore are the warnings when building ROS? @metalglove. Thanks!

@dejongyeong

This comment has been minimized.

Copy link

@dejongyeong dejongyeong commented Jun 1, 2020

The complete script for installing ROS with python3.

# ensure there are no ros packages
sudo apt-get remove --autoremove ros-*

# check for updates
sudo apt update

# ensure /etc/ros removal
sudo rm -rf /etc/ros/

# install the python3 libraries 
sudo apt install -y python3 python3-dev python3-pip build-essential

# Remove python2
sudo apt purge -y python2.7-minimal

# link python -> python3
sudo ln -s /usr/bin/python3 /usr/bin/python

# Same for pip
sudo ln -s /usr/bin/pip3 /usr/bin/pip

# install the ros dependencies
sudo -H pip3 install rosdep rospkg rosinstall_generator rosinstall wstool vcstools catkin_tools catkin_pkg

# initialize catkin build environment
sudo rosdep init && rosdep update

# create catkin workspace
mkdir -p ~/ros_catkin_ws/src && cd "$_/.."

# initialize catkin workspace (will show warning about Extending... ignore that)
catkin config --init -DCMAKE_BUILD_TYPE=Release --blacklist rqt_rviz rviz_plugin_tutorials librviz_tutorial --install

# generate a ros melodic install
rosinstall_generator desktop_full --rosdistro melodic --deps --tar > melodic-desktop-full.rosinstall

# initialize the install
wstool init -j8 src melodic-desktop-full.rosinstall

# setup environment and install dependencies
export ROS_PYTHON_VERSION=3

# install wxPython
pip install -U -f https://extras.wxpython.org/wxPython4/extras/linux/gtk3/ubuntu-18.04 wxPython

# create install_skip file
printf '#/bin/bash\nif [ $(whoami) != root ]; then\n    echo You must be root or use sudo to install packages.\n    return\nfi\n\nfor pkg in "$@"\ndo\n    echo "Installing $pkg"\n    sudo apt-get -my install $pkg >> install.log\ndone' > install_skip

# make file executable
chmod +x install_skip

# install python 3 packages
sudo ./install_skip `rosdep check --from-paths src --ignore-src | grep python | sed -e "s/^apt\t//g" | sed -z "s/\n/ /g" | sed -e "s/\<python\>/python3/g"`

# skip python 2 packages
rosdep install --from-paths src --ignore-src -y --skip-keys="`rosdep check --from-paths src --ignore-src | grep python | sed -e "s/^apt\t//g" | sed -z "s/\n/ /g"`"

# rename all old python links to python3
find . -type f -exec sed -i 's/\/usr\/bin\/env[ ]*\<python\>/\/usr\/bin\/env python3/g' {} +

# remove all depricated install-layout=deb arguments
find ./ -name 'python_distutils_install.sh' -exec sed -i 's/--install-layout=deb//g' {} \;

# install python-empy
sudo apt install -y python-empy

# build ros
catkin build

# export python path
export PYTHONPATH=/usr/lib/python3/dist-packages

# source setup
source ~/ros_catkin_ws/install/setup.bash

Hi, I would like to know how could I catkin_make or catkin build certain package, e.g. video_stream_opencv in src folder?

In the folder video_stream_opencv.
Catkin build —this.
Optionally —no-deps for no dependencies.

Hi @metalglove, I encountered an issue where it doesn't detect any launch files. Any solution to resolve this issue? thanks

@Strong30Impact

This comment has been minimized.

Copy link

@Strong30Impact Strong30Impact commented Jun 18, 2020

Hi @metalglove, I need pyhton3 in ROS for a project and I followed your blog post and its publication on April 2nd and gives me error when I apply the following line:

# install python 3 packages
sudo ./install_skip `rosdep check --from-paths src --ignore-src | grep python | sed -e "s/^apt\t//g" | sed -z "s/\n/ /g" | sed -e "s/\<python\>/python3/g"`
Installing python3-rospkg
Installing python3-mock
Installing python3-empy
Installing python3-catkin-pkg-modules
E: Unable to locate package python3-catkin-pkg-modules
Installing python3-rosdep-modules
E: Unable to locate package python3-rosdep-modules
Installing python3-coverage
Installing python3-rospkg-modules
E: Unable to locate package python3-rospkg-modules
Installing python3-nose
Installing python3-paramiko
Installing python3-pycryptodome
Installing python3-gnupg
Installing python3-defusedxml
Installing python3-pyqt5.qtopengl
Installing python3-cairo
Installing python3-pil
Installing python3-psutil
Installing python3-pyqt5.qtsvg
Installing python3-catkin-pkg
Installing python3-dev
Installing python3-opencv
Installing python3-numpy
Installing python3-matplotlib
Installing python3-opengl
Installing python3-wxtools
E: Unable to locate package python3-wxtools
Installing python3-sip-dev
Installing python3-pydot
Installing python3-pygraphviz
Installing python3-pyqt5.qtwebkit

If I continue with the installation when I do "catkin build" this error occurs:

[build] Summary: All 204 succeeded packages!                                   
[build]   Ignored: 33 packages were skipped or are blacklisted.              
[build]   Warnings: 13 packages succeeded with warnings.                      
[build]   Abandoned: None.                                                     
[build]   Failed: None.                                                     
[build] Runtime: 13 minutes and 13.8 seconds total.                            
[build] Note: Workspace packages have changed, please re-source setup files to use them.
Exception ignored in: <bound method BaseEventLoop.__del__ of <_UnixSelectorEventLoop running=False closed=True debug=False>>
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/trollius/base_events.py", line 395, in __del__
  File "/usr/local/lib/python3.6/dist-packages/trollius/unix_events.py", line 65, in close
  File "/usr/local/lib/python3.6/dist-packages/trollius/unix_events.py", line 166, in remove_signal_handler
  File "/usr/lib/python3.6/signal.py", line 47, in signal

TypeError: signal handler must be signal. SIG_IGN, signal. SIG_DFL, or a callable object

If I try "roscore" at the end of it all gives me error. I followed the suggestions in the previous post and nothing. Could you help me, please? Thanks

@metalglove

This comment has been minimized.

Copy link

@metalglove metalglove commented Jun 18, 2020

Hi @metalglove, I need pyhton3 in ROS for a project and I followed your blog post and its publication on April 2nd and gives me error when I apply the following line:

# install python 3 packages
sudo ./install_skip `rosdep check --from-paths src --ignore-src | grep python | sed -e "s/^apt\t//g" | sed -z "s/\n/ /g" | sed -e "s/\<python\>/python3/g"`
Installing python3-rospkg
Installing python3-mock
Installing python3-empy
Installing python3-catkin-pkg-modules
E: Unable to locate package python3-catkin-pkg-modules
Installing python3-rosdep-modules
E: Unable to locate package python3-rosdep-modules
Installing python3-coverage
Installing python3-rospkg-modules
E: Unable to locate package python3-rospkg-modules
Installing python3-nose
Installing python3-paramiko
Installing python3-pycryptodome
Installing python3-gnupg
Installing python3-defusedxml
Installing python3-pyqt5.qtopengl
Installing python3-cairo
Installing python3-pil
Installing python3-psutil
Installing python3-pyqt5.qtsvg
Installing python3-catkin-pkg
Installing python3-dev
Installing python3-opencv
Installing python3-numpy
Installing python3-matplotlib
Installing python3-opengl
Installing python3-wxtools
E: Unable to locate package python3-wxtools
Installing python3-sip-dev
Installing python3-pydot
Installing python3-pygraphviz
Installing python3-pyqt5.qtwebkit

If I continue with the installation when I do "catkin build" this error occurs:

[build] Summary: All 204 succeeded packages!                                   
[build]   Ignored: 33 packages were skipped or are blacklisted.              
[build]   Warnings: 13 packages succeeded with warnings.                      
[build]   Abandoned: None.                                                     
[build]   Failed: None.                                                     
[build] Runtime: 13 minutes and 13.8 seconds total.                            
[build] Note: Workspace packages have changed, please re-source setup files to use them.
Exception ignored in: <bound method BaseEventLoop.__del__ of <_UnixSelectorEventLoop running=False closed=True debug=False>>
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/trollius/base_events.py", line 395, in __del__
  File "/usr/local/lib/python3.6/dist-packages/trollius/unix_events.py", line 65, in close
  File "/usr/local/lib/python3.6/dist-packages/trollius/unix_events.py", line 166, in remove_signal_handler
  File "/usr/lib/python3.6/signal.py", line 47, in signal

TypeError: signal handler must be signal. SIG_IGN, signal. SIG_DFL, or a callable object

If I try "roscore" at the end of it all gives me error. I followed the suggestions in the previous post and nothing. Could you help me, please? Thanks

It was not my blog post, but I took an interest in this script. I have since moved to ROS neotic (which supports python 3 by default).
http://wiki.ros.org/noetic

@Strong30Impact

This comment has been minimized.

Copy link

@Strong30Impact Strong30Impact commented Jun 18, 2020

Oi @metalglove , eu preciso do pyhton3 no ROS para um projeto e segui a postagem do seu blog e sua publicação em 2 de abril e me dá um erro quando aplico a seguinte linha:

# install python 3 packages
sudo ./install_skip `rosdep check --from-paths src --ignore-src | grep python | sed -e "s/^apt\t//g" | sed -z "s/\n/ /g" | sed -e "s/\<python\>/python3/g"`
Installing python3-rospkg
Installing python3-mock
Installing python3-empy
Installing python3-catkin-pkg-modules
E: Unable to locate package python3-catkin-pkg-modules
Installing python3-rosdep-modules
E: Unable to locate package python3-rosdep-modules
Installing python3-coverage
Installing python3-rospkg-modules
E: Unable to locate package python3-rospkg-modules
Installing python3-nose
Installing python3-paramiko
Installing python3-pycryptodome
Installing python3-gnupg
Installing python3-defusedxml
Installing python3-pyqt5.qtopengl
Installing python3-cairo
Installing python3-pil
Installing python3-psutil
Installing python3-pyqt5.qtsvg
Installing python3-catkin-pkg
Installing python3-dev
Installing python3-opencv
Installing python3-numpy
Installing python3-matplotlib
Installing python3-opengl
Installing python3-wxtools
E: Unable to locate package python3-wxtools
Installing python3-sip-dev
Installing python3-pydot
Installing python3-pygraphviz
Installing python3-pyqt5.qtwebkit

Se eu continuar com a instalação quando eu faço "catkin build", este erro ocorre:

[build] Summary: All 204 succeeded packages!                                   
[build]   Ignored: 33 packages were skipped or are blacklisted.              
[build]   Warnings: 13 packages succeeded with warnings.                      
[build]   Abandoned: None.                                                     
[build]   Failed: None.                                                     
[build] Runtime: 13 minutes and 13.8 seconds total.                            
[build] Note: Workspace packages have changed, please re-source setup files to use them.
Exception ignored in: <bound method BaseEventLoop.__del__ of <_UnixSelectorEventLoop running=False closed=True debug=False>>
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/trollius/base_events.py", line 395, in __del__
  File "/usr/local/lib/python3.6/dist-packages/trollius/unix_events.py", line 65, in close
  File "/usr/local/lib/python3.6/dist-packages/trollius/unix_events.py", line 166, in remove_signal_handler
  File "/usr/lib/python3.6/signal.py", line 47, in signal

TypeError: signal handler must be signal. SIG_IGN, signal. SIG_DFL, or a callable object

Se eu tentar "roscore" no final de tudo, me dá erro. Segui as sugestões do post anterior e nada. Você poderia me ajudar por favor? obrigado

It was not my blog post, but I took an interest in this script. I have since moved to ROS neotic (which supports python 3 by default).
http://wiki.ros.org/noetic

@metalglove But did you get python3 up and running with ROS Melodic?
My code have to run on the nano jetson and I believe there is no ros noetic installation for it, just Melodic.
So it's very important for me to get ROS Melodic up and running with python3

@graemerae

This comment has been minimized.

Copy link

@graemerae graemerae commented Jun 18, 2020

Noob here. Trying to run catkin config --init -DCMAKE_BUILD_TYPE=Release --blacklist rqt_rviz rviz_plugin_tutorials librviz_tutorial --install, I get the following error:

Traceback (most recent call last):
  File "/usr/local/bin/catkin", line 6, in <module>
    from catkin_tools.commands.catkin import main
  File "/usr/local/lib/python3.7/dist-packages/catkin_tools/commands/catkin.py", line 28, in <module>
    from catkin_tools.common import is_tty
  File "/usr/local/lib/python3.7/dist-packages/catkin_tools/common.py", line 23, in <module>
    import trollius as asyncio
  File "/usr/local/lib/python3.7/dist-packages/trollius/__init__.py", line 21, in <module>
    from .base_events import *
  File "/usr/local/lib/python3.7/dist-packages/trollius/base_events.py", line 42, in <module>
    from . import tasks
  File "/usr/local/lib/python3.7/dist-packages/trollius/tasks.py", line 565
    def async(coro_or_future, loop=None):
            ^
SyntaxError: invalid syntax

I am trying to follow the instructions on a Raspberry Pi running Raspbian Buster Lite and had previously installed Ros Melodic from source according to this tutorial. Before trying the instructions above, I simply deleted my existing ros_catkin_ws and catkin_ws.

Noob here. Trying to run catkin config --init -DCMAKE_BUILD_TYPE=Release --blacklist rqt_rviz rviz_plugin_tutorials librviz_tutorial --install, I get the following error:

Traceback (most recent call last):
  File "/usr/local/bin/catkin", line 6, in <module>
    from catkin_tools.commands.catkin import main
  File "/usr/local/lib/python3.7/dist-packages/catkin_tools/commands/catkin.py", line 28, in <module>
    from catkin_tools.common import is_tty
  File "/usr/local/lib/python3.7/dist-packages/catkin_tools/common.py", line 23, in <module>
    import trollius as asyncio
  File "/usr/local/lib/python3.7/dist-packages/trollius/__init__.py", line 21, in <module>
    from .base_events import *
  File "/usr/local/lib/python3.7/dist-packages/trollius/base_events.py", line 42, in <module>
    from . import tasks
  File "/usr/local/lib/python3.7/dist-packages/trollius/tasks.py", line 565
    def async(coro_or_future, loop=None):
            ^
SyntaxError: invalid syntax

I am trying to follow the instructions on a Raspberry Pi running Raspbian Buster Lite and had previously installed Ros Melodic from source according to this tutorial. Before trying the instructions above, I simply deleted my existing ros_catkin_ws and catkin_ws.

Thank you very much for any help!

I solved this one by a hack. Apparently there's a bug in that tasks file for the version of python we both seem to be using. (Are you on a raspberry PI?) I simply edited /usr/local/lib/python3.7/dist-packages/trollius/tasks.py and deleted the 'async' reference at the top of the file and then deleted the entire function at line 565. (it even says it's been deprecated)
Discussed here: catkin/catkin_tools#558

@Strong30Impact

This comment has been minimized.

Copy link

@Strong30Impact Strong30Impact commented Jun 18, 2020

Noob here. Trying to run catkin config --init -DCMAKE_BUILD_TYPE=Release --blacklist rqt_rviz rviz_plugin_tutorials librviz_tutorial --install, I get the following error:

Traceback (most recent call last):
  File "/usr/local/bin/catkin", line 6, in <module>
    from catkin_tools.commands.catkin import main
  File "/usr/local/lib/python3.7/dist-packages/catkin_tools/commands/catkin.py", line 28, in <module>
    from catkin_tools.common import is_tty
  File "/usr/local/lib/python3.7/dist-packages/catkin_tools/common.py", line 23, in <module>
    import trollius as asyncio
  File "/usr/local/lib/python3.7/dist-packages/trollius/__init__.py", line 21, in <module>
    from .base_events import *
  File "/usr/local/lib/python3.7/dist-packages/trollius/base_events.py", line 42, in <module>
    from . import tasks
  File "/usr/local/lib/python3.7/dist-packages/trollius/tasks.py", line 565
    def async(coro_or_future, loop=None):
            ^
SyntaxError: invalid syntax

I am trying to follow the instructions on a Raspberry Pi running Raspbian Buster Lite and had previously installed Ros Melodic from source according to this tutorial. Before trying the instructions above, I simply deleted my existing ros_catkin_ws and catkin_ws.

Noob here. Trying to run catkin config --init -DCMAKE_BUILD_TYPE=Release --blacklist rqt_rviz rviz_plugin_tutorials librviz_tutorial --install, I get the following error:

Traceback (most recent call last):
  File "/usr/local/bin/catkin", line 6, in <module>
    from catkin_tools.commands.catkin import main
  File "/usr/local/lib/python3.7/dist-packages/catkin_tools/commands/catkin.py", line 28, in <module>
    from catkin_tools.common import is_tty
  File "/usr/local/lib/python3.7/dist-packages/catkin_tools/common.py", line 23, in <module>
    import trollius as asyncio
  File "/usr/local/lib/python3.7/dist-packages/trollius/__init__.py", line 21, in <module>
    from .base_events import *
  File "/usr/local/lib/python3.7/dist-packages/trollius/base_events.py", line 42, in <module>
    from . import tasks
  File "/usr/local/lib/python3.7/dist-packages/trollius/tasks.py", line 565
    def async(coro_or_future, loop=None):
            ^
SyntaxError: invalid syntax

I am trying to follow the instructions on a Raspberry Pi running Raspbian Buster Lite and had previously installed Ros Melodic from source according to this tutorial. Before trying the instructions above, I simply deleted my existing ros_catkin_ws and catkin_ws.
Thank you very much for any help!

I solved this one by a hack. Apparently there's a bug in that tasks file for the version of python we both seem to be using. (Are you on a raspberry PI?) I simply edited /usr/local/lib/python3.7/dist-packages/trollius/tasks.py and deleted the 'async' reference at the top of the file and then deleted the entire function at line 565. (it even says it's been deprecated)
Discussed here: catkin/catkin_tools#558

I'm using nvidia's jetson nano. I searched and found no ROS2 installation that already has python3 nor ros noetic. That's why it's important to be able to install. Did you even use some algorithm in python3 and it worked well? Deep Learning algorithm? @metalglove

@graemerae

This comment has been minimized.

Copy link

@graemerae graemerae commented Jun 18, 2020

I knew this was going too well:

Errors     << xmlrpcpp:make /home/pi/ros_catkin_ws/logs/xmlrpcpp/build.make.000.log                         
/home/pi/ros_catkin_ws/src/ros_comm/xmlrpcpp/src/XmlRpcClient.cpp: In member function �virtual std::__cxx11::string XmlRpc::XmlRpcClient::generateHeader(size_t) const�:
/home/pi/ros_catkin_ws/src/ros_comm/xmlrpcpp/src/XmlRpcClient.cpp:330:3: error: �snprintf� is not a member of �std�
   std::snprintf(buff,40,":%d\r\n", _port);
   ^
/home/pi/ros_catkin_ws/src/ros_comm/xmlrpcpp/src/XmlRpcClient.cpp:330:3: note: suggested alternative:
In file included from /home/pi/ros_catkin_ws/src/ros_comm/xmlrpcpp/src/XmlRpcClient.cpp:8:0:
/usr/include/stdio.h:354:12: note:   �snprintf�
 extern int snprintf (char *__restrict __s, size_t __maxlen,
            ^
/home/pi/ros_catkin_ws/src/ros_comm/xmlrpcpp/src/XmlRpcClient.cpp:335:3: error: �snprintf� is not a member of �std�
   std::snprintf(buff,40,"%zu\r\n\r\n", length);
   ^
/home/pi/ros_catkin_ws/src/ros_comm/xmlrpcpp/src/XmlRpcClient.cpp:335:3: note: suggested alternative:
In file included from /home/pi/ros_catkin_ws/src/ros_comm/xmlrpcpp/src/XmlRpcClient.cpp:8:0:
/usr/include/stdio.h:354:12: note:   �snprintf�
 extern int snprintf (char *__restrict __s, size_t __maxlen,
            ^
make[2]: *** [CMakeFiles/xmlrpcpp.dir/build.make:63: CMakeFiles/xmlrpcpp.dir/src/XmlRpcClient.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:137: CMakeFiles/xmlrpcpp.dir/all] Error 2
make: *** [Makefile:141: all] Error 2
cd /home/pi/ros_catkin_ws/build/xmlrpcpp; catkin build --get-env xmlrpcpp | catkin env -si  /usr/bin/make --jobserver-auth=3,4; cd -

Any thoughts?

@graemerae

This comment has been minimized.

Copy link

@graemerae graemerae commented Jun 21, 2020

I knew this was going too well:

Errors << xmlrpcpp:make /home/pi/ros_catkin_ws/logs/xmlrpcpp/build.make.000.log
/home/pi/ros_catkin_ws/src/ros_comm/xmlrpcpp/src/XmlRpcClient.cpp: In member function �virtual std::__cxx11::string XmlRpc::XmlRpcClient::generateHeader(size_t) const�:
/home/pi/ros_catkin_ws/src/ros_comm/xmlrpcpp/src/XmlRpcClient.cpp:330:3: error: �snprintf� is not a member of �std�
std::snprintf(buff,40,":%d\r\n", _port);
^
/home/pi/ros_catkin_ws/src/ros_comm/xmlrpcpp/src/XmlRpcClient.cpp:330:3: note: suggested alternative:
In file included from /home/pi/ros_catkin_ws/src/ros_comm/xmlrpcpp/src/XmlRpcClient.cpp:8:0:
/usr/include/stdio.h:354:12: note: �snprintf�
extern int snprintf (char *__restrict __s, size_t __maxlen,
^
/home/pi/ros_catkin_ws/src/ros_comm/xmlrpcpp/src/XmlRpcClient.cpp:335:3: error: �snprintf� is not a member of �std�
std::snprintf(buff,40,"%zu\r\n\r\n", length);
^
/home/pi/ros_catkin_ws/src/ros_comm/xmlrpcpp/src/XmlRpcClient.cpp:335:3: note: suggested alternative:
In file included from /home/pi/ros_catkin_ws/src/ros_comm/xmlrpcpp/src/XmlRpcClient.cpp:8:0:
/usr/include/stdio.h:354:12: note: �snprintf�
extern int snprintf (char *__restrict __s, size_t __maxlen,
^
make[2]: *** [CMakeFiles/xmlrpcpp.dir/build.make:63: CMakeFiles/xmlrpcpp.dir/src/XmlRpcClient.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:137: CMakeFiles/xmlrpcpp.dir/all] Error 2
make: *** [Makefile:141: all] Error 2
cd /home/pi/ros_catkin_ws/build/xmlrpcpp; catkin build --get-env xmlrpcpp | catkin env -si /usr/bin/make --jobserver-auth=3,4; cd -
Any thoughts?

I solved this by wiping the SD card on my pi and reinstalling from scratch.

@graemerae

This comment has been minimized.

Copy link

@graemerae graemerae commented Jun 21, 2020

The complete script for installing ROS with python3.

# ensure there are no ros packages
sudo apt-get remove --autoremove ros-*

# check for updates
sudo apt update

# ensure /etc/ros removal
sudo rm -rf /etc/ros/

# install the python3 libraries 
sudo apt install -y python3 python3-dev python3-pip build-essential

# Remove python2
sudo apt purge -y python2.7-minimal

# link python -> python3
sudo ln -s /usr/bin/python3 /usr/bin/python

# Same for pip
sudo ln -s /usr/bin/pip3 /usr/bin/pip

# install the ros dependencies
sudo -H pip3 install rosdep rospkg rosinstall_generator rosinstall wstool vcstools catkin_tools catkin_pkg

# initialize catkin build environment
sudo rosdep init && rosdep update

# create catkin workspace
mkdir -p ~/ros_catkin_ws/src && cd "$_/.."

# initialize catkin workspace (will show warning about Extending... ignore that)
catkin config --init -DCMAKE_BUILD_TYPE=Release --blacklist rqt_rviz rviz_plugin_tutorials librviz_tutorial --install

# generate a ros melodic install
rosinstall_generator desktop_full --rosdistro melodic --deps --tar > melodic-desktop-full.rosinstall

# initialize the install
wstool init -j8 src melodic-desktop-full.rosinstall

# setup environment and install dependencies
export ROS_PYTHON_VERSION=3

# install wxPython
pip install -U -f https://extras.wxpython.org/wxPython4/extras/linux/gtk3/ubuntu-18.04 wxPython

# create install_skip file
printf '#/bin/bash\nif [ $(whoami) != root ]; then\n    echo You must be root or use sudo to install packages.\n    return\nfi\n\nfor pkg in "$@"\ndo\n    echo "Installing $pkg"\n    sudo apt-get -my install $pkg >> install.log\ndone' > install_skip

# make file executable
chmod +x install_skip

# install python 3 packages
sudo ./install_skip `rosdep check --from-paths src --ignore-src | grep python | sed -e "s/^apt\t//g" | sed -z "s/\n/ /g" | sed -e "s/\<python\>/python3/g"`

# skip python 2 packages
rosdep install --from-paths src --ignore-src -y --skip-keys="`rosdep check --from-paths src --ignore-src | grep python | sed -e "s/^apt\t//g" | sed -z "s/\n/ /g"`"

# rename all old python links to python3
find . -type f -exec sed -i 's/\/usr\/bin\/env[ ]*\<python\>/\/usr\/bin\/env python3/g' {} +

# remove all depricated install-layout=deb arguments
find ./ -name 'python_distutils_install.sh' -exec sed -i 's/--install-layout=deb//g' {} \;

# install python-empy
sudo apt install -y python-empy

# build ros
catkin build

# export python path
export PYTHONPATH=/usr/lib/python3/dist-packages

# source setup
source ~/ros_catkin_ws/install/setup.bash

I also had massive problems with installing wxTools, so I skipped as I didn't need it (everything I do is headless). I also changed my build to a ros_base instead of a desktop_full.

So if anyone is coming here to successfully build melodic on a headless raspberryPi ZeroW running rasbian buster the changes I needed were:

  1. CHANGE:
    rosinstall_generator desktop_full --rosdistro melodic --deps --tar > melodic-desktop-full.rosinstall
    TO:
    rosinstall_generator ros_base --rosdistro melodic --deps --tar > melodic-base.rosinstall

  2. CHANGE:
    wstool init -j8 src melodic-desktop-full.rosinstall
    TO:
    wstool init -j8 src melodic-base.rosinstall

  3. DELETE:
    pip install -U -f https://extras.wxpython.org/wxPython4/extras/linux/gtk3/ubuntu-18.04 wxPython

  4. CHANGE:
    sudo ./install_skip rosdep check --from-paths src --ignore-src | grep python | sed -e "s/^apt\t//g" | sed -z "s/\n/ /g" | sed -e "s/\<python\>/python3/g"
    TO:
    sudo ./install_skip rosdep check --from-paths src --ignore-src | grep python | grep -v wxtool | sed -e "s/^apt\t//g" | sed -z "s/\n/ /g" | sed -e "s/\<python\>/python3/g"
    (check the back quotes in these lines, the markdown is messing it up - basically added | grep -v wxtool to the chain.

Thanks to everyone here, I've now got ros1 running on a Pi using python3.

@yaobinwen

This comment has been minimized.

Copy link

@yaobinwen yaobinwen commented Jun 24, 2020

Thanks for the instructions! I'm following them and they are helpful! Can I also ask how you figured out the instructions? Did you get them from some other sources? I tried to search in the ROS official website and ROS source code repositories but didn't seem to find them. How did you find the instructions?

@CrazyPopLin

This comment has been minimized.

Copy link

@CrazyPopLin CrazyPopLin commented Jun 24, 2020

(Moved this from the blog for history preservation purpose)
Two errors were encountered during setting up with this tutorial:

  1. When catkin_make:
File "/usr/lib/python3.6/signal.py", line 47, in signal
TypeError: signal handler must be signal.SIG_IGN, signal.SIG_DFL, or a callable object

This will not affect launching roscore just as you posted before.

  1. When rosrun somepacakge somecode.py:
rosrun topic_demo pytalker.py
Traceback (most recent call last):
File "/root/catkin_ws/src/ROS-Academy-for-Beginners/topic_demo/scripts/pytalker.py", line 3, in <module>
import rospy
File "/root/ros_catkin_ws/install/lib/python3/dist-packages/rospy/__init__.py", line 47, in <module>
from std_msgs.msg import Header
File "/root/ros_catkin_ws/install/lib/python3/dist-packages/std_msgs/msg/__init__.py", line 1, in <module>
from ._Bool import *
File "/root/ros_catkin_ws/install/lib/python3/dist-packages/std_msgs/msg/_Bool.py", line 5, in <module>
import genpy
File "/root/ros_catkin_ws/install/lib/python3/dist-packages/genpy/__init__.py", line 34, in <module>
from . message import Message, SerializationError, DeserializationError, MessageException, struct_I
File "/root/ros_catkin_ws/install/lib/python3/dist-packages/genpy/message.py", line 44, in <module>
import yaml
File "/usr/lib/python3/dist-packages/yaml/__init__.py", line 284
class YAMLObject(metaclass=YAMLObjectMetaclass):
^
SyntaxError: invalid syntax

This is an interesting error.. Python seems to be complaining about the python3's metaclass syntax here. And with the traceback log, ROS is trying to run with python3 here but somehow ended up using python2 when the error happens.

Does anyone have any insight?

@zzh112119, Hi, I have same question with you. Is your problem related to YAMLObject Syntax solved by uninstall python2? which installed accidentally after install libboost?

@graemerae

This comment has been minimized.

Copy link

@graemerae graemerae commented Jun 24, 2020

Thanks for the instructions! I'm following them and they are helpful! Can I also ask how you figured out the instructions? Did you get them from some other sources? I tried to search in the ROS official website and ROS source code repositories but didn't seem to find them. How did you find the instructions?

Trial and Error :-)

(Moved this from the blog for history preservation purpose)
Two errors were encountered during setting up with this tutorial:

  1. When catkin_make:
File "/usr/lib/python3.6/signal.py", line 47, in signal
TypeError: signal handler must be signal.SIG_IGN, signal.SIG_DFL, or a callable object

This will not affect launching roscore just as you posted before.

  1. When rosrun somepacakge somecode.py:
rosrun topic_demo pytalker.py
Traceback (most recent call last):
File "/root/catkin_ws/src/ROS-Academy-for-Beginners/topic_demo/scripts/pytalker.py", line 3, in <module>
import rospy
File "/root/ros_catkin_ws/install/lib/python3/dist-packages/rospy/__init__.py", line 47, in <module>
from std_msgs.msg import Header
File "/root/ros_catkin_ws/install/lib/python3/dist-packages/std_msgs/msg/__init__.py", line 1, in <module>
from ._Bool import *
File "/root/ros_catkin_ws/install/lib/python3/dist-packages/std_msgs/msg/_Bool.py", line 5, in <module>
import genpy
File "/root/ros_catkin_ws/install/lib/python3/dist-packages/genpy/__init__.py", line 34, in <module>
from . message import Message, SerializationError, DeserializationError, MessageException, struct_I
File "/root/ros_catkin_ws/install/lib/python3/dist-packages/genpy/message.py", line 44, in <module>
import yaml
File "/usr/lib/python3/dist-packages/yaml/__init__.py", line 284
class YAMLObject(metaclass=YAMLObjectMetaclass):
^
SyntaxError: invalid syntax

This is an interesting error.. Python seems to be complaining about the python3's metaclass syntax here. And with the traceback log, ROS is trying to run with python3 here but somehow ended up using python2 when the error happens.
Does anyone have any insight?

@zzh112119, Hi, I have same question with you. Is your problem related to YAMLObject Syntax solved by uninstall python2? which installed accidentally after install libboost?

Did you check this? If the head of the file is :
#!/usr/bin/env python

Make sure that it points to python3

Or change it to:
#/usr/bin/python3
(or whatever your path is the executable)

@CrazyPopLin

This comment has been minimized.

Copy link

@CrazyPopLin CrazyPopLin commented Jun 24, 2020

@graemerae, thanks for your reply, after changing to #!/usr/bin/python3, it's working, cheer!

@Michdo93

This comment has been minimized.

Copy link

@Michdo93 Michdo93 commented Jun 28, 2020

Hi, I got following error.

Exception ignored in: <bound method BaseEventLoop.del of <_UnixSelectorEventLoop running=False closed=True debug=False>>
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/trollius/base_events.py", line 395, in del
File "/usr/local/lib/python3.6/dist-packages/trollius/unix_events.py", line 65, in close
File "/usr/local/lib/python3.6/dist-packages/trollius/unix_events.py", line 166, in remove_signal_handler
File "/usr/lib/python3.6/signal.py", line 48, in signal
TypeError: signal handler must be signal.SIG_IGN, signal.SIG_DFL, or a callable object

I tried to change this line #/usr/bin/python3 and it doesnt help. Also I tried sudo apt install -y python-empy and other solutions posted here. It doesnt work.

@CrazyPopLin

This comment has been minimized.

Copy link

@CrazyPopLin CrazyPopLin commented Jun 28, 2020

Hi, I got following error.

Exception ignored in: <bound method BaseEventLoop.del of <_UnixSelectorEventLoop running=False closed=True debug=False>>
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/trollius/base_events.py", line 395, in del
File "/usr/local/lib/python3.6/dist-packages/trollius/unix_events.py", line 65, in close
File "/usr/local/lib/python3.6/dist-packages/trollius/unix_events.py", line 166, in remove_signal_handler
File "/usr/lib/python3.6/signal.py", line 48, in signal
TypeError: signal handler must be signal.SIG_IGN, signal.SIG_DFL, or a callable object

I tried to change this line #/usr/bin/python3 and it doesnt help. Also I tried sudo apt install -y python-empy and other solutions posted here. It doesnt work.

Does roscore and a simple python publisher working? This problem should be fine.

@IndustrialEngStudent

This comment has been minimized.

Copy link

@IndustrialEngStudent IndustrialEngStudent commented Aug 20, 2020

Hi I ran into the following problem:

(base) laurah@laurah-HP-EliteBook-840-G1:~/ros_catkin_ws$ sudo ./install_skip `rosdep check --from-paths src --ignore-src | grep python | sed -e "s/^apt\t//g" | sed -z "s/\n/ /g" | sed -e "s/python/python3/g"`
Installing python3-psutil
Installing python33-paramiko
E: Unable to locate package python33-paramiko
Installing python33-rosdep-modules
E: Unable to locate package python33-rosdep-modules
Installing python33-coverage
E: Unable to locate package python33-coverage
Installing python3-rospkg
Installing python3-mock
Installing python3-catkin-pkg
Installing python3-matplotlib
Installing python3-numpy
Installing python33-pydot
E: Unable to locate package python33-pydot
Installing python33-pygraphviz
E: Unable to locate package python33-pygraphviz
Installing python33-pycryptodome
E: Unable to locate package python33-pycryptodome
Installing python33-gnupg
E: Unable to locate package python33-gnupg
Installing python3-opencv
Installing python33-empy
E: Unable to locate package python33-empy
Installing python33-defusedxml
E: Unable to locate package python33-defusedxml
Installing python33-nose
E: Unable to locate package python33-nose
Installing python3-opengl

I fixed this with "sudo apt-get install python3-[package]", now I get this when building:

error: option --install-layout not recognized
CMake Error at catkin_generated/safe_execute_install.cmake:4 (message):
  
  execute_process(/home/laurah/ros_catkin_ws/build/catkin/catkin_generated/python_distutils_install.sh)
  returned error code
Call Stack (most recent call first):
  cmake_install.cmake:171 (include)


make: *** [install] Error 1
cd /home/laurah/ros_catkin_ws/build/catkin; catkin build --get-env catkin | catkin env -si  /usr/bin/make install; cd -

...

[build]   Ignored:   33 packages were skipped or are blacklisted.                                                                                                              
[build]   Warnings:  None.                                                                                                                                                     
[build]   Abandoned: 203 packages were abandoned.                                                                                                                              
[build]   Failed:    1 packages failed.                                                                                                                                        
[build] Runtime: 4.0 seconds total.                                                                                                                                            
Exception ignored in: <bound method BaseEventLoop.__del__ of <_UnixSelectorEventLoop running=False closed=True debug=False>>
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/trollius/base_events.py", line 395, in __del__
  File "/usr/local/lib/python3.6/dist-packages/trollius/unix_events.py", line 65, in close
  File "/usr/local/lib/python3.6/dist-packages/trollius/unix_events.py", line 166, in remove_signal_handler
  File "/usr/lib/python3.6/signal.py", line 47, in signal
TypeError: signal handler must be signal.SIG_IGN, signal.SIG_DFL, or a callable object

I know that others experienced the same. I tried the solutions they used, but it does not help in my case.
What am I doing wrong?

@stephjo

This comment has been minimized.

Copy link

@stephjo stephjo commented Oct 1, 2020

[build] Ignored: 33 packages were skipped or are blacklisted.
[build] Warnings: None.
[build] Abandoned: 203 packages were abandoned.
[build] Failed: 1 packages failed.
[build] Runtime: 4.0 seconds total.
Exception ignored in: <bound method BaseEventLoop.del of <_UnixSelectorEventLoop running=False closed=True debug=False>>
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/trollius/base_events.py", line 395, in del
File "/usr/local/lib/python3.6/dist-packages/trollius/unix_events.py", line 65, in close
File "/usr/local/lib/python3.6/dist-packages/trollius/unix_events.py", line 166, in remove_signal_handler
File "/usr/lib/python3.6/signal.py", line 47, in signal
TypeError: signal handler must be signal.SIG_IGN, signal.SIG_DFL, or a callable object

I am getting the same error after following the recommended steps.I am trying install ros melodic on ubuntu 18.04.Please help.

@Ademord

This comment has been minimized.

Copy link

@Ademord Ademord commented Oct 11, 2020

@metalglove if i change your script from melodic to noetic it should work? i will test it today...

@metalglove

This comment has been minimized.

Copy link

@metalglove metalglove commented Oct 11, 2020

@metalglove if i change your script from melodic to noetic it should work? i will test it today...

There is a supported noetic version for Python3.

@Ademord

This comment has been minimized.

Copy link

@Ademord Ademord commented Oct 11, 2020

@metalglove thanks for taking the time to reply on a sunday :) i worked until 6 am yesterday haha!
I am not 100% sure what I did (well, our life), but I did manage to install on a ubuntu20.04 image, python3.8 and ros-noetic without building the same way you did (that's why I had uncertainty about it working).
If you have time maybe to explain to me why this worked compared to your solution I'd highly appreciate it as a learning experience :)

# Dockerfile
(i installed sudo and created a user above...)
ENV ROS_DISTRO=noetic

# install utilities
RUN apt-get install -y vim bash tmux git wget lsb-release lsb-core

# install the python3 libraries 
RUN apt-get install -y python3 python3-dev python3-pip build-essential

# link python -> python3
RUN ln -s /usr/bin/python3 /usr/bin/python

# Same for pip
RUN ln -s /usr/bin/pip3 /usr/bin/pip

# install ROS dependencies
RUN pip install rosdep rospkg rosinstall catkin_tools catkin_pkg

# install python-empy
RUN apt-get install -y python3-empy

# Install ROS
RUN sh -c 'echo "deb http://packages.ros.org/ros/ubuntu `lsb_release -cs` main" > /etc/apt/sources.list.d/ros-latest.list' && \
    apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-key C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654 && \
    apt-get update && \
    apt-get install -y ros-${ROS_DISTRO}-desktop \
    ros-${ROS_DISTRO}-pcl-ros \
    ros-${ROS_DISTRO}-pcl-conversions \
    ros-${ROS_DISTRO}-image-geometry


# initialize catkin build environment
RUN rosdep init

# adjust user
USER $USER

# fix permissions and update dependencies
RUN rosdep fix-permissions && rosdep update

# i  dont know what to name this comment
RUN echo "source /opt/ros/${ROS_DISTRO}/setup.bash" >> ~/.bashrc
RUN /bin/bash -c "source ~/.bashrc"

# setup catkin workspace << this just makes mkdir, catkin_init_workspace and catkin_make, in brief 
ENV CATKIN_WS ${HOME}/catkin_ws
COPY docker/init_workspace.sh ${HOME}
RUN ${HOME}/init_workspace.sh

Also, I am trying to build a docker image with ubuntu20.04, cuda11 (or 10) with cudnn8 (or 7), but all the available images (official) are either ubuntu18 or below with cuda10, cudnn7, and the ubuntu20 ones have only cuda11 and the package for cudnn8 do not exist yet.. ugh!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.