slack about how many threads by default.txt

Fons van der Plas:pluto:  27 minutes ago
Why does julia not start with the maximum number of threads by default?






Mosè Giordano  23 minutes ago
I can see a couple of reasons:
1. historical
2. using the maximum number of threads available isn't often a good idea

Sebastian Pfitzner  20 minutes ago
mainly 1

Harmen Stoppels  20 minutes ago
but julia uses all threads (or does not set OMP_NUM_THREADS) for BLAS since ancient history

Fons van der Plas:pluto:  19 minutes ago
is the maximum minus one a good idea? i am setting Pluto's default to the maximum and wondering what will go wrong

Mosè Giordano  19 minutes ago
it really depends on the type of workload

Mosè Giordano  19 minutes ago
and if there is hyperthreading

Sebastian Pfitzner  18 minutes ago
Juno has been using core count for nthreads for ages now

Sebastian Pfitzner  18 minutes ago
people have mostly been happy with that

Mosè Giordano  17 minutes ago
"core" as in "physical cores"?

Sebastian Pfitzner  17 minutes ago
yes

Mosè Giordano  17 minutes ago
how do you get that?

Sebastian Pfitzner  16 minutes ago
there's a JS library for that :stuck_out_tongue: and it seemed to work fairly well

Mosè Giordano  16 minutes ago
oh, I see :smile:

Sebastian Pfitzner  15 minutes ago
there's a Julia package too

Sebastian Pfitzner  15 minutes ago
cpuid.jl or something like that (edited) 

Mosè Giordano  15 minutes ago
https://github.com/m-j-w/CpuId.jl?

Mosè Giordano  14 minutes ago
it's unmaintained, doesn't even have the Project.toml :sweat_smile:

Harmen Stoppels  14 minutes ago
You have to be a bit careful on Linux, since it has cgroups to restrict number of cpus usable per process. For instance:
harmen-desktop ~ $ nproc 
16
harmen-desktop ~ $ taskset -c 0-1 nproc
2

Harmen Stoppels  13 minutes ago
so setting number of threads to number of cores is not always correct

Sebastian Pfitzner  11 minutes ago
fair point, but also a bit of an edge case

Harmen Stoppels  11 minutes ago
for instance python's multiprocessing gets this wrong :man-shrugging: multiprocessing.cpu_count() just looks up the number of cores the hardware has

Harmen Stoppels  11 minutes ago
not an edge case in HPC though :stuck_out_tongue:

Harmen Stoppels  10 minutes ago
but maybe HPC is an edge case itself

Sebastian Pfitzner  10 minutes ago
I feel like you shouldn't use your IDEs automatics config on HPC though
:ok_hand:
1


Fons van der Plas:pluto:  9 minutes ago
how about I just use max(Sys.CPU_THREADS() ÷ 2, 1)  as the default and call it a day (edited) 

Mosè Giordano  8 minutes ago
what if one has only one thread? :laughing:

Fons van der Plas:pluto:  8 minutes ago
:slightly_smiling_face: edited
:smile:
1


Théo Galy-Fajou  6 minutes ago
I am happy that Julia don't use all my threads by default as it tends to freeze my computer entirely when I run heavy computations...

Fons van der Plas:pluto:  3 minutes ago
hey theo!

Fons van der Plas:pluto:  3 minutes ago
but threads / 2 is okay then, right?
New

Théo Galy-Fajou  2 minutes ago
Hey :D! Yeah that definitely sounds reasonable!