Skip to content

Instantly share code, notes, and snippets.

@luc14n0
Last active December 30, 2021 04:51
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save luc14n0/1a0b36badf5c3ce1304ff09c3475a05c to your computer and use it in GitHub Desktop.
Save luc14n0/1a0b36badf5c3ce1304ff09c3475a05c to your computer and use it in GitHub Desktop.
Using DNF for upgrading openSUSE Tumbleweed while Curl error (16) is sorted out.

openSUSE vs Curl error (16)

We've been fighting against those Curl errors for a while now (Dec 27, 2021) and it seems that those errors (namely 16 and 52) started bashfully but with time it grew stronger and stronger.

I use Tumbleweed, just like many people, and at almost every geeko # zypper ref or geeko # zypper dup I end up with headaches. When I see a new Tumbleweed Snapshot went out I know what's coming. Since we don't know yet what's really going on that's triggering those errors we have to cope with it until things are sorted out, and DNF might be the solution for now.

What DNF does differently than Zypper

For those wondering, the Curl error (16) is not particular to Zypper, even DNF will run into it (or bare curl commands if you're patient enough). Another thing we know is that if a package fails to be retrieved, trying again will solve the issue because it happens in a sporadic manner, maybe it will take a couple tries, maybe several, but the package will be downloaded eventually.

Some of you may know that Zypper is suppose to automatically retry the download if, for whatever reason, a package can't be retrieved, just like DNF does. Zypper should, by default, try 5 times while DNF 10. However Zypper is not trying at all. In a Tumbleweed VM I have which today has 979 packages to be updated I tried setting download.max_silent_tries to 50 in /etc/zypp/zypp.conf. First geeko # zypper ref I press "enter" Zypper shouts out "Curl error 16" and tells me it tried 5 times (you sneaky sneaky), so that part of the code where it handles the silent tries must be broken in libzypp right now.

Next step I installed DNF and tried to do a distro upgrade. It almost went flawless, even though there were several packages that where caught by the "Error (16)" DNF retried and only one package hit the 10 retries treshold and DNF exited. After another try, again several packages where caught by the "Error (16)", but DNF managed to download all the remaining packages this time. So what can be done for unnattended DUPs?

Setting up DNF

Setting up DNF to retry more than 10 times is easy peasy. First you need to install DNF of course and a helper config to show DNF what your current repos are:

geeko # zypper in dnf libdnf-repo-config-zypp

Now you must be wondering "Oh man! Do I really need to learn DNF commands?" and the answer is NO, not at all. Our fine openSUSE folks already set up some command aliases so we can use some DNF commands just like we do with Zypper. We'll only need two commands:

dnf ref and dnf dup

That's all, for the rest we can keep using Zypper. If someone asks, dnf ref is an alias for dnf makecache and dnf dup is dnf distro-sync.

But before you go upgrading your system while you go somewhere else, you need to do something first:

geeko # echo -e "retries=25\ntimeout=5\nmax_parallel_downloads=4\nfastestmirror=True" >> /etc/dnf/dnf.conf

This will append a new line (note the double >) to dnf.conf with the configuration we want, if you think you're such an unlucky bast... I mean guy/lady you could set it to 0 for unlimited retries, though you're allowed to do it if you blow up some mirrors while upgrading your system, please don't mention my name. I'll deny everything.

P.S. I

In the case you notice the download of a package is taking a while to start, it's just DNF doing its silent retries, it has a default timeout of 30 seconds so depending how frequent you run into the "Error (16)" the overall system upgrade duration can take quite some time. If you're using HTTPS the time will be longer because of the TLS authentication, and sorts. I tested a couple of dnf dups using HTTPS and all went well, but comparing with plain HTTP it took longer.

P.S. II

After a some fine tuning using dnf.conf options like: timeout=5, for a not too long wait for the next retry and max_parallel_downloads=4 for more download at the same time (careful not to abuse it or you'll roast some kittens) and fastestmirror=True for obvious reasons, one can fancy a less overall time taken for their system to upgrade. Shout outs to Sergey Kondakov for providing the input.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment