Skip to content

Instantly share code, notes, and snippets.

@weshouman
Last active October 4, 2023 07:18
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save weshouman/568d1e4d44c7a9454c3cc235d2b1136f to your computer and use it in GitHub Desktop.
Save weshouman/568d1e4d44c7a9454c3cc235d2b1136f to your computer and use it in GitHub Desktop.
apt tips and tricks
apt-cache show gives usually more raw information, but is less suitable for CLI.
## Differences
### apt-cache show
shows the architecture: `Architecture: amd64`
shows the Installed-Size in KiB may be?
shows the Download-Size as size, in Bytes may be?
shows the hashes
shows the Filename
shows the Description-LOCALE and Description-md5
### apt show
shows the Sizes in KB
shows whether the package is manually installed
shows the source
shows the Description without LOCALE # very useful for scripting
## Appendix
Following are the results for wireshark-qt for comparison
`apt-cache show`
```
Package: wireshark-qt
Architecture: amd64
Version: 3.6.7-1
Priority: optional
Section: universe/net
Source: wireshark
Origin: Ubuntu
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Balint Reczey <balint@balintreczey.hu>
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 9290
Depends: libc6 (>= 2.34), libgcc-s1 (>= 3.0), libgcrypt20 (>= 1.10.0), libglib2.0-0 (>= 2.67.3), libminizip1 (>= 1.1), libnl-3-200 (>= 3.2.21), libnl-genl-3-200 (>= 3.2.7), libnl-route-3-200 (>= 3.2.7), libpcap0.8 (>= 1.5.1), libqt5core5a (>= 5.15.1), libqt5gui5 (>= 5.14.1) | libqt5gui5-gles (>= 5.14.1), libqt5multimedia5 (>= 5.6.0~beta), libqt5printsupport5 (>= 5.2.0), libqt5widgets5 (>= 5.15.1), libspeexdsp1 (>= 1.2.0), libstdc++6 (>= 5.2), libwireshark15 (>= 3.6.0), libwiretap12 (>= 3.6.3), libwsutil13 (>= 3.6.1), zlib1g (>= 1:1.1.4), wireshark-common (= 3.6.7-1), libqt5svg5
Recommends: libqt5multimedia5-plugins
Breaks: wireshark (<< 2.0.0~), wireshark-gtk (<< 3.0.0~)
Replaces: wireshark (<< 2.0.0~), wireshark-gtk (<< 3.0.0~)
Filename: pool/universe/w/wireshark/wireshark-qt_3.6.7-1_amd64.deb
Size: 4273670
MD5sum: 8c5e17e90bd5e3bab5422768dec70665
SHA1: fd73f5d236d3f493a67cad4a05b55acee691736b
SHA256: 9e46b2298b3256bb51fbae1f953a173b5074286f7b862d3dc5b993b478ad4dfa
SHA512: 3b248d2f7f189330bbcb753a0860a568a771522d44ff7550e4990d216b727fe03d08f2ae0312a4b5bbdd40b1a59faa7350b2e8c2840372d0022610dcb87bb0c5
Homepage: https://www.wireshark.org/
Description-en_GB: network traffic analyzer - Qt version
Wireshark is a network "sniffer" - a tool that captures and analyzes
packets off the wire. Wireshark can decode too many protocols to list
here.
.
This package provides the Qt version of Wireshark.
Description-md5: 4aa78d5ff5497c82e96e6e723d3c5af6
```
`apt-show`
```
Package: wireshark-qt
Version: 3.6.7-1
Priority: optional
Section: universe/net
Source: wireshark
Origin: Ubuntu
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Balint Reczey <balint@balintreczey.hu>
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 9.513 kB
Depends: libc6 (>= 2.34), libgcc-s1 (>= 3.0), libgcrypt20 (>= 1.10.0), libglib2.0-0 (>= 2.67.3), libminizip1 (>= 1.1), libnl-3-200 (>= 3.2.21), libnl-genl-3-200 (>= 3.2.7), libnl-route-3-200 (>= 3.2.7), libpcap0.8 (>= 1.5.1), libqt5core5a (>= 5.15.1), libqt5gui5 (>= 5.14.1) | libqt5gui5-gles (>= 5.14.1), libqt5multimedia5 (>= 5.6.0~beta), libqt5printsupport5 (>= 5.2.0), libqt5widgets5 (>= 5.15.1), libspeexdsp1 (>= 1.2.0), libstdc++6 (>= 5.2), libwireshark15 (>= 3.6.0), libwiretap12 (>= 3.6.3), libwsutil13 (>= 3.6.1), zlib1g (>= 1:1.1.4), wireshark-common (= 3.6.7-1), libqt5svg5
Recommends: libqt5multimedia5-plugins
Breaks: wireshark (<< 2.0.0~), wireshark-gtk (<< 3.0.0~)
Replaces: wireshark (<< 2.0.0~), wireshark-gtk (<< 3.0.0~)
Homepage: https://www.wireshark.org/
Download-Size: 4.274 kB
APT-Manual-Installed: yes
APT-Sources: http://old-releases.ubuntu.com/ubuntu kinetic/universe amd64 Packages
Description: network traffic analyzer - Qt version
Wireshark is a network "sniffer" - a tool that captures and analyzes
packets off the wire. Wireshark can decode too many protocols to list
here.
.
This package provides the Qt version of Wireshark.
```

apt configurations are located in /etc/apt/ following is a breakdown for the different directories

Main Components

  1. sources.list: This is the main list of repositories that APT will use for software package management. Each line in this file specifies a different repository. Ref: man 5 sources.list

  2. sources.list.d/: This directory allows you to add additional repository lists without modifying the main sources.list file. Each file in this directory should end with .list and contain one or more repository lines, just like the main sources.list file. Ref: man 5 sources.list

  3. apt.conf and apt.conf.d/: These files and directories contains various configuration files that affect APT's behavior. These files are read in alphanumeric order, and they can override each other. Ref: man 5 apt.conf

  4. preferences and preferences.d/: These files and directories are used for APT pinning, a feature that allows you to prioritize packages from certain repositories or specific package versions. Ref: man 5 apt_preferences

  5. trusted.gpg and trusted.gpg.d/: These files and directories contain the public keys for the repositories. APT uses these keys to verify the integrity of the packages it downloads. Ref: man 8 apt-key

  6. auth.conf.d/: This directory can contain authentication information for private repositories, although it's less commonly used. Ref: man 5 apt_auth.conf

Note: General man page for apt is located at man 8 apt

Typical Folder Structure

Here's how the folder structure might look like:

/etc/apt/
├── apt.conf.d/
│   ├── 00aptitude
│   ├── 01autoremove
│   └── ...
├── preferences.d/
│   └── ...
├── sources.list
├── sources.list.d/
│   ├── official-package-repositories.list
│   └── ...
├── trusted.gpg
└── trusted.gpg.d/
    ├── debian-archive-jessie-automatic.gpg
    ├── debian-archive-jessie-security-automatic.gpg
    └── ...

Important Notes

  • Order Matters: In apt.conf.d/, the order in which the files are read can affect the final configuration. Files are read in alphanumeric order.

  • File Extensions: Make sure that files in sources.list.d/ end with .list. Otherwise, they won't be read by APT.

  • Security: Be cautious when adding new repositories or keys to your APT configuration, as malicious or poorly-maintained repositories can compromise your system.

The line deb http://archive.debian.org/debian jessie-backports main which is a typical entry in a Debian a .list file either in sources.list or within the sources.list.d/. This line informs the APT package management system where to find packages for installation, upgrade, and various other operations.
Following is a breakdown for this line.

  1. deb: This keyword indicates that the repository contains binary packages, which are pre-compiled packages that can be installed directly. The other option is deb-src, which indicates source packages.

  2. http://archive.debian.org/debian: This is the URI (Uniform Resource Identifier) of the repository. It specifies where the packages can be downloaded from. This can also be an FTP address like ftp://.

  3. jessie-backports: This is the distribution or "suite" of the repository. In this case, it refers to the backports for the Debian Jessie release. Backports are packages taken from a newer version of Debian and adjusted to work on an older release.

  4. main: This is the component or "section" of the repository. Debian repositories are usually divided into three sections: main, contrib, and non-free.

    • main: Contains free software that complies with Debian's Free Software Guidelines.
    • contrib: Contains free software, but depends on software in non-free.
    • non-free: Contains software that doesn't comply with Debian's Free Software Guidelines.

So, putting it all together:

  • deb: We're using binary packages.
  • http://archive.debian.org/debian: We're downloading from this repository.
  • jessie-backports: We're using the backports for Debian Jessie.
  • main: We're using the main section of the repository, which contains free software.

This line would be used to tell APT where to find packages that are backported from newer Debian releases to Jessie and are in the main section of the repository.


A more advanced example would include using options.
The format of the source line is deb [ options ] uri distribution [component1] [component2] [...]
A source with options could be deb [check-valid-until=no] http://archive.debian.org/debian jessie-backports main Here the [check-valid-until=no] part is an option that modifies the behavior of APT when it interacts with this specific repository.

[check-valid-until=no]

  • check-valid-until: This is the name of the option. It instructs APT to check (or in this case, not to check) the "Valid-Until" field in the Release file of the repository. Normally, APT checks this field to ensure that the metadata and packages from the repository are still considered valid and safe to install. If the date in the "Valid-Until" field is in the past, APT will refuse to install packages from this repository as a safety measure.

  • no: This is the value assigned to the check-valid-until option. Setting it to no disables the check for the "Valid-Until" field. This is useful for archived or outdated repositories where the "Valid-Until" date has passed but you still need to install packages from there.

Why Use This Option?

The check-valid-until=no option is generally used when you're dealing with archived or legacy repositories, like jessie-backports in this case. These repositories are no longer updated, and their "Valid-Until" dates have likely passed, making them invalid by default in the eyes of APT. By setting check-valid-until=no, you're telling APT to ignore this and proceed with fetching packages from this repository.

Caution

Disabling this check can expose you to potential risks, as you're essentially telling APT to ignore a built-in security measure. Make sure you understand the implications and risks before using this option.


Are curious to know about more options?

Certainly! Options in the sources.list file can be quite versatile and allow for a range of customizations. They are usually specified in square brackets [...] immediately after the deb or deb-src keyword. Here are some commonly used options:

Common Options

  1. arch: Specifies the architecture for which packages should be downloaded. Example: [arch=amd64].

    deb [arch=amd64] http://my-repo.com/debian stable main
    
  2. trusted: Specifies whether the repository is trusted. yes or no option.

    deb [trusted=yes] http://my-repo.com/debian stable main
    
  3. signed-by: Specifies a key file that should be used to verify the signature of the packages. Example: [signed-by=/usr/share/keyrings/my-keyring.gpg].

    deb [signed-by=/usr/share/keyrings/my-keyring.gpg] http://my-repo.com/debian stable main
    
  4. allow-insecure: Allows the use of an insecure repository. Use this with caution. yes or no option.

    deb [allow-insecure=yes] http://my-repo.com/debian stable main
    
  5. allow-downgrade-to-insecure: Allows downgrading to an insecure repository. Use this with extreme caution. yes or no option.

    deb [allow-downgrade-to-insecure=yes] http://my-repo.com/debian stable main
    

Combining Multiple Options

We can also combine multiple options, separating them with spaces:

deb [arch=amd64 trusted=yes] http://my-own-repo.com/debian stable main

Caution

While these options provide flexibility, they can also introduce security risks if not used carefully.

Q: When I type apt-cache show, where does debian check for the packages?

A: When you run the command apt-cache show, the system checks the package cache to retrieve and display information about a package.
The package cache is a local repository of metadata about available packages that is stored on your system.
It's used by the package manager (apt in this case) to quickly retrieve information about packages without having to download the entire package files.

The package cache is typically located in the /var/cache/apt directory.
Within this directory, you'll find subdirectories for each repository you have configured on your system.
These subdirectories contain the metadata and index files necessary for apt to provide information about packages, dependencies, and other related data.

When you run apt-cache show, the command accesses the appropriate cache files to display detailed information about the specified package, including its description, version, dependencies, and other relevant data.


Q: what's inside /var/cache/apt/archives?

A: The /var/cache/apt/archives directory contains cached copies of the downloaded .deb package files.
These files are the actual binary packages that apt retrieves from repositories when you install or upgrade software on your system.
By storing these package files locally, apt can avoid re-downloading them if they are already present in the cache.
This can save both time and bandwidth when installing or upgrading packages.

The files in the /var/cache/apt/archives directory have filenames corresponding to the package names and versions.
For example, if you've downloaded the package "example-package" version 1.2.3, you might find a file named example-package_1.2.3.deb in this directory.

If you were to install the same package again or upgrade it to a newer version, apt would check the local cache first before attempting to download the package again from the repository.
If the required package file is found in the cache, it will be used for the installation or upgrade.
If not, apt will download the package from the repository and store it in the cache for future use.

This local package cache helps improve the efficiency of package management operations on your system by reducing the need to repeatedly download the same package files.


Q: What about the content of /var/cache/apt/pkgcache.bin?

A: The /var/cache/apt/pkgcache.bin file is a binary cache file that contains a serialized representation of the package cache metadata.
This file is used by apt to quickly access and search package information without having to parse and process the individual package metadata files from the repositories every time a package-related operation is performed.

The pkgcache.bin file is generated by apt to speed up package management operations, such as package searches, dependency calculations, and generating package lists.
By storing the metadata in a binary format, apt can read and access this data more efficiently than parsing the plain text metadata files stored in the repository.

It's important to note that while the pkgcache.bin file accelerates package operations, it doesn't store the actual package files (.deb files) themselves.
Those are stored in the /var/cache/apt/archives directory, as mentioned earlier.
The pkgcache.bin file specifically stores metadata about available packages, their dependencies, versions, and other related information.

It's generally safe to leave this file alone, as apt manages it automatically.
However, if you encounter issues with package information or if you suspect corruption, you might consider refreshing the cache using commands like apt update or apt-get update, which will regenerate both the binary cache and the individual package metadata files.


Q: Where are the individual package metadata files?

A: The individual package metadata files in a Debian-based system are typically stored in the /var/lib/apt/lists directory. This directory contains plain text files that list the available packages and their associated information from the repositories configured on your system.
These files are generated during the apt update process, which fetches information from the repositories and updates the package cache.

Inside the /var/lib/apt/lists directory, you'll find subdirectories corresponding to the repositories you have configured.
Within these subdirectories, there are files with names that represent the repository they belong to.
These files contain the metadata for the packages available in that repository, including package names, versions, dependencies, and other package-related details.

For example, you might find files like archive.ubuntu.com_ubuntu_dists_focal_main_binary-amd64_Packages or security.debian.org_debian-security_dists_buster_updates_main_i18n_Translation-en.
These files are human-readable but are not meant to be directly edited. They are used by apt to quickly access information about packages without having to query the remote repository servers every time.

The apt update command fetches the latest information from the repositories and updates these files.
The apt-cache and apt commands then use the data in these files to provide package information, perform dependency calculations, and manage package installations and upgrades.

Simulate Update

Run apt update --print-uris: See which uris will be fetched

Compare that with:

# the content of the sources.list
cat /etc/apt/sources.list | grep -v '^\s*$\|^\s*\#'

# Or even with the full list 
(cat /etc/apt/sources.list && cat /etc/apt/sources.list.d/*.list) | grep -v '^\s*$\|^\s*\#'

You'll notice that this gives a hint of which sources would be fetched.

Update

  • Rename the file archive.debian.org_debian_dists_jessie-backports_Release to archive.debian.org_debian_dists_jessie-backports_Release.old.
  • Rename the file /var/cache/apt/pkgcache.bin to /var/cache/apt/pkgcache.bin.old.

Then run apt update.

You'll find that the old files were restored and the new files got removed.

Simulate Upgrade

Let's pick a single upgradeable package, for example, curl

Output of apt-cache show curl | grep Version:

Version: 7.88.1-10+deb12u1~bpo11+1
Version: 7.74.0-1.3+deb11u7
Version: 7.74.0-1.3+deb11u3

Which means that after the last update we know of 3 curl versions.

Then we can simulate upgrading the package by running apt-get install curl --simulate which would output

Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  libcurl4
The following packages will be upgraded:
  curl libcurl4
2 upgraded, 0 newly installed, 0 to remove and 47 not upgraded.
Inst curl [7.74.0-1.3+deb11u3] (7.74.0-1.3+deb11u7 Debian:11.7/oldstable, Debian-Security:11/oldstable-security [amd64]) []
Inst libcurl4 [7.74.0-1.3+deb11u3] (7.74.0-1.3+deb11u7 Debian:11.7/oldstable, Debian-Security:11/oldstable-security [amd64])
Conf curl (7.74.0-1.3+deb11u7 Debian:11.7/oldstable, Debian-Security:11/oldstable-security [amd64])
Conf libcurl4 (7.74.0-1.3+deb11u7 Debian:11.7/oldstable, Debian-Security:11/oldstable-security [amd64])

We can see here that the Installed is 7.74.0-1.3+deb11u3 and it would be upgraded to 7.74.0-1.3+deb11u7

Note: we could run apt-get upgrade --simulate to show all upgrades that would take place

Upgrade

Run the previous section's commands without --simulate

Invasive Upgrade

  • apt-get dist-upgrade: Has some conflict management abilities and it may remove some packages. man apt-get Named dist-upgrade as it could updgrade across packages to the newer distribution
  • apt full-upgrade: Same as apt-get dist-upgrade, with a different naming. man apt Named full-upgrade to avoid thinking that this command would do any distribution upgrades, it only upgrades packages.

Upgrade Logs

To check the logs of the upgrade, run grep "upgrade " /var/log/dpkg.log

Distribution Upgrade

sudo apt update && sudo apt upgrade
sudo reboot
sudo apt install update-manager-core
sudo do-release-upgrade
sudo reboot

Distribution Upgrade Logs

To check the logs of the upgrade, check the content of ls -l /var/log/dist-upgrade

Update Issues

Reaching End Of Life Before Upgrading

Version reached end of life before upgrade, for example having Ubuntu Kinetic 22.10, which reached end of life on 23.07, on 23.09 on can't update or upgrade due to the errors E: Failed to fetch http://de.archive.ubuntu.com/ubuntu/pool/universe/e/endeavour/endeavour-common_42.0-4_all.deb 404 Not Found [IP: 141.30.62.24 80] And when trying to upgrade the dist, one would get

$ do-release-upgrade 
Checking for a new Ubuntu release
Your Ubuntu release is not supported anymore.
For upgrade information, please visit:
http://www.ubuntu.com/releaseendoflife
Please install all available updates for your release before upgrading.

Fix: take a copy of the sources then move the urls manually to the old-releases.

cp /etc/apt/sources.list{,.backup}
sudo sed -i -re 's/([a-z]{2}\.)?archive.ubuntu.com|security.ubuntu.com/old-releases.ubuntu.com/g' /etc/apt/sources.list
sudo apt update

Only then, the guide here could be followed again

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment