YaST Hostnames - Test plan

explanation-ip-addresses.md

Complete /etc/host:

127.0.0.1 localhost
130.57.5.70 suse.com suse.br suse.ae
130.57.5.70 suse.com suse.br suse.ae

::1 localhost ipv6-localhost ipv6-loopback
fe00::0 ipv6-localnet
1001:db8:0:0:0:ff00:10:8000 äbç
2001:db8::ff00:43:9000 АБВГДЕ

# 147.67.34.45 europa.eu
# special IPv6 addresses
# ff02::4 example3

256.0.0.0 invalid
gg::1 also.invalid

Why these values?

There are two common ways to decide on values to test: Boundary value analysis and Equivalence classes.

Boundary value analysis means to take values that are just at the edge of where behavior changes (before and after). This was used for the two negative examples (although I did not test before, just after the border of what is working).

Equivalence class (EC) means that two values can be considered equivalence, the behavior should not change. Therefore, not every imaginable value needs to be tested, just one of every EC.

• 127.0.0.1 localhost Represents a system IPv4 address
• 130.57.5.70 suse.com suse.br suse.ae 130.57.5.70 suse.com suse.br suse.ae Represents a duplicate entry, an entry with one hostname and multiple aliases and a public IPv4 address
• ::1 localhost ipv6-localhost ipv6-loopback Represents a system IPv6 address, an entry with one hostname and multiple aliases and a leading shorthand notation of IPv6 addresses
• fe00::0 ipv6-localnet Represents a system IPv6 address, an entry without any aliases and a shorthand notation in the middle
• 1001:db8:0:0:0:ff00:10:8000 äbç Represents a public IPv6 address, an entry with unicode characters in the hostname and using "0" for groups of all "0"s
• 2001:db8::ff00:43:9000 АБВГДЕ Represents a public IPv6 address, an entry with unicode characters in the hostname and using "0" for groups of all-"0"s
• # 147.67.34.45 europa.eu Represents an IPv4 address that is commented out
• # special IPv6 addresses Represents a comment
• # ff02::4 example3 Represents an IPv6 address that is commented out
• 256.0.0.0 invalid Represents an invalid IPv4 address
• gg::1 also.invalid Represents an invalid IPv6 address

Observations:

• Hostname and IP parts can be validated separately to gain more precision on test results. Validating one equvalence group of IP address and one of hostnames at the same time is more time and performanc efficient, as two different things are validated at once. This is a tradeoff I made because I value test resources higher than precision in this case.

testplan.md

YaST Hostnames Module test plan

Introduction

YaST Hostnames is a module that helps system administrators with configuring their /etc/hosts file. It has a Terminal and a Graphical User Interface and validates input.

Features to be Tested

Inlcuded

• Terminal UI
• Graphical UI
• File reading + parsing
• File saving
• Input validation
• Aborting without damaging the existing content
• Module can handle full or read-only disk
• Reaction times
• Running as normal user

Excluded

• Ease of use
• Memory usage

Test Strategy

The most important component is that saving the configuration creates a working file, i.e. it is not corrupted or damaged. It is equally important that cancellation keeps the old configuration intact. Therefore the biggest focus is on disk IO. Functionality only needs to be tested once, one of the two UIs is used to test the functionality while the other is just checked for visual correctness. Input validation, correct parsing of existing /etc/hosts file and aborting is tested in the GUI, while the TUI only needs to render correctly and have working keyboard shortcuts.

Test cases 1 and 2 will be executed on ARM, Power. All test cases will be executed on x86_64, using Qt GUI on GNOME (Xorg). Additionally, test case 5 is executed using the ncurses TUI and Qt on GNOME (Wayland). On s390, only ncurses is used as UI in the tests.

Test cases

1. File Parsing

   Write the following to /etc/hosts:

   127.0.0.1 localhost
   130.57.5.70 suse.com suse.br suse.ae
   130.57.5.70 suse.com suse.br suse.ae
   
   ::1 localhost ipv6-localhost ipv6-loopback
   fe00::0 ipv6-localnet
   1001:db8:0:0:0:ff00:10:8000 äbç
   2001:db8::ff00:43:9000 АБВГДЕ
   
   # 147.67.34.45 europa.eu
   # special IPv6 addresses
   # ff02::4 example3
   
   256.0.0.0 invalid.example
   gg::1 also.invalid
   

   Check the UI output. Every line after the first with a leading "#" must not show up. Every other entry must show up in the UI output: IP in IP field, first hostname in "hostname" field, all other hostnames in "aliases" field. Duplicates must not appear more than once.

2. File Writing

   Add the following via the UI:

   IP: 100.100.100.1
   Hostname: onehost.example
   Aliases: äbç, АБВГДЕ  
   
   IP: 2000::3000
   Hostname: ipv6-example.intern
   
   IP: 195.135.221.143
   Hostname: static.opensuse.org
   Aliases: html5test.opensuse.org
   

   Disable DNS resolution and ping static.opensuse.org to see if the entry works correctly.

3. Input Validation

   Try to enter the following in the IP field in the UI:

   0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ.:!@#$%^&*[](){}"'
   

   Check that only these characters are in the field:

   0123456789abcdefABCDEF.:
   

4. System entries are protected

   Delete "localhost" entry for IPv4 in the UI. A popup must show up, asking for verification. Ensure nothing changes when cancelling.

5. UI is navigatable

   Check that all text labels are readable and in their correct place. This includes line wrapping. Buttons must be distinct from the background and keyboard shortcuts must be documented. They must work as well.

6. Translations are present and correct

   Run the module in a different language environment and check that no English words are left, except for technical names.

7. full / partition

   Fill / completely and try to open the GUI. An error message should pop up, notifying users that the module is in read-only mode and no changes can be made. There should also a persistent notice that reminds users of read-only mode. Current configuration should still be displayed.

8. read-only /etc/hosts

   Use chmod u=r,go= /etc/hosts to only allow reading for root and start the GUI. . An error message should pop up, notifying users that the module is in read-only mode and no changes can be made. There should also a persistent notice that reminds users of read-only mode. Current configuration should still be displayed.

9. normal user running hostnames module

   After starting the GUI, don't enter the root password but click on "ingore". The module should display that it runs in read-only mode and any changes will be lost on saving.

10. no /etc/hosts

Delete /etc/hosts, then start the GUI. The system defaults (i.e. like a freshly installed system) should appear in the UI.

11. /etc/hosts is replaced while gui is running

Open the GUI, add an entry. Replace /etc/hosts with the version from test case 1. Save the changes made in the GUI. A popup notifying that /etc/hosts was changed should appear, asking the user for comfirmation.

12. Load time is smaller than 15s for /etc/hosts files < than 2k lines

Use a prepared /etc/hosts that contains 2000 lines of IP + Hostname pairings. The time from starting the module until it is usable, i.e. the content is displayed and can be modified.

Acceptence Criteria

All planned tests are executed without faults regarding disk IO, configuration parsing, input validation and keyboard shortcuts.

Test Documentation

The test results are integrated into OpenQA. This can either be as part of an existing job group or a new, YaST sepecific job group. The tests themselves don't have to use OpenQA as long as the results appear in it.