Skip to content

Instantly share code, notes, and snippets.

@KodyKantor
Last active June 30, 2020 18:32
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save KodyKantor/c12159e82f4cfef133fb72d252379102 to your computer and use it in GitHub Desktop.
Save KodyKantor/c12159e82f4cfef133fb72d252379102 to your computer and use it in GitHub Desktop.

Guide for reviewing SAS topology enumeration

Introduction

Rob Johnston previously integrated changes to libtopo to support directed graph topologies. That work, 12330, provided the tools necessary to represent SAS topologies.

This commit addresses 12331. The SAS topology enumeration code relies on a few existing tools, some well known and some less known:

  • libsmp and the associated SMP driver.
  • libsmhbaapi and the associated code in the mptsas driver. libsmhbaapi is a library developed by SNIA with a goal of creating an interface for interacting with HBAs. This is effectively third party code and doesn't follow illumos style guidelines. It ships with illumos and is commonly referred to as SM-HBA API. libsmhbaapi provides us with an API to retrieve topology information from the HBA, and HBA phy link error counters. We tried to keep our usage of libsmhbaapi to a minimum due to its foreign API and questions about maintainability.
  • libtopo HC scheme. The SAS enumeration code calls back into libtopo for retrieving some hardware device properties.

Code review: gerrit.

In the blocks below I'll describe the purpose of the larger file changes and additions.

State of the code

This code is in a 'works on some hardware' state. There are certain assumptions made (often with a code comment stating the assumption) that will limit functionality on hardware with complicated SAS topologies. For example, some places in the code don't account for dual-port SAS targets and topologies that involve SAS multipathing will almost surely result in the software crashing.

We've been able to test this on a number of simple SAS topologies without issues. A large portion of getting this to a 'done' state will involve testing on more exotic hardware and working through design changes necessary to accomodate such hardware.

There are certainly a large number of problems (both in logic and style) with this code as it stands today.

Although I do not have time or hardware to dedicate to developing this further I thought it would be valuable to make sure this code makes it off of my machine.

cmd/fm/sastopo/common/sastopo.c

sastopo.c is the administrator's CLI tool for accessing SAS topology mappings. It consumes the public libtopo interface for interacting with directed graphs, specifically those for SAS topologies. It can pretty-print node properties (manufacturer info, phy link errors) and directed graphs to stdout, or optionally serialize directed graphs into XML for consumption by other tools.

lib/fm/topo/libtopo/common/sas.c

sas.c is the main entrypoint for SAS topology enumeration. There are a handful of useful block comments describing the logic of each piece of the topology enumeration logic. The top block comment should be helpful in describing the general approach to topology enumeration.

It also includes libtopo API wrappers. For example, sas_create_vertex contains the logic to create directed graph vertices and assign property groups to each topo node.

lib/fm/topo/libtopo/common/sas-method.c

sas-method.c contains a number of utility functions. Some wrap logic to get SCSI device pages (e.g. 'scsi_log_sense') and others interact with libsmhbaapi and libsmp to get/set topo node properties (e.g. 'sas_get_adapter_phy_err_counter').

sas.h

sas.h includes additions of hardware page descriptions we can access from SAS devices.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment