Skip to content

Instantly share code, notes, and snippets.

@thinkhy

thinkhy/readnote_uss.txt

Created April 4, 2012 14:22
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save thinkhy/2301530 to your computer and use it in GitHub Desktop.
Save thinkhy/2301530 to your computer and use it in GitHub Desktop.
Download ZIP
Reading note for UNIX System Services z/OS Version 1 Release Implementation
Raw
readnote_uss.txt
#+TITLE: Reading Note for UNIX System Services Implementation
#+AUTHOR: thinkhy
#+DATE: 05/06/2012 Sun
* ********************************************************
* gist: https://gist.github.com/gists/2301530
* Book: z/OS UNIX System Services Implementation
* [[www.redbooks.ibm.com/redbooks/pdfs/sg247035.pdf][Redbook]]
* ********************************************************
** TODO
** DONE
** DONE Read 1.3.8 CLOSED: [4/20/2012]
* 1.1.4 UNIX
** 1> UNIX functionally organized at three levels: kernel, shell and utiliti.
kernel and shell form the os, while utilities have evovled over time to make os more immediately useful to the user.
** 2> Kernel
core of UNIX, consists of a small collection of software:
- processes
- A file system
- Communications(TCP/IP?)
- A means to start the system
Kernel functions: autonomous and responsive
** 3> Processes
A process is the execution
** 4> Signals
processes communicate with each other and with the kernel is through signals.
** 5> Virtual memory
paging and swapping technques similar to MVS.
** 6> Shell
- Bourne shell(sh)
- C shell(csh)
- Korn shell(ksh)
- TC shell(tcsh)
- Bourne Again shell(bash)
** 7> Utilities
utility programs(often refered to as commands)
- Editing
- File maintenance
- Printing
- Sorting
- Programming support
- Online information
* 1.1.5 UNIX file system
A UNIX file system is a data structure or a collection of files.
** Logical file system
tree or inverted pyramid
** Physical file system
superblock, inodes, and data blocks
** File and directory permissions
read, for a directory, read permission enables a user to find out what files are in that directory.
write,for a directory, the user can create and delete files in that d
execute: for a directory, execute permission allows a user to change to that derectory.
** Physical file system
The physical file system is divided first by disk partitions.
* 1.1.6 Parameter files
/etc : SYS1.PARMLIB
* 1.1.7 Daemons
Daemons are like Started Tasks(STCs) in MVS
** run continuously
** handling periodic service requests
** forwards the requests to other programs
* 1.1.8
UIDs and GIDs: a number between 0 and 65,535, where 0 thru 99 might be reserved, UID=0 has sepcial meaning as the superuser.
* 1.1.9 UNIX standards
POSIX ==> IEEE standards, regi
POSIX is an evolving family of standards describing a wide spectrum of
operating system components.
Organization for Standardization(ISO)
X/Open Portability Guides(XPG)
POSIX and 1003.1 are registered trademarks of IEEE
* 1.1.10 MVS and UNIX functional comparison
** pax ==> belong to POSIX Standard? Yes, pax belong to POSIX and XPG4.2.
** Promary configuration: IEASYSxx VS. BPXPRMxx?
** Program products: LNKLST and /bin
** Test programs: STEPLIB VS. /sbin
** Resident programs: LPA vs. sticky bit
* 1.2
** two Open Systems interfaces on z/OS UNIX:
- 1> API: C
- 2> Shell interface
both interfaces are through LE.
** z/OS Shell
modeled after UNIX System V shell, similar with Korn Shell.
It's upward-compatible with the Bourne shell.
** tcsh shell
It's an enhanced but completely compatible version of the Berkeley UNIX C shell, csh.
* 1.2.1 Dub and undub
Dub is a term that means to make an MVS address space known to z/OS UNIX System Service. Once dubbed, an AS is considered to be a "process".
MVS AS become dubbed if they invoke a UNIX service.
A dubbed task is considered to be a "thread".
Undub: inverse of dub. Notice: AS undubbed when the last dubbed thread ends.
* 1.2.2 z/OS UNIX support
* System Services provide:
- XPG4 UNIX 1995 conformance
- Assembler callable services
- TSO/E commands to manage the file system.
- IShell environment
* Application Services
Application Services(FMID HOTxxxx) provides
- TSO/E
- develop and run app.
- Utilities to admin and develop
- dbx
- socket
- rlogin and inetd
- telnet
- full-screen app.(curses support)
* 1.2.3 Interaction with elements and features of z/OS
** WLM, BCP element
The kernel uses WLM to create child processes.
BPXAS PROC in SYS1.PROCLIB: provide a new AS.
type of process
- User processes
- Daemon processes
[what is Cataloged procedure?]
A thread is a single flow of control within a process.
[C] Oh, I got it through Fig 1-4. ASID is produced by WLM service.
** SMF(System Management Facilities), BCP element
- collects data for accounting.
- SMF job and job-step accounting records identify processes by user, process, group, and session identifiers.
- SMF file system records describe file system events such as file open , file close, and file system mount, unmount, quiesce, and unquiesce.
[NOTICE]: Use the JWT value in the SMF parmlib SMFPRMxx to specify when to time-out an idle address space. SMF/WLM does the tracking.
[04/04/12]
** C/C++
- C: c89 command
- C/C++: cxx command
** Language Environment(LE)
- C/C++ run-time library provided with Language Environment.
** Data Facility System Managd Storage(DFSMS)
DFSMS: can be manage data sets used for processing Hierarchical File System(HFS).
- A file hierarchy can consist of:
-> Files
-> Directory
-> Additional local or remote file systems
[C] Here dont mention zFS, zFS is introduced in following chapters.
** Security Server (RACF)
- UID and GID kept in RACF profile
- Equivalent security product (CA-ACF2) can be used.
** Resource Measurement Facility(RMF)
- Collect data used to describe USS performance.
- Can show activity of forked AS separately in report
- OMVS Kernel Activity report
** SDSF
- Monitor printing
- Monitor and control batch job
- M and c forked AS
- Find out users logged on to TSO
[Q] Can SDSF monitor USS process and thread? answer TBD
** TSO
- TSO environment has some useful commands for USS.
-> logically mount and unmount
-> oput and oget
-> OMVS and ISHELL
-> oedit
[C] Shell environment has two commands tso and tsocmd which can invoke TSO commands, also very useful.
** z/OS Communications Services(TCP/IP Services)
- AF_INET and AF_INET6 for BS applications
- AF_UNIX as local socket
- assembler interface provided without C/C++ RTL.
** ISPF
Here it means ISHELL
** BookManager READ/MVS
- ohelp, it's a TSO command
- support BookManager format
** Network File System(NFS)
** zSeries File System(zFS)
It's a UNIX file system, along with HFS.
* 1.2.4 Hardware considerations
** rlogin,[Q] different ? what about?
** optional Suppression on Protection ==> mmap() and fork() copy-on-write
** CHECKSUM hardware improved TCP/IP performance
** semaphore processing improved by PLO(Perform Loocked Operation) instruction.
* 1.2.5 Configuration parameters
The z/OS implementation of UNIX is different from other implementations as it's part of the z/OS operating system. [Indeed, USS appears to be a subsystem of z/OS.
With z/OS, UNIX is just an environment, which also processes other no-UNIX workloads(CICS, IMS,MQ,TSO,batch etc).
- External configuration
BPXPRMxx member of SYS1.PARMLIB: define the environment and the file systems
[Book] z/OS MVS Initialization and Tuning Reference, SA22-7592
- Internal configuration
also USS has /etc directory as other UNIX implementation.
[C] Actually, the '/etc' is very different with what we think.
[Book] z/OS UNIX System Services Planning, GA22-7800, P36 Establishing an /etc file sytem ...
1.2.6 z/OS UNIX file system
can install Virtual FS and Physical FS on USS.
* A VFS server is similar to a POSIX program that reads and writes files, except that it uses the lower-level VFS callable services API instead of the POSIX C-language API. For example NFS.
* PFSs receive and act upon requests to read and write files that they control. The format of these requests is defined by the PFS interface. File requests are routed by the LFS to the appropriate PFS.
- some kinds of PFSs: HFS, NFS, DFS, TFS, zFS, Pipe, Socket.
[R] http://www.kokwind.com/bbs/viewthread.php?tid=264&extra=page%3D2
[Q] 1. NFS belong to both PFS and VFS?
2. Since zFS must be SMS-Managed, HFS is dispensable?
* File system organization
- USS views files as organized data in a heirarchy.
- MVS views an entire file hierarchy as a collection of data sets. Each HFS or zFS data set is a mountable file system.
* File types
- regular file
- character special file
-> terminal /dev/ptypnnnn and /dev/ptypnnnn(Only superuser can create thie file)
-> default controlling terminal for a process(/dev/tty)
-> null, /dev/null. Only superuser can create thie file
-> file descriptor file
-> console. Data written to this file is sent to the console
-> UNIX domain socket name file
-> A Communications Server remote tty file (for example, rtynnnn)
[Q] What's this?
-> The Communications Server character special file (/dev/ocsadmin)
[Q] What's this?
- FIFO, also known as a named pipe.
- symbolic links. In USS, /etc, /tmp, /dev, and /var are symbolic links. An external link can even link to MVS data set([C] maybe through the form of '//xxx.yyy.zzz').
[C] /etc is also a symbolic link for z/OS USS.
[04/06/12]
-----------------------------------------------------------------------
* File security packet
64-byte file security packet(FSP), structure as below.
[UID] [GID] [extattr] [(setUID setGID Sticky)
(owner_r owner_w owner_x group_r group_w group_x other_r other_w other_x)]
[ Access_ACL_exists
File_model_ACL_exists
Directory_model_ACL_exists
]
Remarkably, SetUID This bit only relates to executable files. If on, it causes the UID of the user executing the file to be set to the file's UID. SetGID have similar meaning.
Sticky Bit seems tricky. This bit only relates to executable files. If so, it causes the file to be retained in memory for performance reasons. In z/OS UNIX, it means programs are loaded from LPA(LPA is a swap space.) or LNKLST instead of a HFS file. For a directory, the sticky bit causes UNIX to permit files in a directory or subdirectories to be deleted or renamed only by the owner of the file, or by the owner of the directory, or by a superuser.
[Q] Why? Why's UNIX is designed in this way?
From wikipedia: Typically this is set on the /tmp directory to prevent ordinary users from deleting or moving other users' files.
[R] http://en.wikipedia.org/wiki/Sticky_bit
[04/07/12]
--------------------------------------------------------------------------------
ACL bits ACLs are used together with the permission bits in FSP in order to control the access to z/OS UNIX files and directories by individual users(UIDs) and group(GIDs).
* Executable modules in the file system
- can use chmod command to set sticky bit. When the bit is set on, z/OS UNIX searches for the program in the user's STEPLIB, LPALST, or the LNKLST concanation.
-> STEPLIB: Colon-separated list of dataset names.
e.g. STEPLIB=DSN1:DSN2:DSN3
- extattr command, related extended attributes as below
-> a: run APF authorized if linked AC=1
[C] what are APF and AC?
-> p: program controlled. [C] Need to investigate the attribute.
-> s: run in a shared address space.
-> -: Attribute not set.
BTW, ls -E can display the extend attribute.
* Path and pathname
A pathname can be up to 1023 characters long,including all directory names, file names, and separating slashes.
NOTICE: Using DBCS data in these names is not recommended, it may cause unpredictable results.
* Symbolic and external with a sticky bit
- External links
exec attemp to locate the module in the MVS search order.
For example: ln -e "//'dataset_name(module)'" link_name
It seems like we use external links to access MVS dataset in USS eviroment.
- Symbolic links
[C] As metioned Symbolic link and vnode here, I should take a look at OS book like <<The design and Implementation of the FreeBSD Operation System>>. Here we go.
[04/08/12]
-------------------------------------------------------------------------
* 1.2.7 Address spaces
* OMVS
This AS runs a program that initializes the kernel. STARTUP_PROC statement in the BPXPRMxx member of SYS1.PARMLIB specifies the name of the OMVS cataloged procedure.
[C]In my ADCD z/OS image, the content of BPXPRMxx is as below.
//STEP01 EXEC PGM=IEFBR14
//TEST DD DSN=OMVS.VAR,
// DISP=(NEW,CATLG,DELETE),
// UNIT=SYSALLDA,
// SPACE=(TRK,(15,15,1)),
// DCB=(DSORG=PO),
// DSNTYPE=HFS
[Q] where is STARTUP_PROC statement in ADCD?
* BPXOINIT
The BPXOINIT AS runs the initialization process.BPXOINIT is also jobname of init process.
BPXOINIT AS has two categories of functions
1.Behave as PID(1) of a typical UNIX systems. It's the parent of /etc/rc, and other UNIX AS, dubbed or native AS.
2.Make certain normal kernel calls.( for example mmap() and user ID alias processing).
* BPXAS
programs started by the fork() or spawn() or callable services
* Colony address spaces
Extensions of the kernel AS for physical file systems. Sometimes, some operations can not be done from the kernel.
[04/09/12]
-----------------------------------------------------------------------
* 1.2.8 Accessing z/OS UNIX
It's possible to access UNIX without personal OMVS segments defined.
[C]In the redbook, it metions BPX.DEFAULT.USER facility for temperarily using. But actually default UID/GID can lead to resource integrity issue and forced additional restrictions on the use of some USS functions(kill, pid_affinity, pidxfer, sigqueue, ptrace). So some changes will happen in next version.
The choices to access USS include:
* rlogin or telnet
* TSO OMVS, suject to the limitations of 3270 technology.
* ISPF, explits the full-screen capability of ISPF.
* BPXBATCH, executed from batch JCL.
* 1.2.9 What people like about z/OS UNIX
ATTENTION! here we say z/OS UNIX and other UNIX implimentation.
** Open standard
** Web enable
** Security. RACF is available in z/OS UNIX.
** Workload can be effectively managed. [Q] Why? maybe it 's related to using WLM to supply AS.
** DFSMS, almost infinite disk capacity is available.
** UNIX process, failing process cannot impact other processes.
* 1.2.10 What people don't like about z/OS UNIX
EBCDIC implementation, the encode seems weird for stander UNIX users.
[C] Beside above point, in fact user experience still can't be compared with other UNIX-like os such as Linux due to dificient tools.
* 1.3 z/OS UNIX System Services release history
** In 1991, the US Federal Information Processing Standards (FIPS) Document 151 stated that MVS must incorporate support for popular UNIX interfaces.
[Q] what 's the background of this story?
** POSIX < XPG4 < XPG4.2 (Full UNIX Brnding)
[C] The X/Open Portability Guide(XPG) is a standard for UNIX systems originally published by X/Open Company Ltd. It has a wider scope than POSIX, which is only concerned with direct operating system interfaces.
[R] http://en.wikipedia.org/wiki/X/Open
EBCDIC implementation VS. ASCII. by Ahilan. [8/3/2012]
[04/10/12]
-----------------------------------------------------------------------
1.3 z/OS UNIX System Services release history(Con'd)
1994 --------------------------------------- 1996 (Full UNIX Branding)
MVS/ESA 4.3 and 5.1
MVS/ESA 5.2.2 and OS/390 R1
OS/390 R2
z/OS UNIX has been UNIX branded since 1996.
* 1.3.1 MVS/ESA V4R3 - 1994
** Introduced:
- C API, HFS files,
- APPC/ASCH
[Q] what 's this?
- POSIX Shell
- dbx Debugger
[04/16/12]
-----------------------------------------------------------------------
Here just noted some questions.
* 1.3.2 MVS/ESA V5R1 - 1994
[Q] AD/Cycle C/370(tm) Language Support?
[Q] DCE Base Services?
* 1.3.4 OS/390 V2R1 - 1996
[Q] Internet BonusPak - ICS?
* 1.3.5 OS/390 V1R2 - 1996
* 1.3.6 OS/390 V1R3 - 1997
[Q] Permanent Kernel?
* 1.3.7 OS/390 V2R4 - 1997
Notice: Uses WLM to supply address spaces.
[Q] APPC/ASCH no longer required.
[Q] Message Passing Interface(MPI)
* 1.3.8 OS/390 V2R5
Interestingly, here has a 'highlight'
- F BPXOINIT, SHUTDOWN=FORKINIT
[Q]It seems a big change for USS?
[4/20/2012]
-----------------------------------------------------------------------
* 1.3.9 OS/390 V2R6 - 1998
- Name changed from OpenEdition to OS/390 UNIX System Services
* 1.3.10 OS/390 V2R7 - 1999
- Dynamic creation of character special files
Files such as /dev/fdxx and /dev/ptyzzzz are created based on the
MAXFILEPROC and MAXPTYS setting in BPXPRMxx, respectively.
MAXFILEPROC: the upper bound on the VALUE of n.
- Security enhancements
No longer reuqire a UID=0 user ID(super user) to perform SMP/E
actions. Check the BPX.SUPERUSER FACILITY
UNIXMAP class: for the system to look up a user ID from a UID, or a
group name from a GID.
- Miscellaneous
- ServerPac install IPL eliminated
ServerPac?
* 1.3.11 OS/390 V2R7 - 1999
[C] We have two releases in 1999, should be busy in 1999.
** Magic number support
Magic number here is #!.
If the kernel cann't locate the program in magic number, shell attempts to
process the file as a shell script.
[Q] above feature specified by POSIX std.
** SETOMVS RESET operator command
For FILESYSTYPE, NETWORK, and SUBFILESYSTYPE in BPXPRMxx, add them without
reIPL, but if you change the exsisting value, a reIPL is necessary.
* 1.3.12 OS/390 V2R9 - 2000
** Support for shared HFS
Shared HFS allows read/write data to be shared transparently among
participating systems across a sysplex.
- Changes to the BPXPRMxx parmlib member:
-> SYSPLEX(YES|NO): sysplex environment or local mode
-> VERSION('nnnn'): indicates the release or version of root HFS.
[Q] root HFS need to specify a version number?
[Notice] The parameter cannot be changed dynamically.
- New BPXPRMxx optional keywords on the ROOT and MOUNT parameters:
-> SYSNAME(sysname):
The name of a system in a sysplex that was IPLed with SYSPLEX(YES).
-> AUTOMOVE|NOAUTOMOVE:
If the specified root file system owner goes down,
the root file system can be automatically moved to another system
- A UNIX C shell
the new shell, tcsh, is commonly available on USS.
tcsh has a number of commands designed especially for C programmers.
-> The tcsh shell commands are documented in OS/390 UNIX System Services Command Reference.
-> Usage is documented in USS User's Guide.
- Support for WLM multi-system enclaves
It provides the capability for managing and reporting on work requests
that are executed in parallel on multiple MVS images as single
entities.
- Shared library support
".so" supported.
- New shell commands
mount, chmount, unmount
- Application enablement
Megabyte mapping services greatly reduce the excessive amounts of ESQA
required to support servers that need to access more than 2 GB of
storage.
-> BPX1MMI(__map_init)
-> BPX1MMS(__map_service)
May.6, 2012
--------------------------------------------------------------
- System management features
-> D OMVS
PFS: current configuration of the physical file system.
CINET: displays routing information using the Common INET Pre-Router.
[Q] INET pre-Router?
-> BPXBATSL: like BPXBATCH, except that it dose not require resetting of enviroment variables.
-> provides a controlled way for a PFS to terminate and restart so that
its kernel-resident load module can be deleted and reloaded for APAR
service without a re-IPL.
- Debugging improvements
-> SETOMVS with new parameter SYNTAXCHECK=(xx)
-> With JOBLOG to STDERR support, WTO messages normally targeted to the
JES JESYSMSG file can be redirected to a joblog in the HFS with a new
environment variable: _BPXK_JOBLOG.
-> dbx supports Language Environment debug events for read/write locks
and shared mutexs(LE CEEEVDBG).
** 1.3.13 OS/390 V2R10 - 2000
- XPLINK(eXtra Performance Linkage), which improves the execution
performance and compile times of OS/390 applications written in C/C++.
- Large file support
- Security enhancements to AF_UNIX PFS
INTERESTING! These enhancements allow an AF_UNIX datagram server to
receive the identity of the sender of each message it receives,
providing for better troubleshooting of data passed from the syslog
daemon to the joblog.
FROM daemon to the joblog?
- Message routing capability for the _console() service
Routing and descriptor codes can be specified for messages issued
with the _console() service.
- New features for binary semaphores
The UNDO feature is provided for binary semaphores.
** 1.3.14 OS/390 V2R10 - 2000 Software Refresh
** 1.3.18 z/OS V1R4 - 2002
* Chapter2 Installation
* 2.1 Introduction
z/OS UNIX System Services is a base element and exclusive feature of the
z/OS operating system.
The sequenc of topics is:
** Activating z/OS UNIX in ** minimum mode **
Mini mode just be suitable for a system that runs traditional MVS
workloads. No UNIX service, TCP/IP, or other funtions.
** Activating z/OS UNIX in full function mode.
Full function mode is activated if there is a requirement to exploit
z/OS UNIX.
** Notes
*** SYS1.PROCLIB
Intended to indicate a system procedure data set(library) from where
started tasks(STCs) may be initiated(with SUB=MSTR).
*** SYS1.PARMLIB
Intended to indicate a system parameter data set(library) where system
parameters may be found by z/OS.(Specified by PARMLIB statement of
LOADxx).
* 2.2 Activating z/OS UNIX in minimum mode
P49 (page 69 for PDF file)
[8/13/2012 thinkhy]
Go to Chapter 3 directly, as security skill is more useful.
* Chapter 3. Establish security for z/OS UNIX
** 3.2 Superuser authority
Superuser comes from UNIX, also referred to as root authority.
A superuser can do below things:
-> Pass all z/OS USS security check, but notice that the authority is limited to the z/OS UNIX component.
-> Manage USS processes and files.
-> Have an unlimited number of processes running concurrently.
-> For a started procedure, this is true only if it has a UID of 0.
-> Change identify from one UID to another.
-> Use setrlimit to increase any of the system limits for a process.
*** 3.1.1 Defining superusers with appropriate privileges
There are three ways of assigning superuser privileges:
1> The preferred way - Using the RACF UNIXPRIV class profiles.
2> Using the BPX.SUPERUSER profile in the FACILITY class.
This just allows you to request full superuser authority, no request,
no authority.
3> Assigning a UID of 0 should be given to the most important
administrators.
Notice: do not confuse superuser authority with the MVS supervisor state.
Being a superuser is not related to supervisor state, PSW key 0, and
using APF-authorized instructions, macros, and callable services.
*** 3.1.2 Using the UNIXPRIV class profiles
- Define profiles in the UNIXPRIV class to grant RACF authorization for
certain USS privileges.
- By defining profiles in the UNIXPRIV class, you may specifically grant
certain superuser privileges with a high degree of granularity to users
who do not have superuser authority. So you should minimize the number
of assignments of superuser authority.
SUPERUSER.CHOWN.UNRESTRICTED
SUPERUSER.FILESYS
SUPERUSER.FILESYS.CHOWN
SUPERUSER.FILESYS.MOUNT
SUPERUSER.FILESYS.PFSCTL
SUPERUSER.QUIESCE
SUPERUSER.IPC.RMID
SUPERUSER.PROCESS.GETPSENT
SUPERUSER.PROCESS.KILL
SUPERUSER.PROCESS.PTRACE
SUPERUSER.SETPRIORITY
SUPERUSER.FILESYS.VREGISTER
[8/30/2012 11:02 PM]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment