top linux command for esx and esxi

The top Linux commands
The ESX service console is based on Red Hat Linux and therefore many Linux commands can be used inside it. Here are common Linux commands and some VMware-specific versions of them.

    Find/cat/grep commands find, display and search for files. Find locates specific files, cat displays the contents of files and joins files together, and grep searches for specific text within in a file. These commands help administrators find specific infrastructure elements such as snapshot files and also display log and config files. They can also search for information within files.
    Tail displays the last part of a text-based file and can also monitor output to the file in real time. This command helps monitor log files in real time.
    Service can start, stop and restart services (or programs) that run on the host server. Some common ESX services include mgmt-vmware, vmware-vpxa, firewall, vmware-hostd and vmware Web Access. This command can restart services that hang or following configuration changes.
    Nano and vi edit text files. Nano is a simpler and easier to use editor than is vi, but vi is a more featured and powerful editor. Text editors help edit configuration files on an ESX host.
    Su and sudo commands help control access and prevent the root account from being used. Su elevates the privileges of less privileged user accounts to that of a superuser (or root). Sudo runs commands as another user as specified in the sudoers configuration file.
    ls lists file and directory information. By using certain switches (e.g., ltr) you can display detailed file information, including the owner, size, permissions and the last modified date andtime.
    Df and vdf display file system (partitions) information, including free space. The df command will not display Virtual Machine File System (VMFS) volumes because it cannot read them. Vdf is the VMware version of this command, which will also display VMFS volume information. Both commands can use the H switch, which displays the output in readable form (i.e., as 2 GB rather than as 2,016,044).
    Ps and kill commands can forcibly terminate stuck VMs that will not power on or off. Ps displays the status or processes running on the host. It can use many switches, but the most common is EF, which displays full information about every process running. The kill command is often used with the ps command to terminate specific running processes.
    Ping and vmkping are the most basic network troubleshooting commands. Ping tests network connectivity with other hosts and network devices by sending an Internet Control Message Protocol packet to them and seeing the response. Vmkping is the VMware-specific version of the Ping command. It uses the IP stack of the VMkernel to ping another ESX host's VMkernel port. This command helps troubleshoot VMotion and network storage issues.

The top VMware ESX commands and ESXi commands
These VMware ESX and ESXi commands can be run with the ESX service console (locally or remotely using Secure Shell) or with RCLI (in VMware Infrastructure 3) and vSphere CLI (in vSphere). With the RCLI and vSphere CLI, note that many of the commands have been renamed to vicfg- instead of esxcfg- (i.e., esxcfg-nics.pl and vicfg-nics.pl). Both commands perform the same function, but VMware is trying to migrate from esxcfg- to vicfg-.

    The versatile vmkfstools command is the Swiss army knife of virtual disks and can be used to copy, convert, rename, import, export and resize virtual disk files.
    Esxtop troubleshoots performance problems. It provides real-time and historical performance statistics for CPU, memory, disk and network usage.
    Esxcfg-nics views and configures physical network interface cards (NICs). It displays NIC status and can configure speed and duplex of the NICs.
    Esxcfg-vswitch views and configures virtual switches. It's useful for configuring networking when the vSphere Client cannot be used. The command configures port groups and links physical NICs to them (known as uplinks) anad configures virtual LAN IDs, Cisco Discovery Protocol (CDP) and the MTU of vswitches.
    Esxcfg-vswif and esxcfg-vmknic allow you to view and configure special port groups on vSwitches. Esxcfg-vswif configures the ESX service console network interfaces, which are also known as vswif ports. Esxcfg-vmknic configures VMkernel network interfaces, which are necessary for VMotion and connecting to iSCSI and Network File System network storage devices.
    Vmware-cmd is a versatile command to manage and retrieve information from virtual machines. It can change VM power states, manage snapshots, register and unregister VMs, and retrieve and set various VM information.
    Vimsh and vmware-vim-cmd are complex commands that you should fully understand before using. Vimsh is a powerful interactive shell that allows execution of commands and the ability to display and configure many things. VMware-vim-cmd is a front end of sorts for vimsh that simplifies command usage without having to know the many switches that vimsh requires.
    Vihostupdate and esxupdate update and patch ESX and ESXi hosts. Esxupdate is used on the ESX service console and vihostupdate is used by the RCLI/vSphere CLI. In addition, vihostupdate35 is used to patch ESX and ESXi version 3.5 hosts.
    Svmotion is an RCLI/vSphere CLI command used to initiate Storage VMotion sessions to relocate a VM's virtual disk to another datastore while it is running. In ESX 3.5 this command was the only method to initiate a SVMotion; in vSphere the ability to do this was added to the vSphere Client GUI.
    Esxcfg-mpath displays and sets all paths from a host to its storage devices.
    Esxcfg-rescan lets a host res-can a particular storage adapter to discover new storage devices. This tools is useful when storage devices have been added, removed or changed from a storage network.
    Esxcfg-scsidevs and esxcfg-vmhbadevs display information on the storage devices connected to a host. Esxcfg-vmhbadevs was used in ESX 3.5 and was replaced by esxcfg-scsidevs in vSphere.
    Esxcfg-firewall displays information and configures the built-in firewall that protects the ESX service console. It allows and blocks specific TCP/IP ports between the service console and other network devices.
    The esxcfg-info command provides a wealth of information about the host that it is run on. It can be re-directed to a text file to document host configuration.
    Esxcfg-auth configures Service Console authentication on an ESX host. It can configure authentication to a third-party LDAP or Active Directory server and set various local security options.
    Vm-support is a powerful information gathering tool commonly used in troubleshooting. The command gathers up a large amount of configuration info, log files and the output from many commands into a single .tgz archive file. It can also be used to display VM information as well as kill VMs that are not responding.

Most of the above commands have various syntaxes, options and switches that can be used with them. With them you can usually run a command without any options. For more information on these 25 ESX and ESXi commands, check out the following documentation:

VMware essential commands

One of the biggest challenges I run into working with ESXi hosts is the lack of a real usable CLI. The remote CLI provided by VMware is clunky at best and makes administration a bit of a pain. I did some poking around and managed to find some interesting CLI commands that work for ESXi.

Most of these have no manual page but will output a help message if run by themselves. I would recommend using extreme caution and using a test host to determine the usefulness in your environment before using them on an important system. Also keep in mind that the examples that I give are not the "only" way the commands can be used.

To power on a virtual machine from the command line:

    - Lists all vm's running on hypervisor and provides vmid
    
    vim-cmd vmsvc/getallvms
    
    - List the inventory ID of the virtual machine with the command:

     vim-cmd vmsvc/getallvms |grep <vm name>

    Note: The first column of the output shows the vmid.

    - Check the power state of the virtual machine with the command:

      vim-cmd vmsvc/power.getstate <vmid>

    - Power-on the virtual machine with the command:

      vim-cmd vmsvc/power.on <vmid> 

    - Power-off the virtual machine with the command:

      vim-cmd vmsvc/power.off <vmid> 

    - Reboots vmid referenced from getallvms command
    
      vim-cmd vmsvc/power.reboot vmid
     
Power Off (Hard)

get the world ID of the virtual machine

esxcli vm process list

TestComputer

World ID: 1625788
Process ID: 0
VMX Cartel ID: 1625786
UUID: 56 4d 9e d3 8b ce ab 59-9b 22 ac 87 40 6c 48 c3
Display Name: TestComputer

And kill them

esxcli vm process kill -t [soft,hard,force] -w

esxcli vm process kill -t hard -w 1625788

uspend a vm

vim-cmd vmsvc/power.suspend

Resume a virtual machine

vim-cmd vmsvc/power.suspendResume

Reset a virtual machine

vim-cmd vmsvc/power.reset

Shutdown

vim-cmd vmsvc/power.shutdown

    - Deletes the vmdk and vmx files from disk
    
      vim-cmd vmsvc/destroy vmid
  
    - Puts hypervisor into maintenance mode
    
     vim-cmd hostsvc/maintenance_mode_enter

    - Takes hypervisor out of maintenance mode

     vim-cmd hostsvc/maintenance_mode_exit

    - Registers vm in hypervisor inventory

     vim-cmd solo/registervm /vmfs/vol/datastore/dir/vm.vmx

    - Unregisters vm with hypervisor

     vim-cmd vmsvc/unregister vmid

    - Starts vmware tools installation for VM

     vim-cmd vmsvc/tools.install vmid

    - Provides information about hypervisor networking

     vim-cmd hostsvc/net/info

    - Shows daemons running on hypervisor. Can also be used for configuration.
    
     chkconfig -l

    - Same as linux top for vmware
    
     sxtop

    - List of vmkernel errors
    
     vmkerrcode -l

    - Lists a LOT of information about the esx host

     esxcfg-info

    - Lists information about NIC's. Can also be used for configuration.

     esxcfg-nics -l

    - Lists information about virtual switching. Can also be used for configuration.
     esxcfg-vswitch -l

    - Provides console screen to ssh session
     dcui

    - Vmware interactive shell
     vsish

    - Read System Event Log of server 
     decodeSel /var/log/ipmi_sel.raw

     esxcfg-vmknic -l
     esxcfg-route -l (for defaultgateway:)
     esxcfg-vswitch -l (for switch)

Restart Management, HA Services
/sbin/services restart

Installing Software. List/Install/Uninstall  VIBs (vSphere Installation bundle)

List vibs

esxcli software vib list

Install a vib

esxcli software vib install -v file:/tmp/[NewVIB].vib

Uninstall a vib (determine the Name of the VIB by the list command)

esxcli software vib remove -n VIBname

Install a patch

esxcli software vib install /tmp/[patchName].zip

Network

firewall state

esxcli network firewall get

Firewall rules

esxcli network firewall ruleset list

Firewall activate a Ruleset, i.e. sshClient

esxcli network firewall ruleset set --ruleset-id=sshClient --enabled=true

Only allow an IP Range for the ssh daemon

esxcli network firewall ruleset allowedip add --ruleset-id sshServer --ip-address 192.168.254.0/24

List Kernel Network Interfaces

esxcli network ip interface list

List routing table

esxcli network ip route ipv4 list

Add a route

esxcli network ip route ipv4 add --gateway 10.1.1.254 --network 10.1.2.0/24

To make the route persistent add the command line to a ESXi startup script. In ESXi 5.1

/etc/rc.local.d/local.sh

in ESXi <=5.0

/etc/rc.local

SNMP

Set Community (Communityname: MGMCOM)

esxcli system snmp set --communities MGMCOM

Set Trap destintion (IP Address Management station: 192.168.254.100)

esxcli system snmp set --targets 192.168.254.100/MGMCOM

Send test trap

esxcli system snmp test

State

esxcli system snmp get

enable IPMI as SNMP source

esxcli system snmp set --hwsrc sensors

enable CIM as SNMP source

esxcli system snmp set --hwsrc indications

Enable snmp

esxcli system snmp set --enable true

Virtual disks

extend a virtual disk, i.e. to 40GB

vmkfstools -X 40G /vmfs/volumes/datastore1/Test/test.vmdk

Convert a Workstation based virtual Disk to a VMFS Disk for use at ESXi. Disk to convert: Computer.vmdk
Rename Disk Computer.vmdk to Computer_WS.vmdk
mv Computer.vmdk Computer_WS.vmdk
Clone the disk
vmkfstools -i Computer_WS.vmdk -d zeroedthick Computer.vmdk
Destination disk format: VMFS thin-provisioned
Cloning disk 'Computer.vmdk'...
Clone: 100% done.

Remove Workstation disk

rm Computer_WS.vmdk

Snapshots

Get VMID

vim-cmd vmsvc/getallvms

Vmid  Name File Guest OS  Version  Annotation
36 .......................................

List all snapshots for a virtual machine

vim-cmd vmsvc/snapshot.get 36

Get Snapshot:
|-ROOT
--Snapshot Name : Installation Complete
--Snapshot Id : 1
--Snapshot Desciption :
--Snapshot Created On : 3/15/2013 13:20:34
--Snapshot State : powered off
--|-CHILD
----Snapshot Name : SNAP1
----Snapshot Id : 2
----Snapshot Desciption :
----Snapshot Created On : 5/14/2013 11:46:19
----Snapshot State : powered on

Create a snapshot,  including the RAM of the Machine

vim-cmd vmsvc/snapshot.create 36 "New Snap" "Snap desc" includeMemory

Delete a snapshot, where 36 is the VMID and 2 the Snapshot ID

vim-cmd vmsvc/snapshot.remove 36 2


ESXi Advanced Kernel Settings/Parameters

Get all

esxcli system settings advanced list

Set one, i.e. disable the shell respectively ssh warnings

esxcli system settings advanced set -o /UserVars/SuppressShellWarning -i 1

ESXi Kernel modules

List loaded kernel modules

vmkload_mod  -l

Get a list of all enabled kernel modules

esxcfg-module -q

Get parameters of a kernel modul, i.e. an Emulex FC HBA module

esxcfg-module -g lpfc820

lpfc820 enabled = 1 options = 'lpfc0_lun_queue_depth=8 lpfc1_lun_queue_depth=8 lpfc2_lun_queue_depth=8 lpfc3_lun_queue_depth=8'

Set kernel module parameters, also i.e. the Emulex FC HBA module

esxcfg-module -s "lpfc0_lun_queue_depth=8 lpfc1_lun_queue_depth=8 lpfc2_lun_queue_depth=8 lpfc3_lun_queue_depth=8" lpfc820

Get info of a module with all possible parameters and a description of each

esxcfg-module -i lpfc820

Storage

Get a list of all storage devices

esxcli storage filesystem list

Rename a datastore

vim-cmd hostsvc/datastore/rename OldName NewName

Get a list of all storage pathes

esxcli storage core path list

Get storage pathes for a specific drive

esxcli storage core path list -d naa.600000e00d11000000111a2400000000

Show Storage Array Type and Path selection policies of disk devices

esxcli storage nmp device list

Set Roundrobin(VMW_PSP_RR) path selection policy for a disk device, possible other policies are “Most recently used”:VMW_PSP_MRU or “Fixed”:VMW_PSP_FIXED

esxcli storage nmp device set -P VMW_PSP_RR -d naa.600000e00d11000000111a2400020000

Stat of the visorfs. Here you can find  is a detail description of the visorfs(Hypervisor Filesystem).

vdf

Usage of the visorfs

vdu

Host Services

Enable ESXi Shell

vim-cmd hostsvc/enable_esx_shell

Disable ESXi Shell

vim-cmd hostsvc/disable_esx_shell

Start ESXi Shell

vim-cmd hostsvc/start_esx_shell

Enable the ssh daemon

vim-cmd hostsvc/enable_ssh

Disable ssh daemon
vim-cmd hostsvc/disable_ssh

Start ssh daemon

vim-cmd hostsvc/start_ssh

Enter Maintainance mode

vim-cmd hostsvc/maintenance_mode_enter

Leave Maintanance Mode

----------------------------------------------------------------------------------------------------------------------------------------------

      
Enabling root SSH login on an ESX host

Note: Each SSH connection to an ESX host uses additional Service Console resources. Use caution when using scripts or third party software that create multiple SSH sessions to the ESX Service Console. Excessive use of SSH on an ESX machine may cause the service console to exhibit symptoms of memory exhaustion.

To enable root login for SSH and SCP clients:

    If you have physical access to the ESX host, login to the console of your ESX host as the root user. If you can only connect to the ESX host over the network, connect using an SSH client (such as PuTTY) and log in as a user other than root.

    To create a user in ESX host for using a SSH client:

        Log in to the vSphere Client as a root user.
        Click Users & Groups.
        Right-click on a blank area and click Add.
        Enter a username and password. Confirm your password.

        Note: Starting in ESX 4.0, the password needs to be at least 8 characters in length.

        Select Grant shell access to this user.
        Select root group from the dropdown and click Add > OK.

        Note: By default it assigns to the users group and does not allow SSH access.

    After you are logged in SSH session, switch to the root user with the command:

    su -

    Note: If you do not have any other users on the ESX host, you can create a new user by connecting directly to the ESX host with VMware Infrastructure (VI) or vSphere Client. Go to the Users & Groups tab, right-click on the Users list and select Add to open the Add New User dialog. Ensure that the Grant shell access to this user option is selected. These options are only available when connecting to the ESX host directly. They are not available if connecting to vCenter Server.

    Edit the configuration file for SSH with the command:

    nano /etc/ssh/sshd_config

    Find the line that starts with PermitRootLogin and change the no to yes. You can find this line about 2 pages down from the top.
    Save the file by first pressing Ctrl-O and then Enter.
    Exit with Ctrl-X.
    Restart the sshd service with the command:

    service sshd restart

    Note: Alternatively, use the command:

    /etc/init.d/sshd restart