Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Amazon Elastic Network Adapter (ENA) on CentOS 7
sudo su
yum --enablerepo=extras install epel-release
yum -y install patch dkms kernel-devel perl
yum update
#Required for kernel num 5:
yum --enablerepo=elrepo-kernel -y install kernel-ml-devel
reboot
#After reboot:
sudo su
###############Update kernel START###################
## Update to latest kernel - optional
rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org
rpm -Uvh http://www.elrepo.org/ elrepo-release-7.0-5.el7.elrepo.noarch.rpm
yum --enablerepo=elrepo-kernel install kernel-ml
awk -F\' /^menuentry/{print\$2} /etc/grub2.cfg
#Note: In the list of kernels, identify the latest and remeber it's index (index starts from 0)
#In case the first item in the list (index=0) is the kernel you want to activate, run:
grub2-set-default 0
grub2-mkconfig -o /boot/grub2/grub.cfg
vi /etc/default/grub
#change line: GRUB_DEFAULT=saved to GRUB_DEFAULT=0
reboot
#verify current kernel via:
uname -a
#remove old kernels:
rpm -qa | grep kernel | sort
#for the kernels in the list , that you want to remove:
yum remove REPLACE_WITH_KERNEL_NAME
###############Update kernel END ###################
reboot
sudo su
//TODO fetch latest ena_driver from: https://github.com/amzn/amzn-drivers/releases, this script is based on 2.2.9
cd /tmp
curl -o ena_linux_2.2.9.tar.gz https://codeload.github.com/amzn/amzn-drivers/tar.gz/ena_linux_2.2.9
tar zxvf ena_linux_2.2.9.tar.gz
mv amzn-drivers-ena_linux_2.2.9 /usr/src/ena-2.2.9
#Copy the bellow and paste in command line (lines 52-63):
cat <<EOF > /usr/src/ena-2.2.9/dkms.conf
PACKAGE_NAME="ena"
PACKAGE_VERSION="2.2.9"
AUTOINSTALL="yes"
REMAKE_INITRD="yes"
BUILT_MODULE_LOCATION[0]="kernel/linux/ena"
BUILT_MODULE_NAME[0]="ena"
DEST_MODULE_LOCATION[0]="/updates"
DEST_MODULE_NAME[0]="ena"
CLEAN="cd kernel/linux/ena; make clean"
MAKE="cd kernel/linux/ena; make BUILD_KERNEL=\${kernelver}"
EOF
dkms add -m ena -v 2.2.9
dkms build -m ena -v 2.2.9
dkms install -m ena -v 2.2.9
dracut -f --add-drivers ena
##Verification:
modinfo ena
##### You should get similar info: #####
#filename: /lib/modules/5.7.2-1.el7.elrepo.x86_64/extra/ena.ko
#version: 2.2.9g
#license: GPL
#description: Elastic Network Adapter (ENA)
#author: Amazon.com, Inc. or its affiliates
#srcversion: CBF3F17443FAB1A53E24D56
#alias: pci:v00001D0Fd0000EC21sv*sd*bc*sc*i*
#alias: pci:v00001D0Fd0000EC20sv*sd*bc*sc*i*
#alias: pci:v00001D0Fd00001EC2sv*sd*bc*sc*i*
#alias: pci:v00001D0Fd00000EC2sv*sd*bc*sc*i*
#alias: pci:v00001D0Fd00000051sv*sd*bc*sc*i*
#depends:
#retpoline: Y
#name: ena
#vermagic: 5.7.2-1.el7.elrepo.x86_64 SMP mod_unload modversions
#parm: debug:Debug level (0=none,...,16=all) (int)
#parm: rx_queue_size:Rx queue size. The size should be a power of 2. Max value is 8K
# (int)
#parm: force_large_llq_header:Increases maximum supported header size in LLQ mode to 224 bytes, while reducing the maximum TX queue size by half.
# (int)
#parm: num_io_queues:Sets number of RX/TX queues to allocate to device. The maximum value depends on the device and number of online CPUs.
# (int)
# The below is relevant only if the instance did not have ENA enabled before :
#Stop the instance and from another AWS instance run:
aws ec2 modify-instance-attribute --region THE_INSTANCE_REGION --instance-id THE_INSTANCE_ID --ena-support
#e.g: aws ec2 modify-instance-attribute --region ap-southeast-1 --instance-id i-0123456789abcdef --ena-support
#Create an image from the instance
#Open the image on ENA supportive instance type (e.g. M5 , R4,.. )
#Validate interface:
ifconfig -s -a
#get Iface name, e.g: ens5
ethtool -i ens5
##### You should get similar info: #####
#driver: ena
#version: 2.1.1g ################## the ENA version #############
#firmware-version:
#expansion-rom-version:
#bus-info: 0000:00:05.0
#supports-statistics: yes
#supports-test: no
#supports-eeprom-access: no
#supports-register-dump: no
#supports-priv-flags: no
######################################################################################
# Find this gist useful ?? : Please share your love by clicking the "Star" icon above
######################################################################################
@Mustache84

This comment has been minimized.

Copy link

@Mustache84 Mustache84 commented Jun 26, 2018

just tried this and my M5 hangs and goes into maintenance mode. Any ideas? I've tried your guide, multiple others. They all go into maint mode when i launch them. I've done the delete of /etc/udev/rules.d/70-persistent-net.rules and i've done:
sudo sed -i '/^GRUB\_CMDLINE\_LINUX/s/\"$/\ net\.ifnames\=0\"/' /etc/default/grub
that wasn't in your guide. Maybe I shouldn't do those?

@j-b0n

This comment has been minimized.

Copy link

@j-b0n j-b0n commented Jul 21, 2018

Just, Thank you, it works like a charm, just in the row 8, after the reboot I needed to execute the "sudo su" to be able to copy the ena_driver to /usr/src/ena-1.5.3

@tmblue

This comment has been minimized.

Copy link

@tmblue tmblue commented Sep 28, 2018

Finding errors in 1.5.3, appears others saw the same thing in later 1.4 but I'm seeing them in 1.5.3

CentOS 7.2

[Fri Sep 28 12:35:01 2018] ena 0000:00:05.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 93.
[Fri Sep 28 12:35:01 2018] ena 0000:00:05.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 94.
[Fri Sep 28 12:35:01 2018] ena 0000:00:05.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 95.

Affecting C5 AWS instances so far that I've found.

oldish kernel "3.10.0-693.11.6.el7.x86_64"

Any ideas, and or what can I provide?

Thanks

@Ray33

This comment has been minimized.

Copy link
Owner Author

@Ray33 Ray33 commented Nov 14, 2018

Finding errors in 1.5.3, appears others saw the same thing in later 1.4 but I'm seeing them in 1.5.3

CentOS 7.2

[Fri Sep 28 12:35:01 2018] ena 0000:00:05.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 93.
[Fri Sep 28 12:35:01 2018] ena 0000:00:05.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 94.
[Fri Sep 28 12:35:01 2018] ena 0000:00:05.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 95.

Affecting C5 AWS instances so far that I've found.

oldish kernel "3.10.0-693.11.6.el7.x86_64"

Any ideas, and or what can I provide?

Thanks

Script now updated to 1.6.0

@frank3427

This comment has been minimized.

Copy link

@frank3427 frank3427 commented Dec 13, 2018

Ray,

Have you tried this as a bash script in a Packer build?

@prakash9999in

This comment has been minimized.

Copy link

@prakash9999in prakash9999in commented Feb 9, 2019

I am getting this error : any idea what could be the reason.

_[root@ip-172-31-44-34 tmp]# dkms install -m ena -v 1.6.0

ena.ko.xz:
Running module version sanity check.

  • Original module
    • Found /lib/modules/3.10.0-957.5.1.el7.x86_64/kernel/drivers/net/ethernet/amazon/ena/ena.ko.xz
    • Storing in /var/lib/dkms/ena/original_module/3.10.0-957.5.1.el7.x86_64/x86_64/
    • Archiving for uninstallation purposes
  • Installation
    • Installing to /lib/modules/3.10.0-957.5.1.el7.x86_64/extra/
      Adding any weak-modules
      Failed to install module enanvme_core_
@Ray33

This comment has been minimized.

Copy link
Owner Author

@Ray33 Ray33 commented Mar 25, 2019

Ray,

Have you tried this as a bash script in a Packer build?

No.

@Ray33

This comment has been minimized.

Copy link
Owner Author

@Ray33 Ray33 commented Mar 25, 2019

Updated to ENA v. 2.0.3

@gmogames

This comment has been minimized.

Copy link

@gmogames gmogames commented Mar 27, 2019

For anyone that following this and updated the Kernel to 5.0.4 and is getting the error message "Your kernel headers for kernel 5.0.4... cannot be found at..." when running dkms build -m ena -v 2.0.3

You need to install the kernel-ml-devel package, to do this, run
yum --enablerepo=elrepo-kernel -y install kernel-ml-devel

And proceed with the command that failed.

@guillermo-menjivar

This comment has been minimized.

Copy link

@guillermo-menjivar guillermo-menjivar commented Mar 31, 2019

yup just hit that @gmogames thanks for the heads up

@Ray33

This comment has been minimized.

Copy link
Owner Author

@Ray33 Ray33 commented Jun 16, 2019

Updated to ENA v. 2.1.1

@Ray33

This comment has been minimized.

Copy link
Owner Author

@Ray33 Ray33 commented Jun 16, 2019

For anyone that following this and updated the Kernel to 5.0.4 and is getting the error message "Your kernel headers for kernel 5.0.4... cannot be found at..." when running dkms build -m ena -v 2.0.3

You need to install the kernel-ml-devel package, to do this, run
yum --enablerepo=elrepo-kernel -y install kernel-ml-devel

And proceed with the command that failed.

Updated the script following your commnet.

@aquadrehz

This comment has been minimized.

Copy link

@aquadrehz aquadrehz commented Oct 11, 2019

FYI
Recent success result for modinfo ena
Represent as following

filename: /lib/modules/5.3.5-1.el7.elrepo.x86_64/extra/ena.ko
version: 2.1.1g
license: GPL
description: Elastic Network Adapter (ENA)
author: Amazon.com, Inc. or its affiliates
srcversion: F342F1BEE707726508E3DF8
alias: pci:v00001D0Fd0000EC21svsdbcsci*
alias: pci:v00001D0Fd0000EC20svsdbcsci*
alias: pci:v00001D0Fd00001EC2svsdbcsci*
alias: pci:v00001D0Fd00000EC2svsdbcsci*
depends:
retpoline: Y
name: ena
vermagic: 5.3.5-1.el7.elrepo.x86_64 SMP mod_unload modversions
parm: debug:Debug level (0=none,...,16=all) (int)
parm: rx_queue_size:Rx queue size. The size should be a power of 2. Max value is 8K
(int)
parm: force_large_llq_header:Increases maximum supported header size in LLQ mode to 224 bytes, while reducing the maximum TX queue size by half.
(int)

@Ray33

This comment has been minimized.

Copy link
Owner Author

@Ray33 Ray33 commented Jun 15, 2020

updated to version 2.2.9

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.