Crowdsec is a new project to protect against brute force Crowdsec's architecture allows distributed setups, as most components communicate via HTTP API.
When doing such, a few considerations must be kept in mind to understand the role of each component:
The agent is in charge of processing the logs, matching them against scenarios, and sending the resulting alerts to the local API (container)
The local API (LAPI from now on) receives the alerts and converts them into decisions based on your profile (container)
The bouncer(s) query the LAPI to receive the decisions to be applied (can be installed by .deb .rpm and openwrt package)
intro: https://docs.crowdsec.net/docs/intro
concept: https://docs.crowdsec.net/docs/concepts
bouncer: https://docs.crowdsec.net/docs/bouncers/intro
cscli: https://docs.crowdsec.net/docs/cscli/cscli
The idea is to offer something new that you could use in a distributed architecture
local API is the intelligence to make decisions, it is a crowdsec instances (container) running one time in a cluster The Local API (LAPI) is a core component of CrowdSec to :
Allow CrowdSec machines to push alerts & decisions to a database
Allow bouncers to consume said alerts & decisions from database
Allow cscli to view add or delete decisions
Agent is a crowdsec instances without local API to use in case of a remote server, it reports to Local API (container) with encrypted TLS communication
in Local API and agent we could install
Collection is a way To make user's life easier, "collections" are available, which are just a bundle of parsers and scenarios.
Scenarios are YAML files that allow to detect a specific behavior, usually an attack.
A parser is a YAML configuration file that describes how a string must be parsed. Said string can be a log line, or a field extracted from a previous parser.
You can push alert (alert == notification ). CrowdSec supports notification plugins, meant to be able to push alerts to third party services for alerting or integration purposes. At the time of writing, plugins exists for slack, splunk, and a generic http push plugin (allowing to push to services such as elasticsearch).
To apply a decision of ban you need a standalone software called bouncers (decision == ban)
Bouncer are standalone software pieces in charge of acting upon a decision taken by crowdsec (block an IP, present a captcha, enforce MFA on a given user, etc.)
Communication between Local API, Agents (crowdsec without local API) and bouncers can be protected under TLS, it could be offer a nice distributed architecture, an attacker is banned from the whole network if we have a FW protecting it and running the bouncer on it
It exists a CENTRAL API that you send the banned IP, we need to disable or enable it following the need of the user. The Central API is the service where the Local API pushes signal meta-data and from where it receives the community blocklists.
I actually cover the installation of a simple crowdsec on the same host with a bouncer locally installed. We will use sqlite DB but crowdsec developers ask for a dedicated MYSQL or Postgresql DB
The container run as ROOT because it needs to mount log journal, we will use simple collections and we mount the /var/log/secure as syslog (we could try journald)
podman run -d --replace -v crowdsec_config:/etc/crowdsec \
-v crowdsec_data:/var/lib/crowdsec/data \
-v /var/log/secure:/var/log/secure:Z \
-e COLLECTIONS="crowdsecurity/sshd" \
-p 8080:8080 -p 6060:6060 \
--name crowdsec docker.io/crowdsecurity/crowdsec:latest-debian
then we need to set the acquisition, comment old entries, add this one (I use for now the /var/log/secure with syslog but we could use the journalmatch of journald)
[root@ns8loc ~]# vim /var/lib/containers/storage/volumes/crowdsec_config/_data/acquis.yaml
---
filenames:
- /var/log/secure
labels:
type: syslog
---
restart the container
podman run -d --replace -v crowdsec_config:/etc/crowdsec \
-v crowdsec_data:/var/lib/crowdsec/data \
-v /var/log/secure:/var/log/secure:Z \
-e COLLECTIONS="crowdsecurity/sshd" \
-p 8080:8080 -p 6060:6060 \
--name crowdsec docker.io/crowdsecurity/crowdsec:latest-debian
The container run as ROOT because it needs to mount log journal, we will use simple collections and we mount journald for acquisition
podman run -d --replace -v crowdsec_config:/etc/crowdsec \
-v crowdsec_data:/var/lib/crowdsec/data \
-v /run/log/journal/:/run/log/journal:Z \
-e COLLECTIONS="crowdsecurity/sshd" \
-p 8080:8080 -p 6060:6060 \
--name crowdsec docker.io/crowdsecurity/crowdsec:latest-debian
then we need to set the acquisition, comment old entries, add this one
[root@ns8loc ~]# vim /var/lib/containers/storage/volumes/crowdsec_config/_data/acquis.yaml
---
source: journalctl
journalctl_filter:
- "_SYSTEMD_UNIT=sshd.service"
labels:
type: syslog
---
we could test to filter for a module by adding (not tested)
journalctl_filter:
- "SYSLOG_IDENTIFIER=dokuwiki1"
restart the container
podman run -d --replace -v crowdsec_config:/etc/crowdsec \
-v crowdsec_data:/var/lib/crowdsec/data \
-v /run/log/journal/:/run/log/journal:Z \
-e COLLECTIONS="crowdsecurity/sshd" \
-p 8080:8080 -p 6060:6060 \
--name crowdsec docker.io/crowdsecurity/crowdsec:latest-debian
[root@ns8loc ~]# podman exec -ti crowdsec cscli config show
Global:
- Configuration Folder : /etc/crowdsec
- Data Folder : /var/lib/crowdsec/data
- Hub Folder : /etc/crowdsec/hub
- Simulation File : /etc/crowdsec/simulation.yaml
- Log Folder : /var/log/
- Log level : info
- Log Media : stdout
Crowdsec:
- Acquisition File : /etc/crowdsec/acquis.yaml
- Parsers routines : 1
cscli:
- Output : human
- Hub Branch :
- Hub Folder : /etc/crowdsec/hub
Local API Server:
- Listen URL : 0.0.0.0:8080
- Profile File : /etc/crowdsec/profiles.yaml
- Trusted IPs:
- 127.0.0.1
- ::1
- Database:
- Type : sqlite
- Path : /var/lib/crowdsec/data/crowdsec.db
- Flush age : 7d
- Flush size : 5000
You have a whitelist enabled by default to allow all local IP, disable it
[root@ns8loc ~]# podman exec -ti crowdsec cscli parser remove crowdsecurity/whitelists
you need to create ipset list before to install the bouncer
[root@ns8loc ~]# firewall-cmd --permanent --new-ipset=crowdsec6-blacklists --type=hash:net --option="timeout=0" --option="maxelem=150000"
[root@ns8loc ~]# firewall-cmd --permanent --new-ipset=crowdsec-blacklists --type=hash:net --option="timeout=0" --option="maxelem=150000"
[root@ns8loc ~]# firewall-cmd --permanent --zone=drop --add-source=ipset:crowdsec-blacklists
[root@ns8loc ~]# firewall-cmd --permanent --zone=drop --add-source=ipset:crowdsec6-blacklists
restart firewalld (reload will probably be enough)
[root@ns8loc ~]# systemctl restart firewalld
check the drop zone wil source the created ipset list
[root@ns8loc ~]# firewall-cmd --zone=drop --list-sources
ipset:crowdsec-blacklists ipset:crowdsec6-blacklists
[root@ns8loc ~]# firewall-cmd --zone=drop --list-all
drop (active)
target: DROP
icmp-block-inversion: no
interfaces:
sources: ipset:crowdsec-blacklists ipset:crowdsec6-blacklists
services:
ports:
protocols:
forward: yes
masquerade: no
forward-ports:
source-ports:
icmp-blocks:
rich rules:
verify ipsets exist
[root@ns8loc ~]# firewall-cmd --permanent --get-ipsets
crowdsec-blacklists crowdsec6-blacklists
check ipset are well created with timout 0 and maxelem 150000
[root@ns8loc ~]# ipset -L
Name: crowdsec-blacklists
Type: hash:net
Revision: 7
Header: family inet hashsize 1024 maxelem 150000 timeout 0 bucketsize 12 initval 0x5663c4ac
Size in memory: 472
References: 0
Number of entries: 0
Members:
Name: crowdsec6-blacklists
Type: hash:net
Revision: 7
Header: family inet hashsize 1024 maxelem 150000 timeout 0 bucketsize 12 initval 0x04b07830
Size in memory: 472
References: 0
Number of entries: 0
Members:
install the bouncer on the host (documentation https://docs.crowdsec.net/docs/bouncers/firewall)
[root@ns8loc ~]# curl -s https://packagecloud.io/install/repositories/crowdsec/crowdsec/script.rpm.sh | sudo bash
[root@ns8loc ~]# dnf install crowdsec-firewall-bouncer-iptables
register the bouncer
[root@ns8loc ~]# podman exec -ti crowdsec cscli bouncers add localhost
Api key for 'localhost':
c14a204e5a602d1f53567d299ad14a0a
edit the bouncer configuration, change the mode to ipset and add the api_key to talk with the local api (key from above)
[root@ns8loc ~]# vim /etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml
mode: ipset
pid_dir: /var/run/
update_frequency: 10s
daemonize: true
log_mode: stdout
log_dir: /var/log/
log_level: info
log_compression: true
log_max_size: 100
log_max_backups: 3
log_max_age: 30
api_url: http://127.0.0.1:8080/
api_key: c14a204e5a602d1f53567d299ad14a0a
insecure_skip_verify: false
disable_ipv6: false
deny_action: DROP
deny_log: false
supported_decisions_types:
- ban
#to change log prefix
#deny_log_prefix: "crowdsec: "
#to change the blacklists name
blacklists_ipv4: crowdsec-blacklists
blacklists_ipv6: crowdsec6-blacklists
#if present, insert rule in those chains
iptables_chains:
- INPUT
# - FORWARD
# - DOCKER-USER
## nftables
nftables:
ipv4:
enabled: true
set-only: false
table: crowdsec
chain: crowdsec-chain
ipv6:
enabled: true
set-only: false
table: crowdsec6
chain: crowdsec6-chain
# packet filter
pf:
# an empty string disables the anchor
anchor_name: ""
start the bouncer
[root@ns8loc ~]# systemctl start crowdsec-firewall-bouncer
[root@ns8loc ~]# systemctl status crowdsec-firewall-bouncer
[root@ns8loc ~]# podman exec -ti crowdsec cscli bouncers list
----------------------------------------------------------------------------------------------------------------------------------------------------
NAME IP ADDRESS VALID LAST API PULL TYPE VERSION AUTH TYPE
----------------------------------------------------------------------------------------------------------------------------------------------------
localhost 10.88.0.1 ✔️ 2022-09-06T14:49:02Z crowdsec-firewall-bouncer v0.0.24-el9-rpm-8e00af2c9e83af22deab8c0c49a4ad9b8fc57a3f api-key
----------------------------------------------------------------------------------------------------------------------------------------------------
If you do not have the IP, Type and version of the bouncer, try to restart the crowdsec bouncer rpm : systemctl restart crowdsec-firewall-bouncer
try to do false authentication by ssh or add manually a decision, check inside the ipset -L or eventually in log file of crowdsec-firewall-bouncer in case of failure /var/log/crowdsec-firewall-bouncer.log
add decision
[root@ns8loc ~]# podman exec -ti crowdsec cscli decisions add -i 1.2.3.4
[root@ns8loc2 ~]# podman exec -ti crowdsec cscli decisions list
+----+--------+-------------+-------------------------------+--------+---------+----+--------+--------------------+----------+
| ID | SOURCE | SCOPE:VALUE | REASON | ACTION | COUNTRY | AS | EVENTS | EXPIRATION | ALERT ID |
+----+--------+-------------+-------------------------------+--------+---------+----+--------+--------------------+----------+
| 2 | cscli | Ip:1.2.3.4 | manual 'ban' from 'localhost' | ban | | | 1 | 3h57m44.711601367s | 2 |
+----+--------+-------------+-------------------------------+--------+---------+----+--------+--------------------+----------+
[root@ns8loc2 ~]# ipset -L
Name: crowdsec-blacklists
Type: hash:net
Revision: 7
Header: family inet hashsize 1024 maxelem 150000 timeout 0 bucketsize 12 initval 0x635a7dd9
Size in memory: 536
References: 0
Number of entries: 1
Members:
1.2.3.4 timeout 12588
Name: crowdsec6-blacklists
Type: hash:net
Revision: 7
Header: family inet hashsize 1024 maxelem 150000 timeout 0 bucketsize 12 initval 0x52f94518
Size in memory: 472
References: 0
Number of entries: 0
Members:
remove decision
[root@ns8loc ~]# podman exec -ti crowdsec cscli decisions delete -i 1.2.3.4
you can check decisions with
[root@ns8loc ~]# podman exec -ti crowdsec cscli decisions list
you can check the collections/parsers installed
[root@ns8loc ~]# podman exec -ti crowdsec cscli collections list
[root@ns8loc ~]# podman exec -ti crowdsec cscli parser list
[root@ns8loc ~]# podman exec -ti crowdsec cscli scenarios list
check what it occurs
[root@ns8loc ~]# podman exec -ti crowdsec cscli metrics
[root@ns8loc ~]# podman exec -ti crowdsec cscli alerts list
[root@ns8loc ~]# podman exec -ti crowdsec cscli decisions list
The Central API is the service where the Local API pushes signal meta-data and from where it receives the community blocklists.
You need to login to the website and register your instance from the website (enroll instance with the given key from the website)
for example
[root@ns8loc ~]# podman exec -ti crowdsec cscli console enroll cl7qa8xdn00030vl70wqqdqsfdsgfg
