Add additional section to the SELinux documentation to describe how to collect debug information when SELinux issues are encountered. This is purely a documentation change. Note: Merged seperate from SELinux Policy change in case policy change needs to be cherry-picked to older releases. Change-Id: I7ba3d3c7d84171b503d956eb01e13a680b4d53fc Signed-off-by: Billy McFall <bmcfall@redhat.com>
20 KiB
SELinux - VPP Custom SELinux Policy
Overview
Security-enhanced Linux (SELinux) is a security feature in the Linux kernel. At a very high level, SELinux implements mandatory access controls (MAC), as opposed to discretionary access control (DAC) implemented in standard Linux. MAC defines how processes can interact with other system components (Files, Directories, Other Processes, Pipes, Sockets, Network Ports). Each system component is assigned a label, and then the SELinux Policy defines which labels and which actions on each label a process is able to perform. The VPP Custom SELinux Policy defines the actions VPP is allowed to perform on which labels.
The VPP Custom SELinux Policy is intended to be installed on RPM based platforms (tested on CentOS 7 and RHEL 7). Though SELinux can run on Debian platforms, it typically is not and therefore is not currently being built for Debian.
The VPP Custom SELinux Policy does not enable or disable SELinux, only allows VPP to run when SELinux is enabled. A fresh install of either Fedora, CentOS or RHEL will have SELinux enabled by default. To determine if SELinux is enabled on a given system and enable it if needed, run:
$ getenforce
Permissive
$ sudo setenforce 1
$ getenforce
Enforcing
To make the change persistent, modify the following file to set
SELINUX=enforcing
:
$ sudo vi /etc/selinux/config
:
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=enforcing
:
Installation
To install VPP, see the installation instructions on the VPP Wiki
(https://wiki.fd.io/view/VPP/Installing_VPP_binaries_from_packages). The VPP
Custom SELinux Policy is packaged in its own RPM starting in 18.04,
vpp-selinux-policy-<VERSION>-<RELEASE>.rpm
. It is packaged and installed along
with the other VPP RPMs.
Fresh Install of VPP
If VPP has never been installed on a system, then starting in 18.04, the VPP Custom SELinux Policy will be installed with the other RPMs and all the system components managed by VPP will be labeled properly.
Fix SELinux Labels for VPP
In the case where the VPP Custom Policy is being installed for the first time, either because VPP has been upgraded or packages were removed and then reinstalled, several directories and files will not not be properly labeled. The labels on these files will need to be fixed for VPP to run properly with SELinux enabled. After the VPP Custom SELinux Policy is installed, run the following commands to fix the labels. If VPP is already running, make sure to restart VPP after the labels are fixed. This change is persistent for the life of the file. Once the VPP Custom Policy is installed on the system, subsequent files created by VPP will be labeled properly. This is only to fix files created by VPP prior to the VPP Custom Policy being installed.
$ sudo restorecon -Rv /etc/vpp/
$ sudo restorecon -Rv /usr/lib/vpp_api_test_plugins/
$ sudo restorecon -Rv /usr/lib/vpp_plugins/
$ sudo restorecon -Rv /usr/share/vpp/
$ sudo restorecon -Rv /var/run/vpp/
$ sudo chcon -t vpp_tmp_t /tmp/vpp_*
$ sudo chcon -t vpp_var_run_t /var/run/.vpp_*
NOTE: Because the VPP APIs allow custom filenames in certain scenarios, the
above commands may not handle all files. Inspect your system and correct any
files that are mislabeled. For example, to verify all VPP files in /tmp/
are
labeled properly, run:
$ sudo ls -alZ /tmp/
Any files not properly labeled with vpp_tmp_t
, run:
$ sudo chcon -t vpp_tmp_t /tmp/<filename>
VPP Files
Recommended Default File Directories
Documentation in the VPP Wiki (https://wiki.fd.io/view/VPP/) and doxygen generated documentation have examples with files located in certain directories. Some of the recommend file locations have been moved to satisfy SELinux. Most of the documentation has been updated, but links to older documentation still exist and there may have been instances that were missed. Use the file locations described below to allow SELinux to properly label the given files.
File locations that have changed:
- VPP Debug CLI Script Files
- vHost Sockets
- VPP Log Files
VPP Debug CLI Script Files
The VPP Debug CLI, vppctl
, allows a sequence of CLI commands to be read from a
file and executed. To avoid from having to grant VPP access to all of /tmp/
and
possibly /home/
sub-directories, it is recommended that any VPP Debug CLI script
files be placed in a common directory such as /usr/share/vpp/
.
For example:
$ cat /usr/share/vpp/scripts/gigup.txt
set interface state GigabitEthernet0/8/0 up
set interface state GigabitEthernet0/9/0 up
To execute:
$ vppctl exec /usr/share/vpp/scripts/gigup.txt
Or
$ vppctl
_______ _ _ _____ ___
__/ __/ _ \ (_)__ | | / / _ \/ _ \
_/ _// // / / / _ \ | |/ / ___/ ___/
/_/ /____(_)_/\___/ |___/_/ /_/
vpp# exec /usr/share/vpp/scripts/gigup.txt
vpp# quit
If the file is not labeled properly, you will see something similar to:
$ vppctl exec /home/<user>/dev/vpp/scripts/vppctl/gigup.txt
exec: failed to open `/home/<user>/dev/vpp/scripts/vppctl/gigup.txt': Permission denied
$ ls -alZ
drwxrwxr-x. <user> <user> unconfined_u:object_r:user_home_t:s0 .
drwxrwxr-x. <user> <user> unconfined_u:object_r:user_home_t:s0 ..
-rw-r--r--. <user> <user> unconfined_u:object_r:user_home_t:s0 gigup.txt
Original Documentation
Some of the original documentation showed script files being executed out of
/tmp/
. Convenience also may lead to script files being placed in
/home/<user>/
subdirectories. If a file is generated by the VPP process in
/tmp/
, for example a trace file or pcap file, it will get properly labeled
with the SELinux label vpp_tmp_t
. When a file is created, unless a rule is in
place for the process that created it, the file will inherit the SELinux label
of the parent directory. So if a user creates a file themselves in /tmp/
, it
will get the SELinux label tmp_t
, which VPP does not have permission to
access. Therefore it is recommended that script files are located as described
above.
vHost Sockets
vHost sockets are created from VPP perspective in either Server or Client mode.
In Server mode, the socket name is provided to VPP and VPP creates the socket.
In Client mode, the socket name is provided to VPP and the hypervisor creates
the socket. In order for VPP and hypervisor to share the socket resource with
SELinux enabled, a rule in the VPP Custom SELinux Policy has been added. This
rules allows processes with the svirt_t
label (the hypervisor) to access
sockets with the vpp_var_run_t
label. As such, when SELinux is enabled,
vHost sockets should be created in the directory /var/run/vpp/
.
Original Documentation
Some of the original documentation showed vHost sockets being created in the
directory /tmp/
. To work properly with SELinux enabled, vHost sockets should be
created as described above.
VPP Log Files
The VPP log file location is set by updating the /etc/vpp/startup.conf
file:
vi /etc/vpp/startup.conf
unix {
:
log /var/log/vpp/vpp.log
:
}
By moving the log file to /var/log/vpp/
, it will get the label vpp_log_t
,
which indicates that the files are log files so they benefit from the
associated rules (for example granting rights to logrotate so that it can
manipulate them).
Original Documentation
The default startup.conf
file creates the VPP log file in /tmp/vpp.log
. By
leaving the log file in /tmp/
, it will get the label vpp_tmp_t
. Moving it
to /var/log/vpp/
, it will get the label vpp_log_t
.
Use of Non-default File Directories
VPP installs multiple files on the system. Some files have fixed directory and file names:
- /etc/bash_completion.d/vppctl_completion
- /etc/sysctl.d/80-vpp.conf
- /usr/lib/systemd/system/vpp.service
Others files have default directory and file names but the default can be overwritten:
- /etc/vpp/startup.conf
- Can be changed via the
/usr/lib/systemd/system/vpp.service
file by changing the -c option on the VPP command line:
- Can be changed via the
ExecStart=/usr/bin/vpp -c /etc/vpp/startup.conf
- /run/vpp/cli.sock
- Can be changed via the
/etc/vpp/startup.conf
file by changing the cli-listen setting:
- Can be changed via the
unix {
:
cli-listen /run/vpp/cli.sock
:
}
- /var/log/vpp/vpp.log
- Can be changed via the
/etc/vpp/startup.conf
file by changing the log setting:
- Can be changed via the
unix {
:
log /var/log/vpp/vpp.log
:
}
If the directory of any VPP installed files is changed from the default, ensure that the proper SELiunx label is applied. The SELinux label can be determined by passing the -Z option to many common Linux commands:
ls -alZ /run/vpp/
drwxr-xr-x. root vpp system_u:object_r:vpp_var_run_t:s0 .
drwxr-xr-x. root root system_u:object_r:var_run_t:s0 ..
srwxrwxr-x. root vpp system_u:object_r:vpp_var_run_t:s0 cli.sock
VPP SELinux Types
The following SELinux types are created by the VPP Custom SELinux Policy:
-
vpp_t
- Applied to:- VPP process and spawned threads.
-
vpp_config_rw_t
- Applied to:/etc/vpp/*
-
vpp_tmp_t
- Applied to:/tmp/*
-
vpp_exec_t
- Applied to:/usr/bin/*
-
vpp_lib_t
- Applied to:/usr/lib/vpp_api_test_plugins/*
/usr/lib/vpp_plugins/*
-
vpp_unit_file_t
- Applied to:/usr/lib/systemd/system/vpp.*
-
vpp_log_t
- Applied to:/var/log/vpp/*
-
vpp_var_run_t
- Applied to:/var/run/vpp/*
Debug SELinux Issues
If SELinux issues are suspected, there are a few steps that can be taken to debug the issue. This section provides a few pointers on on those steps. Any SELinux JIRAs will need this information to properly address the issue.
Additional SELinux Packages and Setup
First, install the SELinux troubleshooting packages:
$ sudo yum -y install setroubleshoot setroubleshoot-server setools-console
-- OR --
$ sudo dnf -y install setroubleshoot setroubleshoot-server setools-console
To enable proper logging, restart auditd:
$ sudo service auditd restart
While debugging issues, it is best to set SELinux to Permissive
mode. In
Permissive
mode, SELinux will still detect and flag errors, but will allow
processes to continue normal operation. This allows multiple errors to be
collected at once as opposed to breaking on each individual error. To set
SELinux to Permissive
mode (until next reboot or it is set back), use:
$ sudo setenforce 0
$ getenforce
Permissive
After debugging, to set SELinux back to Enforcing
mode, use:
$ sudo setenforce 1
$ getenforce
Enforcing
Debugging
Once the SELinux troubleshooting packages are installed, perform the actions
that are suspected to be blocked by SELinux. Either tail
the log during
these actions or grep
the log for additional SELinux logs:
sudo tail -f /var/log/messages
-- OR --
sudo journalctl -f
Below are some examples of SELinux logs that are generated:
May 14 11:28:34 svr-22 setroubleshoot: SELinux is preventing /usr/bin/vpp from read access on the file hostCreate.txt. For complete SELinux messages run: sealert -l a418f869-f470-4c8a-b8e9-bdd41f2dd60b
May 14 11:28:34 svr-22 python: SELinux is preventing /usr/bin/vpp from read access on the file hostCreate.txt.#012#012***** Plugin catchall (100. confidence) suggests **************************#012#012If you believe that vpp should be allowed read access on the hostCreate.txt file by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'vpp_main' --raw | audit2allow -M my-vppmain#012# semodule -i my-vppmain.pp#012
May 14 11:28:34 svr-22 setroubleshoot: SELinux is preventing /usr/bin/vpp from read access on the file hostCreate.txt. For complete SELinux messages run: sealert -l a418f869-f470-4c8a-b8e9-bdd41f2dd60b
May 14 11:28:34 svr-22 python: SELinux is preventing /usr/bin/vpp from read access on the file hostCreate.txt.#012#012***** Plugin catchall (100. confidence) suggests **************************#012#012If you believe that vpp should be allowed read access on the hostCreate.txt file by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'vpp_main' --raw | audit2allow -M my-vppmain#012# semodule -i my-vppmain.pp#012
May 14 11:28:37 svr-22 setroubleshoot: SELinux is preventing vpp_main from map access on the packet_socket packet_socket. For complete SELinux messages run: sealert -l ab6667d9-3f14-4dbd-96a0-7a655f7b4eb1
May 14 11:28:37 svr-22 python: SELinux is preventing vpp_main from map access on the packet_socket packet_socket.#012#012***** Plugin catchall (100. confidence) suggests **************************#012#012If you believe that vpp_main should be allowed map access on the packet_socket packet_socket by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'vpp_main' --raw | audit2allow -M my-vppmain#012# semodule -i my-vppmain.pp#012
May 14 11:28:51 svr-22 setroubleshoot: SELinux is preventing vpp_main from map access on the packet_socket packet_socket. For complete SELinux messages run: sealert -l ab6667d9-3f14-4dbd-96a0-7a655f7b4eb1
May 14 11:28:51 svr-22 python: SELinux is preventing vpp_main from map access on the packet_socket packet_socket.#012#012***** Plugin catchall (100. confidence) suggests **************************#012#012If you believe that vpp_main should be allowed map access on the packet_socket packet_socket by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'vpp_main' --raw | audit2allow -M my-vppmain#012# semodule -i my-vppmain.pp#012
From the logs above, there are two sets of commands that are recommended to be
run. The first is to run the sealert
command. The second is to run the
ausearch | audit2allow
commands and the semodule
command.
sealert Command
This sealert
command provides a more detailed output for the given issue
detected.
$ sealert -l a418f869-f470-4c8a-b8e9-bdd41f2dd60b
SELinux is preventing /usr/bin/vpp from 'read, write' accesses on the chr_file noiommu-0.
***** Plugin device (91.4 confidence) suggests ****************************
If you want to allow vpp to have read write access on the noiommu-0 chr_file
Then you need to change the label on noiommu-0 to a type of a similar device.
Do
# semanage fcontext -a -t SIMILAR_TYPE 'noiommu-0'
# restorecon -v 'noiommu-0'
***** Plugin catchall (9.59 confidence) suggests **************************
If you believe that vpp should be allowed read write access on the noiommu-0 chr_file by default.
Then you should report this as a bug.
You can generate a local policy module to allow this access.
Do
allow this access for now by executing:
# ausearch -c 'vpp' --raw | audit2allow -M my-vpp
# semodule -i my-vpp.pp
Additional Information:
Source Context system_u:system_r:vpp_t:s0
Target Context system_u:object_r:device_t:s0
Target Objects noiommu-0 [ chr_file ]
Source vpp
Source Path /usr/bin/vpp
Port <Unknown>
Host vpp_centos7_selinux
Source RPM Packages vpp-19.01.2-rc0~17_gcfd3086.x86_64
Target RPM Packages
Policy RPM selinux-policy-3.13.1-229.el7_6.12.noarch
Selinux Enabled True
Policy Type targeted
Enforcing Mode Permissive
Host Name vpp_centos7_selinux
Platform Linux vpp_centos7_selinux
3.10.0-957.12.1.el7.x86_64 #1 SMP Mon Apr 29
14:59:59 UTC 2019 x86_64 x86_64
Alert Count 1
First Seen 2019-05-13 18:10:50 EDT
Last Seen 2019-05-13 18:10:50 EDT
Local ID a418f869-f470-4c8a-b8e9-bdd41f2dd60b
Raw Audit Messages
type=AVC msg=audit(1557785450.964:257): avc: denied { read write } for pid=5273 comm="vpp" name="noiommu-0" dev="devtmpfs" ino=36022 scontext=system_u:system_r:vpp_t:s0 tcontext=system_u:object_r:device_t:s0 tclass=chr_file permissive=1
type=AVC msg=audit(1557785450.964:257): avc: denied { open } for pid=5273 comm="vpp" path="/dev/vfio/noiommu-0" dev="devtmpfs" ino=36022 scontext=system_u:system_r:vpp_t:s0 tcontext=system_u:object_r:device_t:s0 tclass=chr_file permissive=1
type=SYSCALL msg=audit(1557785450.964:257): arch=x86_64 syscall=open success=yes exit=ENOTBLK a0=7fb395ffd7f0 a1=2 a2=7fb395ffd803 a3=7fb395ffe2a0 items=0 ppid=1 pid=5273 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=993 sgid=0 fsgid=993 tty=(none) ses=4294967295 comm=vpp exe=/usr/bin/vpp subj=system_u:system_r:vpp_t:s0 key=(null)
Hash: vpp,vpp_t,device_t,chr_file,read,write
In general, this command pumps out too much info and is only needed for additional debugging for tougher issues. Also note that once the process being tested is restarted, this command loses it's context and will not provide any information:
$ sealert -l a418f869-f470-4c8a-b8e9-bdd41f2dd60b
Error
query_alerts error (1003): id (a418f869-f470-4c8a-b8e9-bdd41f2dd60b) not found
ausearch | audit2allow and semodule Commands
These set of commands are more useful for basic debugging. The
ausearch | audit2allow
commands generate a set files. It may be worthwhile to
run the commands in a temporary subdirectory:
$ mkdir test-01/; cd test-01/
$ sudo ausearch -c 'vpp_main' --raw | audit2allow -M my-vppmain
$ ls
my-vpp.pp my-vpp.te
$ cat my-vpp.te
module my-vpp 1.0;
require {
type user_home_t;
type vpp_t;
class packet_socket map;
class file { open read };
}
#============= vpp_t ==============
allow vpp_t self:packet_socket map;
allow vpp_t user_home_t:file { open read };
As shown above, the file my-vpp.te
has been generated. This file shows
possible changes to the SELinux policy that may fix the issue. If an SELinux
policy was being created from scratch, this policy could be applied using the
semodule -i my-vpp.pp
command. HOWEVER, VPP already has a policy in place. So
these changes need to be incorporated into the existing policy. The VPP SELinux
policy is located in the following files:
$ ls extras/selinux/
selinux_doc.md vpp-custom.fc vpp-custom.if vpp-custom.te
In this example, map
needs to be added to the packet_socket
class. If the
vpp-custom.te
is examined (prior to this fix), then one would see that the
packet_socket
class is already defined and just needs to be updated:
$ vi extras/selinux/vpp-custom.te
:
allow vpp_t self:process { execmem execstack setsched signal }; # too benevolent
allow vpp_t self:packet_socket { bind create setopt ioctl }; <---
allow vpp_t self:tun_socket { create relabelto relabelfrom };
:
Before blindly applying the changes proposed by the ausearch | audit2allow
commands, try to determine what is being allowed by the policy and determine if
this is desired, or if the code can be reworked to no longer require the
suggested permission. In the my-vpp.te
file from above, it is suggested to
allow vpp_t
(i.e. the VPP process) access to all files in the home directory
(allow vpp_t user_home_t:file { open read };
). This was because a
vppctl exec
command was executed calling a script located in the
/home/<user>/
directory. Once this script was run from the /usr/share/vpp/
directory as described in a section above, these permissions were no longer
needed.