Debugging Server

Debugging Pritunl server

Getting Information For Bug Report

Use the commands below to get information needed for debugging an issue. Refer to Repair Database for information on repairing the database.

pritunl version
printf "dh:\n" && sudo df -h
printf "free:\n" && sudo free -l -m
printf "pritunl lsof: " && sudo lsof -p `pgrep -x pritunl` | wc -l
printf "pritunl-web lsof: " && sudo lsof -p `pgrep -x pritunl-web` | wc -l
printf "pritunl-dns lsof: " && sudo lsof -p `pgrep -x pritunl-dns` | wc -l
printf "pritunl limits:\n" && sudo cat /proc/`pgrep -x pritunl`/limits
printf "pritunl-web limits:\n" && sudo cat /proc/`pgrep -x pritunl-web`/limits
printf "pritunl-dns limits:\n" && sudo cat /proc/`pgrep -x pritunl-dns`/limits
printf "limits:\n" && sudo sh -c "ulimit -Hn; ulimit -Sn"
sudo netstat -tulpn | grep pritunl
printf "pritunl-http:\n" && curl -I http://localhost/check
printf "pritunl-internal:\n" && curl -I http://localhost:9756/check
printf "pritunl-https:\n" && curl --insecure https://localhost

sudo strace -p `pgrep -x pritunl`
sudo strace -p `pgrep -x pritunl-web`
sudo strace -p `pgrep -x pritunl-dns`

Server Synchronized Time

Several components of a multi-server Pritunl cluster rely on accurate synchronized time on all the servers. It is important to ensure all servers in a cluster have the same time.

Process Inspection

To debug CPU usage of the pritunl process py-spy can be used. Run the command below to install py-spy from PyPI into the Pritunl packaged Python build. If sampling rate error is shown reduce the --rate 50 option. This will not work for the Golang based pritunl-web and pritunl-dns processes.

# Install pySpy from PyPI
sudo /usr/lib/pritunl/usr/bin/pip3 install py-spy

# Find pid for process "/usr/lib/pritunl/usr/bin/python3 /usr/lib/pritunl/usr/bin/pritunl start"
sudo ps ax --format user,pid,%cpu,%mem,tty,stat,start,time,command | grep pritunl

# Get live top view of function CPU usage press 4 to sort by total time spent on function
sudo /usr/lib/pritunl/usr/bin/py-spy top --rate 50 --pid <PID>

# Record CPU usage flamegraph to profile.svg press ctrl+c to stop collection and generate output
sudo /usr/lib/pritunl/usr/bin/py-spy record --rate 50 --output profile.svg --pid <PID>

# Dump list of active threads
sudo /usr/lib/pritunl/usr/bin/py-spy dump --pid <PID>

Updated about 5 hours ago