During the holidays I managed to get some time to hack on nsntrace. I have mentioned nsntrace on this blog in the past but you might need a refresher:
The nsntrace application uses Linux network namespaces and iptables magic to perform network traces of a single application. The traces are saved as pcap files and can be analyzed by for instance wireshark or tshark
In May of 2020 nsntrace moved to a collaborative maintenance model and to its own GitHub organisation. This helped a lot with my guilt for neglecting its maintenance. We even managed to get some pull requests merged and some issues solved.
For the new release we landed some notable pull requests.
Handle loopback nameservers
On many systems the nameserver functionality is handled by an application such as systemd-resolved or dnsmasq and the nameserver address in resolv.conf is a loopback address (like 127.0.0.53) where that application listens for incoming DNS queries.
This would not work for us since the network namespace we create has its own separate loopback device and cannot reach the one where systemd-resolved or dnsmasq listens. If somebody reading this has a solution to this problem, please let us know in the comments or via GitHub!
What we ended up doing were two things. First we added a warning output for when we detect only loopback nameservers. And second we offer a new --use-public-dns command line option to override the system resolv.conf in our namespace.
$ sudo ./src/nsntrace ping lwn.net Warning: only loopback (127.x.y.z) nameservers found. This means we will probably not be able to resolve hostnames. Consider passing --use-public-dns to use public nameservers. Starting network trace of 'ping' on interface wlp58s0. Your IP address in this trace is 172.16.42.255. Use ctrl-c to end at any time.
In order to have our own nameservers for our network namespace we will bind mount our own resolv.conf file, with public nameservers from Cloudflare, OpenDNS and Quad9, over the one in /etc/resolv.conf.
But we do not want to affect other applications outside of our namespaced application, so we enter a mount namespace (CLONE_NEWNS) before we do the bind mount.
This, however, is not enough. Before the bind mount we need to remount the root partition with special mount flags to make sure our changes to the subtree are not propagated to the outside mount namespace.
This is basically how the ip-netns tool allows for namespace specific nameservers as well.
Treat lone dash ("-") output filename as stdout
This started as an issue from a user that reported:
I am using nsntrace to dump and analyze a processes network traffic in real time. In order to do that, I need to get the network dump piped to another process, not written to a file.
We got this implemented and we made sure to write all non-pcap output to stderr in the case of a lone dash filename being detected. We also made sure that the application under trace behaves by replacing its stdout file descriptor with stderr, using the dup2 syscall.
This change enables us to do live packet capture using tshark.
$ sudo nsntrace -o - wget lwn.net | tshark -r - -Y 'http.response or http.request' 13 0.467670 172.16.42.254 → 239.255.255.250 SSDP 214 M-SEARCH * HTTP/1.1 16 0.541178 172.16.42.255 → 66.228.47.22 HTTP 200 GET / HTTP/1.1 20 0.747923 66.228.47.22 → 172.16.42.255 HTTP 448 HTTP/1.1 302 Moved Temporarily (text/plain) 41 1.468987 172.16.42.254 → 239.255.255.250 SSDP 214 M-SEARCH * HTTP/1.1 71 2.472515 172.16.42.254 → 239.255.255.250 SSDP 214 M-SEARCH * HTTP/1.1
Give the network namespace a name
To give a network namespace a name in the eyes of tools like ip-netns we need to bind mount /proc/<pid>/net/ns/ to /run/netns/NAME. It is described in the man page of ip-netns:
By convention a named network namespace is an object at /run/netns/NAME that can be opened. The file descriptor resulting from opening /run/netns/NAME refers to the specified network namespace. Holding that file descriptor open keeps the network namespace alive. The file descriptor can be used with the setns(2) system call to change the network namespace associated with a task.
By implementing this, the nsntrace network namespace will show up when we use ip-netns.
$ ip netns list
nsntrace-128411
And we might even use ip-netns to enter the same network namespace as a running nsntrace process.
By adding a snapcraft.yaml file we enable building a snap package of nsntrace and adding it to the snapcraft app store. Which might mean more people can find nsntrace, and that the maintenance team get a bit more control of distribution of nsntrace.
Thanks for reading! And check out the new release!
