Using pkgsrc on Mac OS X

Given NetBSD focus on portability, it’s only logical that pkgsrc is also available on systems other than NetBSD, including Darwin (Mac OS X). Here are some notes showing to bootstrap pkgsrc in unprivileged mode, which means that everything can easily be installed in the user home directory.

Before starting, we need to install Xcode Command Line Tools to get a working compiler.

Fetching and extracing latest pkgsrc stable release

This will create a ~pkgsrc directory :

cd
wget http://ftp.netbsd.org/pub/pkgsrc/stable/pkgsrc.txz
tar xfz pkgsrc.txz

Bootstrapping pkgsrc

Launching the bootstrap script and setting the ABI to 64-bit :

cd pkgsrc/bootstrap
./bootstrap --abi=64 --compiler=clang --unprivileged

This will create and start populating the ~pkg directory where all built packages will be installed.

For a complete list of available options :

./bootstrap -h
===> bootstrap command: ./bootstrap -h
===> bootstrap started: Sat Sep 27 21:59:08 CEST 2014
Usage: ./bootstrap
    [ --abi [32|64] ]
    [ --binary-kit <tarball> ]
    [ --binary-macpkg <pkg> ]
    [ --compiler <compiler> ]
    [ --full ]
    [ --gzip-binary-kit <tarball> ]
    [ --help ]
    [ --mk-fragment <mk.conf> ]
    [ --pkgdbdir <pkgdbdir> ]
    [ --pkginfodir <pkginfodir> ]
    [ --pkgmandir <pkgmandir> ]
    [ --prefer-pkgsrc <list|yes|no> ]
    [ --prefix <prefix> ]
    [ --preserve-path ]
    [ --quiet ]
    [ --sysconfdir <sysconfdir> ]
    [ --unprivileged | --ignore-user-check ]
    [ --varbase <varbase> ]
    [ --workdir <workdir> ]

Adding ~pkg to the path :

export PATH=$PATH:~/pkg/bin:~/pkg/sbin

Fetching security vulnerabilities information :

pkg_admin fetch-pkg-vulnerabilities

Adding some acceptable licenses to our pkgsrc configuration :

echo "ACCEPTABLE_LICENSES+= vim-license" >> ~/pkg/etc/mk.conf

Building packages

Here is how to build a package and clean the working directory and all dependencies :

cd ~/pkgsrc/category/package
bmake install clean clean-depends

Keeping pkgsrc up-to-date

First, we need to build CVS :

cd ~/pkgsrc/devel/scmcvs
bmake install clean clean-depends

We can then update pkgsrc using the following command :

cd ~/pkgsrc && cvs update -dP

Checking for security vulnerabilities in packages :

pkg_admin audit

Installing CA certificates

cd ~/pkgsrc/security/mozilla-rootcerts
bmake install clean clean-depends
mozilla-rootcerts install

For more details, please read the following post : Installing CA certificates on NetBSD.

Using binary packages

For those who prefer using binary packages, please check the Joyent packages repository and Save OS X.

Final words

After running Fink in 2009 on my Mac mini, and then Homebrew since late 2011 on my MacBook Pro, it’s nice to explore alternatives especially since they are not mutually exclusive. It’s in fact a nice idea to combine pkgsrc and Homebrew to get the best of both worlds and access to even more packages.

Lastly, for a comprehensive searchable database of packages, please check the excellent pkgsrc.se.

Fingerprinting DNS servers authoritative for the top 1 million domains

As an experiment, I’ve been using fpdns (version 0.10.0 on FreeBSD/amd64) to fingerprint DNS servers authoritative for the top 1 million domains (according to Alexa).

At first, I had plans to use adnshost to resolve name servers first and then feed the resolved list to fpdns, in order to speed up things and avoid fingerprinting the same host several times. Unfortunately, it seems adnshost doesn’t work that well on large batches and I experienced numerous timeouts and crashes.

Extracting a list of domains from the CSV file

wget http://s3.amazonaws.com/alexa-static/top-1m.csv.zip
unzip top-1m.csv.zip
cut -d "," -f 2 top-1m.csv > domains.txt

As the fingerprinting process will require resolving name servers for each domain in the list, I will be using a local Unbound instance in order to avoid hitting my ISP name servers too aggressively.

Configuring the system to use Unbound as local resolver

After adding our local resolver to resolv.conf :

echo "nameserver 127.0.0.1" > /etc/resolv.conf

We can verify that we are indeed using our Unbound instance :

dig version.bind CH txt +short
"unbound 1.4.22"

Fingerprinting using fpdns

Here is a list of fpdns options we will be using :

-D         (check all authoritative servers for Domain)
-F nchild  (maximum forked processes) [10]

Starting fpdns with 128 child processes :

fpdns -D -F 128 - < domains.txt > fingerprints.txt

Processing output and aggregating results

First, we aggregate results by IP addresses in order to avoid counting results several times (a name server can be authoritative for several different domains) :

cut -d ',' -f 2 < fingerprints.txt | sort | uniq > results.txt

We then aggregate by software and count occurences :

awk -F'[)][:] ' '{print $2}' < results.txt | sort | uniq -c

I used awk here instead of cut as the latest doesn’t allow using more than one character as a delimiter.

Here are the results :

     6 sheerdns  [Old Rules]
     2 3Com Office Connect Remote  [Old Rules]
    57 DJ Bernstein TinyDNS 1.04 [Old Rules]
  5199 DJ Bernstein TinyDNS 1.05 [Old Rules]
    13 Dan Kaminsky nomde DNS tunnel  [Old Rules]
     3 Fasthosts Envisage DNS server  [Old Rules]
     2 Meilof Veeningen Posadis  [Old Rules]
     2 Men & Mice QuickDNS for MacOS Classic  [Old Rules]
     4 Michael Tokarev rbldnsd  [Old Rules]
    29 Microsoft ?  [Old Rules]
   387 Microsoft Windows DNS 2000 [New Rules]
    50 Microsoft Windows DNS 2000 [Old Rules]
    88 Microsoft Windows DNS 2003 R2 [New Rules]
  6373 Microsoft Windows DNS 2003 [New Rules]
    87 Microsoft Windows DNS 2003 [Old Rules]
  1278 Microsoft Windows DNS 2008 R2 [New Rules]
    25 Microsoft Windows DNS 2008 [New Rules]
     2 Microsoft Windows DNS NT4 [Old Rules]
    12 NLnetLabs NSD 1.0 alpha [Old Rules]
 12046 NLnetLabs NSD 3.1.0 -- 3.2.8 [New Rules]
     6 NLnetLabs Unbound 1.4.10 -- 1.4.12 [New Rules]
220751 No match found
    25 Simon Kelley dnsmasq  [Old Rules]
    18 Sourceforge JDNSS  [Old Rules]
     1 TZO Tzolkin DNS  [Old Rules]
  4863 Unlogic Eagle DNS 1.0 -- 1.0.1 [New Rules]
    88 Unlogic Eagle DNS 1.1.1 [New Rules]
    18 ValidStream ValidDNS  [Old Rules]
     1 WinGate Wingate DNS  [Old Rules]
     1 XBILL jnamed (dnsjava)  [Old Rules]
    40 Yutaka Sato DeleGate DNS  [Old Rules]
    13 javaprofessionals javadns/jdns  [Old Rules]

As often with these kind of experiments, results aren’t really exploitable to produce reliable statistics : apparently, it seems that BIND has totally disappeared from the Internet ;)

However, I believe the process is still useful and demonstrates how easy it can be to quickly produce DNS surveys using simple UNIX tools.

DNSSEC validation at the router level with OpenWrt

Over the past few years, I have been exploring various options for doing local DNSSEC validation. Validating locally is necessary in order to avoid DNS answers being forged on the path from the ISP resolvers (or from open validating resolvers) to the local network.

If validating on servers and laptops is a solved problem, doing so on mobile devices such as phones and tablets is still an open question. For these use cases, having a validating resolver running directly on a router is convenient. As it turns out, it’s a pretty simple two steps process to achieve this with OpenWrt.

Disabling Dnsmasq DNS component

Dnsmasq is used within OpenWrt as both DHCP and DNS server, but we only want to use its DHCP component and let Unbound perform DNS resolution.

For doing so, after logging into LuCI, please go to : Network => DHCP and DNS => Advanced Settings.

Set the DNS server port to 0, as shown in the following screenshot :

Listening port for inbound DNS queries set to 0

Installing and configuring Unbound

Using opkg install :

opkg install unbound

The bundled init script supports the following commands :

/etc/init.d/unbound
Syntax: /etc/init.d/unbound [command]

Available commands:
	start	Start the service
	stop	Stop the service
	restart	Restart the service
	reload	Reload configuration files (or restart if that fails)
	enable	Enable service autostart
	disable	Disable service autostart

We start Unbound and enable autostart at boot time :

/etc/init.d/unbound start
/etc/init.d/unbound enable

We also need to fetch (or update) the root anchor manually :

opkg install unbound-anchor
unbound-anchor -a "/etc/unbound/root.key"

Configuring Unbound to validate answers :

# If you want to perform DNSSEC validation, run unbound-anchor before
# you start unbound (i.e. in the system boot scripts).  And enable:
# Please note usage of unbound-anchor root anchor is at your own risk
# and under the terms of our LICENSE (see that file in the source).
auto-trust-anchor-file: "/etc/unbound/root.key"

We can easily verify that our local Unbound instance is answering queries :

dig version.bind CH txt +short
"unbound 1.4.17"

Finally, let’s try to resolve a DNSSEC secured domain :

dig statdns.net +dnssec

; <<>> DiG 9.8.3-P1 <<>> statdns.net +dnssec
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 28028
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 2, AUTHORITY: 3, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 4096
;; QUESTION SECTION:
;statdns.net.			IN	A

;; ANSWER SECTION:
statdns.net.		594	IN	A	46.19.37.108
statdns.net.		594	IN	RRSIG	A 7 2 600 20150901000000 20140901115509 23348 statdns.net. bxWP9OzGs8v6gb9zEJHecQaBkU+BLKr8qRUi6VFPFVoKbZpHwiqkGatb rq7ov2AlnUrjs/a46xiu+bNNx8K9xQvY0f6QrBb/7RUPKPYSRNFiyLkb w9p92QFt5qr4LQL5kpddf/bnJYrVBDP3b6KZ2ph5X5x+C1hDq0HjgqZz EQg=

;; AUTHORITY SECTION:
statdns.net.		594	IN	NS	ns1.statdns.com.
statdns.net.		594	IN	NS	ns2.statdns.com.
statdns.net.		594	IN	RRSIG	NS 7 2 600 20150901000000 20140901115509 23348 statdns.net. FOEKRnI3VkI8EOmBj5xqqSKwdWwdFS24FroZMwBJisN4wSbDrz/EFWaa H0UqPZKKi0ViLM2z0sg1BfEvrDxFb2G4RDGpVsx6uh4BKXeaQ/1KK2Cc IF9gCSGvgyGZMCHd/DI0EeWRU9In3JK+YVI5AGA5qG2oY3IzQl4F1ho9 8RM=

;; Query time: 4 msec
;; SERVER: 10.0.0.1#53(10.0.0.1)
;; WHEN: Sat Oct  4 17:53:53 2014
;; MSG SIZE  rcvd: 445

As we notice, the AD flag is set in the header, meaning the answer is secure.

Window Maker Nostalgia

Window Maker is one of the very few graphical user interfaces which I would call timeless. Despite the fact it hasn’t evolved graphically for almost 20 years, it still looks clean and beautiful by today standards, whereas AfterStep now looks terribly old and outdated.

In fact, it is one of those very few intemporal interfaces which left a huge impact on me over the years :

  • Early Macintosh System Software on the Macintosh Plus, the first GUI I ever used in the mid-eighties
  • GEM (Graphical Environment Manager), customized CGA version on an Amstrad PC 1512
  • FVWM (F Virtual Window Manager), my first X11 love story
  • BeOS, in the late nineties, a clean, bright, and colorful UI which still looks crisp and modern

I loved the default BeOS color scheme so much that I’m using it as my Window Maker theme :

Window Maker running on FreeBSD 10.0, screenshot taken in March 2014

In retrospect, I have very fond memories of the few years during which I used it as my sole window manager, circa 15 years ago. Window Maker allowed the rest of us to experience NeXTSTEP.

Luarocks on FreeBSD

As LuaRocks is unfortunately not part of the FreeBSD ports collection, it has to be compiled manually. As I’m going to need to repeat this process in the foreseeable future, I decided to document the required steps here.

The reason why I’m targeting Lua 5.1 specifically is that I’m installing Luarocks to build rocks to be used with the Nginx Lua module, as I plan to migrate the Telize server to FreeBSD.

Installing Lua and some required packages

pkg install lua51 gmake curl

Fetching, unpacking, and configuring using Curl as downloader

curl -O http://luarocks.org/releases/luarocks-2.2.0.tar.gz
tar xvfz luarocks-2.2.0.tar.gz
cd luarocks-2.2.0
./configure --with-lua-include=/usr/local/include/lua51 --with-downloader=curl
make build && make install
make bootstrap

Building and installing some Lua modules

luarocks build lua-cjson
luarocks build lua-iconv

Verifying that our modules have been built and installed successfully

ls /usr/local/lib/lua/5.1
cjson.so    iconv.so