Monday, July 22, 2019

Compiling Emacs on Mac OS against X11 libraries

Motivation

My place of work has been using Apple as the main, and now basically the only, supported platform for office use. As a long-time (GNU) Emacs user and occasional contributor, I usually compile GNU Emacs from the development (master) sources, which are now thankfully available over Git.
I already feel bad for using a proprietary OS rather than something based on GNU/Linux or another free or at least open source system. For this and other reasons, I try to at least use system libraries from the free/open source community rather than Apple's, for example for the GUI.  So rather than Apple's Carbon UI, I build Emacs against X11 libraries. Previously I could do this with the Gtk toolkit, but the maintainers of the Mac OS port of Gtk I was using stopped supporting its X11 drivers. So I reverted to the older "Lucid" toolkit with X11. (Yes, maybe I'm weird to cling to the X Window System in spite of the obstacles... that's probably because X was an important part of my socialization to Free Software and distributed systems.)

Howto

The prerequisites are
  • a bunch of packages from Homebrew (not all of which may be relevant):
    • autoconf autogen automake cairo d-bus dbus fontconfig freetype gcc gdk-pixbuf gettext git glib gmp gnutls gobject-introspection graphite2 harfbuzz imagemagick intltool ispell jansson jpeg json-glib libffi libpng librsvg libtasn1 libtiff libtool libunistring libxml2 little-cms2 netpbm nettle openjpeg openshift-cli osinfo-db osinfo-db-tools pcre pcre2 pkg-config readline shared-mime-info texinfo zlib
  • an X11 (XQuartz) installation with development headers
  • The Xcode development environment with its CLI tools installed.
I clone Emacs into /var/tmp/emacs/emacs according to the instructions on the Emacs from Git page of the Emacs Wiki. Then I mkdir /var/tmp/emacs/gbuild, cd there, and run
../emacs/configure --verbose \
  --with-x --with-x-toolkit=lucid --with-ns=no \
  --without-makeinfo \
  LIBXML2_CFLAGS=-I/usr/local/opt/libxml2/include/libxml2 \
  LIBXML2_LIBS='-L/usr/local/opt/libxml2/lib -lxml2' \
  --with-jpeg=no --with-gif=no --with-tiff=no \
  --x-libraries=/usr/local/opt/freetype/lib:/usr/X11/lib \
  --x-includes=/usr/local/opt/freetype/include:/usr/X11/include \
  --with-xpm=no \
  PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:/usr/X11/lib/pkgconfig
This gives me the following configuration:
Configured for 'x86_64-apple-darwin18.6.0'.

  Where should the build process find the source code?    ../emacs
  What compiler should emacs be built with?               gcc -g3 -O2
  Should Emacs use the GNU version of malloc?             no
    (The GNU allocators don't work with this system configuration.)
  Should Emacs use a relocating allocator for buffers?    no
  Should Emacs use mmap(2) for buffer allocation?         no
  What window system should Emacs use?                    x11
  What toolkit should Emacs use?                          LUCID
  Where do we find X Windows header files?                /usr/local/opt/freetype/include:/usr/X11/include
  Where do we find X Windows libraries?                   /usr/local/opt/freetype/lib:/usr/X11/lib
  Does Emacs use -lXaw3d?                                 yes
  Does Emacs use -lXpm?                                   no
  Does Emacs use -ljpeg?                                  no
  Does Emacs use -ltiff?                                  no
  Does Emacs use a gif library?                           no
  Does Emacs use a png library?                           yes -L/usr/local/Cellar/libpng/1.6.37/lib -lpng16 -lz
  Does Emacs use -lrsvg-2?                                no
  Does Emacs use cairo?                                   no
  Does Emacs use -llcms2?                                 yes
  Does Emacs use imagemagick?                             no
  Does Emacs support sound?                               no
  Does Emacs use -lgpm?                                   no
  Does Emacs use -ldbus?                                  yes
  Does Emacs use -lgconf?                                 no
  Does Emacs use GSettings?                               no
  Does Emacs use a file notification library?             yes (kqueue)
  Does Emacs use access control lists?                    yes
  Does Emacs use -lselinux?                               no
  Does Emacs use -lgnutls?                                yes
  Does Emacs use -lxml2?                                  yes
  Does Emacs use -lfreetype?                              yes
  Does Emacs use HarfBuzz?                                yes
  Does Emacs use -lm17n-flt?                              no
  Does Emacs use -lotf?                                   no
  Does Emacs use -lxft?                                   yes
  Does Emacs use -lsystemd?                               no
  Does Emacs use -ljansson?                               yes
  Does Emacs use -lgmp?                                   yes
  Does Emacs directly use zlib?                           yes
  Does Emacs have dynamic modules support?                no
  Does Emacs use toolkit scroll bars?                     yes
  Does Emacs support Xwidgets (requires gtk3)?            no
  Does Emacs have threading support in lisp?              yes
  Does Emacs support the portable dumper?                 yes
  Does Emacs support legacy unexec dumping?               no
  Which dumping strategy does Emacs use?                  pdumper
Then I can build Emacs using
make bootstrap
This produces an intermediate "temacs" binary referencing many shared objects:
: 1leinen@macsl[leinen]; objdump -macho -dylibs-used /var/tmp/emacs/gbuild/src/temacs
/var/tmp/emacs/gbuild/src/temacs:
 /opt/X11/lib/libpng16.16.dylib (compatibility version 43.0.0, current version 43.0.0) /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11) /opt/X11/lib/libXaw3d.8.dylib (compatibility version 9.0.0, current version 9.0.0) /opt/X11/lib/libXmu.6.dylib (compatibility version 9.0.0, current version 9.0.0) /opt/X11/lib/libXt.6.dylib (compatibility version 7.0.0, current version 7.0.0) /opt/X11/lib/libSM.6.dylib (compatibility version 7.0.0, current version 7.1.0) /opt/X11/lib/libICE.6.dylib (compatibility version 10.0.0, current version 10.0.0) /opt/X11/lib/libXext.6.dylib (compatibility version 11.0.0, current version 11.0.0) /opt/X11/lib/libX11.6.dylib (compatibility version 10.0.0, current version 10.0.0) /opt/X11/lib/libX11-xcb.1.dylib (compatibility version 2.0.0, current version 2.0.0) /opt/X11/lib/libxcb.1.dylib (compatibility version 3.0.0, current version 3.0.0) /opt/X11/lib/libXft.2.dylib (compatibility version 6.0.0, current version 6.2.0) /opt/X11/lib/libXrender.1.dylib (compatibility version 5.0.0, current version 5.0.0) /usr/local/opt/dbus/lib/libdbus-1.3.dylib (compatibility version 23.0.0, current version 23.11.0) /opt/X11/lib/libXrandr.2.dylib (compatibility version 5.0.0, current version 5.0.0) /opt/X11/lib/libXinerama.1.dylib (compatibility version 2.0.0, current version 2.0.0) /opt/X11/lib/libXfixes.3.dylib (compatibility version 5.0.0, current version 5.0.0) /usr/local/opt/libxml2/lib/libxml2.2.dylib (compatibility version 12.0.0, current version 12.9.0) /usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0) /usr/local/opt/freetype/lib/libfreetype.6.dylib (compatibility version 24.0.0, current version 24.1.0) /opt/X11/lib/libfontconfig.1.dylib (compatibility version 11.0.0, current version 11.2.0) /usr/local/opt/harfbuzz/lib/libharfbuzz.0.dylib (compatibility version 20504.0.0, current version 20504.0.0) /usr/local/opt/gnutls/lib/libgnutls.30.dylib (compatibility version 55.0.0, current version 55.0.0) /usr/local/opt/little-cms2/lib/liblcms2.2.dylib (compatibility version 3.0.0, current version 3.8.0) /usr/local/opt/jansson/lib/libjansson.4.dylib (compatibility version 16.0.0, current version 16.1.0) /usr/local/opt/gmp/lib/libgmp.10.dylib (compatibility version 14.0.0, current version 14.2.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
...and an src/emacs that hopefully doesn't crash. Oops... why am I saying that?
Well, yesterday I used a simpler method that didn't specify /usr/local/opt/freetype explicitly anywhere, but relied on autoconf/pkgconfig to find it. Unfortunately this created a "DLL hell" situation where an older version of the freetype dynamic library from /opt/X11/lib was used instead of the newer on in /usr/local/opt/freetype/lib, leading to crashes when Emacs tried to use certain fonts. Thanks to YAMAMOTO Mitsuharu for helping me find the error in my old build process!