Post-Vacation Update and Build System Observations

2017-05-31

I just got back from a two-week vacation, and the last blog post is over three weeks old. It explained the strategy for building OVM by rewriting Python's build system from scratch.

What's happened since then?

I completed most of that work, and started a few new things, all leading up to an initial OSH release. The work led to ideas for blog posts which I don't have time to write. So in this post, I'll summarize the main ideas (in "Twitter mode").

The OSH App Bundle is Working

I've packaged OSH as a single file with both native code and bytecode, as described in the last post.

However, it's too big and too slow. The bash man page acknowledges that bash is too big and too slow, and right now OSH is even worse:

The source tarball is too big and takes too long to compile, because it's a hefty subset of Python.
Once compiled, it takes too long to start (tens of milliseconds). This is apparently because initializing Python modules is expensive. Each module does some work on import, and this all happens before main().
It takes too long to parse your code. I still need to translate the lexer into a state machine, e.g. via re2c.
It takes too long to run your code. It's an interpreter written in an interpreted language. This can be fixed by compiling OSH into bytecode, although we'll need to change the opcode semantics in many cases.
The resulting binary is too big.

It's not worse than a normal Python program, but I'm (rightly) using the standard of a C program when making these judgements.

This annoys me, but I'm constantly reminding myself that the right strategy is to prioritize completeness and correctness over performance.

In particular, I don't want to block the initial OSH release on performance optimizations. I believe it will have (some) value even if it's bigger and slower than bash, and it's important to release early and often.

After it's released, someone might have better ideas for optimization than I do. That would be nice because a more important goal is to make progress on the Oil language.

An Evaluation of GNU Make

I wrote a Makefile to build both the release tarball and the osh binary. If I were to write a GNU Make experience report, it would elaborate on these issues:

It's extraordinarily easy to write incorrect Makefiles. This is because Makefiles are hard to test, and because Make is missing essential features and concepts. This thread discusses related build system design issues.
I've made use of pattern rules (e.g. %.o : %.c) in three different Makefiles now, and they're useful. They can probably be generalized.
Build actions with multiple outputs should be expressed with pattern rules if you want correct parallel builds. (In Oil, parallel builds a la make -j will be the default.)
gcc -M uses the preprocessor to discover dependencies in C source code, then generates GNU Make fragments, which are often massaged with sed. This interface is poorly designed.
I finally remember what $^, $<, $@, and $* mean in Make. It's unfortunate that these special variables collide so badly with shell's special variables.
Shell here docs also interact badly with Makefile syntax.
I don't like the style of using "fake targets" to set build execution flags.
- .DELETE_ON_ERROR and .SECONDARY should be the default.
- .PHONY and .ONESHELL are bolted-on hacks.

Observations on Build System Design and Implementation

This work also led to observations about build systems in general:

Build systems use too many languages. Make is only a small part of the picture. The CPython build system uses make, shell, non-trivial sed, autoconf/M4, and Python. I replaced the sed with awk for readability. The top-level setup.py script uses distutils to build the standard library.
Build systems perform poorly. The CPython core can be built in parallel with make -j, but the standard library must be built serially with setup.py. This is bad because the larger task is done in a slower fashion.
Build systems are big, and should be treated as real code. CPython's build system is bigger than tinypy, a Python interpreter with an impressive number of features.
Build systems use shell poorly. I've encountered this both in CPython and toybox. This gave me a couple ideas for the series on "Shell: The Good Parts".
Build systems use metaprogramming. For example,
1. Code generation with gcc -M.
2. Expressing build variants with $(eval).
Build "actions" should be pure functions. This has implications for both correctness and performance of builds. Many Makefile bugs are a result of the fact that Make does not have this view of the world.
With this view, you can think of build systems as using partial evaluation.
1. The first step is done on the developer's box: translate the files in the source repo to a source tarball. This step can have non-trivial transformations, like invoking autoconf, or generating code with yacc so the end user doesn't need to install it.
2. The second step is done on the end user's box: compile the source tarball to an OS- and architecture-specific binary. The job of the configure script is to discover parameters for the second evaluation.

Summary

I completed most the work described in the last post, but the result still needs to be optimized.

Rewriting Python's build system produced specific observations about Makefiles, as well as general observations about build systems. I wrote about them while they're fresh in my mind, so I can use them when designing Boil.

In the next post, I'll give an update on project metrics.