Why Sponsor Oils? | blog | oilshell.org

OSH 0.5 - A Release with Many Contributors

This is the latest version of OSH, a bash-compatible shell:

• https://www.oilshell.org/download/oil-0.5.0.tar.xz (gzip version)

Please try it on your shell scripts and report bugs! To build and run it, follow the instructions in INSTALL.txt.

If you're new to the project, see Why Create a New Shell?. OSH can run unmodified shell scripts that are thousands of lines long, as described in the announcement for OSH 0.4.

Summary of Changes

This section summarizes the raw changelog. If you notice anything missing, let me know in the comments.

Platform Support

• Aleks Kamko ported Oil to OS X! This fixed some of the hacks I put in when rewriting Python's build system from scratch.

• Oil no longer segfaults when compiled with GCC 8. This bug affected Arch Linux, Fedora, and other distributions.

  This bug was interesting: Oil inherited C code from CPython that relied on undefined behavior. In GCC 8, this undefined behavior was no longer harmless. See the links in issue 107 for details.

From External Contributors

(in reverse alphabetical order)

• Yorwba
  • Fixed the semantics of arrays when elements are unset (as opposed to empty).
  • Fixed several crashes in the parser.
  • Improved the spec tests.
• Ricardo Grant
  • Tightened up the spec test format.
• Kyle Strand (BatmanAoD)
  • Implemented cd -P and -L.
  • Implemented the pwd builtin.
  • Implemented set -o emacs/vi.
  • Implemented source with arguments. (I also worked on this.)
• Aleks Kamko
  • Reimplemented the glob to regex conversion, which fixed bugs in the ${x/glob/replace} construct. (I also worked on this.)

From Recurse Center Contributors

I pair-programmed with everyone on these changes:

• okay zed
  • Implemented ${BASH_REMATCH[@]}, which are the groups captured during a regex match.
• Emanuel Gerodin
  • Reimplemented the UTF-8 encoding in pure Python, to remove the dependency on Python's unichr() function.
• Caroline Lin
  • The string length operator now counts code points, not bytes.
• Nick Hahn and Caroline Lin
  • Oil can now behave like readlink -f, in the style of busybox. The motivation for this is to provide a cross-platform way to locate files relative to the main script, even in the presence of symlinks. (issue 96)
• Janice Shiu
  • Reimplemented the UTF-8 string length calculation in pure Python.

Other Features

• Implemented the &> and &>> redirects (from bash). Thanks timetoplatypus for the report.
• Add i to $- when the shell is interactive. Thanks BatmanAoD for the report.

Bug reports like these are particularly useful because they let me know where OSH falls short in practice.

Bug Fixes

• Fixed a bug in parsing hex code point constants like \u03bc.
• Fixed crash in unquoted ${#array[@]} (issue 114)
• Fixed make clean in the source tarball.

Under the Hood

• Because of the Unicode changes above, Oil no longer includes unicodeobject.c from CPython. This noticeably reduced build times. See the metrics below.
• Removed other CPython source code from Oil, like complexobject.c and frozen.c.
• Removed almost all dependencies on Python's re module by compiling regexes to C via re2c. After we remove the last dependency, we can strip Python's regex engine out of Oil.
• Many enhancements to OPy. There are too many to list but these documents have details:
  • Building Oil with the OPy Bytecode Compiler
  • opy/README.md
  • I've started publishing OPy test output to the release index.
• The spec tests can now run against stable versions of shells like bash and dash, which means that they're less dependent on the developer's OS. There's still more to do in this area.

Reminder: Zulip

If you're interested in Oil development, please join oilshell.zulipchat.com, as described in this post.

For users of recurse.zulipchat.com: One thing I hadn't anticipated is that you can only be logged into one Zulip instance at a time. I assume that this is due to a browser security issue.

Possible solutions:

• Use two browsers.
• Use their mobile or desktop app, which allow you to switch between instances easily.

What's Next?

When I return to San Francisco in August, I plan to write some blog posts, including another project roadmap.

Feel free to send me feedback in the comments!

Appendix A: Selected Metrics

Here I summarize changes in the tests and metrics published with each release.

• spec tests for 0.5.alpha3: 1007 tests, 840 passing, 119 failing
• spec tests for 0.5.0: 1038 tests, 873 passing, 117 failing

Native code size went down due to the removal of parts of CPython:

• nativedeps for 0.5.alpha3: 165,131 lines
• nativedeps for 0.5.0: 151,645 lines

This is reflected in the smaller binary, which is faster to build:

• ovm-build for 0.5.alpha3: native code size is 1,762,384 bytes
• ovm-build for 0.5.0: native code size is 1,667,688 bytes

Oil's Python source code only got a little bigger:

• src for 0.5.alpha3: 16,504 lines
• src for 0.5.0: 17,096 lines

Excluding comments and blanks:

• cloc for 0.5.alpha3: 8,712 + 139 lines of ASDL
• cloc for 0.5.0: 9,130 + 144 lines of ASDL

The bytecode size went up a bit:

• pyc-bytes for 0.5.alpha3: 1,005,443 bytes
• pyc-bytes for 0.5.0: 1,022,691 bytes