Why Sponsor Oils? | blog | oilshell.org
This is the latest version of Oil, a compatible Unix shell and an experimental new language:
If you're new to the project, see Why Create a New Shell? and the 2019 FAQ.
To build and run it, follow the instructions in INSTALL.txt. Please try it on your shell scripts and report bugs. I'm also looking for feedback on the Oil language, which you can send through Github or Zulip.
oil-native Shows How We'll Optimize OilI've warned that Oil is too slow because it's written in an abstract style with a focus on correctness.
This release takes a major step toward speeding it up. There's a new
oil-native tarball:
with a demo you can run in 10-20 seconds:
$ build/mycpp.sh tarball-demo
...
-rwxrwxr-x 1 andy andy 487584 Dec 8 20:47 _bin/osh_parse.opt.stripped
You can now run _bin/osh_parse.opt.stripped. Example:
+ _bin/osh_parse.opt.stripped -c 'echo "hello $name"'
(command.Simple
words: [
(compound_word parts:[
(word_part.Literal token:(Token id:Id.Lit_Chars val:echo span_id:0))
])
(compound_word
parts: [
...
]
)
]
)
What is this?
_bin/osh_parse is Oil's principled parser automatically
translated to C++. This listing of the tarball shows that it's
pure C++.oil tarball, not the
oil-native one.The next post will cover:
#oil-dev about it.The rest of this post summarizes changes in the last four releases. Since the demo of the Oil language in October, most work has been on translation, but there are also some features and bug fixes.
Contributions:
${s#?} and ${x//?/char}. This algorithm was
tricky and I didn't know how to do it myself!printf %d \'c, which is shell's obscure syntax for the
ord(c) function. The Oil language should simply use
ord(c).Other:
tup(42) is a tuple with one element, instead of Python's confusing 42,
(with trailing comma). Singleton tuples are rare.Contributions:
uninstall script to undo what install does.Other:
${prefix*}, which the homebrew
package manager uses to unset variables starting with a certain prefix. I'd
like more testing of important shell scripts along these lines. See the
#should-run-this label on Github, and How To Test
OSH.oil-native tarball (linked above).py-yajl into the app
bundle.py-yajl, with just 862 lines of C, compared to the original's
1578 lines.oil-native release.pp f(x) to
= f(x). It's like an assignment with nothing on the LHS. As in
assignments, everything to the right of = is parsed in expression
mode.=foo. Most likely you
want to add a space like = foo, or quote it like '=foo'.oil-native tarball.The C++ translation isn't done, but the oil-native demo has made me
optimistic that this large project is feasible. After all, I wrote more
than two years ago that using CPython is the riskiest part of the
project!
But now we have a concrete path forward. (Appendix A summarizes the path I tried and abandoned.) The parser is about 40% of the Oil codebase, and it took two months to translate. Given that, I expect the rest to take two to six months of continuous work to translate.
However, at the current rate, this will likely happen more than six months from now, because there are many other parts of the project.
In short, it would be better if development was more parallel than serial. There are many independent parts of the project.
The requests in How to Help still stand:
oshrc.I continue to maintain issue labels to help new contributors:
There are several important categories of work:
virtualenv for C and shell.
build/dev.sh minimal and then bin/osh run
in an isolated environment. These commands generate Python source code,
build a C extension, and run the resulting plain Python program. I got
a couple PRs but it wasn't clear how to use them. (This is the "dev
build", which is different than release build or the mycpp build.)Overall, I think oil-native is evidence that Oil will work. Try it and
let me know if you disagree!
An abstract shell interpreter can be turned into a production-quality shell. The shell will be compatible with old programs, but it also has powerful Python-like data structures and JSON support.
In other words, Oil is our upgrade path from bash.
If that appeals do you, consider helping out. The most important thing is to try it and figure out what's preventing you from being a user. Users are more likely to be developers!
oil, but it will be
retired when Oil becomes a pure C++ program via mycpp,
ASDL, and other translators.dict,
list, tuple, str, int, bool, but we'll remove the bytecode
interpreter in favor of native code. This solves the "double interpretation"
problem.Let's compare the current release with version 0.7.pre5, released two months ago on October 4th.
The parser benchmarks deserve their own post, so here are the metrics we routinely track.
Most of the development work was on translation, which doesn't affect spec tests:
There were a few new features like the ${prefix@} implementation. And we add
failing tests to expose bad behavior, sometimes before we can fix it.
The Oil language now has JSON support:
Even though there were at least a hundred commits to aid translation, the source code didn't get much bigger:
Physical code:
The yajl dependency for JSON support added almost 5K lines of C code:
The native code size increased by a corresponding amount:
The bytecode shrunk a bit:
These are minor differences compared to the reductions that mycpp should enable in the coming months. All bytecode will be replaced with native code.