Why Sponsor Oils? | blog | oilshell.org
The last post was about stricter behavior for existing shell features. This post is about new features, which I collectively call the Oil language. (Both posts are tagged #zulip-links, since they summarize Zulip threads.)
Casual readers may want to skip this post, since we'll discuss many details that are subject to change. But if you're interested in shaping the design of the Oil language, read on.
The linked threads describe new Oil language features, which you can try now:
$ git clone https://github.com/oilshell/oil.git $ cd oil $ build/dev.sh minimal $ bin/osh osh$ var x = 1 + 2*3 osh$ echo $x 7
After trying them, please ask questions on Zulip! Many of the threads below were motivated by questions. Feedback also helps me prioritize what parts of the language to work on.
Before diving into details, here's an important update on the name "Oil".
I explained in the 2018 FAQ that the Oil project contains two separate languages, OSH and Oil. OSH runs your existing shell scripts, and Oil will be a new language that you can automatically translate them to.
This changed in the last couple months as I've started to implement Oil. Oil is now an upgrade to OSH. It retains compatibility using three mechanisms:
var and func. An entirely new sublangauge can be
introduced to the right of a keywords, via lexer
modes. I'll try to minimize the number of new
keywords, favoring builtins where possible.shopt -s parse-at, which allows you to write
@array instead of "${array[@]}".shopt -s simple-word-eval, discussed below.This change is essential to understand if you're following the project, and the Zulip thread has details. Summary:
While I had a strong idea of the project's design three years ago, it's changed based on real-world experience.
After implementing several Oil features within OSH, it appears that compatibility requires surprisingly few compromises.
To back this up, it would be nice to talk about what a legacy-free
implementation of Oil would look like. I won't have time to work on that since
I'll be maintaining everything in the osh binary, but it's a good thought
experiment.
A related thing I'd like help with is a set of Python-compatible data structures and garbage collector, e.g. what MicroPython has. That's a large project on its own!
Try typing !qefs in #bash on FreeNode. It responds with advice that has
been drilled into every shell programmer's head for decades.
My view is that the default is wrong, and the shell language should be changed.
Oil introduces a new mode shopt -s simple-word-eval, which turns on an
alternative word evaluation
algorithm
that disables word splitting, empty elision, and dynamic globbing. In other
words, you don't need to use double quotes everywhere.
Oil programs should use arrays rather than word splitting (which poorly simulates arrays). But this means we need to improve array syntax and semantics, because shell arrays are confusing and awkward.
shopt -s parse-at is enabled, you can use @flags to splice an
array flags into a command, e.g. ls @flags ~/src.push builtin appends one or more elements to an array, and replaces
awkwkard shell idioms.var myarray = @(bare words).In the future, we'll also allow splicing arrays returned from functions with
echo @arrayfunc(x, y).
var to declare variables and setvar to mutate them (thread)Oil behaves more like other languages in this regard. It's clear from the syntax whether you're declaring or modifying a local or global variable.
OSH has shell assignment builtins like local and declare for compatibility,
but they're deprecated.
eval builtin (thread)Based on a help-bash@ thread, I introduced another strict option shopt -s strict-eval-builtin, which changes eval to take exactly one argument.
In POSIX shell, eval and echo both have the pitfall that they join
their arguments with a space, which is confusing since it's another step after
word splitting! This is another place where the distinction between
strings and arrays is blurred.
echo will be fixed in the near future with shopt -s oil-echo.
Early this year, I rediscovered a 30-year-old bug in all ksh-derived shells, including bash.
OSH doesn't have this problem. Nonetheless, Oil expressions under var and
setvar replace shell arithmetic.
In addition to the Oil language, I'm working on POSIX shell compatibility. The
Smoosh project uncovered several bugs which I've fixed, and it
exposing unimplemented features like times, command -V and export -p.
I could use help with these features!
Here is how I see things going:
HEAD, using the instructions at the top of this
post.I've had the Oil language brewing in my head for over 3 years, so this iterative process should yield a coherent result quickly.
In contrast, I started an rfc/ dir in the Oil repo, but it became apparent
that writing the docs takes longer than implementing the features. That
process may make sense in the distant future.