Why Sponsor Oils? | blog | oilshell.org
I wrote in the last post that we want to stabilize the compatible OSH, and resume work on YSH (formerly known as the Oil language).
I'd like to bring more people into the project, and that requires explaining what we need help with!
For now, I'm dumping links to recent Zulip planning threads. For some reason the titles alone help me think about the project.
I don't expect that readers will follow it all, but I expect to refer to this post in private and public conversations. I may edit it as we go.
Ask me questions here or on Zulip!
I'm itching to work on YSH, but OSH needs to be polished and useful.
help builtin
oshrc.d directory, in the style of Debian, which is a good idea.oil to oils in ~/.config/oil/oshrc. Figure out where to put history files, standard library, and more.(loc_t, command_t) lists, and let's create a nicer syntax like trap --event DEBUG { echo debug }starship.rs or Nix.trap 'echo debug' DEBUG registers code to run in the main interpreter loopcomplete -C could lead into a completion protocolhistory -a and -r.We still need more testing of OSH. I just fixed a good getopts builtin bug reported by a user.
configure / Running real scripts. This was one of the open questions / risks in the last post.
/usr/local/bin/oils-for-unix only, no bash.To accompany the project renaming, we have to rename environment variables and directories:
OIL_GC_STATS → OILS_GC_STATS~/.config/oils/oshrc and ~/.config/oils/yshrcI'll talk about in the next post. We also might switch to date-based versioning, like Oils 23.04 for April 2023.
May 2023 Update: In addition to the help builtin, I plan to overhaul the HTML docs. I started planning here:
We can optimize the shell in Python, or in C++.
Based on #oil-dev > Interpreter Optimizations in Python
loc_t (already started, with contributions from CoffeeTableEspresso and Travis Everett)Token type smaller by removing redundant infoList[Token]Token as value type, not GC objectbenchmarks/compute/fib.sh benchmarkingcompound_word() instances in echo hi to be cached. We could go further than this, but we should keep the code simple. Measurements will help.Metrics / tools we're using for performance:
benchmarks2/uftrace at http://travis-ci.oilshell.org/github-jobs/. Should put this in the /release/$VERSION tree.Based on #oil-dev > Plans for mycpp runtime
Near Term:
next pointer and linked lists. We can reuse the small integer ID from the mark bitmap. It will also have "two levels" -- 32 byte cells and larger blocks.Medium Term
Slab<T> can be "inlined" into the List<T> header. Affects field masks.Str* to Str everywhereLong Term:
This thread has a lot of brainstorming, but I've narrowed it down to the above: GC Pressure Optimization List
Conclusion: There's a lot of fun work here! It would be nice to have someone to own the mycpp runtime.
These are speculative ideas to reduce allocations. Whether they're interesting depends on the results of the custom pool allocator.
Final[T] in Python, and it works well!Foo foo on the stack rather than auto foo = Alloc<Foo>(), and then pass the pointer down the stack.There's been a lot of progress recently. This section is based on these two threads:
oilshell/oil, not their fork myname/oil.List[str] -> List[value_t] refactoring. Could be painful. No thread on it yet.May 2023 update: Shell has history substitution like !! and !$. A few years ago, I implemented it as bash does: with a separate lexing pass that's only active in the interactive shell.
But this causes problems because "I'm excited !!" within double quotes also expands history. Moreover, it conflicts with egg expressions, which use ! for negation.
So I believe we should rewrite history substitution as part of our single-pass, static parsing. In non-interactive shells, it will be easy to treat the history tokens as literals.
I should write a separate post about #oil-discuss > Oil Language 2023. Here are some rough notes for now:
Upcoming breakages:
shopt --set simple_echo, which will be on in YSH.
echo -e -n. This eliminates an escaping issue, and makes code look more familiar.$join(mylist) in favor of more explicit $[join(mylist)], to remove a wart$'\n' to YSTR y"\n". Unify with data languages.mydict->name to mydict.name. A practical concession.May 2023 Updates:
${.builtin echo hi} syntax. This can just be a function var s = builtin_sub(mycmd).setref and proc p(:out).
setref (), and a Ref type like proc p(out Ref).Upcoming features:
func or fn keyword.error () builtin, and think about return (expr).argparse builtin: can we do it as a YSH library, rather than it being in the interpreter? It's a good test case.(*.py) isn't the idiom. Parentheses mean Oil expressions.
| *.py >, which is a bit weird.;; token.@[strs] in expression mode, to be consistent.Data languages:
May 2023 Update: The "task file" pattern should be built into Oils. Make is being used/abused as a task runner, and it introduces a bad textual macro language on top of shell.
Chris and Aidan both improved the used of task files in our own repo. More links:
run.sh Scripts - A section of a 2020 post with many links.This is pretty ambitious, but we've been knocking down tasks at a good rate lately. And I hope to get more help.
I could write these blog posts:
Refer to the last post.
I want to let contributors know that the codebase is in much better shape.
I mentioned a bunch of improvements in the last post. The mycpp/examples all pass; the code is auto-formatted; and we removed hard-coded lists of names. Those names coupled mycpp to Oil.
However, CoffeeTableEspresso did hit a new smell with regard to variable vs. module naming. If you import a name loc, then mycpp will generate the wrong C++ for the local variable loc.
Nice fix:
List[int] rather than int*, which looks like MyPy.List[int]? for Optional[List[int]].