Why Sponsor Oils? | blog | oilshell.org
The last release announcement said that I want to rename:
Yes, using "Oil" for two different things was confusing.
But some readers were confused about not just the names, but the project itself. This isn't too surprising, since some things have changed over the years, and I haven't blogged as much lately. And it's an unusual project.
The new names are supposed to help people understand what the project actually is, but there will be more confusion in the short term. I hope the short answers and links below will help.
There are multiple related goals, which makes the project big. We have a single shell interpreter that runs in two modes:
Use OSH with bin/osh. It runs POSIX shell and bash scripts, almost always without modification.
It's steadily matured for over 5 years, but only recently became fast, with the C++ translation. It's now pretty clear that it will be done, but we need feedback and help to "finish" it.
I think a new implementation of a compatible shell is worthwhile on its own, without a new language. It has more precise error messages, detailed tracing, interactive completion, and more.
Use YSH with bin/ysh. It's a new shell language designed from scratch.
It's for Python and JavaScript users who avoid shell. It has JSON-like data structures, and more.
YSH has existed for over 3 years as the "Oil language", and I'm planning some big updates this year. It's time to write more about it.
You can optionally upgrade from OSH to YSH.
This will take manual work, but it should be worth it for some users.
(More detail at OSH versus Oil, which needs to be renamed, along with many other wiki pages.)
There are still a gazillion shell scripts out there. They live at the foundations of Linux distros, in scientific data pipelines, in your git repositories, and in many other places.
They do things you likely care about, even if don't use them directly.
New projects can ignore OSH, but old scripts may want an upgrade path.
By the way, Oils is the only project with a proposal for evolving shell. Shell was already old and regretted 30 years ago, so we have to do something different if we want a different result.
YSH is designed to be legacy-free, so you can use it without knowing anything about OSH.
But it's still in progress, and there will be breaking changes. On the other hand, OSH is rapidly stabilizing. A major reason for the renaming is to delineate what's stable and what's not.
If you don't care about compatibility, or an upgrade path, you can choose one of the 30 or so shells here:
In contrast to some of those projects, we also aim at the system level — including in containers in the cloud. For example, you might use fish as your personal shell, but your system shell is likely bash, zsh, or a busybox shell.
We extend the bash shopt mechanism, which is similar to upgrade mechanisms in Python (from __future__) and Perl (use strict).
There are more than 20 parsing and runtime options that distinguish OSH and YSH.
But we have bin/osh and bin/ysh so you don't have to remember all of them. There's also a level in between to help you upgrade, giving three levels:
bin/oshbin/osh with shopt --set ysh:upgradebin/osh with shopt --set ysh:all, which is equivalent to bin/yshConversely, the options are the only thing that distinguish OSH and YSH. You can also --unset the ysh:all options in while running bin/ysh, and get the equivalent of bin/osh!
Otherwise they're the same shell, though the idioms make them look very different.
See Global Shell Options for more details.
This busybox-like structure may help some readers:
/usr/bin/
oils-for-unix # executable file, like busybox
osh -> oils-for-unix # symlink
ysh -> oils-for-unix # symlink
I often compare the project to Clang, which supports C, C++, and Objective C all under one roof.
You can use C without knowing C++, and vice versa. Similarly, you can use OSH without knowing YSH, and vice versa.
It's more confusing for implementers, but I don't think it's more confusing for users.
Here are more languages with a smooth upgrade path from their predecessor:
For example, TypeScript arguably makes the JavaScript ecosystem more confusing, because there are more things to learn. But most people think it adds enough to justify its existence.
You also don't have to use TypeScript if you don't want to!
YSH is also designed to be familiar to Python and JavaScript users. And knowledge of say grep, sed, /dev, and /proc transfers directly from OSH to YSH. Again, it's really a single shell with two different modes, and some different idioms.
This could be because I tend to write about what's stable, and YSH has changed a lot. Even the name is changing!
If it feels like YSH has been "under wraps" for awhile, it's because we're going through many iterations to achieve a holistic and coherent design.
We try things in in YSH, and if they don't stand up to experience, we break them. (On the other hand, OSH isn't easy to break because we have exhaustive tests.)
The analogy with C++ only goes so far: we want to avoid the problem where you add features for years, and they have unexpected interactions and inconsistent syntax.
I think YSH is cleaner than C++ or TypeScript because:
shopt allows us to be less constrained by legacyExperimentation is a major reason that Oil is implemented with Python-based DSLs. It would have been impossible for me to design a language that's both compatible and clean-slate using a low-level implementation.
The upgrade path is a big, tricky, and constrained puzzle.
Some of the wrong turns may have also caused confusion.
Remember that we invite you to ask questions on Zulip. YSH will be missing documentation for quite awhile.
Please see the answers in Backlog: Explaining the Oil Project (2021).
The project was definitely harder than I expected, but it's been fun to do new things, and do them in a new way.
And I think it's still needed. For example, on Github in 2022,
github.com)I'd say this is because Unix now "runs the world": there's been an explosion of Linux systems in the cloud, and in embedded devices. Also, Apple OS X is a kind of Unix, and Windows emulates Linux.
So every day, more people are learning shell. And they're encountering the same flaws we noticed in 2016, and in 1993.
Our prototype of YSH shows that those flaws can be fixed. But we need more help to execute it: Oils Roadmap: What We Can Use Help With.
Every release announcement asks people to try Oils and test it. So you tell me! :-)
Right now, I'd say it's most attractive to users who do shell programming, and advanced users who say build Linux distros.
But I've also heard from users who just like it as their every day shell.
I should also add that I want many people to get value out of Oils, but I don't think of it as "product" to be "adopted". It's an open source project that you can adjust to your needs by contributing.
No, programming languages don't work that way :-) I've noticed that some readers have pinned unrealistic hopes on the project, so I want to dispel this myth.
Existing shells will be with us for decades, but some people and systems will start to use different shells, hopefully including OSH and YSH. The gradual upgrade path is intended to make that attractive.
In other words, our arguably complex implementation aims to solve a social "coordination problem". What shell you use isn't entirely an individual choice; it's partly a group choice.
For example, I didn't really choose to learn bash. I learned it because it was attached to Linux, and a huge ecosystem of tools, which let me get a lot of work done.
Returning to the analogies:
OSH is the most bash-compatible shell, by a mile. We don't implement every last quirk and bug, but it still runs unmodified shell scripts.
It's written in high-level, Python-based DSLs, not low level C. (Related: FAQ: Why Not Write Oil in X? on the wiki)
It's now translated to fast native code, with garbage collection. This means the interpreter is memory safe, unlike essentially every other shell.
We have a prototype for YSH / Oil that you can try. People have written real code in it and given valuable feedback. See A Tour of the Oil Language (also to be renamed).
We showed that there can be an upgrade path with shopt, as described above. Most people thought this was impossible 5 years ago!
We finished our first grant from NLnet, paying 50K euros to 3 people. We now have a second grant of 50K euros. If you'd like to be paid to contribute to an open source shell, try Contributing a 1 or 2 line change to start!
I just dumped a big list of Zulip threads in the last post: Oils Roadmap: What We Need Help With.
Ask me questions about any of those things!
In addition to our Contributing wiki, see the Oil README.md.
In one sense, the shell is "just" a Python program on Github, so it's easy to contribute to. If you can write Python and write good tests for it, you can work on Oils.
It's OK to send a rough prototype as a pull request, and actually encouraged! Features based on concrete needs are always interesting.
Useful computer science knowledge:
I'll explain why we're renaming the project to Oils, and the new shell to YSH!
Check out other posts tagged #FAQ, including Why Create a New Unix Shell? (2021).