Why Sponsor Oils? | blog | oilshell.org

Proposed Changes to Oil's Syntax

2020-11-09

This was an appendix to the last post, but I split it up because it deserves feedback on its own.

These features are "cut" for now, in that I'll work on oil-native and garbage collection before they're implemented. But I put hooks in for them to make sure we don't get boxed into a corner.

While doing this, I noticed that we can likely implement them without a new lexer mode. This is good because new lexer modes make the language more subtle, and harder for humans to understand.

Table of Contents
Four Improvements
Prettier Multi-Line Pipelines With the ... Prefix
Multi-line Strings, Including Here Docs
parse_amp for Nicer Redirect Syntax
Inline Eggex $/d+/
Next
Appendix: Deferred Even Further

Four Improvements

Here's a brief description of each proposed change, and the problems that they solve.

Prettier Multi-Line Pipelines With the ... Prefix

I want to avoid this problem:

cat file.txt \    
  | sort \          # I can't put a comment here
  | cut -f 1 \
  # And I can't put one here
  | grep foo

Maybe with something like this:

... cat file.txt    # Implicit continuation until 2 newlines
  | sort            # Comment allowed
  | cut -f 1 
  # This line has a comment
  | grep foo

Multi-line Strings, Including Here Docs

We want here docs like this:

cat << """
name = $myvar   # variable expansion
"""

cat << '''
price is $3.99  # no expansion
'''

Not like this:

cat << EOF
name = $myvar   # variable expansion
EOF

cat << 'EOF'
price is $3.99  # no expansion
EOF

The quoted 'EOF' rule is confusing. Let's just preserve the meaning of single and double quotes.

We should also have multiline strings like this:

echo '''
price is $3.99
line two
'''

They will also strip leading whitespace in a more natural way than shell's obscure <<- operator.

proc p {
  echo '''
  leading space stripped
  price is $3.99
  '''
}

These are like tripled-quoted string literals in Julia. Feedback is welcome!

parse_amp for Nicer Redirect Syntax

The current shell redirect syntax bugs me.

https://github.com/oilshell/oil/issues/841

Now that & is replaced by the fork builtin, I think we can do something like this:

$ echo 'log message' > &2   # print to stderr
$ ls /foo &2 > /dev/null    # shut up stderr

And

$ echo 'to named descriptor' > &left

And maybe:

$ ls /foo &1 &2 > /dev/null    # redirect BOTH
$ ls /foo &1 &2 | tee out.txt  # pipe BOTH

We may also need an fopen builtin to open two streams at once. I need help with these things!

Inline Eggex $/d+/

As mentioned in the first post in this series, shopt --unset parse_dollar enables $/ to be "taken over". Right now it doesn't mean anything, so we can probably use it for "inline eggex".

For example, here is the current usage of eggex:

$ var pat = / d+ /
$ egrep $pat foo.txt  # converted to ERE on interpolation

We could allow this syntax:

$ egrep $/ d+ / foo.txt

Which expands to:

$ egrep '[[:digit:]]+' foo.txt

See the Egg Expressions doc for rationale.

Next

There are two more posts in this series. Hopefully I'll get to them in a timely manner.

  1. Changes to Oil's Semantics. I overhauled shell options, variable scope, and proc.
  2. The Shell Programmer's Guide to errexit. About error handling in shell and Oil.

Let me know what you think of these features! I'm saving space for them in the language.

Appendix: Deferred Even Further

I want a statically-parsed printf:

printf -v mystr '%.2f' "$average"   # awkward shell style
var mystr = ${average %.2f}  # Oil style

And maybe "formatters" for proper escaping:

echo "<p>${user_input|html}</p>"

These should fit compatibly into the ${} language. Almost everything else inside ${} should be deprecated! For example, ${#x} can be written $len(x).

If we implement all of this, I think Oil's syntax would be almost "done"!