Why Sponsor Oil? | source | all docs for version 0.14.2 | all versions | oilshell.org
Warning: Work in progress! Leave feedback on Zulip or Github if you'd like this doc to be updated.
This doc describes every aspect of Oil briefly. It underlies the help
builtin, and is indexed by keywords.
Navigate it with the index of Oil help topics.
This section describes how to use the Oil binary.
bin/oil UsageUsage: oil [OPTION]... SCRIPT [ARG]...
oil [OPTION]... -c COMMAND [ARG]...
bin/oil is the same as bin/osh with a the oil:all option group set. So
bin/oil also accepts shell flags.
oil -c 'echo hi'
oil myscript.oil
echo 'echo hi' | oil
Usage: oil.ovm MAIN_NAME [ARG]...
MAIN_NAME [ARG]...
oil.ovm behaves like busybox. If it's invoked through a symlink, e.g. 'osh', then it behaves like that binary. Otherwise, the binary name can be passed as the first argument, e.g.:
oil.ovm osh -c 'echo hi'
Docstrings look like this:
proc deploy {
### Deploy the app
echo hi
}
The ... prefix starts a single command over multiple lines. It allows writing long commands without \ continuation lines, and the resulting limitations on where you can put comments.
Single command example:
... chromium-browser
# comment on its own line
--no-proxy-server
--incognito # comment to the right
;
Long pipelines and and-or chains:
... find .
# exclude tests
| grep -v '_test.py'
| xargs wc -l
| sort -n
;
... ls /
&& ls /bin
&& ls /lib
|| error "oops"
;
Procs are shell-like functions, but with named parameters, and without dynamic scope (TODO):
proc copy(src, dest) {
cp --verbose --verbose $src $dest
}
Compare with sh-func.
Command or expression:
if (x > 0) {
echo 'positive'
}
case $x {
# balanced parens around patterns
(*.py) echo 'Python' ;;
('README') echo 'README' ;; # consatnt words must be quoted
(*) echo 'Other' ;;
}
Command or expression:
var x = 5
while (x < 0) {
setvar x -= 1
}
Two forms for shell-style loops:
for name in *.py {
echo "$name"
}
for i, name in *.py {
echo "$i $name"
}
Two forms for expressions that evaluate to a List:
for item in (mylist) {
echo "$item"
}
for i, item in (mylist) {
echo "$i $item"
}
Three forms for expressions that evaluate to a Dict:
for key in (mydict) {
echo "$key"
}
for key, value in (mydict) {
echo "$key $value"
}
for i, key, value in (mydict) {
echo "$i $key $value"
}
The = keyword evaluates an expression and shows the result:
oil$ = 1 + 2*3
(Int) 7
It's meant to be used interactively. Think of it as an assignment with no variable on the left.
The _ keyword evaluates an expression and throws away the result:
var x = %(one two)
_ x.append('three')
Think of it as a shortcut for _ = expr (throwaway assignment).
Internal commands (procs and builtins) accept typed arguments.
json write (myobj)
TODO: Implement these
eval (myblock)
assert (x > 0)
Blocks can be passed to builtins (and procs eventually):
cd /tmp {
echo $PWD # prints /tmp
}
echo $PWD
Compare with sh-block.
Initializes a constant name to the Oil expression on the right.
const c = 'mystr' # equivalent to readonly c=mystr
const pat = / digit+ / # an eggex, with no shell equivalent
It's either a global constant or scoped to the current function.
Initializes a name to the Oil expression on the right.
var s = 'mystr' # equivalent to declare s=mystr
var pat = / digit+ / # an eggex, with no shell equivalent
It's either global or scoped to the current function.
At the top-level, setvar creates or mutates a variable.
Inside a proc, it mutates a local variable declared with var.
Creates or mutates a global variable.
Mutates a variable through a named reference. See examples in doc/variables.md.
Oil uses JavaScript-like spellings for these three "atoms":
true false null
Note that the empty string is a good "special" value in some cases. The null
value can't be interpolated into words.
var myint = 42
var myfloat = 3.14
var float2 = 1e100
#'a' #'_' \n \\ \u{3bc}
Oil strings appear in expression contexts, and look like shell strings:
var s = 'foo'
var double = "hello $world and $(hostname)"
However, strings with backslashes are forced to specify whether they're raw strings or C-style strings:
var s = 'line\n' # parse error: ambiguous
var s = $'line\n' # C-style string
var s = r'[a-z]\n' # raw strings are useful for regexes (not eggexes)
var unicode = 'mu = \u{3bc}'
Lists have a Python-like syntax:
var mylist = ['one', 'two', 3]
And a shell-like syntax:
var list2 = %(one two)
The shell-like syntax accepts the same syntax that a command can:
ls $mystr @ARGV *.py {foo,bar}@example.com
# Rather than executing ls, evaluate and store words
var cmd = %(ls $mystr @ARGV *.py {foo,bar}@example.com)
{name: 'value'}
var myblock = ^(echo $PWD)
var myexpr = ^[1 + 2*3]
var s = 's'
var concat1 = s ++ '_suffix'
var concat2 = "${s}_suffix" # similar
var c = %(one two)
var concat3 = c ++ %(three 4)
var concat4 = %( @c three 4 )
var mydict = {a: 1, b: 2}
var otherdict = {a: 10, c: 20}
var concat5 = mydict ++ otherdict
a == b # Python-like equality, no type conversion
3 ~== 3.0 # True, type conversion
3 ~== '3' # True, type conversion
3 ~== '3.0' # True, type conversion
not and or
Note that these are distinct from ! && ||.
+ - * / // % **
~ & | ^
Like Python:
display = 'yes' if len(s) else 'empty'
Like Python:
myarray[3]
mystr[3]
TODO: Does string indexing give you an integer back?
Like Python:
myarray[1 : -1]
mystr[1 : -1]
Like Python:
f(x, y)
~ !~ ~~ !~~
Not implemented.
Append a string to an array of strings:
var mylist = %(one two)
append :mylist three
This is a command-mode synonym for the expression:
_ mylist.append('three')
Pretty prints interpreter state. Some of these are implementation details, subject to change.
Examples:
pp proc # print all procs and their doc comments
var x = %(one two)
pp .cell x # print a cell, which is a location for a value
The .cell action starts with . to indicate that its format is unstable.
Run a block of code, stopping at the first error (i.e. errexit is enabled).
Set the _status variable to the exit status of the block, and returns 0.
try {
ls /nonexistent
ls | wc -l
diff <(sort left.txt) <(sort right.txt)
var x = 1 / 0
}
if (_status !== 0) {
echo 'error'
}
# Shortcut for a single command
try grep PATTERN FILE.txt
case $_status in
(0) echo 'found' ;;
(1) echo 'not found' ;;
(*) echo "error $_status" ;;
esac
Runs a command and requires the exit code to be 0 or 1.
if boolstatus egrep '[0-9]+' myfile { # may abort
echo 'found' # status 0 means found
} else {
echo 'not found' # status 1 means not found
}
It takes a block:
cd / {
echo $PWD
}
It takes a block:
shopt --unset errexit {
false
echo 'ok'
}
Execute a block with a global variable set.
shvar IFS=/ {
echo "ifs is $IFS"
}
echo "ifs restored to $IFS"
Save global registers like $? on a stack. It's useful for preventing plugins from interfering with user code. Example:
status_42 # returns 42 and sets $?
push-registers { # push a new frame
status_43 # top of stack changed here
echo done
} # stack popped
echo $? # 42, read from new top-of-stack
Current list of registers:
BASH_REMATCH aka _match()
$?
_status set by the try builtin
PIPESTATUS aka _pipeline_status
_process_sub_status
Runs a named proc with the given arguments. It's often useful as the only top level statement in a "task file":
proc p {
echo hi
}
runproc @ARGV
Like 'builtin' and 'command', it affects the lookup of the first word.
Registers a name in the global module dict. Returns 0 if it doesn't exist, or 1 if it does.
Use it like this in executable files:
module main || return 0
And like this in libraries:
module myfile.oil || return 0
Make declarations about the current file.
For files that contain embedded DSLs:
use dialect ninja # requires that _DIALECT is set to 'ninja'
An accepted declaration that tools can use, but isn't used by Oil:
use bin grep sed
Oil adds buffered, line-oriented I/O to shell's read.
read --line # default var is $_line
read --line --with-eol # keep the \n
read --line --qsn # decode QSN too
read --all # whole file including newline; var is $_all
read -0 # read until NUL, synonym for read -r -d ''
When --qsn is passed, the line is check for an opening single quote. If so, it's decoded as QSN. The line must have a closing single quote, and there can't be any non-whitespace characters after it.
write fixes problems with shell's echo builtin.
The default separator is a newline, and the default terminator is a newline.
Examples:
write -- ale bean # write two lines
write --qsn -- ale bean # QSN encode, guarantees two lines
write -n -- ale bean # synonym for --end '', like echo -n
write --sep '' --end '' -- a b # write 2 bytes
write --sep $'\t' --end $'\n' -- a b # TSV line
The preferred alternative to shell's &.
fork { sleep 1 }
wait -n
The preferred alternative to shell's (). Prefer cd with a block if possible.
forkwait {
not_mutated=zzz
}
echo $not_mutated
Write JSON:
var d = {name: 'bob', age: 42}
json write (d)
Read JSON into a variable:
var x = ''
json read :x < myfile.txt
TODO: describe
TODO: when
Option in this group disallow problematic or confusing shell constructs. The resulting script will still run in another shell.
shopt --set strict:all # turn on all options
shopt -p strict:all # print their current state
Options in this group enable Oil features that are less likely to break existing shell scripts.
For example, parse_at means that @myarray is now the operation to splice
an array. This will break scripts that expect @ to be literal, but you can
simply quote it like '@literal' to fix the problem.
shopt --set oil:upgrade # turn on all options
shopt -p oil:upgrade # print their current state
Enable the full Oil language. This includes everything in the oil:upgrade
group.
shopt --set oil:all # turn on all options
shopt -p oil:all # print their current state
Disallow break and continue at the top level, and disallow empty args like
return $empty.
Failed tilde expansions cause hard errors (like zsh) rather than silently
evaluating to ~ or ~bad.
TODO
When strict_nameref is set, undefined references produce fatal errors:
declare -n ref
echo $ref # fatal error, not empty string
ref=x # fatal error instead of decaying to non-reference
References that don't contain variables also produce hard errors:
declare -n ref='not a var'
echo $ref # fatal
ref=x # fatal
For compatibility, Oil will parse some constructs it doesn't execute, like:
return 0 2>&1 # redirect on control flow
When this option is disabled, that statement is a syntax error.
TODO
TODO
TODO
Allow the r prefix for raw strings in command mode:
echo r'\' # a single backslash
Since shell strings are already raw, this means that Oil just ignores the r prefix.
TODO
TODO
If a process that's part of a pipeline exits with status 141 when this is option is on, it's turned into status 0, which avoids failure.
SIGPIPE errors occur in cases like 'yes | head', and generally aren't useful.
TODO:
ARGVReplacement for "$@"
_DIALECTName of a dialect being evaluated.
_this_dirThe directory the current script resides in. This knows about 3 situations:
oshrc in an interactive shellosh myscript.shsource builtinIt's useful for "relative imports".
The version of Oil that is being run, e.g. 0.9.0.
_statusSet by the try builtin.
try ls /bad
if (_status !== 0) {
echo 'failed'
}
_pipeline_statusAlias for PIPESTATUS.
_process_sub_statusThe exit status of all the process subs in the last command.
_match()_start()_end()len(mystr) is its length in byteslen(myarray) is the number of elementslen(assocarray) is the number of pairsGiven an array of strings, returns a string.
var x = ['a', 'b', 'c']
$ echo $join(x)
abc
$ echo $join(x, ' ') # optional delimiter
a b c
index(A, item) is like the awk functionappend() is a more general version of the append builtinextend()keys()values()shvar_get()TODO
These functions give better syntax to existing shell constructs.
shquote() for printf %q and ${x@Q}lstrip() for ${x#prefix} and ${x##prefix}rstrip() for ${x%suffix} and ${x%%suffix}lstripglob() and rstripglob() for slow, legacy globupper() for ${x^^}lower() for ${x,,}strftime(): hidden in printfTODO
TODO