Running Bash Completion Scripts with OSH

Since late August, I've written a lot of code to make OSH a better interactive shell.

Right now, I'm catching up on writing release notes for the three OSH releases I've made since then.

They turned out to be verbose, so this post is a short summary of the motivation and major changes. It also gives you a taste of what bash completion scripts look like.

Why Work on the Interactive Shell?

I explained in the FAQ that Oil treats Unix shell first as a programming language, and second as a text-based user interface.

What Can OSH Now Run?

The next post will list the many changes required to make this work. For example, array assignment like words[$j]=x is often used in completion scripts, but rarely used in "normal" shell scripts.

There are still many parts of completion that don't work. But getting to even this point was difficult because completion scripts are the hairiest of hairy shell scripts. Appendix A shows two examples.

Why Not Develop a Better Completion System?

After looking at that code, you might wonder why I don't develop a nicer system with the Oil language.

It sounds appealing on the surface, but after seeing how complex git completion is — ~3000 lines in bash, and ~7000 lines in zsh — I think it's better to reuse existing work.

Leaving aside git, the bash-completion project provides completion logic for dozens of common Unix commands, and it consists of tens of thousands of lines of code developed over nearly two decades!

zsh has also "boiled the ocean" in parallel over the last two decades. I don't see the use in recapitulating that with OSH.

(By the way: It would be nice to emulate the zsh completion system so we can take advantage of that work as well. If you're familiar with how it works, please leave a comment.)

This post gave you a general idea of what I've been doing, and why. The next post will be a release announcement with details.

Appendix A: Portions of Bash Completion Scripts

Completion scripts often look like their own dialect of bash. Arrays, associative arrays, extended globs, eval, and dynamic scope are common.

# This function performs file and directory completion. It's better than
# simply using 'compgen -f', because it honours spaces in filenames.
_filedir()
{
    local IFS=$'\n'

    _tilde "$cur" || return

    local -a toks
    local x reset

    reset=$(shopt -po noglob); set -o noglob
    toks=( $( compgen -d -- "$cur" ) )
    IFS=' '; $reset; IFS=$'\n'

    if [[ "$1" != -d ]]; then
        local quoted
        _quote_readline_by_ref "$cur" quoted

        # Munge xspec to contain uppercase version too
        # http://thread.gmane.org/gmane.comp.shells.bash.bugs/15294/focus=15306
        local xspec=${1:+"!*.@($1|${1^^})"}
        reset=$(shopt -po noglob); set -o noglob
        toks+=( $( compgen -f -X "$xspec" -- $quoted ) )
        IFS=' '; $reset; IFS=$'\n'

        # Try without filter if it failed to produce anything and configured to
        [[ -n ${COMP_FILEDIR_FALLBACK:-} && -n "$1" && ${#toks[@]} -lt 1 ]] && {
            reset=$(shopt -po noglob); set -o noglob
            toks+=( $( compgen -f -- $quoted ) )
            IFS=' '; $reset; IFS=$'\n'
        }
    fi

    if [[ ${#toks[@]} -ne 0 ]]; then
        # 2>/dev/null for direct invocation, e.g. in the _filedir unit test
        compopt -o filenames 2>/dev/null
        COMPREPLY+=( "${toks[@]}" )
    fi
} # _filedir()

__git_reassemble_comp_words_by_ref()
{
    local exclude i j first
    # Which word separators to exclude?
    exclude="${1//[^$COMP_WORDBREAKS]}"
    cword_=$COMP_CWORD
    if [ -z "$exclude" ]; then
        words_=("${COMP_WORDS[@]}")
        return
    fi
    # List of word completion separators has shrunk;
    # re-assemble words to complete.
    for ((i=0, j=0; i < ${#COMP_WORDS[@]}; i++, j++)); do
        # Append each nonempty word consisting of just
        # word separator characters to the current word.
        first=t
        while
            [ $i -gt 0 ] &&
            [ -n "${COMP_WORDS[$i]}" ] &&
            # word consists of excluded word separators
            [ "${COMP_WORDS[$i]//[^$exclude]}" = "${COMP_WORDS[$i]}" ]
        do
          # Attach to the previous token,
          # unless the previous token is the command name.
          if [ $j -ge 2 ] && [ -n "$first" ]; then
              ((j--))
          fi
          first=
          words_[$j]=${words_[j]}${COMP_WORDS[i]}
          if [ $i = $COMP_CWORD ]; then
              cword_=$j
          fi
          if (($i < ${#COMP_WORDS[@]} - 1)); then
              ((i++))
          else
              # Done.
              return
          fi
        done
        words_[$j]=${words_[j]}${COMP_WORDS[i]}
        if [ $i = $COMP_CWORD ]; then
            cword_=$j
        fi
    done
}