Request

I'm very happy to see that carapace has bridges for some of the more popular python cli framework completion tools (click, argcomplete)

Every time one of these bridges is invoked, it executes the named python cli tool (ie watson in the example documentation) in a zsh completion context and lets the python cli framework's own cli completion logic produce completion candidates for carapace to use

The only problem i have with this approach is the heavy performance cost. every time completion is requested for one of these python tools, it executes the python cli app in a subprocess, which imports the whole tree of python modules depended on by that cli tool (most of which won't be needed), constructs the entire argument parser tree generated by the CLI framework (ie click or argparse/argcomplete) and then executes completion code in python to generate completions.

This is fine for small CLI apps, but quickly becomes VERY noticable to end users for larger CLI apps with nested trees of commands (sometimes >3 second delay between user hitting <TAB> and carapace returning values)

My idea to solve this is to pre-generate carapace Specs for each of my python cli tools, to be installed as part of the cli tool installation process. I'm writing a tool at the moment which takes a click.Group object, introspects it, and generates a carapace Spec yaml file

So far, this approach is proving very promising! every feature / capability expressible in a click CLI parser can be directly mapped to a carapace Spec option.

The only place where this falls short is in custom completion functions. Many python CLI frameworks allow you to specify python functions to execute during shell completions. These functions take in some object representing the current state of the parser (ie. which flags have been parsed so far, what positional arguments have been provided so far, their values, etc), as well as a string representing the current word being completed. These functions are expected to return a list of strings representing viable completion candidates.

here's a concrete example:

def completion_function(ctx, param, incomplete):
    return [k for k in os.environ if k.startswith(incomplete)]

@click.command()
@click.argument("name", shell_complete=completion_function)
def cli(name):
    click.echo(f"Name: {name}")
    click.echo(f"Value: {os.environ[name]}")

Proposed solution

I would love to see a custom macro that could be invoked something like this:

name: example-python
completion:
  positional:
    - ["$carapace.tools.python.CallFunction(my_package.my_module:completion_function)"]

which, when invoked, would do the following:

1. serialize all carapace variables (flags, args, everything available via https://carapace-sh.github.io/carapace-spec/carapace-spec/variables.html) into a JSON string
2. exec something like python -c 'from python_package.python_module import completion_function; print(completion_function("<carapace variables json string>"))
3. collect the results in much the same way you would with the "exec" macro (ie. ["$(echo -e 'a\nb\nc')"])

Anything else?

we can already very nearly get there with:

name: example-python
completion:
  positional:
    - ["$(python -c 'from python_package.python_module import completion_function; print(completion_function(\"${C_VALUE}\")))"]

but:

1. there's no effective way to pass in all carapace values
2. the nested brackets and double-nested double-quotes are gnarly, hard reason about, and all too easy to mess up.

Also, as a side note, i think this feature would be of value to anyone using carapace-bin, not just python CLI app authors.

With this feature, anyone writing custom carapace spec files could leverage this to write custom completion functions