Official "help" as top-level schema like "manifest"?

[christianbundy]

Hey @dominictarr!

I noticed that muxrpcli is calling help() a bunch of times to collect all of the usage info for each command, and I'm wondering whether you'd be open to a single top-level help() method sort of like how we do manifest().

Would you be open to something like this? I suppose I'm also wondering which usage definition schema you're using -- I'm using yargs and would love to be able to just plug in the usage info.

[dominictarr]

we already have that mostly:

no args produce help output:

createLogStream|get|publish|add|status|progress|version|whoami|createHistoryStream
plugins # manage ssb plugins
friends # track what feeds are following or blocking each other
blobs # retrive, store, and share blobs
query # query ssb database with map-filter-reduce queries
links2 # query messages that link to other messages or feeds
ooo # gossip ssb messages without replicating the entire feed, aka "Out of Order"
search # full text search within ssb messages
identities # manage multiple ssb feed identities from a single ssb instance
deviceAddress # advertise an publically reachable address for an ssb device
peerInvites # invite peers to your ssb network

ssb-server help returns raw help data:

{
  "description": "append only-log database for secure-scuttlebutt",
  "commands": {
    "createLogStream": {
      "type": "source",
      "description": "stream of all locally stored messages, in order received",
      "args": {
        "gte": {
          "type": "timestamp",
          "description": "only results greater than or equal to timestamp",
          "optional": true
        },
        "lte": {
          "type": "timestamp",
          "description": "only results less than or equal to timestamp",
          "optional": true
        },
        "gt": {
          "type": "timestamp",
          "description": "only results greater than  timestamp",
          "optional": true
        },
        "lt": {
          "type": "timestamp",
          "description": "only results less than timestamp",
          "optional": true
        },
        "reverse": {
          "type": "boolean",
          "test": "boolean",
          "description": "output is reversed",
          "optional": true
        },
        "live": {
          "type": "boolean",
          "test": "boolean",
          "description": "include live results",
          "optional": true
        },
        "old": {
          "type": "boolean",
          "test": "boolean",
          "description": "include old results",
          "optional": true
        },
        "keys": {
          "type": "boolean",
          "description": "include keys",
          "optional": true
        },
        "values": {
          "type": "boolean",
          "description": "include values",
          "optional": true
        }
      }
    },
    "get": {
      "type": "async",
      "description": "retrive a locally stored message",
      "args": {
        "id": {
          "type": "string",
          "test": {}
        },
        "private": {
          "type": "boolean",
          "description": "decrypt private messages, defaults to false"
        },
        "meta": {
          "type": "boolean",
          "description": "include key,value,timestamp defaults to false"
        }
      }
    },
    "publish": {
      "type": "async",
      "description": "publish a message",
      "args": {}
    },
    "add": {
      "type": "async",
      "description": "append a valid message",
      "args": {}
    },
    "status": {
      "type": "sync",
      "description": "show internal system statuses",
      "args": {}
    },
    "progress": {
      "type": "sync",
      "description": "show internal progress",
      "args": {}
    },
    "version": {
      "type": "sync",
      "description": "show version numbers",
      "args": {}
    },
    "whoami": {
      "type": "sync",
      "description": "print main identity",
      "args": {}
    },
    "createHistoryStream": {
      "type": "source",
      "description": "output messages from a feed in order",
      "args": {
        "id": {
          "type": "FeedId",
          "optional": false,
          "description": "a ssb feed identity"
        },
        "seq": {
          "type": "SequenceNumber",
          "optional": false,
          "description": "sequence number to stream from"
        },
        "limit": {
          "type": "number",
          "description": "max number of messages to output",
          "optional": true
        },
        "keys": {
          "type": "boolean",
          "description": "include keys",
          "optional": true
        },
        "values": {
          "type": "boolean",
          "description": "include values",
          "optional": true
        }
      }
    }
  }
}

see https://github.com/ssbc/muxrpc-usage

[christianbundy]

My understanding is that the top-level help() is returning usage info for ssb-db, but if you have plugins with their own namespace then you have to call:

• help()
• gossip.help()
• blobs.help()
• ebt.help()

It's very convenient that we don't have to do this for manifest(), and I'd love the same behavior for help() if that's the official usage guide (though I'd personally prefer usage() because I'd expect that ssb-server gossip help returns your CLI usage text rather than muxrpc usage text (yargs/yargs#1490)).

[dominictarr]

@christianbundy the top level help (a part of muxrpc-usage) aggregates the lower level X.help output.

ssb-server gossip help prints help output, but only because gossip is not actually a command and gossip.help doesn't do anything. instead of doing ssb-server {command} help I would do ssb-server help {command} as this order is consistent with both git and npm

[dominictarr]

also, cli usage text and muxrpc usage text are the same because everything is directly analogous. it's only different if you call a sbot method from inside the same process, direct javascript, not through muxrpc.

[christianbundy]

the top level help (a part of muxrpc-usage) aggregates the lower level X.help output.

Maybe I've misunderstood, but it seems like you have to call your chosen help() manually. The way I'm doing it is here.

In other words, I really like that you can call manifest() once and get the entire manifest with all of the nested groups. Could we do something similar for help() or usage()? I'd love to avoid making a bunch of individual help() calls to describe the entire API.

[dominictarr]

well, each plugin needs to be responsible for it's own help, because the plugins can change independently. that's how the manifest works too, but it's aggregated by secret-stack. I think that secret-stack could do that with help too, but it's complicated a little bit by how the db stuff has it's own help, I think you could make something that works around that though

[christianbundy]

Right. I was thinking that maybe instead of using the help method, we could make a top-level property called "help" (or "usage", which I think is more specific), which is aggregated in exactly the same way as the manifest. I've moved this issue to the secret-stack issue if you think that would be the right place to do it.

[dominictarr]

methods seems easier than a property because muxrpc doesn't give you a way to expose properties, only methods, so a remote instance can't see the properties... we also have things like out-of-process plugins, that can expose a method but not a property.

[christianbundy]

Yes, I'm proposing that we handle it exactly like manifest, where secret-stack aggregates them. It should, of course, be exposed over muxrpc as a method, but I'm proposing that the usage is added as a top-level secret-stack property exactly like how we're currently handling manifest.

[dominictarr]

To be honest, it seems like you are just proposing to change this for the sake of changing it.
It's also a breaking change, so would require coordinated changes to a bunch of modules (11! ssbc/ssb-server#652)

but it doesn't make any measurable improvement, it's not faster, or more scalable, it's just a different way of doing it. The effort of a breaking change is greater than the effort of a new feature.
The benefit of a breaking change may make that worthwhile, if the change enables new features or other improvements, but the benefit to this change seems quite low...

why not put that effort into something that will make a bigger difference?

for this feature, why not just add a help method on the top level secret-stack that aggregates all the others, that's just a simple feature addition.

as for calling it "usage" vs "help" I can think of many unix commands that have a --help option or help command but I don't know any that call that command usage. Certainly, several use the word "usage" when you do something wrong but if I wanted to figure out how a command works, I'd ask it for help before usage.

[christianbundy]

To be honest, it seems like you are just proposing to change this for the sake of changing it.

😕

I appreciate the honesty, but this was a bummer to read.

I feel like I've been repeating myself a bunch, but the problem I'd like to solve is that aggregating the usage data is difficult, and it seems very similar to the [easy + solved] manifest data aggregation.

for this feature, why not just add a help method on the top level secret-stack that aggregates all the others, that's just a simple feature addition.

That would be fine and I think that backward-compatibility is important, but it feels weird that secret-stack plugins have a convention to pass a JSON payload that's coupled with the manifest property, except you can't pass it as a top-level property and it has to be a function. I'm not proposing that we break backward-compatibility, I'm proposing that we try to make the usage text easier to use.

I'd be curious to hear if there are others who have implemented muxrpc-usage support and what their experiences have been like. Is there anyone else we should reach out to?

why not put that effort into something that will make a bigger difference?

As always, I'm concerned about the accessibility of the stack and I'm raising a red flag where I've found a hazard that beginners are likely to stumble over. I appreciate your work on muxrpc-usage and it's a great improvement over the raw manifest, but it doesn't feel like a first-class citizen in secret-stack. To be more explicit: I like muxrpc-usage so much that I want to make sure it's super easy for secret-stack plugin authors and interface developers.

for this feature, why not just add a help method on the top level secret-stack that aggregates all the others, that's just a simple feature addition.

Yes, that's exactly what I'm proposing in the first comment of this issue. To ensure there are no miscommunications, there are three changes that are separate but related:

• Secret-Stack method: secret-stack should implement help() that calls all deeper help() and aggregates so that it doesn't need to be implemented by every tool that uses muxrpc-usage
• Secret-Stack property: secret-stack should implement this help as a property that plugin authors can use so that they don't have to inject the function help () { return json } thing in their plugin's init() function.
• Rename to usage: I'm much less passionate about this one, but I think that a function that returns usage info should be called usage rather than help, since help creates a naming conflict with the CLI tool(s)

as for calling it "usage" vs "help" I can think of many unix commands that have a --help option or help command but I don't know any that call that command usage. Certainly, several use the word "usage" when you do something wrong but if I wanted to figure out how a command works, I'd ask it for help before usage.

If you're writing something like muxrpcli from scratch this makes sense, but libraries like optimist/yargs/commander all implement their own help so running ssb help is ambiguous between:

• ssb --help (formatted usage info from the CLI tool)
• ssb help (JSON usage info from muxrpc)

This isn't critical, but it's a pain point for anyone trying to use a CLI library with muxrpc-usage.

[dominictarr]

I'm sorry, It was a bummer to write also. It's more important to be honest about hard feelings though.

Okay you say it's hard to aggregate the help... but the best argument that it's easy was just to implement it
https://gist.github.com/dominictarr/28f65288b5991bb9f109150930a5f6a4
I wasn't sure where to put the module because the way I've done it in muxrpcli I don't actually need the whole thing at one time. One of the awkward things about the usage before, was that it dumped everything, a large markdown file, and that much content wasn't useful all at once.

Oh: this just occured to me: help is usually a command that you use to get information about something. usage is output that happens automatically when you use a command incorrectly.
It's usually very terse.

[dominictarr]

Btw, on the property: this is awkward on process plugins...

They expose the manifest as a json, but I didn't want a straight json for the help data, because there is a lot of repetition (because it's a good thing for args to be consistent) and that would be awkward to maintain, and a build step is annoying too.

I'm not sure if anyone has actually using process plugins yet, but we added them to support people working in different languages, so I needed to support that.

[stale]

Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?