Recognize DeclareSynonym, with cascade of related changes

[gwhitney]

This change began with recognizing DeclareSynonym and cascaded to involve a
number of related changes. First, in order for DeclareSynonym to be useful,
there must be some way to specify the manual item type for the identifier
being declared. Hence this change also implements a @ItemType command to allow
that. (Adding @ItemType also achieves the main recommendaton in issue #74.)

In documenting these two changes, it seemed best to have their documentation
in gap/Parser.gi next to their implementation, on the general principle that
documentation should be as close to the relevant code as possible. Further, in
an effort at dogfooding, it seemed best for that documentation to be in
AutoDoc format. This turned into a change in which doc/Comments.xml became
doc/Comments.autodoc and became much smaller, as most of the documentation
moved to gap/Parser.gi and gap/Markdown.gi.

Being able to use Autodoc to document most of Autodoc then entailed a few
other changes. First, Markdown-like syntax with backticks for code expressions
that would suppress the AutoDoc behavior of the quoted items was
essential. Second, it was useful to add an @index command for generating the
index entries. And third, reassembling Chapter 2 of the manual from portions
in several files necessitated allowing sections and subsections inside a chunk
to move with that chunk, and be inserted into the chapter or section at the
point at which the chunk was inserted.

(An implementation note on this last change. It was greatly facilitated by
merging the current_item ambient variable with the context_stack. The
current_item was in essence functioning as the top of the stack anyway. Having
the top of the stack in one variable and the rest of the stack in another made
some of the code a bit confusing. This change touched a large fraction of code
in the Parser, but in the end improved the locality of reference in that code
considerably, and I think made it clearer what is going on during parsing.)

In the course of these changes, it was trivial to add the ability to have code
blocks which add at the current location, with no @insertcode necessary. So
that change is included here as well.

Finally, with so many changes, extensive testing was clearly critical. So this
commit also beefs up the worksheet.tst, and adds a new test dogfood.tst which
consists of generating the AutoDoc manual and verifying that it matches the
expected contents of the manual. I also switched to verifying the produced XML
rather than the html as the XML determines the html anyway, and this provides
a cleaner point for testing, separated from any changes in GAPDoc.

I do understand that this turned into more of a complex of changes than would
ordinarily be packed into a single commit. But this portmanteau seemed a
fairly natural stable point in which the motivating changes
(DeclareSynonym/@ItemType) are made, documented, and tested in a cohesive way.

I would be happy to do the work of separating this out into more than one
successive change to AutoDoc if that would help advance these changes. Please
just let me know what pieces you'd like it carved up into, and in what order.

Thanks very much for your consideration.

Resolves: #60, #74, #174, #175, #176, #177, #178.
It also makes further progress on #48, in adding the backtick code quotes, and
on the testing issues #57 and #152. It also touches on Issue #112, in that to
use makedoc.g in testing, I needed to add a backdoor to prevent it from
QUITting. Note also that the motivation in the original post for Issue #144
was the need to document DeclareSynonym.

This change is on top of all of my recent pull requests, except for #172, as they were also necessary in order to successfully convert Comments.xml to Comments.autodoc and have the test harness.

[fingolfin]

This PR is really to full with too many changes for my taste, making it very difficult to review. It also contains various other PRs that should be reviewed and merged first; and finally, of course it now conflicts with master, as we merged the testing code, but also heavily modified it.

I suggest to first get all your other PRs in. Then discuss how to proceed with this one. It certainly would help if it was done in steps, with each change in a separate commit (which could all be in a single PR). Like this (order may be different, and this list is not complete, just an idea)

• one commit which adds support backticks in AutoDoc markup
• one commit which adds the AutodocFilesOption comment to the manual
• one commit which moves parts of doc/Comments.xml into AutoDoc comments inside source files
• a separate commit which converts the reminder of doc/Comments.xml into doc/Comments.md (splitting this in two commits makes it much easier to verify that nothing was lost
• ...

[fingolfin]

Insert an empty line before this? Or is this intentionally joined to the preceding comment?

[fingolfin]

Please don't use the horrendous GAP "namespacing" feature. Instead, add an explicit AUTODOC_ prefix to the function name (and consider using BindGlobal to assign it, to avoid user's from accidentally messing with it).

[gwhitney]

I am confused. GAP does have a namespacing problem, the existence of all of these AUTODOC_ functions illustrates it perfectly, they shouldn't be in a global namespace, and the "trailing @" convention is the "official" mechanism that GAP has designed for dealing with the issue. I recognize that it doesn't actually create different namespaces, but at least it provides a systematic convention for dealing with the issue in the meantime (rather than this package using AUTODOC_ prefixes, and that package creating a single global record with with all the internal functions as entries, and that other package doing something else), and if/when GAP actually got proper namespaces, you could imagine it backward-compatibly supporting the "trailing @" via a real namespace system so that existing code using that would still work but with the actual namespaces, whereas all of the individual workarounds will simply have to be rewritten to take advantage of such a new feature. So I'd love to learn what's so horrendous about the "trailing @"? Any comments (or pointers to other posts on the topic) would be extremely welcome, especially as in writing the racks package I have adopted it whenever I need a helper function in one of my .gi files that should not be exposed outside the package. If there's a real problem with it I would rather change directions on that sooner rather than later, as it will only get harder to change the more the code develops. Thanks!

[fingolfin]

"... creates the section ..." what is "the" section this is referring to? Perhaps "a" section is meant?

Also, it doesn't always create a section, does it? So perhaps: "This method returns a section object for [INSERT MEANINGFUL TEXT, like "... for the given " -- but note that this example may be nonsense, I am not anymore familiar with the code here....]. If the section already exists, this is returned; otherwise, a new section node is created and returned. In the latter case, it is added as a child..."

[fingolfin]

Same comment as above applies here.

[gwhitney]

Yes, I think as you can tell from my commentary, I expected that this PR incorporated too many items to be accepted as-is/all at once. Mostly I wanted to lay out a direction that you might find worthwhile, and it was hard to elucidate each item without showing the whole collection, and plus I really did need advice on the best way to divide this into properly atomic PRs, and in what order they should be submitted. I completely agree with your proposal that the rest of the PRs should be resolved and then I should revisit this one, breaking it up along the lines you just proposed, and I will do exactly that. I just had one question about the namespacing in the comments above, which I'd be glad to hear your views on whenever convenient. Other than that, feel free to close this PR, assured that I will return to the items contained in it when the others are resolved.

[fingolfin]

By now I updated all your other PRs and made them OK-to-merge (at least from my point of view). But this one is difficult. It has a ton of conflicts, and it does far too much in one commit. I don't agree with the justification for that either; it seems rather clear to me that a lot of these changes could be moved into separate commits (and even separate PRs). E.g.

• add test suite (this code has already been merge)
• add @ItemType with a test
• convert doc/Comments.xml into doc/Comments.autodoc
• move parts of doc/Comments.autodoc into source code
• merge current_item ambient with the context_stack
• the actual fix for DeclareSynonym
• ability to have code blocks which are inserted at the place they appear
• ... perhaps more, I can't tell, the final commit does so many things at once ...

I tried cherry-picking just the last commit, but it seems rather hopeless at this point. So (unless you re-appear), I might try to take inspiration from this PR to re-implement a few of the above points in separate PRs.