WIP: rebased and modified PR #179

doc/Comments.autodoc

@@ -0,0 +1,173 @@
+@Chapter Comments
+@ChapterTitle &AutoDoc; documentation comments
+
+You can document declarations of global functions and variables, operations,
+attributes etc. by inserting **&AutoDoc; comments** into your sources
+before these declarations.
+An &AutoDoc; comment always starts with `#!`.
+This is also the smallest possible &AutoDoc; command.
+If you want your declaration documented, just write
+`#!` at the line before the documentation. For example:
+
+@BeginLogSession
+#!
+DeclareOperation( "AnOperation",
+                  [ IsList ] );
+@EndLogSession
+
+This will produce a manual entry for the operation `AnOperation`.
+<P/>
+
+Inside of &AutoDoc; comments, **&AutoDoc; commands**
+starting with `@` can be used to control the output &AutoDoc; produces.
+
+
+@Section Declarations
+@SectionTitle Documenting declarations
+
+In the bare form above, the manual entry for `AnOperation` will not
+contain much more than the name of the operation. In order to change
+this, there are several commands you can put into the &AutoDoc; comment
+before the declaration. Currently, the following commands are provided:
+
+@InsertChunk documenting_declaration_commands
+&nbsp;<Br/>&nbsp;<Br/>
+As an example, a full &AutoDoc; comment with all the most common options could
+look like this:
+
+@BeginLogSession
+#! @Description
+#! Computes the list of lists of degrees of ordinary characters
+#! associated to the <A>p</A>-blocks of the group <A>G</A>
+#! with <A>p</A>-modular character table <A>modtbl</A>
+#! and underlying ordinary character table <A>ordtbl</A>.
+#! @Returns a list
+#! @Arguments modtbl
+#! @Group CharacterDegreesOfBlocks
+#! @FunctionLabel chardegblocks
+#! @ChapterInfo Blocks, Attributes
+DeclareAttribute( "CharacterDegreesOfBlocks",
+        IsBrauerTable );
+@EndLogSession
+
+@Section Recognized declarations
+@InsertChunk recognized_declarators
+
+Note that in the particular case of `DeclareSynonym`, &AutoDoc; has no way to
+infer the argument list, so when documenting such a declaration, you must use
+the `@Arguments` command to supply that information.
+
+@Section Others
+@SectionTitle Other documentation comments
+
+There are also some commands which can be used in &AutoDoc; comments
+that are not associated to any one particular declaration. This is useful for
+additional text in your documentation, examples, mathematical chapters, etc..
+
+@InsertChunk other_commands
+@EndSection
+
+@InsertChunk titlepage_commands
+
+@Section PlainText
+@SectionLabel Plain
+@SectionTitle Plain text files
+
+AutoDoc plain text files work exactly like AutoDoc comments, except that the
+`#!` is unnecessary at the beginning of a line which should be documented.
+Files that have the suffix .autodoc will automatically regarded as plain text files,
+while the commands `@AutoDocPlainText` and `@EndAutoDocPlainText`
+(<Ref Subsect="Subsection_PlainCommands"/>) mark regions in other files which
+should be regarded as plain text &AutoDoc; parts. All commands can be used
+in a plain text section of a file, without the leading `#!`.
+
+@Section Groups
+@SectionLabel Groups
+@SectionTitle Grouping
+
+In &GAPDoc;, it is possible to make groups of ManItems, i.e., when documenting
+a function, operation, etc., it is possible to group them into suitable chunks.
+This can be particularly useful if there are several definitions of an operation
+with several different argument types, all doing more or less the same to the arguments.
+Then their manual items can be grouped, sharing the same description and return type information.
+Note that it is currently not possible to give a header to the Group in the manual,
+but the generated ManItem heading of the first entry will be used.
+
+Note that group names are globally unique throughout the whole manual.
+That is, groups with the same name are in fact merged into a single group, even if they
+were declared in different source files.
+Thus you can have multiple `@BeginGroup/@EndGroup` pairs using the
+same group name, in different places, and these all will refer to the same group.
+
+Moreover, this means that you can add items to a group via the `@Group` command
+in the &AutoDoc; comment of an arbitrary declaration, at any time.
+
+The following code
+@BeginLogSession
+#! @BeginGroup Group1
+
+#! @Description
+#!  First sentence.
+DeclareOperation( "FirstOperation", [ IsInt ] );
+
+#! @Description
+#!  Second sentence.
+DeclareOperation( "SecondOperation", [ IsInt, IsGroup ] );
+
+#! @EndGroup
+
+## .. Stuff ..
+
+#! @Description
+#!  Third sentence.
+#! @Group Group1
+KeyDependentOperation( "ThirdOperation", IsGroup, IsInt, "prime );
+@EndLogSession
+
+produces the following:
+
+<ManSection Label="Group1">
+  <Oper Arg="arg" Name="FirstOperation" Label="for IsInt"/>
+  <Oper Arg="arg1,arg2" Name="SecondOperation" Label="for IsInt, IsGroup"/>
+  <Oper Arg="arg1,arg2" Name="ThirdOperation" Label="for IsGroup, IsInt"/>
+ <Returns></Returns>
+ <Description>
+ First sentence.
+ Second sentence.
+ Third sentence.
+ </Description>
+</ManSection>
+
+@Section Level
+@SectionLabel Level
+
+Levels can be set to not write certain parts in the manual by default.
+Every entry has by default the level 0. The command `@Level` can
+be used to set the level of the following part to a higher level, for
+example 1, and prevent it from being printed to the manual by default.
+However, if one sets the level to a higher value in the autodoc option of
+<C>AutoDoc()</C>, the parts will be included in the manual at the specific
+place.
+
+@BeginLogSession
+#! This text will be printed to the manual.
+#! @Level 1
+#! This text will be printed to the manual if created with level 1 or higher.
+#! @Level 2
+#! This text will be printed to the manual if created with level 2 or higher.
+#! @ResetLevel
+#! This text will be printed to the manual.
+@EndLogSession
+
+@Section MarkdownExtension
+@SectionTitle Markdown-like formatting of text in &AutoDoc;
+
+&AutoDoc; has some convenient ways to insert special format into text, like
+math formulas and lists. The syntax for them are inspired by Markdown and LaTeX,
+but do not follow them strictly. Neither are all features of the Markdown
+language supported. The following subsections describe the Markdown-like
+conventions that may be used.
+
+@InsertChunk markdown_conventions
+
+@EndSection

doc/Tutorials.xml

@@ -122,7 +122,7 @@ DeclareOperation( "ToricVariety", [ IsConvexObject ] );
 ]]></Listing>
 
 For a thorough description of what you can do with &AutoDoc;
-documentation comments, please refer to chapter <Ref Chap="Comments"/>.
+documentation comments, please refer to chapter <Ref Chap="Chapter_Comments"/>.
 <P/>
 
 <!--
@@ -326,7 +326,7 @@ is named <File>manual.xml</File>.
 The files <File>PackageInfo.g</File>, <File>makedoc.g</File>, 
 <File>doc/title.xml</File> and <File>doc/PackageName.xml</File> 
 (if it exists) will be altered by this procedure, 
-so it may be wise to keep backup copies. 
+so it may be wise to keep backup copies.
 <P/> 
 You should have copies of the &AutoDoc; files <File>PackageInfo.g</File> 
 and <File>makedoc.g</File> to hand when reading these instructions. 

gap/DocumentationTree.gd

@@ -47,9 +47,12 @@ DeclareOperation( "DocumentationTree", [ ] );
 DeclareOperation( "StructurePartInTree", [ IsTreeForDocumentation, IsList ] );
 DeclareOperation( "ChapterInTree", [ IsTreeForDocumentation, IsString ] );
 DeclareOperation( "SectionInTree", [ IsTreeForDocumentation, IsString, IsString ] );
+DeclareOperation( "SectionAsChildOf", [ IsTreeForDocumentation, IsList, IsObject ] );
 DeclareOperation( "SubsectionInTree", [ IsTreeForDocumentation, IsString, IsString, IsString ] );
+DeclareOperation( "SubsectionAsChildOf", [ IsTreeForDocumentation, IsList, IsObject ] );
 DeclareOperation( "DocumentationExample", [ IsTreeForDocumentation, IsList ] );
 DeclareOperation( "DocumentationExample", [ IsTreeForDocumentation ] );
+DeclareOperation( "DocumentationIndexEntry", [ IsTreeForDocumentation, IsString, IsString ] );
 DeclareOperation( "DocumentationDummy", [ IsTreeForDocumentation, IsString, IsList ] );
 DeclareOperation( "DocumentationDummy", [ IsTreeForDocumentation, IsString ] );
 DeclareOperation( "DocumentationCode", [ IsTreeForDocumentation, IsString, IsList ] );

gap/DocumentationTree.gi

@@ -112,6 +112,14 @@ BindGlobal( "TheTypeOfDocumentationTreeCodeNodes",
         NewType( TheFamilyOfDocumentationTreeNodes,
                 IsTreeForDocumentationCodeNodeRep ) );
 
+DeclareRepresentation( "IsTreeForDocumentationIndexNodeRep",
+                       IsTreeForDocumentationNodeRep,
+                       [ ] );
+
+BindGlobal( "TheTypeOfDocumentationTreeIndexNodes",
+            NewType( TheFamilyOfDocumentationTreeNodes,
+                     IsTreeForDocumentationIndexNodeRep ) );
+
 ###################################
 ##
 ## Tools
@@ -173,22 +181,26 @@ InstallMethod( DocumentationTree, [ ],
 end );
 
 ## create a chapter, section or subsection
-InstallMethod( StructurePartInTree, [ IsTreeForDocumentation, IsList ],
+## This first helper, given tree and context, returns a list of two entries.
+## The second entry is a node to represent the structure part at that context.
+## The first is true if that structure part did not yet exist, in which case the
+## node returned as the second entry is "floating," i.e., not yet added
+## anywhere in  the tree. If the first entry is false, the node existed in the
+## tree earlier (and is presumed to have been added in some appropriate place).
+StructurePartMaybeFloating@ :=
   function( tree, context )
-    local label, parent, new_node, type;
+    local label, new_node, type;
 
     if IsEmpty( context ) then
-        return tree;
+        return [ false, tree];
     fi;
 
     # if the part already exist, use that
     label := AUTODOC_LABEL_OF_CONTEXT( context );
     if IsBound( tree!.nodes_by_label.( label ) ) then
-        return tree!.nodes_by_label.( label );
+        return [false, tree!.nodes_by_label.( label )];
     fi;
 
-    parent := StructurePartInTree( tree, context{[1..Length(context)-1]} );
-
     new_node := rec( content := [ ],
                      level := tree!.current_level,
                      name := context[ Length( context ) ],
@@ -203,8 +215,20 @@ InstallMethod( StructurePartInTree, [ IsTreeForDocumentation, IsList ],
     ObjectifyWithAttributes( new_node, type, Label, label );
 
     tree!.nodes_by_label.( label ) := new_node;
-    Add( parent!.content, new_node );
-    return new_node;
+    return [true, new_node];
+end;
+
+# This method creates the structure part and adds it at the current position
+# in the next higher structure part.
+InstallMethod( StructurePartInTree, [ IsTreeForDocumentation, IsList ],
+  function( tree, context )
+    local parent, trial;
+    trial := StructurePartMaybeFloating@( tree, context );
+    if trial[ 1 ] then
+        parent := StructurePartInTree( tree, context{[ 1..Length(context)-1 ]} );
+        Add( parent!.content, trial[ 2 ] );
+    fi;
+    return trial[ 2 ];
 end );
 
 ##
@@ -231,6 +255,22 @@ InstallMethod( DocumentationExample, [ IsTreeForDocumentation ],
     return node;
 end );
 
+##
+InstallMethod( DocumentationIndexEntry,
+               [ IsTreeForDocumentation, IsString, IsString ],
+  function( tree, ikey, entry )
+    local node, label;
+
+    node := rec( content := [ entry ],
+                 key := ikey,
+                 level := tree!.current_level );
+    label := Concatenation( "Index_", String( AUTODOC_TREE_NODE_NAME_ITERATOR( tree ) ) );
+    ObjectifyWithAttributes( node, TheTypeOfDocumentationTreeIndexNodes,
+                             Label, label );
+    tree!.nodes_by_label.( label ) := node;
+    return node;
+end );
+
 ##
 InstallMethod( DocumentationDummy, [ IsTreeForDocumentation, IsString, IsList ],
   function( tree, name, context )
@@ -437,12 +477,46 @@ InstallMethod( SectionInTree, [ IsTreeForDocumentation, IsString, IsString ],
     return StructurePartInTree( tree, [ chapter_name, section_name ] );
 end );
 
+## This method creates the section and if it is not already in the tree,
+## adds it as a child of the specified node (rather than as a child of its
+## chapter in the documentation tree). This is useful for implementing chunks,
+## which can move sections around.
+InstallMethod( SectionAsChildOf, [ IsTreeForDocumentation, IsList, IsObject ],
+  function( tree, context, parent )
+    local section_try;
+    if Length( context ) <> 2 then
+        Error( "Request for section at ill-formed context", String( context ) );
+    fi;
+    section_try := StructurePartMaybeFloating@( tree, context );
+    if section_try[ 1 ] then
+        Add(parent!.content, section_try[ 2 ]);
+    fi;
+    return section_try[ 2 ];
+end );
+
 ##
 InstallMethod( SubsectionInTree, [ IsTreeForDocumentation, IsString, IsString, IsString ],
   function( tree, chapter_name, section_name, subsection_name )
     return StructurePartInTree( tree, [ chapter_name, section_name, subsection_name ] );
 end );
 
+## This method creates the subsection and if it is not already in the tree,
+## adds it as a child of the specified node (rather than as a child of its
+## section in the documentation tree). This is useful for implementing chunks,
+## which can move subsections around.
+InstallMethod( SubsectionAsChildOf, [ IsTreeForDocumentation, IsList, IsObject ],
+  function( tree, context, parent )
+    local subsection_try;
+    if Length( context ) <> 3 then
+        Error( "Request for subsection at ill-formed context", String( context ) );
+    fi;
+    subsection_try := StructurePartMaybeFloating@( tree, context );
+    if subsection_try[ 1 ] then
+        Add(parent!.content, subsection_try[ 2 ]);
+    fi;
+    return subsection_try[ 2 ];
+end );
+
 #############################################
 ##
 ## Write functions
@@ -661,6 +735,15 @@ InstallMethod( WriteDocumentation, [ IsTreeForDocumentationExampleNodeRep, IsStr
     AppendTo( filestream, "]]></", inserted_string, ">\n\n" );
 end );
 
+##
+InstallMethod( WriteDocumentation, [ IsTreeForDocumentationIndexNodeRep, IsStream ],
+  function( node, filestream )
+    if node!.level > ValueOption( "level_value" ) then return; fi;
+    AppendTo( filestream, "<Index Key=\"", node!.key, "\">" );
+    WriteDocumentation( node!.content, filestream );
+    AppendTo( filestream, "</Index>\n" );
+end );
+
 ##
 InstallMethod( WriteDocumentation, [ IsTreeForDocumentationCodeNodeRep, IsStream ],
   function( node, filestream )

gap/Magic.gd

@@ -219,9 +219,20 @@
 #!
 #!         <Mark><A>files</A></Mark>
 #!         <Item>
+#!             <Label Name="AutodocFilesOption"/>
 #!             A list of files (given by paths relative to the package directory)
 #!             to be scanned for &AutoDoc; documentation comments.
 #!             Usually it is more convenient to use <A>autodoc.scan_dirs</A>, see below.
+#!             However, the files in this list are always scanned **before**
+#!             the ones found by <A>scan_dirs</A>, and they are scanned in
+#!             the order listed, so this option can be useful for controlling
+#!             the order that &AutoDoc; processes files, which in turn
+#!             can be useful to ensure that your manual is presented in the
+#!             desired order. For example, you might use this option to
+#!             designate a "master file" which declares all chapters in the
+#!             desired order. Those chapters can then be filled in by
+#!             documentation in your source files, which may be processed in an
+#!             order not relevant to the flow of your documentation.
 #!         </Item>
 #!
 #!         <Mark><A>scan_dirs</A></Mark>