xmp 4.3.0

[AliceLR]

Open issues:

• -l#/--loop # optional parameter: Add Loop Amount #14 (implemented, currently in testing)
• Playlists: Option to read a module playlist from a file/stdin? #25 (undecided whether or not to kick it down the road)

Remaining junk that doesn't really qualify for individual issues:

• AHI. Working on getting a toolchain set up, but literally all of them seem to have problems so far (AROS nightlies rely on hardcoded work directories, adtools is missing its binutils repo and lacks a convincing tarball fallback currently, bebbo amiga-gcc at least compiles but I think may require extra flags somewhere, as linking -lxmp fails?, amigaos-cross-toolchain requires an environment with 32-bit GCC 5).
  • General cleanup: Fix style/indendation issues in sound_ahi.c. #85
  • 32-bit support + new functions: Add 32-bit support to the Amiga AHI driver. #86
• Other drivers that didn't get 32-bit support: switch from manually editing options->format to using the new functions.

[AliceLR]

bebbo amiga-gcc at least compiles but I think may require extra flags somewhere, as linking -lxmp fails?

m68k-amigaos-gcc test.c -I$AMIGADEV/include -L$AMIGADEV/lib -lxmp -lm
/opt/amiga/lib/gcc/m68k-amigaos/6.5.0b/../../../../m68k-amigaos/bin/ld: /opt/amiga/lib/libxmp.a(tempfile.o): in function `unlink_temp_file':
/home/mzx/libxmp/src/tempfile.c:171: undefined reference to `unlink'
/opt/amiga/lib/gcc/m68k-amigaos/6.5.0b/../../../../m68k-amigaos/bin/ld: /opt/amiga/lib/libxmp.a(mkstemp.o): in function `libxmp_mkstemp':
/home/mzx/libxmp/src/mkstemp.c:74: undefined reference to `getpid'
/opt/amiga/lib/gcc/m68k-amigaos/6.5.0b/../../../../m68k-amigaos/bin/ld: /opt/amiga/m68k-amigaos/lib//libc.a(lib_a-gettimeofdayr.o): in function `_gettimeofday_r':
lib_a-gettimeofdayr.o:(.text+0x10): undefined reference to `_gettimeofday'
collect2: error: ld returned 1 exit status

(Retrying building some of the toolchain, this is presumably user error. edit: happened again with a fresh rebuild of Newlib...)

edit: worked around these (and signal() being missing for some reason on the xmp-cli side) and got xmp to compile, at least. It hangs when I try to run it with FS-UAE, though.

[sezero]

Re: m68k-amigaos:

The issue is possibly with the toolchain, but did you specify the crt flavor
you want, perhaps? I don't know Bebbo's toolchain, but without -noixemul
or -mcrt=clib2, my toolchain defaults to ixemul posix emulation library.

How does the attached ones behave, by the way? One is built with vbcc, the
others built with gcc-2.95 one with -noixemul and the other -mcrt=clib2

m68k-amigaos.tar.gz

[AliceLR]

bebbo amiga-gcc is supposed to default to plain Newlib (doesn't necessarily mean this happened). I did try the other CRTs and they all seemed to be missing some function or another. I'll try -noixemul or the tarball and report back..

edit: per bebbo's docs and my own testing, -noixemul selects libnix20, which lacks dirent.
edit: I'm wrong, -noixemul needs to be provided to both CFLAGS and LIBS.
edit: xmp (bebbo amiga-gcc, -noixemul) still hangs when I run it from AmigaShell, but at least it compiled without hacks this time.

[sezero]

-noixemul needs to be provided to both CFLAGS and LIBS.

Yes. And remember to build libxmp the same way you build xmp-cli

xmp (bebbo amiga-gcc, -noixemul) still hangs when I run it from AmigaShell, but at least it compiled without hacks this time.

OK. How do my builds behave, btw?

[AliceLR]

OK. How do my builds behave, btw?

Haven't tried yet, sorry—will follow up when I get a chance.

[AliceLR]

OK. How do my builds behave, btw?

clib2: prints to stdout, then prompts for suspend/reboot. This also happens when attempting to play a module (no playback occurs).
libnix: prints to stdout without hanging or crashing. Playing a module works, but it hangs when the module ends. Opening a second shell and running xmp with a module again breaks audio.
vbcc: Software Failure screen.

[sezero]

Sigh.. Well, I had given up on my m68k-amigaos toolchains for some time in another project, and it's no good here either. I won't be doing anything other than testing build successes with them from now on.

[AliceLR]

With a little bit of coaxing (really just replacing URLs) I got the old adtools amigaos-cross-toolchain for m68k to build. My -noixemul build also plays correctly and crashes on exit (this time, with a Software Failure rather than a hang). We may both have working toolchains for libnix/-noixemul and this may be a libxmp bug, but I'll need to look at it some more. (It could also be an issue with my AmigaOS VM.)

[AliceLR]

7d4ed11, -noixemul results in the same hang on exit (no Software Failure) as the above build. be0dc04, ba9cab9, and 38b6ce7 with -noixemul gets the Software Failure on exit (it might just be random variation...).

edit: Even the file drivers (-dfile, -dwav, -daiff) crash, so whatever this is isn't specific to the AHI driver. Placing return 0 at some points in the code reproduces the exit hang rather than a Software Failure. Advancing to the next module when two modules are provided as arguments will also cause a Software Failure(?!).

edit: inserting goto end before xmp_play_frame() stops the crash... 😬

[AliceLR]

...

STACK

Format:	STACK [<stack size>] 
  
 Template:	SIZE/N 
  
 Purpose:	To display or set the stack size within the current Shell. 
  
 Path:		Internal 
  
 Specification: 
  
 When you run a program, it uses a certain amount of stack, a special area in 
 memory allocated for the program. The stack required for a program should be 
 given in the program's documentation. However, if a program causes a system 
 failure, you may wish to experiment with various stack sizes. 
  
 Commands that perform operations that consist of multiple levels may require 
 additional stack space. 
  
 Stack sizes generally range from 4000 to 25,000 bytes. If the stack size is 
 too small, a system failure may occur. Too large of a stack size may take 
 too much memory away from other system functions. 
  
 Caution  If you  run out of stack space, you  may receive a Software 
 Failure message. If you have altered the stack for the program that caused 
 the Software Failure message, try increasing the stack size. 

Is there a way we can hint the stack size that Workbench 3.1 recognizes?

edit: the options for this look a bit dire. Newer Workbench shell versions can detect a stack cookie, which might be worth adding at minimum.

[sezero]

Ah, forgot the stack size can be a problem because I didn't need to use the stack command on aros.

And didn't know about the stack cookie thing either.

We have this piece of code in our amiga-specific sources (amigaos3, aros, morhpos -- don't
know if its suitable for amigaos4, though), maybe if it'd be useful here?
https://github.com/sezero/uhexen2/blob/m68k-amigaos/engine/hexen2/sys_amiga.c#L38-L70
And the aros-specific part:
https://github.com/sezero/uhexen2/blob/master/oslibs/aros-x86/misc/include/incstack.h

[AliceLR]

The aros-specific part looks essentially like code I saw elsewhere—that method of setting the stack is supposed to be good for Workbench 2.04 onward and just requires a main() wrapper.

The code to automatically pull in swapstack.o (essentially does the same thing as the manual StackSwap code, from what I understand) may help. There are several instances of swapstack.o in the toolchain root and, as far as I can tell, no way to get the compiler to automatically pull it in (I don't know why the stack size isn't just a custom compiler flag).

edit: actually, the aros code uses NewStackSwap, which was added after StackSwap (due to safety concerns I think?).

[AliceLR]

This fixes the crash on my end, likely needs a lot more testing (whichever libnix/WB version the default is with the old adtools m68k toolchain):

diff --git a/src/main.c b/src/main.c
index f1694dfe..dcaf8de9 100644
--- a/src/main.c
+++ b/src/main.c
@@ -18,6 +18,7 @@
 #include <sys/time.h>
 #endif
 #if defined(XMP_AMIGA)
+#include <proto/exec.h>
 #include <proto/timer.h>
 #include <time.h>
 #endif
@@ -45,6 +46,25 @@
 #include <windows.h>
 #endif
 
+
+/* AmigaOS and derivatives require manually setting the stack size. */
+#if defined(XMP_AMIGA)
+#define MIN_STACK_SIZE 65536
+
+/* AmigaOS 3.2, 4.x */
+extern const char stack_cookie[];
+const char stack_cookie[] = "$STACK:65536";
+
+#if defined(__CLIB2__)
+extern int __stack_size;
+int __stack_size = MIN_STACK_SIZE;
+#else
+extern ULONG __stack;
+ULONG __stack = MIN_STACK_SIZE;
+#endif
+#endif
+
+
 static const struct sound_driver *sound;
 static unsigned int foreground_in, foreground_out;
 static int refresh_status;
@@ -234,7 +254,7 @@ static void load_error(const char *name, const char *filename, int val)
 }
 
 
-int main(int argc, char **argv)
+static int real_main(int argc, char **argv)
 {
 	xmp_context xc;
 	struct xmp_module_info mi;
@@ -655,3 +675,36 @@ int main(int argc, char **argv)
 
 	return 0;
 }
+
+int main(int argc, char **argv)
+{
+	/* Do not use any stack variables in this function. */
+	static int ret;
+
+#if defined(XMP_AMIGA)
+	static struct StackSwapStruct sw;
+	static struct Task *mytask;
+	static long stack_size;
+
+	mytask = FindTask(NULL);
+	stack_size = (UBYTE *)mytask->tc_SPUpper - (UBYTE *)mytask->tc_SPLower;
+	if (stack_size < MIN_STACK_SIZE) {
+		sw.stk_Lower = AllocVec(MIN_STACK_SIZE, MEMF_PUBLIC);
+		sw.stk_Upper = (ULONG)sw.stk_Lower + MIN_STACK_SIZE;
+		sw.stk_Pointer = (APTR)sw.stk_Upper;
+		StackSwap(&sw);
+	}
+#endif
+
+	ret = real_main(argc, argv);
+
+#if defined(XMP_AMIGA)
+	/* Restore original stack. */
+	if (stack_size < MIN_STACK_SIZE) {
+		StackSwap(&sw);
+		FreeVec(sw.stk_Lower);
+	}
+#endif
+
+	return ret;
+}

Also, I'm not sure that this usage is safe, even when avoiding stack variables in this manner.

(16384 results in a crash with a basic test XM, 32768 works fine, I doubled it for safety. The main culprit seems to be the libxmp load.c:45 stack-allocated buffer for calculating MD5s. There's another 16k stack buffer in the depackers, too, but they should never co-exist. Lowering these both to 2048 makes a stack size of 8192 work for this test module, but not the default 4096.)

Also, setting the stack to 32768+ also fixes my bebbo toolchain builds.