Gens4All with Z80 Emulation

[Ian Micheal]

if your getting the upto date pvrbuffers i have had no problem with it at all and it only gives me proper vram this included complex opengl games using it.. If i was to use the old version few projects would not even run with the 3.9mb vram max of the old file using opengl

also would not use it as it was first put there in 2002 using magic numbers with no why or how they got those numbers


all pvr kos example have been tested with the new one with out fail also all gldc example tested with out fail included cocoball using simulant

[Ian Micheal]

I uploaded a video showing how it's currently working.

https://youtu.be/JHXp3MywIzQ

I looked into the clicking sounds I mentioned before, and it seems to be an issue with SDL. I dumped the raw data sent to SDL and played it back with Audacity, and there wasn't any clicking. So the issue seems to be with SDL. I thankfully won't have to try to debug the sound emulation. I tried replacing SDL with KOS streaming functions, but they don't seem usable. I had trouble getting them to work. Looking through the source for it, the implementation seems to require (but doesn't mention in the documentation) a buffer greater than 2048 samples, or about 1/10th of a second at 22KHz, for the streaming to work. I had the buffer set to about 1/50th of a second, and the streaming library thought that there wasn't enough to bother loading more samples. That minimum buffer size is way too big to be usable for an emulator. My next goal is getting a lower latency streaming replacement to work, without clicking.

I could write simple rom loader if you wanted but I'm sure you could do that..

I was thinking of having a ROM list file, which would the user to customize the order of the list and store game settings. If the game needs any emulator hacks to work correctly, they could be specified in the list instead of trying to autodetect based on the ROM title, which would be useful for playing hacks. An entry might look something like this:

Code: Select all

Game: Sonic the Hedgehog 3 & Knuckles
Rom: s3&k.gen
Controller: 2p_3button_std
Samplerate: 22050
Emu: sonic_smps_z80_hack shadowhilite_spr
Savename: GENS4ALL_S3K
Savetitle: Sonic 3 & Knuckles
Savetitlevmu: Sonic3&K
Saveicon: s3kicon.bin

There could be multiple lists, so one might have commercial games, another might have hacks, or something.

I'm more worried about the emulation quality at the moment than getting any working loader, though. I'm not really good at graphic design. If anyone wants to come up with ideas for how it should eventually look, feel free.

Could you explain what is needed to build it? Would like to test it

All you should need, with the included object files, is a KallistiOS environment. If you want to make clean and build the source from scratch, the 68000 core requires GLib 2 to compile. You need the development files for GLib, not just the binaries. On Xubuntu, that should be "libglib2.0-dev".

I noticed KallistiOS recently got an update to the PVR driver that looks like it would break 3D under certain conditions. It fixes one memory leak, but mistakes confusing but correct code for a second leak. The vertex buffer and one part of the OPB end up overlapping, and that part of the OPB is set a a very small size. It probably wouldn't affect Gens4All, but if you have rendering errors, use an older version of pvr_buffers.c and recompile KOS.

If your using the crusty sdl included with kos then yes you have clicking the threading is wrong in among other things I updated sdl_dreamhal which fixes all these problems

example perfect sound using it


sdl Dreamhal provided for sound same as video above no tick less impact
in the video it also has 2x the blit speed of the current i have benchmarks but you not using blit speed

I could plug this in get no tick sound common problem on the old sdl stereo channels are wrong also the threading

I have perfect clean sound as shown in video

You have to put this in by hand and remove the old headers or they will conflict..

It would fix the problem but that's not up to me you may have problems geting the above working I'm use to it..

this is for kos2.0 and gcc 4.7 i dont use gcc 9 which also might be your problem with the old sdl

[TapamN]

Thanks!!, got to build it fine, just need to get my custom "coders cable" to test it. Just to ask, without a rom loader I supose that, for now, it won't work on GDemu right?

You'll have to make a seperate image for each individual ROM, but if you change the "ROMDIR" define to "/cd/", set the correct ROM filename in the run call, and put the ROM on the disc image, it should be able to load from "disc".

Edit: Didn't notice the second page.

if your getting the upto date pvrbuffers i have had no problem with it at all and it only gives me proper vram this included complex opengl games using it.. If i was to use the old version few projects would not even run with the 3.9mb vram max of the old file using opengl

also would not use it as it was first put there in 2002 using magic numbers with no why or how they got those numbers

all pvr kos example have been tested with the new one with out fail also all gldc example tested with out fail included cocoball using simulant

The magic numbers were the size of the dynamic OPB. It's like the size of the vertex buffer, but for the pointers to each strip in a tile, rather than the actual strip data. The dynamic OPB was originally set to be much larger than it needed to be, but it's still needed. It looks like the way the new buffers are set up, the vertex buffer and dynamic OPB overlap. If enough vertex data is submitted that the buffers overwrite each other, the PVR will behave unpredictably, possibly hanging. Whether or not anything happens depends on how much is submitted, and where on screen the polygons are.

Or maybe I'm misreading the new KOS PVR code. I'll try making some tests at some point to make sure.

If your using the crusty sdl included with kos then yes you have clicking the threading is wrong in among other things I updated sdl_dreamhal which fixes all these problems

example perfect sound using it


sdl Dreamhal provided for sound same as video above no tick less impact

Ok, I'll try it.

[Ian Micheal]

Thanks!!, got to build it fine, just need to get my custom "coders cable" to test it. Just to ask, without a rom loader I supose that, for now, it won't work on GDemu right?

You'll have to make a seperate image for each individual ROM, but if you change the "ROMDIR" define to "/cd/", set the correct ROM filename in the run call, and put the ROM on the disc image, it should be able to load from "disc".

Edit: Didn't notice the second page.

if your getting the upto date pvrbuffers i have had no problem with it at all and it only gives me proper vram this included complex opengl games using it.. If i was to use the old version few projects would not even run with the 3.9mb vram max of the old file using opengl

also would not use it as it was first put there in 2002 using magic numbers with no why or how they got those numbers

all pvr kos example have been tested with the new one with out fail also all gldc example tested with out fail included cocoball using simulant

The magic numbers were the size of the dynamic OPB. It's like the size of the vertex buffer, but for the pointers to each strip in a tile, rather than the actual strip data. The dynamic OPB was originally set to be much larger than it needed to be, but it's still needed. It looks like the way the new buffers are set up, the vertex buffer and dynamic OPB overlap. If enough vertex data is submitted that the buffers overwrite each other, the PVR will behave unpredictably, possibly hanging. Whether or not anything happens depends on how much is submitted, and where on screen the polygons are.

Or maybe I'm misreading the new KOS PVR code. I'll try making some tests at some point to make sure.

If your using the crusty sdl included with kos then yes you have clicking the threading is wrong in among other things I updated sdl_dreamhal which fixes all these problems

example perfect sound using it


sdl Dreamhal provided for sound same as video above no tick less impact

Ok, I'll try it.

kazade: Yeah I think they're misreading it a bit - we're definitely missing the dynamic opb area, and that will result in glitches if you render enough stuff, but I don't believe there's an overlapping area

You have got the very latestest version ? as what your talking about was solved?

If you cant get my version of sdl to work i can try doing it for you since i have been dealing with dreamcast and sdl since 2002 and solving that tick your talking about on all my ports..

only problem i see is i use gcc4.7 and your compiled core will prollly not work i have not tried yet

[Ivan Guber]

Gens4all with EarthWorm Jim. Tested on Dreamshell, work fine.

Image for GDEMU
https://disk.yandex.ru/d/HngSiBoUHtBuYA
Image for Dreamshell
https://disk.yandex.ru/d/gW94ipeQk40ApA

[fafadou]

If I understand well dear @Ivan, he's working on sound, maybe earthworm Jim doesn't use some particularities we can have with Sonic 2 ?

Thanks for your release and of course thanks to @TapamN for improving the gens4all.

[Ivan Guber]

Tested some games, and i can say, it's best Genesis emulator for Dreamcast! Better, than GenesisPlusDC. Video

he's working on sound, maybe earthworm Jim doesn't use some particularities we can have with Sonic 2 ?

Jim work great and sound work great (see video, choose 720p to best quality).

[kremiso]

thank you TapamN for this !
there is surely hard work behind, congrats
another Chui one sectioned and improved

gold for a retronostalgic as i am, curious to test the added Z80 bass power

also thank you Ivan for the video and the test build, excellent

[TapamN]

kazade: Yeah I think they're misreading it a bit - we're definitely missing the dynamic opb area, and that will result in glitches if you render enough stuff, but I don't believe there's an overlapping area

You have got the very latestest version ? as what your talking about was solved?

If you cant get my version of sdl to work i can try doing it for you since i have been dealing with dreamcast and sdl since 2002 and solving that tick your talking about on all my ports..

only problem i see is i use gcc4.7 and your compiled core will prollly not work i have not tried yet

My interpretation of the buffer allocation code works like this:

The buf->opb variable in the KOS PVR driver does two things. It specifies the high end of the dynamic OPB, and the low end of the static OPB. The dynamic OPB starts right below buf->opb, and grows DOWN to buf->opb - buf->opb_size. The lines "buf->opb_size = 0x50580 + polybuf; outaddr += buf->opb_size;" in the old code were allocating space for the dynamic OPB. The new code sets the size of the dynamic OPB to be the size of the static OPB, and doesn't allocate any specific area for the dynamic OPB. As the dynamic OPB grows down, it overlaps the high end of the vertex buffer. With the old code, the TA would prevent either one growing to the point that they overlap, but now the TA doesn't do that.

The function pvr_sync_reg_buffer() in pvr_misc.c sets the TA OPB registers. The names of the constants in KOS don't really indicate what they do, so I've commented them:

Code: Select all

    PVR_SET(PVR_TA_OPB_START,   buf->opb);	//Static OPB start
    PVR_SET(PVR_TA_OPB_END,     buf->opb - buf->opb_size);	//Dynamic OPB end. It grows down, see the minus?
    PVR_SET(PVR_TA_VERTBUF_START,   buf->vertex);
    PVR_SET(PVR_TA_VERTBUF_END, buf->vertex + buf->vertex_size);
    PVR_SET(PVR_TA_OPB_INIT,    buf->opb);	//Dynamic OPB start (high end of dynamic OPB)

The math for the buffers in the current driver works out like this:

• PVR_TA_VERTBUF_END = buf->vertex + buf->vertex_size
• PVR_TA_OPB_INIT = buf->vertex + buf->vertex_size (== buf->opb)
• PVR_TA_OPB_END = buf->vertex + buf->vertex_size - buf->opb_size (== buf->opb - buf->opb_size)

There is a register to read the position of the next dynamic OPB allocation, PVR_TA_OPB_POS, which would be useful for testing and seeing how much of the dynamic OPB is used. For some reason, you need to left shift it by two to get the real address. ("PVR_GET(PVR_TA_OPB_POS) << 2") You could be able to see the position starts below PVR_TA_VERTBUF_END and decreases as more OPB blocks are allocated.

To make an OPB stress test program, I would do this:

• Draw a lot of polygons with a small vertex buffer, so that you don't need to submit so much to get the vertices and dynamic OPB to meet up.
• Draw everything really close together, so the OPB buckets they touch keep overflowing and require dynamic OPB allocations.
• Draw triangle strips, not individual triangles or sprites. There seems to be a PVR feature that combines multiple individual triangles or sprites into a single OPB entry, but strips always seem to get their own entry and would use up the OPBs faster.
• Use a bin size of 8, so that the static OPB gets used up quickly.

When I said that one leak was fixed, I was talking about the line "outaddr += buf->opb_size; /* Do we _really_ need this twice? */". But looking at the old code again, it looks like the old OPB initialization code DID have a second bug. "buf->opb_size = 0x50580 + polybuf;" should have been just "buf->opb_size = 0x50580;", I think? Polybuf was set to the end of the vertex buffer, so it would set the dynamic OPB size to be the size of the vertex buffer plus 0x50580. (And a dynamic OPB of size 0x50580 is already overkill.) So it seems I was wrong, and there was a second leak, but the fix for the leak introduced a different bug. (By my reading, at least.)

As for the clicking, I tried replacing my default KOS SDL library with yours, but the clicking remained. I double checked a couple times to make sure it was using the new SDL. Maybe the issue is with Gens4All?

The SDL audio callback I'm using in Gens4All currently looks like this.

Code: Select all

void
proc (void *user, Uint8 * buffer, int len)
{
	user=audiobuf;
	if(!user) {
		memset(buffer,len,0);
		return;
	}
	if (audio_len < len) {
		//printf("U"); fflush(stdout);
		//Audio buffer underflow, copy what we have, pad out the rest of the buffer with zeroes
		SDL_LockAudio ();
		memcpy (buffer, user, audio_len);
		memset(buffer + audio_len, 0, len - audio_len);
		SDL_UnlockAudio ();
		 
		if (soundrecenabled) {
			//If recording, make a copy of what was sent to SDL
			memcpy (soundrec + soundrecpos, user, audio_len);
			soundrecpos += audio_len;
		}
		
		audio_len = 0;
	} else {
		//printf("f"); fflush(stdout);
		//We have more samples in the emulated buffer than SDL needs.
		SDL_LockAudio ();
		memcpy (buffer, user, len);
		SDL_UnlockAudio ();
		
		if (soundrecenabled) {
			//If recording, make a copy of what was sent to SDL
			memcpy (soundrec + soundrecpos, user, len);
			soundrecpos += len;
		}
		
		//Remove the part of the buffer that was sent to SDL
		audio_len -= len;
 		memmove (user, (unsigned char *) user + len, audio_len);
	}
	if (soundrecenabled && (soundrecpos > 500*1024)) {
		printf("Rec over\n"); fflush(stdout);
		soundrecenabled = 0;
	}
}

The variable soundrecenabled was used to enable the dump test I did to check if the clicking was in the emulated audio stream. Are the lock and unlock audio calls needed here? I didn't notice any changes whether or not they were there. The "samples" parameter of SDL's audio struct is set to 1024.

Using the printfs, I've checked that the emulated buffer is not underflowing when the clicking occurs. It underflows for the first few calls when the emulator first starts, but when the music for Phantasy Star 2's title screen plays (with clicking), it's printing out "f", so SDL seems to be getting good data.

On Phantasy Star 2, the clicking happens on a regular interval multiple times a second in the left channel. A lower sample parameter in the SDL audio init struct results in more frequent clicking, and a lower frequency results in less frequent clicking. So clicking seems tied to how often the audio buffer needs to be filled. Clicking doesn't happen when the Genesis is silent.

It makes me think that maybe there's an off-by-one error in SDL when it copies the samples into sound RAM, so something like one byte of an old sample remains by accident?

[dark]

Great work! This looks fantastic!