Thanks @syntax_x for pointing me in the right direction

For some reason this game doesn't work with MODE2/2352 bins, only with MODE1/2048 isos!!
The easiest way to get it running on USB-GDROM is to:
1.) get the TOSEC gdi (NTSC or PAL, doesn't matter)
2.) convert all the bins to isos with the help of bin2iso.exe
3.) delete the old bins
and you're done. You don't even have to edit the gdi file (i.e. replace 2352 values with 2048 ones). It just works like that.