Burn Duo-R/PCE-CD/tg16 CD discs in Linux

burnDuor.bash

#!/usr/bin/env bash

_infile="$1"
_basename="${_infile%%.zip}"
_tocfile="${_basename}".toc

CUECONVERT=cueconvert

unzip -tq "${_infile}" || exit 1
mkdir -p "${_basename}"
cd "${_basename}"
unzip ../"${_infile}"
_cuefile="$(find ./ -iname '*.cue')"
"${CUECONVERT}" -i cue "${_cuefile}" > "${_tocfile}"

# Kind of gnarly sed command
sed -E -i ':a ; $!N ; s/TRACK MODE1_RAW\s+FILE/TRACK MODE1_RAW\nDATAFILE/ ; ta ; P ; D' "${_tocfile}"

cdrdao simulate --driver generic-mmc \
       --swap \
       --speed 16 \
       "${_tocfile}" || exit 1
cdrdao write --driver generic-mmc \
       --swap \
       --speed 16 \
       "${_tocfile}" || exit 1

As far as I can tell so far, this gets around at least two bugs:

1. In cdrdao
   The manual claims that it will only assume little endian for wave (.wav) files, and yet it seems to conclude that these are little endian even though they're bins. That could be because they're AUDIO tracks (?) and CUE files are terrible (see 2).
2. Both cue2toc and cueconvert seem to have some issues with these "[PC Engine CD-ROM2 System and TurboGrafx-CD 2016 Reference Set]" files. That could be because CUEs are terrible, though. These probably work fine with whatever 25yo software works with them in Windows 95 or w/e. But, for 1: cdrdao does not "support" data file seeking, and these have start times on data files in them. Secondly, or wherever we are numerically, all these are listed as AUDIO tracks, but they're all raw data. That seems to be what we want, though. Oh, that was the other thing, cueconvert seems to think these are "CD_DA" when it converts them to toc files. Surprisingly, "CD_DA" would indicate something like "Music only," but it doesn't seem to make a difference, and I didn't want to tempt fate by changing them to CD-ROM.

This is old stuff that was probably never fully sorted out in the first place. CUE files were never really standardized. I still plan to use cdrdao to read it back in toc format and see what it looks like. OH, and finally, a lot of people on the internet talk about "size," "brand," and "write-speed," but, to be absolutely honest, I've found that not to matter so far with my awesome cap-refurbished, rgb'd Japanese NEC Duo-R. Some of my CD-Rs are Maxells, some are Virbatum. There are people in forums insisting on TY. Nah. Another weird thing: This pack of "ROMS" I got (the multi-bin, 1 cues) from teh internets, it's TOCs don't seem to match what I see on NECSTASY at all (https://www.necstasy.net/uk/). I haven't tried using necstasy TOCs with the same binary files because, well, the above has worked out for me with this set, anyway. It's also worked for some random zips I got off the internet, so I'll be sticking with it until I need to try something else. I'll try to write up more technical details later, and improve the script. Think of this as a howto for now. I added the final write at the end without testing it, cus I usually just do that part manually so far. It's worked on 5 games so far, though!

Oh, in case it wasn't clear: The bin files are little endian, most likely because that's what CDs used. That's why we have to do the "--swap" in cdrdao. Whoever wrote them wrote them as little endian OR cdrdao just assumes that it should write them as little endian (I haven't checked to see which on yet, but I will). Even Apple's AIFF used big endian when MacOS was PowerPC, but then they made a hacked AIFF format that changed the byte order, etc. more at this page and more about CD formats and some stuff about CD-ROM, but remember that tg16 CD/PCE CD/CD-ROM-ROM/Duo-R discs predate the CD-ROM standard (/yellow book/), and may have even been around early (/red book/) standard from what I remember. These were the first video game consoles to use CDs. The size must have been incredible! Hard Drives on computers were much smaller than that back then. It's pretty amazing that we can still make these things burn, honestly! Now, I want to really make sure that I help preserve this shit.

EDIT: I just checked, and HuC6280 is an improved version of the WDC5C02, which is an improved version of the 6502, which is...little endian. So, this likely has more to do with the CD/CD-ROM format and the way they were ripped than anything else. Besides, the audio tracks that were being written at the wrong endianness are music from the CD. I'm looking closer at hex dumps of the disc tracks from the original dumps I used, and I'm going to compare them to a rip of the same version of the disc I built so that I have a better idea of what's actually going on :). I still think it has something to do with how they were ripped and the CUE file leading the toc convertor and subsequently cdrdao to think it needed to burn the tracks one way (e.g., from a WAV file instead of from raw audio data (PCM)). Looking at the specifications for ISO9660, it doesn't seem like that would matter because it actually says they should encode "one after the other" either directly or through "path tables." However, none of these files look like ISO in any way shape or form, and like I said, these discs/readers kind of predate the CD-ROM format, let alone ISO9660. I'll update this gist with my findings and write a blog post about it soon enough.

EDIT: Yeah, mystery has been solved. The bin tracks are raw 16-bit CD PCM in le. SoX plays them when I give it the right parameters (no headers, remember? So I have to tell it 2 channels, 16 bit, 48k (44.1)--basically what a CD is). So, in this case, whoever or whatever software ripped these from the CD did the byteswap. It's interesting because the dumpers/archivers who made these think that they're perfect rips ;). They're not if they're byte swapped, ya know? Anyway, so cdrdao assumed that they were in big endian format because that's the RedBook CD Audio spec for raw PCM digital sound, and it just happily burned them without doing the byte swap. Telling cdrdao to --swap is actually doing the right thing and turning them into big endian samples and writing them to the disc.

Oh, and being the first consoles to use CDs...the music in Gate of Thunder really is dope!

Sample run (I changed to do the real write at 32x, though, just to see how it worked):

$ ~/consolehacking/burnDuo.bash ./Lords\ of\ Thunder\ \(USA\).zip 
No errors detected in compressed data of ./Lords of Thunder (USA).zip.
Archive:  .././Lords of Thunder (USA).zip
TORRENTZIPPED-FBDF10AF
  inflating: Lords of Thunder (USA) (Track 01).bin  
  inflating: Lords of Thunder (USA) (Track 02).bin  
  inflating: Lords of Thunder (USA) (Track 03).bin  
  inflating: Lords of Thunder (USA) (Track 04).bin  
  inflating: Lords of Thunder (USA) (Track 05).bin  
  inflating: Lords of Thunder (USA) (Track 06).bin  
  inflating: Lords of Thunder (USA) (Track 07).bin  
  inflating: Lords of Thunder (USA) (Track 08).bin  
  inflating: Lords of Thunder (USA) (Track 09).bin  
  inflating: Lords of Thunder (USA) (Track 10).bin  
  inflating: Lords of Thunder (USA) (Track 11).bin  
  inflating: Lords of Thunder (USA) (Track 12).bin  
  inflating: Lords of Thunder (USA) (Track 13).bin  
  inflating: Lords of Thunder (USA) (Track 14).bin  
  inflating: Lords of Thunder (USA) (Track 15).bin  
  inflating: Lords of Thunder (USA) (Track 16).bin  
  inflating: Lords of Thunder (USA) (Track 17).bin  
  inflating: Lords of Thunder (USA) (Track 18).bin  
  inflating: Lords of Thunder (USA) (Track 19).bin  
  inflating: Lords of Thunder (USA) (Track 20).bin  
  inflating: Lords of Thunder (USA) (Track 21).bin  
  inflating: Lords of Thunder (USA) (Track 22).bin  
  inflating: Lords of Thunder (USA) (Track 23).bin  
  inflating: Lords of Thunder (USA).cue  
Cdrdao version 1.2.3 - (C) Andreas Mueller <andreas@daneb.de>
/dev/sr0: PLEXTOR DVDR   PX-Q840U       Rev: 1.05
Using driver: Generic SCSI-3/MMC - Version 2.0 (options 0x0000)

Starting write simulation at speed 16...
Pausing 10 seconds - hit CTRL-C to abort.
Process can be aborted with QUIT signal (usually CTRL-\).
WARNING: No super user permission to setup real time scheduling.
Turning BURN-Proof on
Enabling JustSpeed.
Writing track 01 (mode AUDIO/AUDIO )...
Writing track 02 (mode MODE1_RAW/MODE1_RAW )...
Writing track 03 (mode AUDIO/AUDIO )...
Writing track 04 (mode AUDIO/AUDIO )...
Writing track 05 (mode AUDIO/AUDIO )...
Writing track 06 (mode AUDIO/AUDIO )...
Writing track 07 (mode AUDIO/AUDIO )...
Writing track 08 (mode AUDIO/AUDIO )...
Writing track 09 (mode AUDIO/AUDIO )...
Writing track 10 (mode AUDIO/AUDIO )...
Writing track 11 (mode AUDIO/AUDIO )....
Writing track 12 (mode AUDIO/AUDIO )....
Writing track 13 (mode AUDIO/AUDIO )....
Writing track 14 (mode AUDIO/AUDIO )....
Writing track 15 (mode AUDIO/AUDIO )....
Writing track 16 (mode AUDIO/AUDIO )....
Writing track 17 (mode AUDIO/AUDIO )....
Writing track 18 (mode AUDIO/AUDIO )....
Writing track 19 (mode AUDIO/AUDIO )....
Writing track 20 (mode AUDIO/AUDIO )....
Writing track 21 (mode AUDIO/AUDIO )....
Writing track 22 (mode AUDIO/AUDIO )....
Writing track 23 (mode MODE1_RAW/MODE1_RAW )...
Wrote 553 of 553 MB (Buffers 100%  99%).
Wrote 246869 blocks. Buffer fill min 93%/max 100%.
Flushing cache...
Simulation finished successfully.
Cdrdao version 1.2.3 - (C) Andreas Mueller <andreas@daneb.de>                                            
/dev/sr0: PLEXTOR DVDR   PX-Q840U       Rev: 1.05                                                        
Using driver: Generic SCSI-3/MMC - Version 2.0 (options 0x0000)                                          
                                                                                                         
Starting write at speed 32...                                                                            
Pausing 10 seconds - hit CTRL-C to abort.                                                                
Process can be aborted with QUIT signal (usually CTRL-\).                                                
WARNING: No super user permission to setup real time scheduling.                                         
Turning BURN-Proof on                                                                                    
Enabling JustSpeed.                                                                                      
Executing power calibration...                                                                           
Power calibration successful.                                                                            
Writing track 01 (mode AUDIO/AUDIO )...                                                                  
Writing track 02 (mode MODE1_RAW/MODE1_RAW )...                                                          
Writing track 03 (mode AUDIO/AUDIO )...                                                                  
Writing track 04 (mode AUDIO/AUDIO )...                                                                  
Writing track 05 (mode AUDIO/AUDIO )...
Writing track 06 (mode AUDIO/AUDIO )...
Writing track 07 (mode AUDIO/AUDIO )...
Writing track 08 (mode AUDIO/AUDIO )...
Writing track 09 (mode AUDIO/AUDIO )...
Writing track 10 (mode AUDIO/AUDIO )...
Writing track 11 (mode AUDIO/AUDIO )....
Writing track 12 (mode AUDIO/AUDIO )....
Writing track 13 (mode AUDIO/AUDIO )....
Writing track 14 (mode AUDIO/AUDIO )....
Writing track 15 (mode AUDIO/AUDIO )....
Writing track 16 (mode AUDIO/AUDIO )....
Writing track 17 (mode AUDIO/AUDIO )....
Writing track 18 (mode AUDIO/AUDIO )....
Writing track 19 (mode AUDIO/AUDIO )....
Writing track 20 (mode AUDIO/AUDIO )....
Writing track 21 (mode AUDIO/AUDIO )....
Writing track 22 (mode AUDIO/AUDIO )....
Writing track 23 (mode MODE1_RAW/MODE1_RAW )...
Wrote 553 of 553 MB (Buffers 100%  99%).
Wrote 246869 blocks. Buffer fill min 93%/max 100%.
Flushing cache...
Writing finished successfully.

So, when I originally wrote these without --swap I just heard noise for music tracks. Here's the relevant section of the cdrdao man page that lead me to suspect something like this, in the section about the toc language/file format:

              Audio files may have raw or WAVE format with 16 bits per sample, 44.1 kHz sampling rate,
              stereo. Raw files must have the layout 'MSBLeft  LSBLeft  MSBRight  LSBRight  ...'  (big
              endian byte order). WAVE files are expected to have little endian byte order. The option
              --swap reverses the expected byte order for all raw and WAVE files. Only filenames  with
              a  ".wav"  ending are treated as WAVE files, all other names are assumed to be raw audio
              files. Use tools like sox(1) to convert other file formats to supported formats.

So, either it's misinterpreting these *.bin files as WAV (possibly due to the way the cue file is converted to toc) or it's raw audio data ripped as little endian that cdrdao is assuming is big endian and reversing it normally (when we don't use --swap). Heh. It's the nature of endianness that it's hard to tell, but I should be able to compare my two versions and dig around to find out. I've seen recommendations on forums for PSX and PCE CD-ROM ripping to be done with a similar "reverse" flag (driver flag 0x2 or something in generic-mmc on cdrdao), so it could be that these were ripped in reverse to begin with :).

Oh yeah. I'm guessing these work with emulators and that those way be the main target and not making real physical CDs. Would also be educational to experiment with mednafen and the originals compared with a rip of my build.

SoX is awesome. Gave me a big clue right in the manual.

.cdda, .cdr

‘Red Book’ Compact Disc Digital Audio (raw audio). CDDA has two audio channels formatted as 16-bit signed integers (big endian)at a sample rate of 44.1 kHz. The number of (stereo) samples in each CDDA track is always a multiple of 588.

Looking forward to digging into it more later. I'm beat today.

mystery has been solved. The bin tracks are raw 16-bit CD PCM in le. SoX plays them when I give it the right parameters (no headers, remember? So I have to tell it 2 channels, 16 bit, 48k (44.1)--basically what a CD is). So, in this case, whoever or whatever software ripped these from the CD did the byteswap. It's interesting because the dumpers/archivers who made these think that they're perfect rips ;). They're not if they're byte swapped, ya know? Anyway, so cdrdao assumed that they were in big endian format because that's the RedBook CD Audio spec for raw PCM digital sound, and it just happily burned them without doing the byte swap. Telling cdrdao to --swap is actually doing the right thing and turning them into big endian samples and writing them to the disc.

Hey, this worked perfectly! Thank you!