PCSX2 vs Hardware Texture Differences (with DMA dumps)

ps2-homebrew-texture-issue.md

PCSX2 vs Hardware Texturing Difference (Homebrew Confusion)

I have noticed different ST coordinate behavior on PCSX2 vs hardware PS2 for texturing.

Problem Description

ST coordinates on hardware behave as if they have 1/2 the texel width and height. Eg, only half of the texture can be addressed in the ST coordinate range [0,1] on each axis (so only 1/4 of the total texture can be addressed!)

Here are images captured on emulator vs hardware demonstrating the problem:

Demo 1

In demo 1 the top left corner has ST coordinate (0,0), the bottom left has ST coordinate (1,1)

PCSX2

PS2

Demo 2

In demo 2 the top left corner has ST coordinate (1,1), the bottom left has ST coordinate (2,2).

PCSX2

PS2

How To Run Yourself

Checkout this commit of this repo: https://github.com/phy1um/ps2-homebrew-livestreams/tree/a16848c3bf0ad479755b3ab1213e3e3c4b995176

Then either compile using pre-installed PS2SDK and extras, or for simplicity do:

make docker-image
make assets
make docker-elf

If you have PCSX2 installed and on your path, test easily with make run

To run on hardware manually launch the file dist/test.elf with ps2client. There are Makefile rules for this - PS2HOST=x.x.x.x make runps2 and make resetps2.

This program cycles between 2 different ST coordinates for the bottom-right corner as you press X. First (1,1) then (2,2).

Check the function sprite(...) defined in script/draw2d.lua for how the buffers are being constructed. It's easy to poke around and change things if you understand how GS works (I hope!)

Annotated DMA Transfer (to GS)

The following is an annotated capture of the buffer being sent to GS each frame, from PCSX2 stdout.

-- clear the screen (from PS2SDK)
1000000000000001 000000000000000e
000000000003200f 0000000000000047
1000000000000001 000000000000000e
000000000003200f 0000000000000047
1000000000000002 000000000000000e
0000000000000000 000000000000001a
0000000000000400 000000000000001b
1000000000000002 000000000000000e
0000000000000006 0000000000000000
3f800000802b2b2b 0000000000000001
2400000000000014 0000000000000055
00000000f1f9ebf9 000000000dfaedf9
00000000f1f9edf9 000000000dfaeff9
00000000f1f9eff9 000000000dfaf1f9
00000000f1f9f1f9 000000000dfaf3f9
00000000f1f9f3f9 000000000dfaf5f9
00000000f1f9f5f9 000000000dfaf7f9
00000000f1f9f7f9 000000000dfaf9f9
00000000f1f9f9f9 000000000dfafbf9
00000000f1f9fbf9 000000000dfafdf9
00000000f1f9fdf9 000000000dfafff9
00000000f1f9fff9 000000000dfa01f9
00000000f1f901f9 000000000dfa03f9
00000000f1f903f9 000000000dfa05f9
00000000f1f905f9 000000000dfa07f9
00000000f1f907f9 000000000dfa09f9
00000000f1f909f9 000000000dfa0bf9
00000000f1f90bf9 000000000dfa0df9
00000000f1f90df9 000000000dfa0ff9
00000000f1f90ff9 000000000dfa11f9
00000000f1f911f9 000000000dfa13f9
1000000000000001 000000000000000e
0000000000000001 000000000000001a
### my stuff begins here ###
-- set texture registers
1000000000000004 000000000000000e
-- set TEXA to 0x80,0x80 (no transparency by default)
0000008000000080 000000000000003b
-- set TEX1...
0000000000000101 0000000000000014
-- set TEX0
-- image has width=64, height=64, PSM=32 so we expect to see 
-- TBW = 0x1
-- the TW and TH fields set to 0x6
-- this looks correct to me!
0000000598007499 0000000000000006
-- set PRIM register for SPRITE with texturing flag on
0000000000000016 0000000000000000
-- 1 loop setting 6 registers - ST, RGBAQ, XYZ2x2
6000000000000001 0000000000512512
-- ST = 0,0 Q = 1.0
0000000000000000 000000003f800000
-- RGBA = (0x80,0x80,0x80,0x80)
0000008000000080 0000008000000080
-- top left corner coordinate
00007e8000007880 0000000000000000
-- ST = 1.0, 1.0 Q = 1.0
3f8000003f800000 000000003f800000
-- RGBA = (0x80,0x80,0x80,0x80)
0000008000000080 0000008000000080
-- bottom left corner coordinate
00008b0000008500 0000000000000000
-- draw finish
1000000000008001 000000000000000e
0000000000000001 0000000000000061

Obvious Solution

The obvious but not very satisfying way to solve this is to simply add 1 to the TW and TH fields of the TEX0 register when running on real hardware. This is equivalent to doubling the width and height. This sucks but is easy to implement and produces the correct results on hardware (but incorrect results on PCSX2).

PS2 Snapshots with TW+1, TH+1

ST coordinates (0,0) to (1,1)

ST coordinates (0,0) to (2,2)

I am not sure if there is a difference in my DMA buffer data between PS2 and PCSX2 as ps2client is not printing to stdout. CI would appreciate if someone ran this code on hardware and pastee a DMA buffer dump from a single frame in the comments for comparison!

Hey there, this is interesting. Is there any chance you can provide a precompiled version in your dist folder?

Download link to pre-compiled dist/ folder:
EDIT: link removed, see updated version in comment below

Thanks, but uhh, I just ran the ELF on PCSX2 and this was the result

Just double checked hardware mode, that was correct too.

Edit didn't realise there was more

Second test result

Third test result

Which ones are incorrect (aside from the obvious not stretching out the final pixels like the PS2, maybe that's because you haven't set the clamping modes? (If so the clamping done will be whatever it was when you ran ps2client, PCSX2 won't have set those. I didn't check the code to see if you have)

Ah sorry the first test is confusing things. My demo numbering here didn't account for the first "test" in the script which had ST coordinates (0.5, 0.5). I didn't have any screenshots of this one. I've removed the confusing test case, running this will now have tests that correspond to my Demo 1 and Demo 2 in order.

Hope this makes sense. Try again with this version:
https://mega.nz/file/RuoWHYLY#QTY9PbY3joQpd440H_Yo0X2Hw2vnQqLUxlOVD-uHzhk

Yeah now it's doing it as you showed. however I'm confused.

the point in ST being 0-1 is it's normalised, being 0 is the top left of the texture and the bottom right of the texture is 1, it shouldn't be 1/4. If we were this broken, 90% of games would look incorrect, so I suspect something else is going on here, but I couldn't tell you what right now, I'll look in to it at some stage :)

Okay, I think I've found one thing, at least which is weird!

this block

0000000000000000 000000000000001a <-- sets PRMODECONT to "PRMODE" mode, which masks some parts of PRIM
0000000000000400 000000000000001b <-- Enables FIX fragment value control on DDA (this is a write to PRIM)
1000000000000002 000000000000000e
0000000000000006 0000000000000000 <-- Sets PRIM Type to Sprite
3f800000802b2b2b 0000000000000001 <-- Set RGBAQ
2400000000000014 0000000000000055 <-- Tag to say the following data is all vertex data
00000000f1f9ebf9 000000000dfaedf9 <-- WHAT THE HECK!??
00000000f1f9edf9 000000000dfaeff9
00000000f1f9eff9 000000000dfaf1f9
00000000f1f9f1f9 000000000dfaf3f9
00000000f1f9f3f9 000000000dfaf5f9
00000000f1f9f5f9 000000000dfaf7f9
00000000f1f9f7f9 000000000dfaf9f9
00000000f1f9f9f9 000000000dfafbf9
00000000f1f9fbf9 000000000dfafdf9
00000000f1f9fdf9 000000000dfafff9
00000000f1f9fff9 000000000dfa01f9
00000000f1f901f9 000000000dfa03f9
00000000f1f903f9 000000000dfa05f9
00000000f1f905f9 000000000dfa07f9
00000000f1f907f9 000000000dfa09f9
00000000f1f909f9 000000000dfa0bf9
00000000f1f90bf9 000000000dfa0df9
00000000f1f90df9 000000000dfa0ff9
00000000f1f90ff9 000000000dfa11f9
00000000f1f911f9 000000000dfa13f9

So yeah, it uses FIX mode in PRIM for the DDA, which I don't believe we emulate (nor have I ever found anything that does use it lol), and I have no CLUE What that data is, the values are certainly much different values to when you draw your sprite.

I'm not sure if this is the issue, but that's definitely something PCSX2 doesn't support :D If I'm honest, I'm not even sure of its behaviour

A couple of other things I've noticed

XYOFFSET: You don't set this, so it is whatever ps2client sets it to
SCISSOR: You don't set this either, so it's set to whatever ps2client uses
FRAME: You don't set this either, so again it'll be whatever ps2client set it to, i guess you're just lucky it draws to the where the DISPFB framebuffer is pointing :P

That opening part of my drawbuffer is for clearing the screen in strips/patches, it comes from PS2SDK so not sure why it sets FIX like that lol. The XYZ data you're labelled "WHAT THE HECK" is whatever batches of geometry they clear the screen with, not sure why it isn't done with one big rect but I assume there is a good reason.

XYOFFSET, SCISSOR and FRAME are set with a PS2SDK function called in some C setup in gslua.c, the PS2SDK source is here:

https://github.com/ps2dev/ps2sdk/blob/11500fe1dcec03592e71406cbd9d34b39aa52589/ee/draw/src/draw.c#L61-L79

I override XYOFFSET after this to be (2048 - width/2, 2048 - height/2) after this PS2SDK setup.

Ah okay, that would explain why they aren't in your printouts! I still wonder what the FIX function does differently, I mean I guess it's working, right??

I need to run this test up on my PS2 at some point and see what it's doing on there, Everything seems to look okay, so this could be very strange.

Hey, sorry in the MASSIVE delay in getting back to you, but I think I've solved the mystery.

So you are setting a 64x64 texture and displaying that whole texture but stretched to 200,200 -> 400, 400, right?

Okay, so in your draw2d.lua as part of draw:sprite you do this

self.buf:settex(0, pb, pw, tex.format, math.floor(log2(tex.width)), math.floor(log2(tex.height)), 0, 1, 0, 0, 0)

the problem is, on the PS2, because floats SUCK and aren't as accurate as IEEE ones, it is setting W and H to 5 (32x32), PCSX2 sets it to 6 (64x64), however PCSX2 can simulate this by using the interpreter (which kinda happens to suck too, in a different way lol).

I suspect if you replaced the floor with a round, it would probably get a more accurate result as I'm thinking the log2 is returning 5.999999 or something :) The other option is putting a bias on it, maybe of just a pixel or two, might be enough.

I hope this helps!