VCE YUV rom table dump
Okay, I think I got this figured out. I compared the patent table of the RGB to YUV matrix rom to that of just YUV clipped to 5bits. There’s rounding in there. Not just rounding, but some weird bias in the first all blue level entries (didn’t seem to follow any logic that I could see). Something I really didn’t expect to see that. That makes it a little problematic for trying to model the YUV conversion process.
You could just set a weight to round all values that aren’t in the patent table, to 5bit. But there’s another way.
The VCE doesn’t handle complete quadrature modulation inside the chip. The last step is done externally, right before the amp. Since it’s just a simple addition (you just simply tie the lines together. Remember; amplitudes add). But the 90 degree phase shift and the amplitude modulation of subcarrier sinewave is (handled internal, that is). So, you actually have all three independent Y,U,V outputs. Even the color burst reference is on its own output (but that’s irrelevant for this).
What that means is; is that I can cut the trace of either U or V component of the subcarrier and easily just look at the amplitude modulation. It’s 5bit, so I should be able to clearly see the stepping on my oscilloscope if I stretch it vertically. I’ve already removed U and V independently on my Duo unit before (it’s pretty cool looking :D). So, I just need to do this for Y, then U, then V.
So, I can reliably dump the whole table by hand. That only downside is that I’d be writing 1536 entries down on paper. But yeah, beggers can’t be choosers (that would be me 😉 ). I’ll take this over $600+ decap and micro scope pics.
Edit: Oh yeah, I wanted to mention that just looking at the first few and last few entries in the patent matrix rom table – the Luma is full range. It’s not compressed to 16 – 225. It’s full 0 to 255 (in 5bit steps). That’s important to know. Though, I just figured that out a few hours before hand when messing with the capture card. But it’s nice have a direct second source confirmation of this.