N64 Controller
Line protocol
The console/controller communication is performed using a one-wire, half-duplex serial protocol. The data line idles high. Data is transmitted at a 250KHz baud rate, with a '1' bit represented by 1uS low, 3uS high, and a '0' bit represented by 3uS low, 1uS high. Data is transmitted MSB first. Transmissions begin at the first negative edge and continue until a stop bit is transmitted. Stop bits are only 3uS, which is important to pay attention to, especially when operating in host (i.e. console) mode, because the controller can (and often will) respond immediately. The console transmits a stop bit as 1uS low, 2uS high, the controller transmits as 2uS low, 1uS high.
It is possible to implement this communication on a microcontroller by (ab)using a standard UART module. Although this line protocol is not standard UART, it is possible to very closely mimic it by converting 2 bits of controller data into 1 byte of UART data. Unfortunately, due to the start and stop bits which result in a 10-bit-per-byte transmission, it is not possible to get the exact duty cycle we need, so instead of 1uS and 3uS, we get 0.8uS and 3.2uS, respectively. However, this seems to still work just fine.
In order to wire this up, we need to tie the UART's Rx and Tx pins together using a Schottky diode, as shown here
Once everything is wired up, data can be sent and received using the following values:
N64 | UART
0b00 | 0x08
0b01 | 0xE8
0b10 | 0x0F
0b11 | 0xEF
(Note that the N64 data is MSB first, UART data is LSB first)
One thing to be careful of when doing this is that because the Rx and Tx lines are connected together, any data sent out from the UART will also be received back, so if you are using Rx interrupts, they should be disabled during transmission, and the Rx buffer should be cleared at the end of the transmission before re-enabling them (or, if you can disable the Rx function entirely, that works too). You also need to be very careful about the timing when doing so, because stop bits are only 3uS (2 low, 1 high) which means you can't just read out the Rx buffer normally. On the PIC18F25K42, I did this by waiting for the TX buffer to be empty, indicating that the stop bit had been loaded into the Tx shift register, then waiting by manually polling the Tx pin until it went low, then high. Only then did I re-enable the UART's Rx buffer and interrupt. The timing is pretty tight because the controller can respond immediately after the 1uS high period, but I was able to make it work.
Data protocol
The console polls the controller by sending a 1-byte command. Some commands are followed by parameters, which may vary in length. The controller response varies, depending on the command.
Command 0x00 [Status]
The controller responds with 3 bytes. For a standard gamepad, the first two bytes are 0x05, 0x00. The third byte contains bit flags. 0x04 indicates an address CRC error in the last communication. This flag is cleared upon transmission, so it is only reported once. 0x02 indicates a controller back has been removed. If no pak is inserted at power-on, the value is 0[1]. 0x01 indicates a controller pak (or rumble pak) is inserted. When a pak is inserted or removed, the new flag is set before the current one is cleared, so they are both set for one polling cycle.
[1] This may be due to a glitch in my testing setup, other documentation claims this bit is set to 1 at power-on. It may also be due to an as-of-yet undocumented command which my test bench consistently triggers at start-up. When I start up my test bench, I receive 0x00 for the third status byte, even after unplugging and re-plugging the controller itself (without resetting the test bench). Then, once I insert and remove a controller pak, I start to receive 0x02, even after unplugging/re-plugging the controller (again, without resetting the test bench). So, the state of this bit is actually latched even through a power cycle of the controller itself. Really weird... I've tried sending multiple stop bits in a row and holding the data line low for >1ms to try and recreate what might be happening to that line on startup, but so far I'm unable to recreate the result intentionally through any other means than resetting my test bench.
Command 0x01 [Poll]
The controller responds with 4 bytes containing the button and axis states.
[1] Reset is '1' when L+R+Start are all pressed. When Reset is '1', L and R are both reported as '1' but Start is reported as '0'.
[2] Unknown, always '0' for normal gamepads
Command 0x02 [Read]
The command is followed by a two-byte address and CRC (reads are always 32-byte aligned, so the lowest 5 bits of the address are masked off, and the CRC is stored there). The controller responds with 32 bytes of data read from the specified address and a CRC byte. If no controller pak is inserted, the controller responds with 33 bytes of 0x00, regardless of the validity of the address CRC. If there is a controller pak inserted, but the address CRC is incorrect, the controller responds with 32 bytes of 0x00 and a CRC of 0xFF. The OEM controller responds to this command within 7uS of the host stop bit (10uS between stop bit negedge and first data bit negedge).
Command 0x03 [Write]
The command is followed by a two-byte address and CRC in the same format as command 0x02, then 32 bytes of data. The controller responds with a CRC byte, regardless of whether or not the address CRC is valid. However, if the controller pak removed flag is set (status bit 1), the CRC is XORed with 0xFF. The OEM controller responds to this command within 4uS of the host stop bit (7uS between stop bit negedge and first CRC bit negedge).
Command 0xFF [Reset]
Recenters the joystick at its current position, then responds identically to command 0x00 (including clearing the address error flag if set).
Joystick encoder
Pinout:
Pin 1 on the left(marked with a white wire)
P1: Y1
P2: Y0
P3: X1
P4: Gnd
P5: Vcc
P6: X0
Waveforms
Each waveform is an attempt to quickly move the joystick from its resting centered position all the way to the full extent of a single axis. The irregularity in pulse width is entirely due to inconsistent speed in moving the joystick. Due to imperfect movement, there are a few pulses on the axis I wasn't testing, they can be ignored.
Legend
D0: X0
D1: X1
D2: Y0
D3: Y1
X-
X+
Y-
Y+
Since the gear arm does appear to pivot around the same central axis as the joystick arm, I believe that it is linear, relative to the angle of the joystick. To calculate the relationship between angle and encoding value, I first need to extrapolate the number of gear teeth on the arm. I was able to get a decent photo of the partial gear and rotate multiple copies of the image in order to produce a complete gear.
It appears that the gear arm contains 60 teeth. This gear arm then drives a smaller gear on the encoder disc, containing 18 teeth, a 10:3 gear ratio. The encoder disc contains 80 slits, or 1 slit per 4.5 degrees.
Based on this, a single encoder step represents 4.5 * 3 / 10 = 1.35 degrees deflection of the joystick. This corresponds to a +/-24.3 degree travel.
In addition to determining that the relationship between stick angle and quadrature pulses is linear, by removing the joystick and manually sending quadrature pulses to the controller, then polling via command 0x01, I was able to confirm that the relationship between quadrature pulses and reported joystick position is also linear. A single pulse increments/decrements the position by a value of 4. The position is reported as a signed (two's complement) 8-bit integer. The OEM controller is capable of reporting values ranging from 0x80-0x7F, but the actual joystick does not cover the entire range. In my testing, the joystick appears to report approximately +/- 18 steps (shown above in the logic analyzer screenshots), for a range of 0xB8-0x48 (+/-72), but that may vary from one joystick to the next, depending on wear.
On power-on, the X-axis initializes to 0xFE, while the Y-axis initializes to 0x00. After resetting via command 0xFF, both axes report 0x00. The reported value does not wrap around, i.e. continuing to send pulses results in reported values 0x00, 0x04, 0x08, . . . 0x7C, 0x7F, 0x7F, 0x7F and 0x00, 0xFC, 0xF8, . . . 0x84, 0x80, 0x80, 0x80.