I thought only the automatics from those years used the 4.10 final drive? Or was it a 4.11?

The manual Impreza RS used 4.11(111111...), dunno if that's a doable swap for you, you'd have to find a front diff that would swap in, the rears are all interchangeable pretty much as far as the ring and pinion are concerned.

Diff numbers are a fun exercise in mathematics...

The final drive of the stock SVX is listed as 3.54, but is actually 3.5454545454545454 and on into infinity. The 3.90 gears are actually just 3.90. It's an interesting mathematical problem if you decide to use a reduction center diff like the '02+ WRX, it's 3.90 in the front, 1.1:1 in the center, and 3.54545454 in the rear. Makes me want to do the pee-pee dance.