After taking the rear bearing apart I got to see why it was making so much noise. Honestly the thing was roaring at 30 mph.
This is the outter race of the old bearing:
My guess is that water got into the bearing and rusted out the inside. A testament to how long to car sat on a lot/in someone's driveway. Some of the rollers are also completely roughed up. There's general small scratches and nicks all around the inside of the race.
Once my new one arrives it'll all go back together with new nuts and bolts, all of the rust polished off the knuckle and the knuckle painted. I'm worried that I'm going to be speeding though. The bearing was sort of my audio speedometer. Without the noise I might end up going a bit faster than I mean to.