My 2004 GX470 has the top "u" shape for the 2" square opening for a hitch. I question if there is a hitch
available that will handle a bike rack without going for the full 5000# capacity trailer hitch.

There is currently a big selection (well, a selection of four is big) of hitches for your 04 GX470 at http://www.etrailer.com/hitch-2004_Lexus_GX470.htm where I bought a hitch for my 00 LS some years ago.

All the hitches are 5,000# or more. I've noticed that hitches from Lexus and Toyota, when available, sometimes fit and look better than the ones from the aftermarket.

I primarily use the hitch on my 00 LS to carry a bike rack although I sometimes pull a small trailer and have the car wired for trailer lights using an inexpensive Modulite harness that takes its power off the fusebox in the engine compartment. Here is a photo of the Draw-Tite hitch on my LS with the super cheap bike rack I bought almost 20 years ago.