The A4 is the chip and was first used in the iPad, the iPhone 3GS has a regular ARM chip. The A4 is an in-house designed extension to the basic ARM core, much like the Scorpion processor in the Snapdragon platform used in some Android devices.
Like the Scorpion, the A4 takes the basic ARM used in devices like the 3GS and optimises it, including h.264 decoding and improvements in power consumption to give it a longer battery life. It will be interesting to see what the A4 actually brings the iPhone 4, in terms of performance and battery life, over the 3GS.