You can only turn on/off the vibration feature if you are using a DualShock 3 controller. Not all games have DualShock 3 compatibility so and therefore turning on/off the vibration feature will have no effect. Changing the setting on one controller will affect the vibration settings for all controlls that support vibration feedback.
To turn on/off the vibration feature, press the PS button and select [Controller Settings]. Then, select [Vibration Function] and select "On"/"Off" to turn on/off to change the vibration settings.