These items generally give you some kind of a boost to your stats, or add a special passive ability. They take effect immediately once picked up, and continue for the rest of the game (except for several items that do not provide any lasting effect).

Edmund McMillen split them up in his blog into the subcategories "weapon items" (Such as Number One),"stat padding items" (Such as Growth Hormones), "passive items" (Such as Mitre), and Familiars (Such asLittle Gish).