The shotgun is picky on what it can be modded to fire. I know it can fire arrows, and explosive arrows. I know it can't fire bullets, and I'm not sure about grenades or clusters.
If you want a 5 shot auto grenade thrower, just modify an automatic weapon. Take your pick from the Mp5, AK, Steyr, Minimi and the Minigun. Set its ammo to 5, and bullet style to either 2 or 4.