Grazing and plant poaching are also factors leading to the rare Prince Albert vygie dwindling, researchers said