If in doubt, always call your vet or the Animal Poison Line immediately.
Like all stone fruit pits, plum stones contain amygdalin. They can also lodge in the intestines causing a dangerous obstruction. Always remove before feeding.
If the pit was swallowed, contact your vet : an x-ray may be needed to check for blockage.