This situation is the perfect example of the random process that is called genetic drift. A minor, usually, form of evolution that works best in small populations or in situation as you have described above.
I hope you get this answer right! Have a great day!:DDDD :)!