Hmm.. During these time when for obvious health reasons we as honest rule abiding golfers are required to leave the flags in. Then the situation you described would be a holed shot. Period. If one golfer out of the 4 doesn’t agree then the majority would rule over that call. of it is a 50/50 split then it would have to go to a final ruling by the tournament officials.
Once the restrictions due to COVID are over and the option to pull or not pull are reinstated then I can see that the situation as described would be a missed shot. Since it was a choice to leave the pin in and then having to deal with the consequences of that decision.
Again during these times when required to leave the pin we all should be flexible enough to understand and make the needed exceptions to the sport to be able to still play and compete on even ground.
In addition , Yes there Used to be a rule and penalty for not pulling the Pin when on the green, but remember that rule no longer exists so there are new rules and possible scenarios that will come about due to it.
Rule 13.2a(2), “There will no longer be a penalty if a ball played from the putting green hits a flagstick left in the hole.” Players will still have the option to remove the flagstick or to have someone tend the pin and remove it after the ball is struck.