It's not possible to delete the Default Price Formula, but you can edit it. You need to always have an active Price Formula.
So, when no other formula is assigned to a product the Default Price Formula will be automatically applied.
Each store has its own default formula.