You can just delete the row that the student belongs to.
Yes but if I try to insert a row (to preserve alphabetical order) it won’t calculate correctly
You can delete rows but not add rows. To preserve alpha order you can use the sort option on the add-on menu > GradeBook > sort
And that will keep the grades with the correct student?