Fisher suffered their first defeat under manager Ajay Ashanike as they were knocked out of the second round of the FA Vase by Glebe last weekend.
After going unbeaten in six games in October after previous boss Allan Fenn stepped down, the Fish were two goals behind at the break at St Paul’s.
Antony Dwyer gave the visitors the lead from the penalty spot in the fifth minute, before Toby Ajala doubled the advantage ten minutes before half-time.
Luke Thomas scored for the hosts six minutes into the second half, but Glebe held on to their slim lead to progress.
Fisher are back in SCEFL Premier Division action this Saturday when they travel to Punjab United.
Kick-off at the Elite Venue in Gravesend is 3pm.
Image: @brockleylion