unit: Fix test failures with glib 2.39.0

unit: Fix test failures with glib 2.39.0

glib 2.39.0 made this change:

   - g_source_remove() will now throw a critical in the case that you
     try to remove a non-existent source.  We expect that there is some
     code in the wild that will fall afoul of this new critical but
     considering that we now reuse source IDs, this code is already
     broken and should probably be fixed.

This patch fixes the test suite to keep better track of whether sources have
already been removed and avoid double-removals.