The past perfect part of the sentence always shows the event that happened first. We hadn’t finished cleaning by the time the guests arrived. By the time the guests arrived, we hadn’t finished ...