"And I saw no temple therein: for the Lord God the Almighty, and the Lamb, are the temple thereof." Revelations 21:22. The Lamb seems to be the Christ's lawful assembly which is said to be part of the Household of God. The Papal claim of being the Vicar of Christ might be that which obscures the significance of being of the Christ's lawful assembly. It is rather clear that Jesus remains alive having ascended, the saints remain ambassadors/agents on the planet. Perhaps it possible for a king to be in Christ and his subjects to be without?
This might give insight:
The Church: the Household of God