https://en.wiki.forgeofempires.com/index.php?title=Player%27s_Ranking#Buildings_points
According to the above link, building's ranking points are calculated by: Width * Length * Building Type Factor * Age Factor.
Does this mean that shrinking a Rogue Hideout would subtract your Ranking Points...