The GE difficulty will increase accordingly every time you do anything to add to your A/D boosts whether it is from special event buildings or leveling up GBs, especially the level 4 difficulties. I've found this to be the case through nearly 17 levels (or higher) on Zeus, CdM, CoA and even TA at level 11 currently. Inno makes sure you will never get "ahead of the game" in the GE..although those A/D bonuses do help you considerably in plundering your neighbors successfully, which is and will always be the one "pet" area of this game to Inno.