While there are many reasons why a city would want to host the Olympic Games, most have to do with money and prestige: The Olympics bring international attention to the host city for two weeks.
Consequently, the city has to look its best, but preparations for the spotlight go beyond the cosmetic and often entail the adoption of social values shared by the international community, or, at least, that part of the international community with power and influence. Conversely, when Sochi hosted the Winter Olympics last year, the Russian government undermined the games' image by passing a law beforehand prohibiting the "promotion" of gay lifestyles, a move that offended many people. Some European officials ended up boycotting the opening ceremony in protest.
But social engineering isn't as straightforward as civil engineering. Ever since Tokyo won the right to host the 2020 Games, various groups have been talking about what the city, and Japan as a whole, needs to do before all these foreign visitors show up. Infrastructure-wise, there is talk about speeding up the renovation of train lines and expressways to facilitate transportation in and around Tokyo, expanding wireless broadband service and programming more ATMs to accept foreign credit cards. One group even wants to remove the expressway that was built above Nihonbashi prior to the last Tokyo Olympics in 1964 to make the area more attractive.