The logo, called Harmonized Chequered Emblem, replaces the first choice which was thrown out last year after the designer was accused of plagiarism.
The designer denied stealing the idea.
Organisers said the new design used traditional Japanese colours and patterns to represent the intercultural themes of the Games.
"It incorporates the message of 'unity in diversity'," they said, and the idea that the Games "seek to promote diversity as a platform to connect the world".
When designer Asao Tokolo found out he had won the re-opened contest he said "my mind has gone blank".
"I put a lot of time and effort into this design as though it was my own child."
The first design was rejected after Belgian artist Olivier Debie alleged it copied his design for a theatre logo.
The Games organising committee never agreed to the allegation of plagiarism but said there were too many doubts over the emblem for it to be used.
The logo dispute came shortly after Tokyo decided to scrap the designs for the main Olympic stadium because of spiralling construction costs.
A cheaper design, by Japanese architect Kengo Kuma, was chosen in December last year.

The average mileage for four-wheeled vehicles stood at 7,900 miles (12,700km) in 2013, official figures showed.
This represents a 14% fall from a figure of 9,200 miles in 2002, the first year for which data is available.
Average annual mileage has dipped steadily in the intervening period, with mileage rising only in 2007.
The average figure for 2013 fell from the previous year even though the distance travelled when commuting to and from work increased from 2,500 miles in 2012 to 2,800 miles last year.
Business mileage dropped from 800 miles in 2012 to 700 miles in 2013. In 2002, this figure stood at 1,300 miles.
RAC Foundation director Professor Stephen Glaister said: "While it is important to see whether individuals are driving more or less than before, official figures show overall traffic volume is on the rise again following the end of the recession.
"With the population expected to increase by 10 million over the next two decades congestion is going to get worse, not better."
AA president Edmund King said: "The weather in 2013, spikes in fuel prices and an increase in online shopping have all contributed to the drop in car use.
"Although the coldest March in 50 years will have contributed to the fall in petrol sales in 2013, by far the biggest cause over the first quarter was the third 8p to 10p-a-litre price swing in 12 months."
The figures also showed that people in England travelled 6,592 miles on average in all modes of transport last year, including walking and travelling as a passenger in a car or van.
This figure is lower than the headline 7,900-mile figure for drivers, which applies to each vehicle, many of which are driven by more than one person.
Each person on average walked 184 miles and travelled 288 miles by bus.
The largest users of cars and vans were those living in rural areas, who walked only 97 miles on average last year.
Those living in London walked the most, averaging 215 miles, while those in the West Midlands walked the least, clocking up only 161 miles.
Car and van occupancy has not changed in recent years with the 2013 average being 1.57 people per journey, the same as in 2012 and 2011.

Gloucestershire Police said they had received allegations that Opus Fine Art in Stow-on-the-Wold had been "dishonestly retaining or selling" art.
The gallery deals in works by high profile artists including Damien Hirst.
A police spokesman said investigations were ongoing to establish ownership of the seized property.
The Independent newspaper reported the owners, Donald Smith and Emma Poole, were believed to have moved to the south of France.

25 October 2013 Last updated at 09:08 BST
The mummies of an adult and a child were found at an old religious site in the capital Lima.
Experts also found other offerings including the remains of guinea pigs and jars with cat designs.
They're being described as one of the most important finds in more than 30 years.
Check out Ricky's report to find out more about mummies and where else in the world they've been found...
Japan's Olympic organisers have unveiled the new official logos of the 2020 Tokyo Olympic and Paralympic Games.
[NEXT_CONCEPT]
Drivers in England are clocking up the lowest number of miles on average since records began 12 years ago.
[NEXT_CONCEPT]
A large number of works of art have been seized from a Cotswold art gallery after allegations of fraud and theft.
[NEXT_CONCEPT]
Two ancient mummies more than a thousand years old have been found in Peru in South America.