There was a time when it even had its own International Olympic Committee (IOC) member. The IOA was founded in 1961, largely at the urging of Greek IOC member Ioannis Ketseas and his German friend Carl Diem. It fulfilled the wish of Baron Pierre de Coubertin for an Olympic studies centre. Since then it has become a place of pilgrimage to connect with the true spirit of the Olympic Movement.

In 1966, Prince George of Hanover became IOA President after the death of Ketseas, and an IOC member as a result.

This was possible because of a modification to the Olympic charter in 1966. In a new clause added to rule 10, it stated that "the President of the International Olympic Academy may be elected for the duration of his presidency".

At the Hotel Excelsior in Rome, IOC President Avery Brundage proposed the election of Prince George Wilhelm of Hanover.

The day after the election, Brundage read a telegram from the Prince "communicating his thanks for his election in connection with his presidency of the IOA".

"I hope that the IOA will one day be the bridge between the IOC, the National Olympic Committees and the International Federations. I am convinced that such an institution can only do fertile work in close co-operation," said Prince George early in his presidency.

The Prince was in good company. Also elected at that session was Juan Antonio Samaranch, later to lead the entire Olympic movement.

It was perhaps of little surprise that the Prince was attracted to the IOA. He had been inspired by Salem School, founded by German educationalist Kurt Hahn who later established Gordonstoun, a school favoured by Prince Philip, Duke of Edinburgh.

Prince George later became headmaster at Salem. When he was elected IOA President, it was still very close to nature. Lectures were held in the open air and participants slept in tents pitched on the hillside beneath Mount Kronos and a stone's throw from the site of the Olympic Games of antiquity and archaeological site of Ancient Olympia.

Brundage was delighted with the appointment and, at a meeting that October in Mexico City, told his fellow Executive Board members that the IOA was "the Mecca of the Olympic movement, an excellent way of publicising the Olympic Movement among youth", and insisted: "The idea is still of great importance for the Olympic movement, endeavours to create a real International Olympic Academy should be stimulated, the more so as now we have an ex-officio member of the IOC who is President of the International Olympic Academy."

Brundage undertook to make contact with IOC members in Greece "in an endeavour to elevate the standards of the International Olympic Academy and its educational programme".

The IOC also took a much greater interest with the establishment of a fully fledged IOC Commission chaired by the Dane Ivar Vind in 1967.

Yet the same year, a military coup put the Greek Government on a collision course with the President of the IOA and the Olympic Movement.

There was talk in Lausanne of a separate Olympic institute. Nigerian Sir Ade Ademola insisted, "if there was to be a complete break with the IOA in Olympia the IOC should set up its own". Prince George offered support for the idea which did not endear him to the new regime in Greece.

In 1968, IOC executive board minutes also reveal his suggestion to abolish national anthems and flags.

“I propose, on the contrary, that just like in ancient Greece, each winner be saluted by same flourish of trumpets," he wrote. It echoed an idea proposed by Brundage a few years earlier.

"No Greek should even think of the abolition of national colours," countered Hellenic Olympic Committee President (HOC) Theodosios Papathanasiadis.

As a German, Prince George had broken precedent as the first non-Greek IOC President and paid tribute to the Olympic contribution made by Greece.

"Not being Greek, I think I may say something that my Greek friends would never express, the realisation of the International Olympic Academy is a generous gift of Greece to the world."

Yet now he found that his German nationality became a weapon to be used against him. Some newspapers described him as a "foreigner". Nor did it help that he was uncle to King Constantine.

The Greek Government's general secretary for sport, Lieutenant-Colonel Konstantinos Aslanidis, wrote: "As a Greek, my thought has been that we must make every effort for the IOA to be Greek and to remain Greek, thus helping Greece in its turn by projecting it internationally. If the Prince remains as President of the IOA it will be damaging for Greece if he does not adapt himself to the Greek spirit. The Prince administers the IOC as a German thinking like a German."

Aslanidis insisted he feared the "Olympic spirit would disappear".

Matters were now very fraught.

Although the Prince was re-elected for another term as IOA President, HOC general secretary Epaminondas Petralias was forced to inform him that because of a change in Greek law, it was now "unconstitutional" for a foreign national, albeit one married to a Greek, to remain IOA President.

"The HOC had tried to have the law amended so he could remain president," noted IOC records.

Prince George visited Athens to clarify the situation and later gave evidence to the IOC's own Executive Board. He was followed by Petralias and IOA Dean Otto Szymiczek. They explained the legal situation in Greece.

Petralias tried to smooth the waters and told the IOC his nation "would always welcome and appreciate the advice and direction of the IOC and that there was already excellent co-operation between the IOA and the IOC Commission under Mr. Vind".

The IOC members were clearly annoyed that they had not been consulted, but minutes reveal they appreciated "all the efforts of the Hellenic Olympic Committee towards the development of the IOA and the spreading of the Olympic idea. It emphasised that a means must be found of ensuring that the IOC has a systematic influence on the Academy".

There was no way back for the Prince, however. In 1971, minutes of the IOC session held in Luxembourg noted that: "The resignation had been received of Prince George of Hanover. President Brundage remarked he had been forced to tender his resignation because of Rule 11, which states 'The President of the Olympic Academy may be elected a Member for his term of office'. The President said that Prince of Hanover would be thanked for his excellent work and co-operation."

Norbert Muller, a distinguished professor of Olympic history, insisted: "Prince George was brilliant in preserving the modern Olympic spirit and the ideas behind it."

Never again would a non-Greek serve in the role and never again would the IOA President automatically have a place on the IOC.

Even so, Petralias, who succeeded the Prince as IOA President later, also became an IOC member. Professor Nikolaos Nissiotis and Nikos Filaretos both trod the same path as the IOC Commission was extended to include all Olympic education and culture.

The renovations at the Academy are timed to coincide with the 2021 IOC Session in Athens. During their visit, IOC members will travel to Ancient Olympia and spend time at the Academy, just as they did in 1961, the year it all began.

They will no doubt ponder the words of current IOC President Thomas Bach who insisted: "This Olympic Academy is a link to our roots and heritage."