I'm in a big bind here – I love rugby (even went to the school where it was invented) and I especially love Fiji rugby (I'm writing this within arms reach of a jersey signed by the 1997 Sevens Rugby World Cup-winning team, and a framed photo of the 15s side that beat the British Lions in 1977).

But sometimes you have to think with your head and not your heart. And I believe that by admitting rugby into the Olympics, the International Olympic Committtee (IOC) are propping up a political structure of governance that is discredited and needs to be swept away to save the game we all love.

Political power within the International Rugby Board (IRB) is vigorously and jealously manipulated by a self-selected elite of first-world nations. But don't just take my word for it. Last year an independent study, co-authored by top UK legal firm Addleshaw Goddard, found that seven per cent of the IRB's member unions controlled 62 per cent of the voting power. Put another way, 90 per cent of the unions (think, developing world) had less than a quarter of the votes. The system is rigged in favour of the few, over the many.

The IRB does have a Congress and it also has an Executive Committee but slotted in between is the 26-member Council, the apex of power – rugby's equivalent of a boardroom, and expressly referred to in the game's constitution as the sport's supreme authority.

Sixteen votes, two-a-piece, come from the so-called Foundation Unions of England, Scotland, Wales, Ireland, France, South Africa, Australia and New Zealand. Compared to this permanent bloc, the 10 votes shared, one each, between Japan, Italy, Argentina, Canada and the six regional confederations, are almost an irrelevance.

Impoverished Fiji, for instance, has one vote among 11 others within the Federation of Oceania Rugby Union (FORU) to select the Pacific's IRB representative (currently, the Samoan lawyer Harry Schuster). Australia and New Zealand also vote in the same FORU election as well as having two seats apiece in the Council.

And what of the IRB's Congress, where all the member unions attend? Little more than a talking shop, concluded Addleshaw Goddard, in a highly critical report entitled Putting Rugby First. Noting the Congress meets only every two years and has no formal legislative powers, consequently the wider membership - in one sense equivalent to the full shareholders register of world rugby - has extremely limited power and fewer opportunities to influence the direction of world rugby.

If the rest of the sports affiliated with the Olympic Movement are doing things differently, with greater accountability and transparency, why does rugby have this mania for control and privilege?

The answer is fear. The painful truth is rugby is not in good health across the Council membership.

Some examples: the number of senior male players in Scotland may soon tip below 10,000; in Australia almost all of the Super 14 matches were out-rated in the critical New South Wales TV market by the National Rugby League Under-20 Toyota Cup (that's right, no-name teenagers playing league); in England, a month into the Premiership season and only two sides had scored try-bonus points and five out of the 12 teams were averaging less than a try a match (prompting headlines in the Daily Telegraph such as, Is It Me, Or Is The Rugby Rubbish?)

Everybody on the Council is fretting about something, from the weakness of their local currency to inroads made by rival sporting codes. Which is why the 20 delegates representing national unions view issues through the prism, not of what is good for THE game but what is good for MY game.

This mutual suspicion is what collapsed long-overdue attempts to rewrite the laws of the game (the so-called ELVs). And why, as rugby enters its first economic recession since turning professional, the game's lack of leadership is imperilling its very existence as an entertainment choice.

Not only have the ELVs been bounced into touch but a raft of long-overdue areas of reform have also been neglected because decision-making has become such a fraught and deadlocked process.

The IRB concedes the political structure is skewed in favour of some nations but adds, it is not unreasonable to argue that those that provide the bulk of players and money into the game should have the bulk of the representation.

The bulk of players? Well, that isn't true. Fiji, with a population one-sixth of Scotland, has more senior male players and a consistently higher IRB ranking but is only represented at arm's length by a regional confederation.

Perhaps quality of players? Not true either. Japan, Italy and Canada hold three precious Council votes but between them have only once reached the final eight in all editions of the Rugby World Cup. Samoa has made it to the knockout stages in three of the five previous tournaments, yet has no direct vote.

The tragedy of rugby is that it started at the same time as football, from the same place, has a similar colonial heritage and a deliberately low cost of entry and participation.

But rugby's consistently backward-looking governance has denied the sport the spectacular trajectory enjoyed by football.

The teams likely to win the 2011 Rugby World Cup are the exact same sides you would have backed 100 years ago had there been a tournament. And those sides belong to the very unions in whose grasp all political and legislative control is permanently vested. Yet with all that power, rugby is in, or teetering near, apparent crisis within most of the council membership.

And that's why the IOC should have sent a clear message to the IRB Council. Play fair with your governance. Loosen your grip, open things up, and once you've created a level playing field, then come and join the Olympic Movement.

London-born Charlie Charters was marketing manager of the Fiji Rugby Union from 2001-04 and helped create the Pacific Islanders test side. He is now a writer, and his debut book Bolt Action, a thriller, will be published by Hodder & Stoughton in 2010