BSA Decisions Ngā Whakatau a te Mana Whanonga Kaipāho

---

Summary

The performances of the four leaders during the second Holmes Special – Leaders'

Debate, broadcast on 7 October 1996, were ranked at the conclusion of the programme

using the answers given by the invited audience of uncommitted voters. The leaders

were given an order of first through to fourth.

Mr Archer complained to Television New Zealand Ltd that while the information gained

justified naming the winner of the debate, it did not justify ranking the other leaders.

He maintained that to use it for this purpose was misleading.

Explaining that the audience was asked which leader performed best and who had won

the debate, TVNZ said the results shown agreed with the conclusion to be drawn from

the audience's use of the "worm". It declined to uphold the complaint.

Dissatisfied with TVNZ's decision, Mr Archer referred the complaint to the

Broadcasting Standards Authority under s.8(1)(a) of the Broadcasting Act 1989.

For the reasons below, the Authority upholds an aspect of the complaint.

Decision

The members of the Authority have viewed the item complained about and have read the

correspondence (summarised in the Appendix). As is its practice, the Authority

determines the complaint without a formal hearing.

On 7 October 1996, five days before the general election, TVNZ broadcast the second

Leaders' Debate. The debate was monitored by a selected audience of 120 uncommitted

voters who recorded their reactions continuously by way of the electronic "worm". The

movement of the "worm" was monitored by TVNZ staff as the debate progressed and,

after the debate concluded, extracts were shown to illustrate the points of analysis made

by the commentators.

Observing that he was not, and never had been, a member of a political party, Mr

Archer complained to TVNZ about the explanation provided to the answer to one

question. The invited audience, he recalled, was asked to indicate which of the four

leaders had won the debate. However, he continued, after giving the results as to

which of the leaders was thought to be the winner by the audience, the other leaders

were ranked second, third, and fourth. Mr Archer considered that the ranking was

inaccurate and misleading. He gave the following illustration:

In explanation, let us take the most extreme scenario. It was stated that Mr Peters

came fourth with 9.6%. That means that 90.4% of the sample audience did not

think that Mr Peters won the debate but in a hypothetical extreme all of that

section of the audience might have felt that he was second. If 9.6% rated him as

having won and 90.4% (= 100%) felt he was second then there is no way that he

could be fourth.

Now I accept that it is not likely that those who thought that Mr Peters did not win

the debate all thought that he was second but it is likely that a reasonable number

did think so. Similarly some would have put him third.

TVNZ assessed the complaint under standards G4, G6, G7, G14 and G19 of the

Television Code of Broadcasting Practice. The first three require broadcasters:

G4   To deal justly and fairly with any person taking part or referred to in any

programme.

G6   To show balance, impartiality and fairness in dealing with political matters,

current affairs and all questions of a controversial nature.

G7   To avoid the use of any deceptive programme practice in the presentation of

programmes which takes advantage of the confidence viewers have in the

integrity of broadcasting.

The other ones provide:

G14  News must be presented accurately, objectively and impartially.

G19  Care must be taken in the editing of programme material to ensure that the

extracts used are a true reflection and not a distortion of the original event or

the overall views expressed.

Outlining the arrangements for the debate, TVNZ advised:

After watching the debate right through, the invited audience was asked to

indicate who in their view performed best, and then who had won the debate.

In assessing your complaint, we asked the question: what would a viewer of

normal sensibilities have taken from the figures broadcast as a result of that last

question.

We believe that the viewer would deduce that the greatest number felt Ms Clark

had won, a smaller number that Mr Anderton had won, a smaller number still that

Mr Bolger had won, and the remainder picking Mr Peters as the winner. This,

we submit, is exactly what the electronic gadget told us.

It also pointed out that if TVNZ had confined itself to naming the winner, then other

members of the media and viewers would have imposed the ranking making use of the

percentages which were shown.

By way of summary when declining to uphold the complaint, TVNZ wrote:

We reject your suggestion that the presenters were either mischievous or naive.

They trod a sensible path in providing sufficient information for an audience to

get a sense of how each leader had performed without becoming bogged down in

a detailed statistical analysis – a role for which television is not well suited because

of the inability of the viewer to check back over figures as he or she might do in a

newspaper or magazine.

When he referred his complaint to the Authority, Mr Archer dealt with TVNZ's

comment (recorded above) that the answer to the questions indeed recorded which

proportion of the audience believed each leader had won the debate. He added:

A viewer presented with the wording of the question and the figures then shown

would have deduced the result as stated here by TVNZ. The problem is that

TVNZ did not leave it at that – what TVNZ then stated was –

"so who did they [the invited audience] think actually won ...

The winner was once again Helen Clark ...

Right behind her Jim Anderton

Third was Prime Minister Jim Bolger ... up from his fourth placing

last time

Last this time Winston Peters down from a second placing last time

..."

Mr Archer argued that, given the way the results were in fact presented:

These statements changed the whole emphasis of what was being presented to the

viewing audience and would tend to override viewers' initial reaction almost

before they had had that reaction. Further, faced with a new technique I suggest

that the average viewer would be more ready to accept the interpretation put

forward by TVNZ without really questioning the basis of that interpretation.

In its response to Mr Archer, TVNZ discussed the "worm" technology and Mr Archer

commented on some of these matters when he referred his complaint to the Authority.

As the complaint explicitly focusses on one precise aspect of the use of the "worm", the

Authority has not addressed these other matters.

When responding to the Authority on Mr Archer's referral, TVNZ maintained that it

was fair to rank the four leaders, and that it was supported in this view by Colmar

Brunton – a professional polling organisation – and Alan Lee of Auckland University's

Department of Statistics. Mr Lee, TVNZ added, said that the issue was not a statistical

one, but one of semantics.

In his final comment, Mr Archer emphasised that his complaint was concerned only

with the way TVNZ used and interpreted the data from the audience's response to one

question. He declined to comment on the reports from Colmar Brunton and the

Department of Statistics as he did not know what they had been asked. He repeated the

substance of his complaint:

It was legitimate for TVNZ to ask their invited panel "Who do you think won the

debate?" and it was fair to show the combined individual answers for each leader

as a percentage and to show those percentages on screen all at the same time.

It was not fair to then tell viewers that the leader who received the lowest support

in the question of "Who won?" was therefore the leader who had done worst in

the debate overall – ie had come fourth. Because, for example, 90.4% of the

panel thought Mr Peters did not win the debate did not necessarily mean that those

same 90.4% thought he came last. They, in fact, did not express an opinion

about Mr Peters' performance other than to say he did not win.

That is not semantics, that is a simple fact.

The Authority has considered this complaint in some detail. On the one hand, it agrees

that Mr Archer's point is valid. The invited audience was not asked to rank the leaders.

They were asked to name the person whom they believed had won the debate. TVNZ

screened this information by way of a percentage for each leader, and then used that

information to rank the leaders. If the use of the "worm" was presented as a reliable

index for measuring the audience's reaction, then the Authority would be inclined to the

view that the ranking process involved a breach of standards.

On the other hand, the Authority accepts that the "worm" was not advanced as a

scientifically accurate gauge. From this approach, it can be seen as an adjunct to

entertainment – a "rough and ready tool" used to assist in the analysis. Because of its

limited validity, and questionable relevance, the "worm" at most was only a guide

which was used to confirm impressions gained by watching the leaders in operation.

From this perspective, a breach of standards is unlikely to follow.

In its determination of this complaint, the Authority has considered carefully the way in

which the "worm" was promoted. Whereas it was not claimed to be unquestioningly a

reliable and accurate instrument, it was advanced as more than merely a device to

entertain viewers. Moreover, it was used in a situation where the leaders were taking

part in an election campaign to determine leadership of the country.

The "worm", the Authority concludes, was presented by TVNZ as a piece of modern

technology which would provide substantive assistance to the analysts. It was also said

to record accurately the reactions of the invited studio audience.

In view of the purposes for which the viewers were told that the "worm" would be

used, the Authority is of the opinion that viewers could expect that the results would

reflect more or less accurately the responses of the invited audience. The audience was

asked to nominate the leader whom they felt had won the debate. The audience was not

asked to rank the leaders' performances. The presenters recorded the percentages of the

audience's response to the question put. In using those answers to rank the leaders, the

Authority considers that the broadcast distorted the answers provided by the audience.

Accordingly, the complaint is upheld as a breach of standard G19.

The standards nominated overlap to some degree. The Authority decides that the

concerns raised are in this instance appropriately subsumed into standard G19.

For the reasons above, the Authority upholds the complaint that one

aspect of the analysis of the Holmes Special – Leaders' Debate,

broadcast by Television New Zealand Ltd on TV One on 7 October 1996,

breached standard G19 of the Television Code of Broadcasting Practice.

Having upheld a complaint, the Authority may impose an order under s.13(1) of the

Broadcasting Act 1989. In view of the precise matter raised in the complaint, and the

wide ranging nature of the Leaders' debate and the ample opportunity for viewers to

decide for themselves how each participant performed, the Authority considers that it is

not appropriate to impose an order.

Signed for and on behalf of the Authority

Judith Potter
Chairperson
23 January 1997

Appendix

N E Archer's Complaint to Television New Zealand Ltd – 15 October 1996

Mr Archer of Rotorua complained to Television New Zealand Ltd about an aspect of the

analysis broadcast at about 9.00pm following the Leaders' Debate on TV One on 7

October 1996.

Pointing out that he was not, and never had been, a member of any political party, Mr

Archer focussed on the interpretation of one question. He accepted that it was

legitimate to ask the members of the sample audience to record which of the leaders they

felt had won the debate and to broadcast the result. The broadcast would announce to

viewers which leader the invited audience believed had won the debate. However, Mr

Archer did not accept that TVNZ was justified in asserting that the other named leaders

had come, respectively, second, third and fourth. He wrote:

That assumption/presumption was misleading and could have had an unfair

influence on those who heard the claim made and who did not hear or did not

understand the reasoning, or lack of reasoning, behind it.

Mr Archer continued:

I consider that TV One, and its presenters, were either simply naive (in their

understanding of what they were dealing with) or mischievous in the way in

which they interpreted the information which they had. I suspect they were

simply naive.

Mr Archer noted that, according to the item, Mr Peters came fourth with 9.6%.

Whereas from the question asked, it in fact meant that 90.4% of the audience did not

think that Mr Peters came first. It could also have meant that 90.4% of the audience

believed that he came second. Mr Archer added:

Now I accept that it is not likely that those who thought that Mr Peters did not win

the debate all thought that he was second but it is likely that a reasonable number

did think so. Similarly some would have put him third.

Quite bluntly it is totally unacceptable for TV One to state that Mr Peters came

fourth in the debate simply because less people thought he had won it. A similar

argument as just outlined would apply to the other leaders - Mr Anderton and Mr

Bolger.

As part of his contention, Mr Archer commented:

As an aside – the presentation team should have been aware that their

interpretation was incorrect as they had just spent some time explaining how the

"worm" had indicated that certain leaders had done consistently well (in worm

terms) in the debate and others had not. These explanations did not agree with the

way in which the team interpreted the final analysis.

TVNZ's Response to the Formal Complaint – 30 October 1996

Assessing the complaint about some of the analysis on the Holmes Special - Leaders'

Debate under standards G4, G6, G7, G14, and G19 of the Television Code of

Broadcasting Practice, TVNZ explained that 120 uncommitted voters had watched the

entire debate and had recorded their reaction continuously by way of the electronic

"worm". The movement of the "worm" was monitored by TVNZ staff during the

debate and extracts were used during the analysis to illustrate the points made by the

presenters. TVNZ reported:

After watching the debate right through, the invited audience was asked to

indicate who in their view performed best, and then who had won the debate.

In assessing your complaint, we asked the question: what would a viewer of

normal sensibilities take from the figures broadcast as a result of that last

question?

TVNZ provided the following answer:

We believe that the viewer would deduce that the greatest number felt Ms Clark

had won, a smaller number that Mr Anderton had won, a smaller number still that

Mr Bolger had won, and the remainder picking Mr Peters as the winner. This,

we submit is exactly what the electronic gadget told us.

We do not believe that our "normal viewer" would expect a survey along the lines

you suggest which would involve analysis of the second, third and fourth choices

of each of the voters concerned. Remember the question put to the voters was -

who won the debate?

Noting that the responses to the questions, who performed best and who won the

debate, were consistent, TVNZ maintained that the "worm" was advanced as no more

than an adjunct to the debate and as a "temperature gauge". TVNZ acknowledged that

the "worm" technology was in its early stages, and maintained that had it declined to

name a winner, other members of the media would have advanced an opinion. TVNZ

continued:

We reject your suggestion that the presenters were either mischievous or naive.

They trod a sensible path in providing sufficient information for an audience to

get a sense of how each leader performed without becoming bogged down in a

detailed statistical analysis – a role for which television is not well suited because

of the inability of the viewer to check back over figures as he or she might do in a

newspaper or magazine.

With reference to the specific standards cited in the complaint, TVNZ maintained that no

one had been treated unfairly, that there was no imbalance, or inaccuracy, and that there

was nothing deceitful. It did not accept that the extracts shown were a distortion of the

views of the invited audience and it declined to uphold the complaint.

Mr Archer's Referral to the Broadcasting Standards Authority – 19 November 1996

Dissatisfied with TVNZ's decision, Mr Archer referred his complaint to the

Broadcasting Standards Authority under s.8(1)(a) of the Broadcasting Act 1989.

He pointed out that his complaint focussed on one specific aspect of the analysis, and

he considered much of TVNZ's reply which discussed the use of the "worm" to be

irrelevant. He continued:

Although it may seem a relatively minor point I bring the matter to the attention of

the Authority because the impact of the TVNZ action, if it were to become

standard practice, could be far greater than initially appears. I consider that what

TVNZ did as a final summary and how it presented that summary was ultimately

misleading and could have had considerable effect on viewers' subsequent actions

when the broadcast was made less than 5 days before election morning.

He was seeking, he added, guidelines which would apply in future.

Mr Archer then dealt with a number of points made by TVNZ in reply to his complaint.

He was not objecting to ranking the leaders but to the misleading way it had been done

on this occasion. He said that a viewer might have accepted TVNZ's explanation in its

letter when it wrote that what the question disclosed was:

"We believe that the viewer would deduce that the greatest number felt Ms Clark

had won, a smaller number that Mr Anderton had won, a smaller number still that

Mr Bolger had won, and the remainder picking Mr Peters as the winner. This,

we submit, is exactly what the gadget told us".

However, Mr Archer observed, the following was the summary broadcast in the item:

"so who did they [the invited audience] think actually won ...

The winner was once again Helen Clark ...

Right behind her Jim Anderton ...

Third was Prime Minister Jim Bolger ... up from his fourth placing last

time

Last this time Winston Peters down from a second placing last time ..."

Mr Archer contended:

These statements changed the whole emphasis of what was being presented to the

viewing audience and would tend to override viewers' initial reaction almost

before they had had that reaction. Further, faced with a new technique I suggest

that the average viewer would be more ready to accept the interpretation put

forward by TVNZ without really questioning the basis of that interpretation.

Pointing out that he had not suggested any particular survey technique, Mr Archer said

that the viewer was not aware of the minute-by-minute movement of the "worm" during

the debate. He added that he was not objecting to the use of the "worm" – just one

specific aspect of the analysis. Accordingly, he had not addressed much of the material

TVNZ advanced. Nevertheless, in response to TVNZ's denial that the item was

mischievous or naive, he stated:

Viewers who watched the debate, or even just the bulk of the summary, would

have indeed gained some idea of how the leaders performed. I make no

suggestion that TVNZ indulge in a detailed statistical analysis - that is a decision

for them. However, to assert that various leaders came first to fourth based on

data which did not provide a basis for that assertion is not to provide sufficient

information to a viewing audience but, in fact, to provide misleading

"information".

Mr Archer maintained his complaint that the broadcast for the reasons given, breached

standards G4, G6, G7, G14 and G19. He concluded:

The aim of this complaint has not been to end or prevent the use of the "worm" or

similar techniques but to ensure that all aspects of such use are legitimate, well

thought out and that their use does not cause prejudice of any kind to any party

subject to them.

TVNZ's Response to the Authority – 2 December 1996

TVNZ advised the Authority that it had little to add, other than commenting:

We believe it was fair, given the data provided by the worm, to rank the four

leaders as Tonight did. That was also seen as fair by the professional polling

organisation, Colmar Brunton. To double check the validity of "Tonight's"

conclusions we have today sought the advice of the Head of the Department of

Statistics at Auckland University, Mr Alan Lee, who also thinks the ranking was

fair. He says it is not a statistical issue at all and believes that Mr Archer may be

dealing in semantics which do not stand close examination.

We emphasise again that Tonight told viewers of the pros and cons of using a

panel of uncommitted voters this way and suggest that the viewing audience was

thus provided with the context on which they could judge the result.

We believe our method of ranking the debate's winners and losers was fair and

reasonable.

Mr Archer's Final Comment – 11 December 1996

Emphasising that his complaint focussed on TVNZ's use and interpretation of the data

supplied by the audience when answering one particular question, Mr Archer

maintained that he was concerned with fairness. He said that he was reluctant to

respond to the comments from Colmar Brunton or Mr Lee as he was unaware of the

questions they had been asked. He summarised his complaint:

It was legitimate for TVNZ to ask their invited panel "Who do you think won the

debate?" and it was fair to show the combined individual answers for each leader

as a percentage and to show those percentages on screen all at the same time.

It was not fair to then tell viewers that the leader who received the lowest support

in the question of "Who won?" was therefore the leader who had done worst in

the debate overall – ie had come fourth. Because, for example, 90.4% of the

panel thought Mr Peters did not win the debate did not necessarily mean that those

same 90.4% thought he came last. They, in fact, did not express an opinion

about Mr Peters' performance other than to say he did not win.

That is not semantics, that is a simple fact.