The following are the outputs of the real-time captioning taken during the Eleventh Annual Meeting of the Internet Governance Forum (IGF) in Jalisco, Mexico, from 5 to 9 December 2016. Although it is largely accurate, in some cases it may be incomplete or inaccurate due to inaudible passages or transcription errors. It is posted as an aid to understanding the proceedings at the event, but should not be treated as an authoritative record.
***
>> MARKUS KUMMER: Let's start and let the recording come in whenever it's ready. Okay. Okay. We are ready, let's get started. Feedback, please. Yes.
>> ANDREA SAKS: Personally I think we made a lot of progress compared to how we did last year, I really enjoyed it and I think it was the right format and the moderation was well done and I want to thank you and congratulate you. I think it was as you said done in a very gracious way. Thank you. And yes, I share the impression that we could have used another half hour maybe and that's it.
>> AVRI DORIA: I just want to build on that. I actually think we could have done another whole hour and a half given the content of your research papers, given the questions that you put out two, three questions plus the survey questions, I think we could have easily done another round or if slightly shorter, another two rounds of questions, especially as the sort of, where things were combined and sort of building on each other just sort of started at the end of that first round, so.
>> AUDIENCE: I tend to agree with Avri. Would you like to comment, Tatiana?
>> AUDIENCE: Yes, absolutely. There were some good questions of yours or some points in your papers which could have fit very well into the questions from the floor, and it would have been nice to have another half hour. And to jump between remote participants, me, remote participants and me, but I understand the time constraints but if you can split it into two or maybe do it longer next time it might work well and from me thank you very much for your papers and for your questions. They really helped a lot. Like after I read everything, after I saw your questions, I understood the focus, I understood so clear what you care about, how I can approach this. So it's a joint effort. Thank you.
>> MARKUS KUMMER: Thank you. Maarten asked for the floor.
>> MAARTEN BOTTERMAN: The moderation made it an interesting whole for some of us there may not have said everything we want to say, but that's a good thing because an hour norm would no have been enough. So appreciation for that. The other thing I appreciated was hearing things from the other Dynamic Coalition about very good practice things. I loved the fact that a group of students were involved. I loved the fact that there were ways to communicate with Government in another, which brings me to the suggestion that we share what we deem to be our own best successes in running the DC, the best elements of that, and share best practice in that way. I'm very willing to write an email in what I think we do well, and I'm looking forward to read those from you on what you think you do well. And one of the things that came out for me and I bring it back to my Dynamic Coalition for sure is I'm going to offer our good practice paper to universities like, hey, do you want to discuss this with your students and love to have the feedback.
>> MARKUS KUMMER: Thank you. Other comments? Yes, please.
>> AUDIENCE: Yes, I would also like to sort of plus one and endorse all of the comments about it being a really good session and definitely significant progress from last time, fantastic moderation. Thank you. I have to say I'm in the short and sweet camp, which means that even though there were questions remaining, et cetera, I thought the session worked really well at 90 minutes, and I think it will be easier to attract more people if we keep it at 90 minutes rather than three hours because the dynamic or two hours because the Dynamic Coalitions are not the only spaces where these issues are being discussed.
It's one of the spaces within the IGF.
>> MARKUS KUMMER: Thank you. Well, as I said earlier, I think it's better if you are still left on your appetite. I mean, nothing worse than if you sort of run out of steam, but we left in a way on a high. There was still a lot to be discussed and I'm sure discussions will go on so I think that's a good thing as it is. Yes. Marianne.
>> MARIANNE FRANKLIN: Yes, plus one for me, I was in the audience so I was getting a sense of how it came across. Very, very smooth, professional and I think, again, an hour and a half is enough. What we were missing, perhaps, on the program was some sort of title. DC main sessions is kind of obscure. And I think that's really important because then we can spread the word more. I'm obviously Tatiana, fantastic, and that you studied all of the papers. That's best practice in itself.
And what was the other thing I noticed? Yes, about students. We found the IRP coalition they have been extraordinary energizer and if it weren't for them they provided us with new material, and they are just totally becoming engaged to IGF and are thrilled to be able to be there live, which is a real, a real plus for the remote participation if it's rebuilt over the year so in that sense it was good. I felt it needed a little bit higher profile and a snappy title but I wouldn't want to stretch is out longer for the reason that you said Markus, but well done, everyone.
>> MARKUS KUMMER: And finding a snappy title may not be that easy as we are covering a broad area, but I agree DC main session is not very sexy.
>> AUDIENCE: This year might have been the first year there was a newcomers track too. I think we should encourage a tie between the newcomers track and the DC main session because some of the points that Tatiana made about isn't this the same as that and haven't you been very different were helpful to newcomers to the IGF so to the extent that the session could be advertised through the newcomers track or some other thing, I think that could be helpful.
>> MARKUS KUMMER: Luca.
>> LUCA BELLI: I liked the format and I think it was much more dynamic than last year, so very good moderation. I just wanted to point that there is also Dynamic Coalition for youngsters or youth Dynamic Coalition so we should involve them in the session. I know that for students for youngsters maybe to prepare an outcome document is not the he'diest thing but we maybe could try to provide them our outcome and having their feedback, and I know last year, I had the chance of having two interns that were in the youth at IGF group, so they brought a lot of youngs, but this year unfortunately I didn't have the chance but we have to try to involve these youth programs to try to have them and to give feedback to us to do a better job and better outcomes.
>> MARKUS KUMMER: Thank you good suggestion. Indeed they did not provide a paper that's why they were not part of the exercise. Yes, Eleanor.
>> AUDIENCE: Sorry. I just wanted to say that they were a little bit disorganized and they sort of came in to the main session planning process too late to be participants in the main session, but once they sort of realized what was going on, they came together very quickly and they have sort of been rebooting their activities now. They did produce a report, and they, and they have been meeting and talking much more regularly. So I think, you know, from now on, we will see them being more active.
>> MARKUS KUMMER: Thank you and I think Luca's idea was to involve them in the process, maybe not necessarily produce a paper if they don't have one, but Christopher, yes.
>> AUDIENCE: Two comments and a question. One is I believe in short and sweet. And we do have all separate main sessions, so there is ample opportunity to discuss it at length. This is more lightning presentations. Second, I agree with Luca, I think there is not just Dynamic Coalition, but I would like to take a suggestion just to throw it out there, can we create an event where we meet them early in the week like maybe day zero in an informal mix and match session?
Because one of the things we did in our booth we put up a sign that yesterday youth welcome here. They are wandering around lost. They don't have a home, and the more we orient them early and get them in conversations with some of the more active organisations here, I think that would be helpful. The question is are we just talking about the session we just ended or are we talking about the Dynamic Coalitions generally here, because I have a comment about how feedback I received from my main session.
>> MARKUS KUMMER: Well, my ‑‑ I thought we are still fresh our memories, let's just have a fresh reactions to our main session and I think it is all very helpful and I see that quite a few, Olivier was first.
>> OLIVIER MJ CREPIN‑LEBLOND: Two things, the fact that our moderator walked behind us freaked me out. No, but more importantly, the concern I had was there appears to be a sort of disconnect between the ideas, rating sheets and the session itself. I know that it was mentioned in the beginning, but then afterward I'm not quite sure we got, we made full use of that, and I wonder whether if we had more time and I'm not saying whether we should have more time or whether this was enough time. If we did have more time we could actually submit some of the questions from the ideas rating sheet to the audience and with either a show after hands or a show of green or red card.
>> MARKUS KUMMER: Thank you for that. Jeremy my was not able to join us as he had another commitment. I'm losing track of the cue, I think you were next and then Marianne.
>> NICOLAS ECHANIZ: This is Nicolas. I was thinking of the title you mentioned and if we had a more interesting title and maybe this session were scheduled earlier in the program, it would also, I think, get more people involved in the Dynamic Coalition's actives because I think this actually would have worked very well as a presentation of the Dynamic Coalitions so during the rest of the IGF people know more about what we are about. So that's my proposal.
>> MARKUS KUMMER: Thank you. That's a valid point about the scheduling. I can't remember, maybe the idea was let's have the Dynamic Coalition meet first and then feed into the main session but do it the other way around, have the main sessions as appetizer feed into the Dynamic Coalition might indeed make more sense. Marianne and then Luke ca.
>> MARIANNE FRANKLIN:: Before I forget, Twitter hashtags, it wasn't clear where the hash was. I couldn't find it.
I was using hashtag IGF 2016. And also I just had a question, is it IGF policy that Twitter ‑‑ oh, that Twitter walls, I think they are great to have, I think they line things up but is there a policy to moderate them? Why do they need to moderate at all?
>> MARKUS KUMMER: There is the question there may be taboo issues that are not allowed to be mentioned in a UN context. We are not talking about just a gate keeper to see what's coming up or not, but that was in a sense, it's a little bit of a can of worms. We asked more than once for a Twitter wall, but Avri, maybe you want to comment, you are the most active on that and I think also the most frustrated person by that.
>> AVRI DORIA: Part of the reason for moderated is since anything can come from anybody on Twitter, you want to avoid the UN wants to avoid certain, certainly profanity or pictures or anything that occurs out a Government, et cetera. So they have always required a mechanism with some sort of delay or some sort of moderation. And those tools are available. We just weren't able, the human rights group got something going, and if we had known what they got going, we could have used what they had, but I only found out about it when I was sitting in their session earlier in the day. So that's the reason why we have to do that at least a delay where someone has time to stop the expletives, stop the dirt because you can send anything in Twitter.
>> AUDIENCE: Just in terms of general education about this event, Twitter walls come and I think they are great, okay, point taken. And then some basic guidelines are included so that people understand why it would be moderated because it's counter intuitive to how Twitter works and that's also educational.
>> MARKUS KUMMER: Should I put it that way there are other sensitives apart from profanity, there are political sensitivity. Who was next now? I'm sorry. Tatiana.
>> TATIANA:: Well, first point about Twitter wall, from my experience with moderation, it reproduces itself, when those who are Tweeting can see the Twitter wall, when it's projected somewhere, other wise it will be mostly silent, especially if the session is not super controversial. This is the first point. I would like to excuse, I would like to say sorry for walking behind you. It's a big stage, and I decided to do this to maintain the substance and keep you awake, but honestly, the reason behind this is I don't stay, I don't like standing moderators, because it's a kind of ‑‑ there should be some movement on stage, I believe. Moderator is not just a tape recorder, it is someone who at least provides connection between the floor and the stage, and the set is so terrible when you see that, you know, you are like above the audience. So I decided to move here and there because if I move in front of you, there is an opportunity for me to turn with my back to you and this is a killer for the session, so I'm sorry for this but next time I'm going to do it again.
>> MARKUS KUMMER: You may have freaked out Olivier but I think you did it well and you moved very elegantly, you were floating across the stage. Luca.
>> Luca: I think there is also a lot of room for improvement in the survey to have more feedback, because if we want the IGF participants to provide us feedback, they have to understand, they have to know that the survey exists and if we go on the IGF website, we have to go on IGF community intersessional work then the Dynamic Coalition, and then Dynamic Coalition you find them and I think a link directly from the home page could be feasible and much easier to have feedback.
>> MARKUS KUMMER: Of course, Avri.
>> AVRI DORIA:: I think I agree on the page, but there may be other ways to make sure that people can get to the link, because the page is constantly turning so I think it's good to get it on the front page, probably good to get it on the schedules, good to get it on signs around the place. So I try to push it as much as possible in various meetings, you know, whether there were ways to post it so that people could just get it. I think you are right about keeping it on the page and I will work on that for next time, but I don't even think that that necessarily will be enough to get, to get enough people to read the papers. I think the other part was, is some of the people obviously won't have read the papers until maybe today it will be enough for them to be interested enough to go back, so that's part of the, us advertising them in the meeting and such.
>> MARKUS KUMMER: We did have some out on the DC booth, but according to Jeremy there was very little feedback, although we had it right from the beginning.
>> AUDIENCE: I want to share our experience of the idea rating survey last year when we did it for the first time. We actually put it out, remember the deadlines were extended by a month, and we put it out on social media. We put it out on a number of general lists, et cetera, but honestly people have been resistant to filling out the survey. I don't know if that's because of the platform itself, the user interface, or what the issue is.
So I would like us to also keep the possibility open of testing it this year, and then deciding what we want to do next year.
>> MARKUS KUMMER: Yes, please.
>> AUDIENCE: Did anybody go to an Internet Society session where they asked you to contribute to their survey and they gave you a card that told you where to go. Could we do something like that for the surveys just so that people can take it away with them and they will know where to go because I still have mine and I got this the first day.
>> AUDIENCE: Have you gone to the survey?
>> AUDIENCE: I have, but I thought it was such a great idea that I'm taking it as evidence.
>> MARKUS KUMMER: Yes, and please, I can't remember all of the names so please state your name.
>> AUDIENCE: Aye Jutta Croll. Founding member of the Dynamic Coalition on child online safety. And going back to that year I believe it was the second IGF that it was founded. I found the idea of Dynamic Coalition very thrilling and I think we might consider when we talk about the title of the session, DC Main Session, not everybody knows the acronym. And it could be much more dynamic, and when I went back to tell my staff at the organisation, we are now part of a Dynamic Coalition, everybody was asking what is that? What does it mean? And I think we can stress that a little bit, explore it a little bit, what does it mean Dynamic Coalition? What could we do? It's a good name and good format, but it needs also to be explained to all of the other IGF participants.
>> MARKUS KUMMER: In a way you are already touching on what was part of Eleonora's agenda for the session but it's fine, we are having the discussion, our memory is still fresh, we had a good session, we all are fairly pleased with how it went and all of the work that people did too. So it's great to have this brain storming immediate Luca.
>> LUCA BELLI: One point on the survey, the survey, it has been useful to put the option of having, identifying the stakeholder group and I think also the regional that has been put, but I think the fact of asking people to put their name, that is something that will dissuade people to take the survey. And it's a basic principle for democracy, I mean you vote, you give your name but you don't say that you have voted for something else, so I would suggest not to ask people's name if we want to have feedback.
Another comment I had about the name Dynamic Coalition. I like the name Dynamic Coalition, but maybe it would be more understandable for people if we called them IGF coalitions rather than Dynamic Coalition. Because Dynamic Coalition for us it's immediate to understand what it is, but for someone never in contact with the IGF process, if you say IGF coalition it's much more understandable than Dynamic Coalition.
>> MARKUS KUMMER: Well, (Indiscernible) doesn't agree and I was going to say there may be some resistance against naming them Dynamic Coalitions. Andrea you don't agree,.
>> ANDREA SAKS: When you change a name, you have a brand. The best way to go forward is to take the brand and make it more accessible and more visible. If you change brands, and, you know, somebody is going to mix up this, that and the other, so I would be against that, and also you are perfectly right, there is a problem with labeling it IGF. So also we weren't even recognized in the beginning. We were considered upstarts, and horrible little creatures that can't influence the IGF. So rather than push the boat out, what you want is, as I say, you said it, it isn't very sexy, just DC.
We could have put Dynamic Coalitions, and we could have put a caveat, which is on a kind of, you click on it and it drops down and you can see the names of the Dynamic Coalitions aand what they are. So that if there is, if you put that and you have access to the Web, you can look at what's there. So rather than try and become fashionable, catchy, sexy, or whatever, let's be more explanative and more accessible so if there is a dropdown box, people will see that. That would be my suggestion.
>> MARKUS KUMMER: Thank you. Christopher and then Marianne.
>> CHRISTOPHER YU: I agree with the idea of rating sheets or whenever we are calling them now. To me the spirit of the IGF is bringing together multistakeholder groups and it's not, you get consensus in more than one and different views on the other. It's important that we capture in those the breadth of it because the whole point is to engage the whole community because unless it has that consensus, it's not going to succeed. What was striking to me, the one comment I got in my own main session, and this is an interesting blind spot, it struck me from the other ones I met.
I had three of the different stakeholder groups there. I had no Government on the stage. And I know I have attended sessions sometimes we do, sometimes we don't. And I just, I thought they were right. And it's one of these things on a commitment level, if we are going to get the output, and so I have done a lot of the diversity work of different kinds and you start with the people who are having the commitments, but we do understand we started inviting people outside that group because ultimately people we need to engage in the work and unless we get their attendance, we can't get their buy in. So one of the challenges in terms of, I think this is once caused a best practices discussion, as Dynamic Coalitions, we need to, I think every convener has to take responsibility for active outreach to make sure that the participation is representative so the full range of voices is there.
It can be quite frustrating it. It can be hard to get parts of the stakeholder groups to participate, but I think if this type of organisation is going to have the kind of impact that we hope, it's going to require us as Dynamic Coalitions to commit to that because otherwise we will talk amongst ourselves, which, has it's own rewards in its own way but will not lead to the kind of outcomes we are committed to trying to ‑‑ the kind of dialogue and interaction and consensus building we are trying to promote.
>> MARKUS KUMMER: Marianne.
>> MARIANNE FRANKLIN: DC Main Session subtitle Presenting New Work or New Work Proposals, even that provides a lead into why someone might want to come to the main session. About the survey, I ‑‑ I'm sorry, I'm terribly sorry. I didn't realize. My apologies. My mom tells me this all of the time. I'm sorry. About the survey, yes, I made an intervention last year and I will follow up anonymity is absolutely vital for a survey to work. It's pointless to have a survey and ask people, you can ask people to volunteer their name but it's pointless and it's a shame because a lot of work is being put into that, and we were developing it quite well this year by being asked to have these propositions. That was a good disciplining and focusing exercise. We got a lot from the IRP coalition, but to not anonymize it is inappropriate on not just Democratic grounds, it's inappropriate on research grounds. And it actually unfortunately invalidates the tool as a research approach. Before we get into the discussion of whether we want to rate like we are rating, I just need to under score that point. It's a shame really, but let's see how they go this month.
>> MARKUS KUMMER: Did you have remote comments? Yes, please.
>> REMOTE AUDIENCE: I think that there is one people on line, but I don't know why it is not to begin to speaking. I don't know. It's okay.
>> MARKUS KUMMER: Thank you. Olivier.
>> OLIVIER MJ CREPIN‑LEBLOND: Just a quick suggestion, I was a bit disappointed with the interaction with the audience. Yes, we had questions at the end, but I was still under the impression that a lot of them were checking emails and this sort of stuff Luca and I were counting. We got different numbers.
We have to explore ways to engage the audience more and to get more immediate feedback from them, and I just wonder, and this is just something I'm throwing out there, I remember, I can't remember if it was the first or the second Russian IGF they had these boxes that everyone had with A, B, C, D. People while discussions were going on could press buttons and if they strongly agreed they would press A and you had a screen behind the speaker, not in front of the speaker, behind the speaker that would show how people were agreeing or disagreeing with what was being said. It was quite amusing and it get people quite interested with the sessions because they felt they were really getting on well.
And the moderator could see what is going on and he would say it looks like people are not agreeing with you. It can be done with apps these days. You get a simple app on the phone and everybody is connected. The people are doing their emails, they could actually go and interactively agree, disagree and this sort of stuff. Just a thought.
>> MARKUS KUMMER: Good suggestion. Tatiana.
>> AUDIENCE: I agree with Olivier about disengagement of the audience. There were several factors, first of all, the stage itself, it is the positioning of the room and the Chairs. I just, from my moderation experience, I just know that these rooms with a stage are always less inclusive. Whatever happens, you can try to run around with small devices, whatever, the only way to actually engage is when moderator comes down from the stage, runs around the audience, and the session starts with open floor for questions.
That's the only way to engage these big audiences. All other format, I mean, of course, using the tools, I agree, might be super helpful, yes, when you have enough participants in the audience. When we decide to count there was like ten people or something.
So I believe that for the future, you might, you might explore this format. Collect the questions from the audience during the Dynamic Coalitions but I don't know how it will fit well with the presentation, because I don't think that all of the people are able to read 12 papers, and it just doesn't make sense to start without presentation because it's not an open mic session.
So there is a tension between these two.
>> MARKUS KUMMER: Good comment. The room definitely is not ideal. This kind of, it reminds me of the very first IGF. The audience was ‑‑ we don't have audience, we have participants, but this root creates an audience. It makes it very passive.
>> AUDIENCE: I felt this wall, and I have huge moderation experience and my most miserable experience is always connected with these big rooms with stage and audience.
>> MARKUS KUMMER: Yes, please.
>> OLIVIER MJ CREPIN‑LEBLOND: I wasn't saying that the audience wasn't listening at all. I found that at the beginning they were listening less than at the end which is a good point. There was an increase in involvement from the audience, but I really want more audience feet back. I want the audience to talk, to do things and, of course, if we have to first present our work, it would be really great to have interactive feedback that comes to us at a faster rate than having to work wait for one hour before we start getting questions and feedback.
>> MARKUS KUMMER: Thank you. Well, we knew the challenge when we started, and we actually managed quite well by reducing the individual time to what we did. It was a challenge, but would you like to comment, Andrea.
>> ANDREA SAKS: I'm thinking back to your comment whether you or Malcolm started captioning first, and ‑‑
>> MARKUS KUMMER: The ITU copied the IGF.
>> ANDREA SAKS: Which they couldn't have, they had me and I was there before the IGF. But anyway, we will argue this later, but this is not the point. When I was talking, I was going to say something about the room configuration. We were on a stage, and what didn't used to happen is that we didn't have the screens in front, so the speakers could see them. And that was what I was going to say, if you get into a smaller room or a different kind of room, you might not have that facility because not only could we see the speaker speaking, we could see the captioning and it was at either end.
We didn't have that before unless we fought like mad for it. I think we did have it in Istanbul because I screamed and I got the boys to do it for me. I get the boys to do everything for me. I am the grandma.
The thing is that you have to think about what the features are that you have that make it accessible before you start changing the room. I think the room was too big because we had too many spaces, but maybe we should look at filling the room rather than, and also it isn't accessible with those stairs, but the thing is you are tied to the venue. So when we choose a venue, maybe we have to look at the main session rooms, because the ones in, what was it the tropical place we went to. Not that one, the one before. Bali.
That was a good main session room. That was terrific. That was all I wanted to say.
>> MARKUS KUMMER: One more comment and can we then conclude. Yes.
>> NICOLAS ECHANIZ: Two things, one, maybe if we want to be more inviting our title would be something more like meet the Dynamic Coalition in the IGF. So we are using IGF without making it a trademarks for us, and one other thing, and that would go well with moving it earlier in the schedule. Meet the Dynamic Coalitions, then we start working. And another thing from our experience, it might be interesting to start the session with one general question and invite just two or three people from the audience, like just asking what is your idea of what Dynamic Coalitions are, for example, and just invite two or three people to engage and define what's the idea because maybe there is a lot of people there that actually don't have a clue of what it is, and they are going to just to find out.
And starting with a question is always good to get the people engaged from the beginning.
>> MARKUS KUMMER: Thank you. Well, I don't think we need to carry on the discussion for much more. I mean, I think we all agree that the title wasn't the best possible title. So next year, we have to think of something else, whether or not we get a main session that will be up to Avri to fight for. But let's have this kind of meeting in any case, if not the main session, then let's have it a workshop or a super Dynamic Coalition session.
But I think it was a very good exchange, and in whatever form we should have that also next year. And improve on what we had this year. Clearly we need another title, and we need to try and make a better engagement with the other participants and whether that will be to continuation of the, what was it called, the idea survey I think this year, not writings. Rating sheets, but I would have one concrete question now. I think what do we do to rescue the surveys from this year? And I take it we should anonymize it and maybe extend the deadline and try to push it out through respective networks among your participants to get feedback. Would that be a fair summary of what I heard. Maarten?
>> MAARTEN BOTTERMAN: Thank you. I like questionnaires surveys and such. I do research as well and it helps if you can get people to answer. The think the second year in a row, we show that we can't get people to answer, so in that way it's not useful. I don't think on the last day or the fourth last day of the IGF you are going to be ail to do anything to save that, and in that way despite liking the theoretical value, for me the practical value is such that I don't intend to invest a lot of time in that.
>> MARKUS KUMMER: Thank you. It was what I heard that you did that last year, could you explain again.
>> BISHAKHA DATTA: What we did last year we found before the IGF we got very few responses and I think Jeremy did a little summary during the session and we did a campaign to basically get responses but let me tell you the positives and negatives. We basically said, look, at least we should get X number of responses. So we reached out to our networks, but the thing is that also means that we don't necessarily have the full diversity that we need.
So it's possible to game the survey in a sense because you can, yes. And the other thing that I wanted to say is that I think I share Maarten's feedback that theoretically I like the idea of a survey, but practically I'm not sure that it ties in that much to our work and perhaps one of the reasons is like, for example, I'm just going to use the example of Net Neutrality. I think Net Neutrality lends itself to a survey because I think people really have strong opinions, et cetera, et cetera. It's a very hot topic.
Honestly, gender and Internet Governance to a point lends itself to a survey, but maybe if we could at least next year or at some point make it easier, is it possible to change the, like you know, we all do surveys online. Is it not possible to use an easier interface Markus. I don't know how this works. Can't we make it like other surveys online?
>> MARKUS KUMMER: I have no answer to that one, and I would have to look to maybe Avri or Jeremy who is not here, but I see Luca was going to speak and then Maarten again.
>> LUCA BELLI: I think to comment we have room to save the survey and to anonymize it and to leave it open until the end of December or the beginning of January so I think we need to do this and we have the possibility to have concrete results.
>> MARKUS KUMMER: Marianne and Christopher.
>> MARIANNE FRANKLIN: Yes, I think there is nothing, all surveys need time to build up steam so I'm happy that we have another morning. I'm happy to here that's anonymized, then a lot of us, at least I can speak for myself and some of us in the Coalition would feel a lot happier about pushing it. We can get what we can from this particular round of survey. It is different, it is better, it's better constructed. But back to Bishaka's point what is is the tool for? It needs to have a reason an objective that makes sense. So we could continue with the survey next year and we need to decide what we want to get out of it. But the very valuable input we got from that interactive platform last year where all of the documents were open for online comment, and it was online comment. I find that extraordinarily valuable so one can have a range of feedback mechanisms, quantitative one is like a survey, simple polls and qualitative ones. Both together are very powerful. If they are designed and it is clear what purpose they are serving and perhaps tailored for each coalition's needs so that there is a feedback mechanism built into this process. We understand this, but one side does not always fit all.
>> AUDIENCE: I like the idea about data. My whole project is about data. I just popped on the surveys and looked. None of them achieved double digits as of today, and I don't know. It's tough. And by definition, we are not, what our ability to reach the community is largely determined by our Dynamic Coalitions already, the people are in. The attempt is to survey the broader IGF community. I find it's going to be very hard. And so my gut tells me that if that's the feedback we are looking for, we may not get it. The online documents I think are very, it's a great idea and I have seen that. One of the things I have experienced is if we do that I think there is a need to be transparent.
There are times that I have given input into documents and it's not actually, it gets projected, you know. No explanation, no even acknowledgment that there were suggestions, and if we are going to use that as a mechanism, there should be, there should be a way of creating dialogue because we are not really voting organisations. It's more consensus building and getting conversations going moreso than trying to get, you know, it's the process of the feedback that's really valuable and surfacing that and going into the actual outcome document.
>> AUDIENCE: May I respond. Last year the tool we had was very good, and it was transparent in this case it wasn't anonymized which I think in this sense was right. I thought it was a powerful tool we have under estimated and I would like to see it back on line again whatever tool we used. I'm not sure what the brand was.
>> MODERATOR: Maarten has been waiting patiently.
>> MAARTEN BOTTERMAN: First, I don't think it's the quality of the survey that's the problem. I'm happy to keep it open until the end of the month or to the end of the January, and if we manage to reach the double digits the first digit may still be a one. So that will not impress me, particularly because it's quantitative which doesn't make really sense. I do appreciate the use of the documents that we had last year. It led to some input from corners I hadn't known and for that it's important, you know how to use it also as a moderator. I agree that you reply on the comments rather than just change it without saying something. Thanks to Avri, I learned how to be better in that, because I wasn't used to it either. I can imagine that many of us are not used to it. So the document is a plus, yes. For me if you want to do the survey again next year, or again deliver a couple of questions, happy to do that effort. But let's see how much better we did in terms of responses than last year.
I don't think it's much better, and where do we put our efforts and our money? Maybe there is better ways like the qualitative feedback on documents itself.
>> MARKUS KUMMER: Thank you. Jerry asked for the floor.
>> AUDIENCE: My first obvious question is will the survey be accessible to the likes of myself, for instance, registering for IGF, there was a (Indiscernible) so I couldn't register myself. And the second question I think is it has been intimated what is the purpose of the survey in if there was a dynamic coalition on Irish people and it got bad marks, are we going to throw the Coalition out? What is the purpose of it? And do the results go back to the Coalitions? Can people who have voted see the results or is it just within IGF? I don't know the purpose of it.
So I think some clarity there, and also another thought that flashed into my head a few moments ago when we were talking about the name of the session. Interactive dynamic evaluation of activity session or ideas. Just a thought.
>> MARKUS KUMMER: Thank you.
>> ANDREA SAKS: Your Captcha which is the robot discerning little things that what you said it didn't get through and you couldn't use it. My name is spelled Saks. Thank you very much, and the thing is that there is a way of doing that and Gonila has information on that so maybe you could ask Gonila because I think everybody needs to read the Dynamic Coalition guidelines please. I bet nobody in the room has read them. You have. You get a prize. You would read them because I would kill you if you didn't read them.
So but I don't think anybody else has, and I think it's important. And that's an interesting test because how effective are we and accessibility is always like, oh, yes, right, she is going around preaching, but the thing is are we successful within our own group? So are we surveying ourselves for what purpose?
>> MARKUS KUMMER: The point was it should provide feedback to the Dynamic Coalition.
>> ANDREA SAKS: We already know that nobody is reading our ‑‑
>> MARKUS KUMMER: Now you are being cynical, but Gonela had an explanation.
>> AUDIENCE: Just quickly about Captcha, I understand why it's there because it means that robots can't get onto the site, and but there are ways around it. And it's not providing an audio version of Captcha because that is not workable either. But there is ‑‑ it's an easy way to do it is just to have a question, one plus one equals what? And if a person puts 2, it's fine. That's all that's necessary. So there is clear, you can say you can have some other text questions that everyone knows the answer to like what is the color of the sky. Presumably it's usually blue, I mean in some places like Ireland it's usually gray, but that's what I mean anyway. So there is a very easy work around to make Captcha accessible.
>> MARKUS KUMMER: Avri wanted to come in.
>> AVRI DORIA: I wanted to answer the question of what was the original purpose. I think they were originally suggested when all of the Dynamic Coalitions were sort of unknown. And so we did the papers with the editing, and then we had the suggestion of doing the surveys. I think we have tried the surveys two times. We have done the paper editing at least once. And that's, it was just methods to get more feedback, and to see which ones work and which ones don't. And I think one of the comments made about some tools being more appropriate for one Dynamic Coalition than another may be actually a clue that we don't necessarily need one tool that everybody is undergoing. We have got two tools now. We have tested them.
We kind of know how they work. Maybe we will come up with another tool to experiment with next year to see if we can start building a tool kit of things to get Dynamic Coalition as and every kind of Dynamic Coalition will probably have a different mechanism that helps get wider feedback. So I just wanted to try and answer the question.
>> AUDIENCE: I just wanted to underline the transparency that you have mentioned. I have done some research in participation online and off line, and the question is that people need to understand what they are, why they are asked to fill in that questionnaire or the survey or the writing sheets, and if it's necessary that we need to explain in this room what was the purpose of the writing sheets, then we have done something wrong. It needs to be obvious why we do that, and it needs to be obvious to all of the people that should fill in that writing sheet. So the transparency to my opinion is key to get more than, more than to get answers from people attendtion the IGF and beyond.
>> MARKUS KUMMER: Right, yes, please.
>> NICOLAS ECHANIZ: Regarding the online document commenting system, I think in our case, we were working with the declaration, it felt like a downgrade from what we were doing, which we were actually elaborating that document already. And I think maybe an interesting platform would be to adopt Wikipedia's media Wiki platform which many people are already used to which has a discussion section which is very useful as was commented here, and maybe, and that's already there to adopt, so just a suggestion.
>> MARKUS KUMMER: Thank you. Well, it's quarter to 6:00 and I think we have to stop at 6:00 so I think we need to come to some closure of some sort, and I take it there seems to be little appetite for squeezing much out of the survey, but at the same time some people may wish to continue, so why don't we leave it open to each. One comment made, I think, struck a cord with me, you said if you push it within your network, then you maybe get the biased result, you don't get the broader feedback that you are actually looking for. And also the comment that if you have to explain it among this group, then it's more difficult to explain to people who are not part of this discussion.
But so I saw a few hands.
>> OLIVIER MJ CREPIN‑LEBLOND: How was the survey publicized because if we publicized it in our main session today, I don't think we will get that many more answers. Was there any Twitter storm? Was there any, and if it was on Twitter, why didn't each one of us reTweet the Tweet?
>> AVRI DORIA: it's been being broadcast for a while. It has been Tweeted. It was when I gave the talk for setting the scene. The whole time I was talking the URL was on the board behind me. I Tweeted it during the session. I Tweeted it a couple of hours before the session. But nobody repeats my Tweets. So that's why there wasn't a Twitter storm, but certainly there was a bunch of Tweeting of it. It has been put out at various times. It was included in most of the announcements about the session. So we got to find a better way, but it wasn't just sat on.
>> MARKUS KUMMER: And we also have the booth and we said let's make use of that. We didn't have that last year, but there was very little interest, very little appetite.
>> AUDIENCE: Just following up on the Twitter comment actually I think one of the factors that I think it makes it reTweet extensively and I would like to interest use Smita who is my colleague and who has attended a number of the DC calls. We shared the load. So one of the things that we found is that because when you Tweet it, you don't go directly to that specific survey. You go to the page, and then you have to tell people like please find the DC‑GIG survey. That's another barrier to like you know normally when you give a link, you go directly to that.
>> MARKUS KUMMER: Yes.
>> AUDIENCE: You mentioned a booth. I was going to ask, what happened, was it a success? Was it worth doing it? It's the first year we are having a DC booth so can we have like a ‑‑
>> MARKUS KUMMER: Marianne.
>> MARIANNE FRANKLIN: I think it was great. We need coffee and biscuits and cookies. I thought the booth was a great idea. Please, let's have it again.
>> AVRI DORIA: And we need coffee or chocolates or something. We can't steal coffee, but maybe we can do chocolates.
>> AUDIENCE: Also from how Christopher Yu worked they didn't only have coffee. They have special people who really went out and socialized with us, like seriously they were all over the place.
>> MARKUS KUMMER: Well, they have the special group there, young people. So, you know, okay. Let's think about what we can do better. As I said, Eleanorra had distributed and it's post the on the website kind of agenda for this session, organizational best practices, co‑facilitators and Secretariat role coordination moving forward and all of the questions but obviously we will not have time to go into that.
My main thing was now what do we do with the survey and that a fair summary to say, okay, let it sleep and let it, let's see what comes out of it, but I think the general feeling is if you are in a single digits then it's really not worth pushing it too much.
But Luca doesn't agree.
>> LUCA BELLI: I'm not sure that was the general feeling. I think the generally feeling was to keep it open until the end of December and anonymize it but maybe I'm misinterpreting the general feeling. Maybe we can raise hands.
>> MARKUS KUMMER: That was my first reading of the discussion, okay, let's anonymize it and leave it open, but then I felt there was quite a lot of push back on the value of the survey. But we can do that. We can do that. We can anonymize it and leave it open and see what comes out of it. Okay.
>> OLIVIER MJ CREPIN‑LEBLOND: I have retreated Avri's Tweets which were not retreated so now you have one retreat.
>> MARKUS KUMMER: Christopher.
>> AUDIENCE: If we are going to change the instrument, we should dump the prior data because by having the requirement that you put a name on it, you have introduced bias into your data you would have to segment them into different, it's just textbook, I mean. If you are going to do that ‑‑ you are not losing very much, but it's a question of then we will get the feedback which is responding to prompts through certain means, but if you, because there is such an effect on asking you to, there is a known fact by asking you to put it you are going to create a sample bias between the two populations which you shouldn't combine the statistics. It's just textbook survey design. We are not losing very much.
>> NICOLAS ECHANIZ: My question is do we know if all of the feedback that has been expressed so far was by people that were ‑‑ because we had two versions. We have already changed it, right, we had the first version that was, we could not say anything in what stakeholder group we belonged to or where are we from or who we are and then the second version where we had the possibility to leave feedback, and say who we are and something like that, and then how we are going to change it again. Right.
>> MARKUS KUMMER: Have you followed it that closely? I have not. But I mean the point is valid. If you want to have a feedback we cannot change course. What I heard was we would like to anonymized survey otherwise it's not much value, but then if we do that, then we should discard all we have received so far, and then you put off the people who went to the trouble of filling in the form.
>> LUCA BELLI: Why don't we have two results and combine them or we have two results saying those are the people that have done it.
>> MARKUS KUMMER: We have survey A and survey B, it's comparing apples and pears.
>> LUCA BELLI: The first is people willing to say who they are and the second is for everyone.
>> ANDREA SAKS: I don't know why we can't put not obligatory, somebody could put it down and they don't have to because lots of times you fill out forms and they will say not required. Luca, we agree on something.
>> MARIANNE FRANKLIN: That would work without us getting too convoluted and the second technical point 101 is single digits, double digits, anything under a 100 does not warrant percentage rating for future rating.
>> AUDIENCE: This is a class, you need benchmarks so you need aggregate statistics because they tend to be four plus. That's just the nature. The scale is, the statement, do you agree, one, degree, five degree. And you can't actually take three as the median because three is never the median. The standard way you look at this if there is a bias you tend to be around 4 and if the graduation is between there so you have baseline either, so you if you do student satisfaction number, you compare it to the universe of all answers to that question across everyone or you look historically to the same questions done before, because I looked at mine.
They are all like 5s and maybe a couple of 4s across the board. I looked at all of the others they are all basically 5s and 4s across the board and that's the nature of the survey instrument we have chosen. To make sense of what that means, you actually have to understand what the baseline expectation of a response to a neutral question is because it's not going to be dead on 3. That's the nature of the measurements.
>> MARKUS KUMMER: Avri just checked the survey.
>> AVRI DORIA: I gave one of you all fives and sent it in without signing a name and it went through just fine.
>> MARKUS KUMMER: So the anonymizing is not really an issue.
>> AUDIENCE: So we have to change the instrument. If I got to that and saw the request, I wouldn't submit. That's the request is the ask, even though they can do it, if you don't make a clear ask, it's going to change the response characteristics of the people you are polling.
>> AUDIENCE: Why don't we just add after the name case not mandatory?
>> AUDIENCE: If you want to keep it open, you would say it's optional and make that clear on there. Segregate the data so you don't do it and then keep it open. You could try to do that. You still have to figure out what the baselines are going to be to make sense the data, but this is textbook survey.
>> MARKUS KUMMER: Let's talk to Jeremy and see how we can best make use of that and let's keep it open until the end of December. And let's take stock after that. Having said that, I think that brings us on the Agenda Item the way forward and my suggestion will be then that we follow our regular pattern and schedule a call in early January next year and then we have also more feedback from the survey and see whether it was worthwhile or not.
And I don't think we have much time to do much more today. I think it was a good, an animated discussion with our memories still being fresh of what I consider a good session, and with that, I would like to thank you Saul for your active participation., for your excellent presentations and for your discipline for sticking to and not exceeding the limited time slot allocated to you. Thank you, and I wish you an excellent rest of this meeting and have an excellent evening. Thank you. Bye‑bye
(Concluded at 0556).