Nokia Beta Labs, Shaping the future together

Hi Tommi,
sorry but I only agree on the “express our gratitude for them” and do not agree with the remaining 3 points.
I agree that you need to optimise the way you work as in every workplace (I’m guessing that was the aim of your boss request) but I disagree with this new "hierarchy of people". The reason is that I feel that "promoting" people into this hierarchy instead of promoting ideas is fundamentally wrong in an Internet age; it's the old way. The more layers you add, the less influential is the "mass" to the initial stages where the most important choices are made, which defeats the point of open beta stages. I feel the focus and efforts should be on Internet tools that
1) increase your audience by lowering the entry threshold.
* Things like an online emulator to test the application will help you collect feedback from even more people (people that cannot be bothered to install the app, but still have an opinion :-) ).
* Anonymous forums (http://en.wikipedia.org/wiki/2ch). Etc…
Ideally the bigger your audience, the more chances people's ideas will reflect the majority of people needs.
2) Optimise the flow of communication to promote the best ideas. I agree with the voting system, but it should be done, in a controlled way, by the mass not Nokia and hidden as someone said. I think you need to profile the voter before voting (by gender, age, etc to process the final results so that they match the consumer niche). In general, We should vote the need, Nokia should vote the solution.
3) Optimise the collection of feedback. You started this process with feedback forms.

Just my cents.
-Reda

After reading the comments left, I think most are focusing on the ratings and not on the benefits to your team. I hope your karma system works and gives your team more time to continue their fantastic work in creating the future.

WHAT CAN YOU DAONLOD TO THES WEB

how I can get a ulead photo express again in my cell phone?, because this application is had been deleted before n i'm too like this application. thank's for your replay

Linda@indonesia

I think it's a great idea and is likely to lead to much higher engagement with the beta community. The maemo team recently rolled a karma system out for the tablet community - you might want to check that out on Maemo.org.

Yes, i think this is an good idea.
If i understand you correctly, the points are not given from the comunity, but from the developers. Sounds fair to me1

Very good idea... I'll be happy to participate...

1...2...3.... GO !

Very very nice! :)

I think this is a good idea.

I'm all for making it easier for developers, do it!

Tommi,

In one year you'll arrive exactly to the point where you started the whole blog(s)/feedback thing.
Be very careful with this approach.

Great weekend to all of you!

I think it sounds good. Only thing that comes to my mind is that the feedback from active Beta Labs contributors isn't always the same as the needs of "normal" users. These Karma levels might enhance it even more. You've written about this earlier and I hope you haven't forgotten it. But if you keep this in mind and have other sources where you can get these "normal" opinions, I believe it will work out fine. With this system you will atleast know how to answer your boss. :)

First of all, the word "beta" has lost it's meaning ages ago. I'm not at all surprised that your Beta Labs is getting so popular. It's simply because you're offering also a few good and working apps here, something that people want. Heck, Gmail's still called "beta", too, with tens of millions of users.

Second, a karma system is really bad idea. What will happen is that positive comments without any real contribution will get plus votes, while critical though well justified comments will get negative votes. Also, I can't think why the number of comments would necessarily be a positive thing - usually the more comments the less content there is. Do you want quality or quantity?

But for today's question, how to identify the elite contributors. I think you should, while you can, identify them in-house from the content of their comments, and then contact those people privately and do some background and experience questionnaire and then discuss these things privately.

Very good idea, Tommi.

You said the ratings would be +1 and -1. I'd also add the option for 0, and it would sound like this:
+1=Useful
0=Changes nothing about the project
-1=Totally useless or inadequate
That way, the contributors "popularity" doesn't go down just because they repeat something that others have said before or is considered to make little difference.

I remembered that thing you did a while back, with Nokia internal testers. It could be applied here too. Make the app available to select contributors 1 or 2 days before it becomes public so you can be sure it works in terms of stability and get an early warning in case of problems.

And as a very modest idea of implementing such a system, I came up with this:
Since we can leave our email addresse on the feedback form page, you could create a database with the addresses (one that checks if an addresse is already in it and if it's not adds it). So, when the associated feedback is read, the rating it receives is associated with the addresse, and thus you can easily see who the top contributors are.

Sorry if that idea is confusing. Anyway, good luck with this initiative!

http://forums.pligg.com/pligg-mods/10560-digg-style-comment-rating-syste...

Why is it that you want the -1 ?

As far as I can understand you should only have the +1 option.

Reasons :
1. Positivism in general
2. A lab only wants the ups. And why should you measure the things that suck ?
3. Easier to develop
4. Easier to see numbers and reports

Good luck to you !

Tommi,
i think it is a good idea overall. Karma system will be helpful to developers.
Just be aware of contributors' ratings, this might eliminate "fresh" heads from the discussions. But their opinion can be valuable because it mostly coincide with opinions of final users.

Good luck!

I agree with what nbelousov said, ratings might discourage people to participate. So the best thing to do is keep them private. After all, only the Beta Labs staff needs to know the status of the contributors.

What about feedback that is inconvenient to developers? For example I fed back re Sports Tracker that the zoom keys should be consistent across Nokia apps, using Nokia Maps as the de facto standard.
So far this seems to have been ignored, but from a usability point of view to users of multiple apps, this is v important.
Can the developer's assessment be relied on in such circumstances? My experience indicates perhaps not.
It's a big turn off to have feedback ignored, and I think you need to make sure that inconvenient but valid feedback is valued, acted on and acknowledged. In my case this has not happened and that's disappointing and discourages further participation.
Karma is 2 way! A system that works one way only shows only half the picture.

I very much like the idea. Nokia is very good at listeing to its users and this might help you key in on certain users who continually provide top notch feed back.

As far as the concern about negative comments, I can understand the concern however I would hope that the community is mature enough to try and keep things constructive. I like the system which Engadget uses, however sometimes things do get out of control over there. Thankfully this is a much more controlled environment, should work better here.

Oh and the pre-beta for a few members does happen from time to time. The Mobile Web Server currently has a non-public going on for something which may very well end up on this blog in a week or two. Expanding this sort of idea though might help avoid some negative PR. The initial fiasco surrounding PC Suite's latest overhaul might have been avoided. Thankfully the situation was handled fairly well an nothing terrible came of it.

fully agree with kanta's comments - drop the beta label (as it's already covered by 'Labs') to lower the threshold yet avoid the dynamics of 'karma-like' community features and contact quality contributers privately

Thanks, I'll check out the maemo system! (and ask them how they did it)

I strongly believe that the +1 / -1 votes should become from Nokia development teams, not from the community. After all, the main purpose of this site is to gather feedback that could impact the development. And the only one who can tell what kind of feedback has an impact, and what doesn't, is the development team. And in any case, I don't believe anymore in letting the community do the voting (in product development issues). Just look at what has happened with Dell in www.ideastorm.com. There, half of the most-voted suggestions are focused around Linux vs. less than 3% Linux market share in real life.

> In one year you’ll arrive exactly to the point
> where you started the whole blog(s)/feedback thing.

How come?

> feedback from active Beta Labs contributors isn’t
> always the same as the needs of “normal” users

We are not looking for "normal" users here. Ideally, Beta Labs is about early adopters and lead-users = people who already now ask things that the mainstream would need in, say, 2010. Nokia has way better mechanisms for doing unbiased studies with mainstream users.

> karma system is really bad idea. What will happen is that positive
> comments without any real contribution will get plus votes, while critical
> though well justified comments will get negative votes

Hope not. It is waste of time - and destructive to your morale - to read comments like "this sucks and you are a moron" without any kind of constructive elements. But if something critical is well justified, I have seen that the message spreads inside Nokia like wildfire. Therefore, I'm confident that people who write constructive critique would win karma points.

> Do you want quality or quantity?

Both. But quality is much more important.

> I think you should, while you can, identify them in-house
> ...and then contact those people privately

Hmm... You might be right. Maybe we should keep the karma system private (not reveal the status of people in public).

> +1 and -1. I’d also add the option for 0

Yeah, but we don't need to do it explicitly. Teams should need to click voting buttons only when something is particularly insightful/moronic.

> email address on the feedback form page

That's a good unique identifier for the karma database.

> Why is it that you want the -1 ?

A small minority of the feedback is just rambling mess. I want to save time from the Nokia developers, not having to read this kind of stuff.

1) Let logged on users visually sort/sequence the http://www.nokia.com/betalabs/applications to publish their most liked applications - see example @ http://www.namastenetizen.com/collatodo/wp-sortable/?tag=software

--> up/down , promote/demote does not really help much. Any ranking should be relative to the entire list..

2) Effect: New users get to know immediately what is hot and not !

3) Let users comment be the feature requests and again let other logged on users visually sort/sequence the feature requests

4) When some one's feature request gets to the top of the comment list, they gain more karma :-)

This seems like a sensible idea to me as it would cut down on the signal to noise ratio. Some caution is needed as such a system might mean that some of the best users might not necessarily fit in with the grading system (e.g. a user who only makes one comment every now and then, but each one is golden). That said you could also use this system to identify such golden comments and users (i.e. versus someone who comments a lot, most of which are good and therefore end up with lots of positive karma). In other words its probably worth recognising that some +1 comments are more valuable than others.

I also think you might need to be careful about identifying elite users as a continuous group across all beta labs participants. Some people are going to be more interested in some areas that others. While I imagine there would be some overlap between the elite users for products a and b it would not be total. A universal elite system might miss some of the best elite users for a particular product. Perhaps you could include an application (generic area) id when giving karma. That way you break down your elite users into more than one group.

Overall though I would have thought something like this can only be a good thing.

Tommi,

Can I draw your attention to my post of December 31st? You will see that one of the things that I said was:
"It’s a big turn off to have feedback ignored, and I think you need to make sure that inconvenient but valid feedback is valued, acted on and acknowledged. In my case this has not happened and that’s disappointing and discourages further participation."
Yet These comments themselves have aslo not been responded to, so it would appear that these too have not been valued, acted on or even acknowledged.
Would you care to put this right, and provide a response?