Nokia Beta Labs

7a in San Fran can't be 9a NYC, there's 3 hours difference. 7a San Fran would be 9a CST (Chicago/Dallas), and 10a NYC.

@Joseph: could you explain why would it be a bad move?

> You want neutrality not favorism, competetion

I definitely don't want to promote unhealthy competition. Not to speak of playing favorites. But I need some way to indicate which of the opinions are likely to be worth attention, and which are not.

Having read tens of thousands of feedback submissions, let me assure you that not every opinion is equally valuable. Quite often, 90% of the insights. And, on a bad day, half of the feedback feels like this:
http://www.theonion.com/content/news/local_idiot_to_post_comment_on.

Of course, if you want to understand the general opinion, you need to listen everyone - or a representative sample of everyone. But Beta Labs has never been intended as a representative sample. What we have here, is an extremely insightful and innovative, but also a very biased community (for example, 96% are male).

The purpose of Beta Labs is not to make everyone feel happy for being heard. Instead, the main purpose is to help Nokia create better apps for everyone. The approach we have chosen here, is to let the most insightful "lead-users" have a direct conversation with the Nokia R&D teams. Because they are busy folks, and don't have time to listen or talk with everyone, we have to choose. And, to me, a formal reputation system feels like the best shot we have.

Anyhow, these were just my quick thoughts... What do you think?

Ilkka, how many people were in the chat?

Wow, IRC! I haven't touched that since the last century! Goes along with last-century operating systems and GUI's right? Will the new site be crippled to ensure compatibility with outdated WebKit-based browsers?

Freeware S60 IRC client:
http://mirggi.net/

> So whats UP…..Not gonna Post the Session

Sure thing: (abridged)

----------------------
[2008-09-26 17:14:04] TommiVilk: ok, let's go to concrete issues.

[2008-09-26 17:14:14] TommiVilk: #1 Categorization

[2008-09-26 17:14:48] TommiVilk: assuming that we have 50+ beta apps in the portfolio, how would you like us to categorize them? Some alternatives:

[2008-09-26 17:14:59] TommiVilk: 1) Only chronoligcally

[2008-09-26 17:15:30] TommiVilk: 2) According to application area (maps, PIM, entertainment, ...)

[2008-09-26 17:15:57] TommiVilk: 3) According to technical maturity (Stable, Some known issues, Early release)

[2008-09-26 17:16:57] TommiVilk: 4) Pre-commercial vs. experimental (for example, Maps and Music =

[2008-09-26 17:18:48] bheetebri: 1 and 2 make most sense to me, and if you ask me don't have to be mutually exclusive.

[2008-09-26 17:20:34] bheetebri: having a tag cloud with categories on the right hand side to filter...

[2008-09-26 17:20:56] Rafe-AAS: Maybe think about broad categories - after all you may only 2 or 3 to get list in each down to manageable numbers... e.g. stand alone application, service applications?

[2008-09-26 17:21:36] Rafe-AAS: Tag clouds are fairly meaningless unless the value of a tag is obvious

[2008-09-26 17:28:40] Rafe-AAS: More openness is good, expectation management might be good from the Nokia end too.

[2008-09-26 17:29:18] Rafe-AAS: but I suspect I'd tend to provide feedback about what interested me most :)

[2008-09-26 17:29:46] iseth: Yeah, if we know that the app may be included built-in on future devices, we're more inclined to check it out and provide feedback... at least I might

[2008-09-26 17:29:47] TommiVilk: To sum up:

[2008-09-26 17:29:48] * Arjun (7aa9b18e@webchat.mibbit.com) joins #nokiabetalabs

[2008-09-26 17:30:13] TommiVilk: 1) we should do better expectation management (pre-commercial vs. experimental apps)

[2008-09-26 17:30:28] TommiVilk: 2) figure out a good categorization ourself :-)

[2008-09-26 17:30:38] peltolai2: or design :)

[2008-09-26 17:30:51] TommiVilk: Let's go to second major topic

[2008-09-26 17:31:00] TommiVilk: 2: Reputation system

[2008-09-26 17:31:45] TommiVilk: First of all, we don't want to make this competitive: people climbing up the top-100 contributors ladder

[2008-09-26 17:32:20] TommiVilk: Nokia tried it in discussions.europe.nokia.com, and it backfired magnificently - people started voting each other down, in an unhealthy competition

[2008-09-26 17:32:36] TommiVilk: Instead, we would want to give badges to people with certain levels

[2008-09-26 17:32:46] TommiVilk: Now, I'm not sure how to do it

[2008-09-26 17:33:21] wsisti: how about based on the content of bug reports

[2008-09-26 17:33:21] TommiVilk: To start, we were thinking a couple of levels like (1) newbie, (2) labber, (3) bronze contributor (4) silver contributor (5) gold. Or whatever

[2008-09-26 17:34:17] TommiVilk: but then, I would love to give honorary badges to people like Rafe. He writes much of his ultra-valuable feedback at allaboutsymbian.com. And I would want to recognize this too somehow.

[2008-09-26 17:36:06] wsisti: i think the more "reps" you get the more access to pre-realse software. That way you can help shape where things go

[2008-09-26 17:36:12] Rafe-AAS: I think the last of those is the most important. I think 'Rewards' make for the wrong motivations. Do they need to be public? Though I do see the other side too. Its nice to get recognition.

[2008-09-26 17:36:46] TommiVilk: yes. For example, we could have 100 people in some insider list, to whom we could send our software a couple of days before public release

[2008-09-26 17:36:47] Rafe-AAS: i.e. identifying people who gives good feedback for more contribution opportunities should be number 1 consideration.

[2008-09-26 17:38:45] ktneely: So maybe some good way to tie a blog post on a personal or blog like allabout... directly into an app's comments/conversation page would be useful

[2008-09-26 17:39:40] Rafe-AAS: Presumably time resources are an issue so perhaps some automated system based on feedback submitted gives you a list of top contributors (ideally with some human score on quality of feedback). On a monthly basis use that list to ive out the badges (privately or publicly)

[2008-09-26 17:39:42] Arjun: Most of ppl write a feedback only when they encounter a probem

[2008-09-26 17:40:05] bheetebri: my reward for contributing is seeing how useful it is for Nokia. Participation [from Nokia] in the feedback process, in forums or so, is quite important and motivating.

[2008-09-26 17:43:12] Rafe-AAS: so automation is a must - for those that provide lots of feedback, but design in a way to add extra recognition? for the 1 out 1000 bit of feedback which is a gold nugget

[2008-09-26 17:43:28] bheetebri: I guess that you are right, but I do think that one way or the other it is important... things like these chats or something like this could be interesting and maybe not to time consuming for the team.

[2008-09-26 17:44:54] bheetebri: community is important... also for you, as I think you'll get better feedback.

[2008-09-26 17:45:01] iseth: Something like a hybrid of what Slashdot and Digg do

[2008-09-26 17:46:23] iseth: I don't think it matters too much what they [badges] look like, as long as the community knows what each one [badge] means

[2008-09-26 17:47:02] Rafe-AAS: Popular suggestion are not always the best ones - people don't know what's good for them. But yes ajax voting widget on idea (maybe class feedback as bug / idea / question) is good.

[2008-09-26 17:48:18] TommiVilk: also, experiences by Dell etc. have shown that if it is only users voting for suggestions, it doesn't provide intelligent prioritization

[2008-09-26 17:50:07] bheetebri: value or activity needs to be quantifiable, if you want it to be an automatic badging system

[2008-09-26 17:51:00] bheetebri: I don't get a lot further then something like: number of posts, number of threads started, and number of responses to a started thread. should be a reasonable indicator of activity and quality of the activity

[2008-09-26 17:51:27] iseth: Right - if a comment/post gets 10 positive votes, the posting user gets a point. Once that user reaches x number of points, they "rank up" or receive a badge

2008-09-26 17:53:23] Rafe-AAS: How much of the ranking equations need to be public? Surely just the end result. That helps stop people trying to game the system!

[2008-09-26 17:56:00] TommiVilk: quick poll: would you want a profile page of you visible to all, showing some background info + your full contribution history at beta labs? (yes/no)

Yes from all

[2008-09-26 17:58:00] Rafe-AAS: i.e. again i don't think its really necessary - nice to have - but maybe only the individual can see it?

[2008-09-26 17:57:49] TommiVilk: ---------5 minutes left in chat------------------

[2008-09-26 17:59:09] TommiVilk: ------Any final wishes for the new website?--------

[2008-09-26 17:59:15] Rafe-AAS: Yes - then we can tare it to shreds in front of you :)

[2008-09-26 17:59:28] bheetebri: i want more trial apps :-)

[2008-09-26 18:01:13] Rafe-AAS: Website - ahcievement section - this is what beta labs has achieved / explain co-creating. Oh and can we have a picture of you and Illka in white [lab] coats (maybe dissecting a phone) as a PR picture?

[2008-09-26 18:01:31] TommiVilk: Rafe: good point!!

[2008-09-26 18:01:46] TommiVilk: action point for Illka ;-)

[2008-09-26 18:01:50] peltolai2: ahaha

[2008-09-26 18:02:03] iseth: Rafe - that sounds like a great idea! The accomplishments thus far, plus a "intro to the team" kind of thing would be cool

[2008-09-26 18:04:01] wsisti: top contributors could get dev builds for testing specific problems. TiVo does something ismilar

You could have used Tangler.com for your chat session.

Just a suggestion.. =>

So whats UP.....Not gonna Post the Session :(

Hey Tommi,
Sorry for the late reply.
I do agree with you on many of the topics, if you do remember i spoke to you about it via email.
Yes a profile is good, but not an open profile. I see 2 viewing systems, one for testers and one for your team.
Test'es side would have a normal profile but with out a rank, of contributing, of post marks and that sorts. The reason for this is simple, u want everyone to to contribute to each other rather than leavin bias opinions to each other, or to whom i should remark or not remark, or ask him an not him for help. In a sense a pushes everyone to chat to each other blindly, but will show more productivity. Now on the side of your team it is different story. Of course i agree in a raking system, a detailed one in fact. But not of one rank. Your beta trials encompass a broad rage of applications and certain tastes are required for each. I think beta testes should be placed in many ranks, from strengths and weakness. You have one who is on the tech end, knowledgeable on the back end of application, where one is more media eccentric, user/UI friendly, out of the box thinker, and so on. Its valuable to you to recognize their contribution to certain projects.
I would like to know do you use location as a factor.
Location is very important(well background may play a change) but over-all location speaks for itself. For example i am from the silicon valley, so hence where nokia's competitors. Or USA/Europe views change, not in a bad way, but compelling information, i know nokia scores high in this.
The Sex, yes majority is male, but have you made in any strides of appealing to women?
Beta Labs is not open to the consumer market, a stronger presence (from front page nokia websites, a portal built-in device, on download, and i remember a mobile sites is in works ;) ) Yes this will spike up your drive-bye hits, but will recruit valuable testers. Create somethin specific to appeal to the women, be it media, something flashy, fashion, Cooking...so on. But its not there, u have their attention, but somethin simple is neeeded to direct their feedback(not saying they dont know whats up, but just in general.

Maybe for all testers profile to show their beta status privately to see where they would like to improve, media, user friendly, tech, so forth. Somethin like nokia pilots, you can see your over-all work.

Hmm....i guess this is what all comes to mind, but keep in mind my answers are based on as much as i understand of the direction you heading and your goals(not aware of full situation and road map) Joseph Asmar

@MiGs
Yeah, Tangler actually seems pretty good. I like how it looks and works. What I don't like is that it requires registration. I'm not sure if I would like to register for a service for a one hour chat...

@horia stanescu
I counted 12 at one point.

@virpi
IRC was easy to set up for a one-hour chat session. It served well for the purpose.

Opps, ahhhh i shoulda stayed quite. hehe