senior editors and baseline quality
senior editors and baseline quality
I'm really glad to see that these two ideas have made it into the final recommendations for quality. I'm a little disappointed about the lack of detail though. After all the discussions, I'm surprised that there weren't at least a few consensus ideas for how to measure these two things.
My impression is that we have not discussed enough. I suggest we continue the discussions and try to converge to more concrete ideas.
A good starting point would be to summarize these discussions:
Senior editors:
Baseline quality (more scattered):
- Thread:Talk:Task force/Wikipedia Quality/Let's set a palatable objective
- Thread:Talk:Task force/Wikipedia Quality/Measuring quality (narrow focus)
- Thread:Talk:Task force/Wikipedia Quality/Measuring quality (narrow focus)/reply (19)
Then we know that we'll have something that at least approaches consensus.
I'm concerned that the recommendations don't really have any sort of defined process for how to determine such things. It is nice to put forth a goal or a result, but it is another matter to define a path to said goal or result. There is also the matter of being in good standing, and many people have differing ideas of what content contribution is. Ottava Rima 03:24, 20 January 2010 (UTC)
Yes, that is a concern. The discussion focused on the goal of a baseline for quality, and the main agreement reached was to keep it simple. The criterium should be not-violating-NOR,-NPoV-or-V-policies.
However, there was not much discussion on how this should be set up. Personally I am very concerned that in areas where there is a strong bias (unitedly held by the owners of the WikiProject) it actually is pointless to hope that it is possible to mark articles as not meeting this baseline. The same forces that put in flaws, in the first place, will keep them from being marked. It tends to be a vicious circle. Perhaps somebody will have a bright idea on how to do this, but I have not seen one yet. - Brya 05:53, 20 January 2010 (UTC)
In the FA process, they frown upon featuring articles that are entirely reviewed by a single WikiProject. The idea being that as you get a bunch of different opinions in there, you're more likely to weed out bias. That, combined with a commonly held idea of "NPOV", keeps our best articles in pretty good shape compared to most trash on the Internet.
Baseline quality is harder. If we want a lot of articles to meet the baseline, we need a process that's going to be a lot less bureaucratic than featured articles. But that means it's going to be a bit more vulnerable to the whims of individuals. This is a hard one to deal with. Maybe the standard for baseline quality will unfortunately be soft on POV: that a baseline article should be neutral in that it covers multiple viewpoints in a neutral tone, but not necessarily neutral overall.
@Brya (this one's especially meant for you): projects are, in my vision, not factories that produce FA or GA, or other content. They are forums where people with the same speciality can meet and discuss. One problem (I have said this many times here) with Wikipedia is it attracts a community of editors with many technical but little social and communicative skills. Therefore, I believe the key words are discussion, cooperation and communication. Discussion leads to intelligent, critical enquiry, and is ultimately the only way forward to improve quality. Fear of others and their views or actions leads to conflict, angst, and stalemates where quality can no longer increase.
A statement that "discussion, cooperation and communication" are key, may sound very fine, but it does not necessarily mean anything, unless there is clarity about the objective and at least some mutual respect. Discussion may lead to many things, not necessarily positive. As to ultimate ways, well, I tend to keep away from religion. I do not see any other way towards improving quality than actually putting in the work.
My experience on the English Wikipedia is that Wikiprojects tend to be clubs of users who among themselves have come to an internal agreement to close themselves (and 'their' part of Wikipedia) off from reality, for example by exempting themselves from NpoV-, NOR- and V-policies.
Obviously, that is not necessarily the case for all Wikiprojects, but it does happen. A mere idea (for a Wikiproject, or for a "senior editor"), by itself, does not have particular value. It is all in how it is implemented and how it helps towards making the overall goal come true. - Brya 06:33, 22 January 2010 (UTC)
The general rule is that two people know more than one, and in the same way: two wikipedias know more than one. Yet that doesn't mean what they say is necessarily true. That's why we should have and follow guidelines about encyclopaedicity, neutrality, verifiability, balance and article coverage.
The tasks this task force envisioned for meta-projects are different from the things projects at wp-en are doing. Nevertheless, I am a member of the wikiproject:geology at wp-en and have its talk page on my watchlist. That particular project is working fine. When a user posts a certain question, I may answer. There are some 30 other geologists active on wp-en that may do the same.
Two people may know more than one if it comes down to a simple addition, but some of the things they know will contradict each other; the things they agree on knowing will be a lot less than what one person knows. As to the useful work they produce, there are many cases where two (or ten) people together do not produce half (or a tenth) the amount of work one person can do (if he knows what he is doing). It all depends. - Brya 05:43, 23 January 2010 (UTC)
Indeed, the idea is not to keep such a project as an elite club, but more as a driving force. A simple example: today as an admin I had to make a decision on an article about a computer game. I have myself no idea about the computer games. The deletion review was not active, only the author of the article voted (obviously, against deletion). What I did I went to the computer games project, asked their opinion, and eventually deleted the article. That was my responsibility, but eventually their expertise.
The problem is we're arguing in the abstract. We've just said "create a quality baseline" and "create a senior editor position". There's hundreds of bad ways to do both of those things, starting with the idea that any random editor can slap the tag "senior" on themselves after a certain number of edits, and that you can go in and say "this article looks like quality to me." If that's where this is going, then of course Brya is right.
This takes me back to the original question: what are the specifics for senior editors, and baseline quality? We should pick one and try to hash it out.
Yes, that would be desirable. The basic ideas, by themselves, could turn into a force-for-evil as easliy as a force-for-good. - Brya 05:55, 23 January 2010 (UTC)
Should we begin with baseline quality, then? We have two competing goals: finding a scheme that is quick and efficient enough that we can review and improve a huge number of articles, while simultaneously making it hard to exploit (especially by groups who have an agenda in seeing their POVs protected as "quality contributions").
Perhaps, although I haven't had any ideas pop up on how this could be done. However, it did strike me that the optimum way to give senior editors a chance would be to restrict gameability:
- Start by electing a cadre of ten (or twenty of whatever) editors with a proven commitment to core values (etc). These should be elected by a very wide margin of confidence (80% or better?).
- Any further senior editors are appointed by co-optation of the existing senior editors, who will also select for a proven commitment to core values (etc).
- Optional: Have a de-motion mechanism in place in case a senior editor fails to adhere to the core values (etc). Obviously this should be robust as every PoV-warrior would like to cry wolf at every opportunity.
This is not all that likely to be adopted (after all, Wikipedia is a democracy!), but it is simple and, if executed right, should maintain exclusivity. - Brya 04:42, 24 January 2010 (UTC)
Yeah, I think anything that's an election is a bad idea. Either you have some kind of open election process, in which case senior editors become about popularity. Or you do it like you suggested, where only a few people have voting rights, in which case it will be very exclusive. There isn't a snowball's chance in hell that the community will buy into that.
Much more palatable is to come up with some objective milestone. Something that any highly competent editor would be able to earn, and that would be difficult to game for editors who "don't get it".
(I don't think we could make it impossible to game. But we could make it hard. And then use a review process to deal with the rest of the problem cases.)
I did not say it was a very palatable idea, all I said was that it was the approach that would be most likely to achieve the desired results.
Basically there appear to be the following methods of selecting:
- open election
- a test to be filled in
- a number of edits to be made
- the human eye
As you pointed out open election will tend to be about popularity. It will be impossible to design a test that checks for anything but smartness and knowledge (a willingness to cheat will help pass the test). Obviously it is not hard at all to run up 10.000 edits or so without adding even a bit of content or showing anything but a drive to make edits.
As what we want is proven commitment to core values (plus proven ability), there is no substitute for the human eye (also basic to the whole idea of Wikipedia as opposed to something that is gathered by a bot or algorithm), the best way to check for the qualities wanted are other senior editors (it takes one to know one); second best would be to accept one of the existing mechanisms for checking for ability, that is having written FA-articles (plus perhaps a bit more). This is less than ideal, but better than anything else I have seen suggested. - Brya 04:46, 25 January 2010 (UTC)
Yeah, a lot of paths keep leading us back through FA. Yeah, featured articles involve a "vote" too. But you're judging content, not people. And the content is being judged up against some pretty well-defined criteria. I'd say that featured articles are one of our most reliable standards, and you'd be hard pressed to find something more consistent on the Wikimedia projects.
If someone has a few featured articles, they probably have a strong understanding of neutrality, verifiability, research, writing...
The only thing missing is whether that editor is disruptive or collaborative. I mean, a featured article is a couple of weeks of work, and anyone can play nice to get a few FAs under their belt. But then I think the best way to deal with this is to look for people who have previously been blocked or sanctioned, and have some kind of extra check to see if their behavior has improved.
As I said, it is less than ideal, but better than anything else I have seen suggested. I am very hesitant about blocking: somebody on this Wiki said that if you have never been blocked you are proven never to have stuck your neck out (you just do not care). And to some extent this is true, you can easily be blocked for the best of reasons: that is the way the system works.
What is wanted is proven commitment to core values; and at least somebody who has written a FA will have been exposed to them and is almost certain to have given them some thought. That does not equate to commitment, but it is at least something. - Brya 05:09, 25 January 2010 (UTC)
Also true: a lot of blocked users have stuck their neck out and paid for it, and then learned their lesson. That said, I think a block is a red flag. There ought to be a review process for editors who have such a red flag.
I just do not see any route except for elections, with may be some suffrage and some preconditions. Everything else is unstable against fraud.
Yes, elections is where it is likely going to end up. But it probably also is one of the worst possible scenarios (number of edits would be worse), leading to more-of-the-same. In that case what already is going well will keep going well; what is going badly will get worse; and where there is great controversy things will remain deadlocked. Popularity is not much of a way to establish the degree of commitment to core values (I have seen an editor who lived and breathed an excentric PoV, to whom NOR-violations were as common as pie, etc, etc make admin by a landslide). I don't believe elected senior editors will do much good. - Brya 05:21, 28 January 2010 (UTC)
Fine, but what other options do we have? Appointment by arbcom would be fine, but takes too much effort, this is not what arbcom has been elected for. I agree that counting # of edits is even worse.
I think the best we can do is to have a system like this:
- A fast track for volunteers who have met some objective criteria that actually shows an understanding of quality
- An election for volunteers who meet that objective criteria, but have been blocked (e.g.: they qualify, but need to explain themselves)
- An election for volunteers who haven't met that objective criteria, but feel they have done enough other good work to skip it
The fast track is key. It avoids the popularity contest and the politics. I don't think we can eliminate the need to have the community double-check certain editors, but maybe we can use it for specific cases.
That looks like a way to combine all the weak points? I am not sure what these "objective criteria that actually shows an understanding of quality" could be. Anything that can establish a commitment-to-core-values will by definition be subjective. What would be possible is to include some of the existing (subjective) tests such as the FA-procedure.
I guess that a workable process to guarantee a minimum of what is wanted would be to require an editor to have written an FA-article and start the vetting procedure from there. - Brya 06:02, 29 January 2010 (UTC)
Well, in my opinion, if we want to avoid the popularity contest, we should just PROHIBIT these stupid votes "Support" without any explanations. Only the vote "Support for the following reason: " (for instance, the editor, did not participate in edit warring, has not been blocked for disruptive behavior etc). And then make a suffrage of 500? edits in the main space (I believe a FA is too much).
Yeah, that's what I'm getting at. FA's focus on the editor's contributions, rather than the editor's personality. And we'd reserve the vote to include exceptionally good editors without FAs, or bad editors who have FAs but have been blocked in the past for being jerks.
A number of edits is meaningless. I have seen editors put in 10.000 edits without adding a iota of content. Writing a FA is not meaningless, thus it is a usable criterium. It is less than ideal, somebody could be very suitable without having written an FA, and not everybody who wrote one will have all the desired qualities. Still, it does not look like too much to ask, that any candidate wrote at least a FA. - Brya 05:28, 31 January 2010 (UTC)
How would you define "wrote"? Our articles are written by many people. That criteria is subjective and open to being gamed.
I would not define "wrote". Firstly a strict definition is likely impossible, as probably no FA is 100% the work of a single editor, but most FA's are mainly the result of a single editor's effort.
Secondly, it is mostly irrelevant as in my view writing a FA is by itself not sufficient to qualify for senior editor. It is sort of a minimum to be up for consideration: I would like to see extra requirements added. - Brya 04:59, 1 February 2010 (UTC)
There would still need to be some sort of approval process, managed by bureaucrats. And part of that approval process would be showing that you've been a major contributor to however many FAs. Maybe by having someone who worked on the article nominate you, or maybe by providing some "diffs".
And then there would be a recall process for anyone who was deceitful, and became a senior editor by lying about their contributions to a FA.
All that would make it very unlikely that someone could game the FA standard.
The lesson that Wikipedia is an uncaring and capricious bureaucracy? It is equally possible to argue that nobody should become a senior editor if he has not been blocked for standing up for something he believed in (he is a whimp). Arguing abstracts won't lead to much. - Brya 05:11, 28 January 2010 (UTC)
About exclusiveness, I am not all that pessimistic. After all, the ArbCom is also quite exclusive, as is the WMF board. There is a correlation between exclusivity and effectiveness.
What we want requires some fairly exclusive traits (commitment to core values, proven ability). In addition the senior editor must be willing to put in the time and effort to use those traits for the common goal. It looks to me that exclusivity is what is needed. If it is not exclusive then it won't work.
What is to be prevented is a closed-off, stale structure. There should always be the possibility for new blood. If that is a concern it would be possible to build in a term-of-office or an inactivity rule (the latter would likely be more effective).
Exclusivity may not be palatable, but it is pretty much inescapable. Otherwise, it becomes just another userbox to add to a userpage. - Brya 04:59, 25 January 2010 (UTC)