Testwiki:Featured article review/Monty Hall problem/archive3

From testwiki
Revision as of 14:11, 20 November 2018 by imported>WOSlinker (fix small tag)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search
The following is an archived discussion of a featured article review. Please do not modify it. Further comments should be made on the article's talk page or at Wikipedia talk:Featured article review. No further edits should be made to this page.

The article was removed by Dana boomer 14:51, 13 June 2011 [1].

Monty Hall problem

Review commentary

Notified: User:Rick Block, User:Gill110951, User:Glopk, (top 3, the next two are topic banned) Wikipedia talk:WikiProject Statistics Wikipedia talk:WikiProject Mathematics Wikipedia talk:WikiProject Television Game Shows Wikipedia talk:WikiProject Game theory Wikipedia talk:WikiProject Psychology

WP:WIAFA concerns (1a, 1b, 1c, 1d, possibly 1e, 2a) are detailed below. But before delving into the details, I'll point out that there are major, long-term disagreements among a number of editors on how to present the topic involving the tension between WP:MTAA and WP:NPOV that have been going on for years, and culminated in an ArbCom case where two of the long-term contributors have been sanctioned, and several placed on restrictions or reprimanded. Unfortunately, the departure a couple of ArbCom-sanctioned editors has changed nothing of substance in topics in disagreement between the remaining editors. The issue that WP:FAR should be concerned with is that those disagreements have negatively impacted the end product: the article itself. Below, I organize my presentation by topic rather than WIAFA criteria, because some topics involve multiple criteria (even butting heads depending how one chooses to favor clarity or their view of npov) but I'll point them out. Before continuing, I disclose that I have contributed a couple of large-block-size diffs to this article, and a few minor ones. Of course, whether a change is semantically major or not doesn't exclusively depend on how much text is changed, so below I review the revision of the article before I had contributed anything, particularly because several long-term contributors to this article think that some of my edits were not an improvement.

1) Lead clarity vs. npov issues on the multiple interpretations/variants of the natural language statement leading to distinct mathematical problems. (WIAFA 1a, 2a, and 1d) Can you tell what do the words "randomly" and "overall" mean in the following chunk of the lead (emphasis mine)?

Template:Cquote

The part of the lead quoted I have quoted is supposed to communicate a mathematical result, as opposed to the vos Savant version just above it. Can you tell if the emphasized words are there for a purpose or are superfluous? What is hidden behind those two is the mathematical equivalent of WP:WEASEL wording that is trying hide a dispute in the interpretation of vos Savant's words, as I tried to explain [2]

Template:Cquote

Rick Block writes that the distinction between the two interpretations (and thus two mathematical problem-objects, one a subset, i.e. particular case of the other) is not important enough for the lead. It may not be immediately apparent why this is also a WIAFA 2a issue, but it becomes evident once you try to read the rest of article: the lead simply fails to prepare the reader for the problem variations which the proponents of various methods argue that their method is "the best". David Hilbert said "He who seeks for methods without having a definite problem in mind seeks for the most part in vain." The major problem variants don't have to be in the lead, but they should be certainly be stated before the several solutions are given, because these also try to convince the reader that the other approaches are wrong or superfluous.

There are plenty of secondary sources that make this separation, e.g. Rosenhouse (2009) Template:ISBN by chapter, but not Wikipedia. In a similar vein, User:Kmhkmh argues that even presenting one of the variants ahead of the other is "nothing but a subtle POV pushing". If you wonder how this could possibly be so, the answer is the next paragraph.

2) Another lead issue is that MHP made it to the pages of the New York Times (the first time around) in no small part because mathematicians disagreed on what math problem vos Savant's words should translate to, with some of them proposing even other variants besides the above two. Of course, I'm not proposing a list in the lead, but a large part of MHP's notability is due to its confusing language, at least in its original formulation, which should be said in the lead (WIAFA 2a/1d).

3) Using a degenerate case to illustrate the use of the "best" proof method for a more general problem/variant. (Like insisting on solving 0x2+2x+1=3 using the quadratic formula). I argue that doing this is confusing for the reader (WIAFA 1a) and I gave a list of RSes not doing this (besides Rosenhouse), i.e. who explicitly introduce the general case or a particular non-uniform (usually deterministic) strategy for Monte, (e.g. always showing preference for one of the doors when he has a choice) before using a general method. Rick Block however says that doing so in not npov (WIAFA 1d trumping 1a), arguing that the majority of math sources do this. Even assuming this is true (a claim for which he provided no evidence), it's still not clear that a head count is the best selection criteria for proofs. I tried to engage him in a more detailed discussion on how he compares two proofs for authority by asking him to compare one from Ken Binmore with one (Morgan's) Rick seems to favor, but so far without getting any reply on that.

(I'll stop with giving WIAFA callouts from here on because they are obvious for the remainder of this review.)

4) Inconsistent terminology throughout the article. Examples include referring to overall/average probability but also calling the case/path probability total. Requests to synchronize terminology from different sources met with "I'd rather not change it". As a results, the article is a terminological pastiche, contravening WP:MOSMATH.

5) Disorganized presentation and tangentiality. Before and after giving some solutions with quote/paraphrases whether the "simple" solutions are wrong, without ever trying to present this matter coherently (because that would require explicitly stating several problem variants). Text reads like "blah, blah, blah, did I mention the simple solutions are wrong?, blah, blah, ... , did I mention the simple solutions are wrong?, blah, blah, ..., did I mention the simple solutions are wrong?, blah ..."

6) The above is the result of endless POV wars between true believers in the various solutions, who frankly seem to be clueless that are talking about different problems. ArbCom didn't ban all of them, unfortunately, only the worst offenders. The counterpoint to the oft repeated (from one source!) claim that simple solutions are wrong, i.e. "the simple solution is right in the case of equal conditional probabilities (because the overall probab. is their average)" has been recently deleted from the article as "unverifiable" even though it was cited. Quite amusing chutzpah, given that several other sources concur with that, e.g. Rosenthal 2005/2008 [3] and those are free on-line. Ironically Rosenhouse cites Wikipeida for inspiration when making this point at p. 52 in his book. I guess this makes him completely unreliable per WP:CIRCULAR! He read the WP:Wrong version too! Someone email Science (journal) (which published a reviewed of the book Template:Doi) right away! Never mind he is a math prof at James Madison University, and can probably evaluate whether a argument like this is convincing or not.

7) The problem variants from given the large table are poorly organized and some are of questionable relevance. Never mind they repeat the snuck-through-the-back door variant that this-or-that solution was really solving. E.g. "The host acts as noted in the specific version of the problem." I have no idea what that refers to. Or "The host opens a door and makes the offer to switch 100% of the time if the contestant initially picked the car, and 50% the time if she didn't. Switching wins 1/2 the time at the Nash equilibrium." If you assume that the host strategy is fixed, it's a little silly to speak of a Nash equilibrium. It's a one-player game against nature, something that many game theory books don't even consider a game. Some decent secondary source like Chun 1999 or Rosenhouse should be used to organize variants.

8) Excessive formulism (for lack of a better term) is one of the Bayes' solution #1. Self-evident eye sore.

9) The "Sources of confusion" section is confusing if not downright POV. There are two issues: people being confused after being presented a definite math problem, and people being confused by the ambiguous formulation(s). No attempt is made to separate these. The implicit assumption there is that those making different assumptions about the game are idiots (other than Morgan of course, we are again reminded that the simple solutions are wrong!), including the profs from the NYT piece, and sources like Chun 1999.

10) Poorly researched from a formal sciences perspective (i.e. limited to STAT 101). Trivial variations between a bunch of proofs are presented as something of note. My note on the lack of serious game theory treatment in the article, which (I think) stymies understanding and only prolongs the absurd discussions, have been met with repetitions of the same obsessive "unconditional vs. conditional" mantras which have nothing to do with this issue. As David Eppstein put in on the ArbCom page, the so-called "advanced solution" using Bayes' theorem is the "basic of basics" as far as probability & decision theory is concerned.

Bayes' theorem vs is like mom's meat grinder next to a food processing plant when up against extensive-form games and Markov decision processes, (no need to skin the pig or chop the meat manually before the machinery takes over). As Ken Binmore puts it, once you formulate it as an EFG you hardly have to (creatively) think at all, meaning you just apply a well known algorithm to solve it. (Same goes for MDP or formulating it as a Bayesian game). Sources usable for this:

The fact that the above two are equivalent approaches is non-trivial in general, a result that played no small part in these guys getting a Nobel prize.

  • Handbook of weighted automata, Template:Doi, pp. 527-536 -- MDP approach, iterated value solution (uses MHP as running example to introduce the notions) This works because Monty has only one move.

10) Issues related to interpretation of probability not discussed. Suggested source: Georgii Template:ISBN (3rd ed.) pp. 54-56 (More correctly these are framed as issues stemming from what can or cannot be assumed common knowledge (logic). Crucially, the definition of a game like EFG assumes the rules are known by all player.) Also Rosenhouse pp. 84-88, but it's less useful. Olofsson Template:ISBN pp. 50-52 discusses it the same way as Georgii, but with less formalism.

11) Issues stemming from bounded rationality not discussed. E.g., Template:Doi Related to this, Rosenhouse p. 135-136 discusses "feeling bad about switching" (Olofsson also mentions this), and with Chun 1999 codifies this as an alternate game where the Von Neumann–Morgenstern utility does not equal the lottery probability. Chugh and Bazerman 2007 [5] is a good overview here.

12) For the uniform problem a combinatorial argument (with 6 layouts by numbering the goats) is given in Rosenhouse p. 54 (taken from Williams Template:ISBN pp. 73-74). Richard Isaac discusses a slightly different sample space counting approach in Template:ISBN, pp. 8-10. These are sufficiently different proofs to include I think. (Isaac also discusses the Gillman, OMG subtle POV variant, if you're curious, on p. 27)

I hope some article improvements come out of the above, but I'm not holding my breath. A fair number of editors repeat on talk the article is fine. Others make weird edits with strange if not misleading summaries reminiscent of WP:ARBPIA articles and stonewall to perfection on talk. Of course, this article may well deserve its FA star as "the best Wikipedia could ever produce on this topic given its social dynamics", but the answer to the question: "is this article a good presentation of the topic based on the sources available", the answer is clearly no in my mind. Overall the article reads to me like it was produced by a committee of humanities journalists who read a few math articles, and cobbled them together without really understanding what they are saying or trying to integrate them in a coherent (mathematical) presentation.

And as a courtesy for my time investment, please do not edit what I wrote above to either strike anything or interject your replies between my paragraphs. I have numbered the issues, so you can easily address them in the unlimited space below, and let the FA(R) director/delegates decide. Tijfo098 ([[User talk: |talk]]) 23:23, 7 April 2011 (UTC)

Tijfo, I cannot seem to find a notification from you of the possibility of a FAR on the article talk page. This notification is required as the first step in the FAR process. If you have not made this notification, and allowed time for the (obviously) active editors to work on the article, then this review should be placed on hold. FAR is for articles that have degraded to the point that they no longer meet the FA criteria, and do not have editors interested in working on them, as shown by a lack of response to the initial notification. Dana boomer (talk) 00:32, 8 April 2011 (UTC)
I do not see that requirement (that I must explicitly "threaten" with a FAR) at WP:FAR [6], only "In this step, concerned editors attempt to directly resolve issues with the existing community of article editors, and to informally improve the article. Articles in this step are not listed on this page." I have tried that. See the Talk:Monty_Hall_problem/Archive_23. The same discussion has been going in circles since then on the non-archived talk. I hoped the remaining participants would come up with something other than reiterating what they have been saying for 22 talk page archives. Slim chance of that, I'd say. Tijfo098 (talk) 01:18, 8 April 2011 (UTC)

@Tijfo098: Since I'm incorrectly cited above, I'd correct that and clear up some possible misunderstandings as well.

  • I did not argue that "presenting one of the variants ahead of the other is 'nothing but a subtle POV pushing'". Instead I've argued declaring one variant as "the" (canonical) MHP and the others "mere" variants is subtle POV pushing, because it suggests to reader that the variants are solving slightly different problems from that ("original") one posed by Whitaker in vos Savant's column (note its Whitaker's words/problem not vos Savant's), which is not simply not correct as some of those variants deal precisely with Whitaker's problem rather than a modified version. The crux is here that Whitaker's problem is ambiguous and there are different possibilities to address the ambiguity leading to different solutions or "variants". Whether this squabbling over the use of the word variant is of any importance depends how the article is written. If the lead is written as suggested above, I have no issue with that. However in the past there was a push to move anything regarding "variants" (and the ambiguity of the problem) completely out of the lead and first chapters ("keept it simply for the less educated reader"). That means a reader not reading the complete article but just the lead and/or first chapters would not have been aware of the ambiguity of the problem, various variants adressing the same ambiguous wording and the disagreement among mathematcians themselves, instead he would have learned only about the spat between vos savants and some acdemics and the related media storm and switching is the best strategy.
  • I agree that the article appears a bit like patchwork (as a result of the "eternal disagreements". Imho the best solution would be if all old editors voluntarily withdraw from the article (other than commenting) and a few new qualified editors attempt a complete overhaul. However you'll have to keep in mind that the MHP somehow works like magnet attracting plenty of editors who feel the need to leave "their" mark on the article often with a almost religious fervor. So even if the overhaul succeeds, unless the article is not closely guarded chances are over time it will turn into a patchwork again.

--Kmhkmh (talk) 03:24, 8 April 2011 (UTC)

Well, a fair number of sources: Rosenhouse (chapter title is "Classical Monty"), Isaac ("The most common version of the problem ..." p. 27), Rosenthal (gives other names to the variants like Monty Small) do make a choice as to what they consider the common interpretation; I can easily scrounge for more RS remarks like that. I do see the point that others (sources and Wikipedians alike) don't see it that way; they see their version as canonical. I think it's a minor issue of wording and perhaps WP:INTEXT attribution. Instead, at least two long term contributors hold the intelligibility of the article hostage over this issue. Tijfo098 (talk) 12:01, 8 April 2011 (UTC)
Their line of POV pushing is to argue that old papers, who usually have more citations simply because of that (first-mover advantage), are cited because they are right. Even if they the had a calculation error that stood [officially] uncorrected for 20 years, which may well have affected the authors' judgement of the simple solutions, because their fancy solution gave a different numerical result. Oops. Tijfo098 (talk) 12:01, 8 April 2011 (UTC)
There is also a problem of POV pushing based on Kraus and Wang. The Bayes formula lovers insist that K & W empirically prove that most people don't get the simple proof(s). [7] At the same time, these Wikipedians ignore the fact that K & W also show that while common people "buy" the Bayes's formula proof, they are utterly unable to apply it to a similar problem (considered a more realistic test of comprehension by K & W). So K & W is an RS when it agrees with their POV, but not when it doesn't. [8] Classic signs of POV pushing right there. No point in me arguing with these guys ad infinitum. (The diffs I gave are representative, but the same arguments are being repeated on the current talk page by the same main contributors. Typical edit from one of them: [9]) Tijfo098 (talk) 12:01, 8 April 2011 (UTC)
Well as I said, with your suggested lead I have no issue with use of the term of variant in there, but my comment to referring to (older) tendencies by (other) authors. I agree that there is also "pov pushing" from other side(s) as well, the problem is that too many editors insist on their personally favoured way to read & treat MHP and try to marginalize everything else and various sources just serve as tools in that battle. Editors often don't seem to care what a particular source is saying overall, but they just pick snippets suiting their agenda. Imho a stable and reasonably good article will only be achieved if all participants (including us) are willing accept that the article cannot match their preferred treatment of MHP. Since there seem to be various righteous and almost religious beliefs and a lot of invested ego from all sides, I think the article would need to rewritten by new authors, old authors might help out by providing information but they need to refrain from editing and give up on seeing their favored treatment in the article. Doing so (egowise) might be easier with respected new authors rather than "giving in" to other side of the longterm conflict.--Kmhkmh (talk) 12:41, 8 April 2011 (UTC)
We're in agreement here. Tijfo098 (talk) 12:57, 8 April 2011 (UTC)
Yes! The nail is hit on the head. Richard Gill (talk) 10:33, 9 April 2011 (UTC)
  • I agree that the article, as it currently stands, is not representative of "the very best of Wikipedia". Its present structure was "frozen in time" in the middle of an edit war (the "Aids to Understanding" section was being bounced up by and down). The content of several paragraphs is obscure, if not contorted (my favorite is: "they do not explicitly address their interpretation of vos Savant's rewording of Whitaker's original question (Seymann)."). Because of said edit war, and the intervening edit semi-freezes during mediation and arbitration, no clarity is shed any longer on the criticism of the simple solution. Overall I agree with Tijfo098's assesment, although not with his assesment of the editors' motives. So yes, this is 'not an FA, it hasn't ben for quite some time. glopk (talk) 01:16, 10 April 2011 (UTC)
  • Comment. I think a desirable outcome here would be a clear consensus of a path for article improvement. Normally, this is what a featured article review achieves. If, as seems likely, that path is still very contentious on certain points, then I think FAR may not be the right instrument to deal with these (real or perceived) problems with the article. It seems to me that the sort of criticisms raised in the FAR here are the kinds of things that would be, for the most part, better addressed on the discussion page of the article first. Sławomir Biały (talk) 13:22, 10 April 2011 (UTC)
This is an overly optimistic view of the editing environment on Wikipedia. The plain reason I started this FAR is that I think the article fails WIAFA in many ways, and that no reasonable time frame for addressing the issues seems forthcoming. Endless circular discussions on talk page (that have been going on for at least two years) is not a criteria for promoting or keeping articles as FA. In fact it's one for demoting them (1e), assuming the disagreements are in good faith, of course. Now that ArbCom has filtered out the perceived (really) bad apples, I was expecting progress, but alas that doesn't seem realistic. Tijfo098 (talk) 16:14, 10 April 2011 (UTC)
Perhaps that is true in this case. But I think it should be a priority in this process to develop some clarity on what needs to happen in order for the article to regain its FA status. I have some sense that the recent bout of petty quarrels is probably not that significant in the grand scheme of the FA status. Perhaps, if there is a clear path laid out, editors can work together to achieve a common goal, rather than working at cross purposes. Although the achievability of this outcome also relies on a healthy dose of optimism. Sławomir Biały (talk) 21:45, 10 April 2011 (UTC)
  • Comment Glopk hits the nail on the head here. The article is its current form is definitely not FA-class - but it has been on semi-freeze status during a protracted mediation and arbitration process. I suspect nearly all of Tijfo098's comments could be addressed by simply editing the article, since most are completely unrelated to the disagreements still running rampant on the talk page. -- Rick Block (talk) 17:30, 10 April 2011 (UTC)
  • Comment I read the article a year or two ago and was pretty impressed with it at the time. I looked at it again during the recent arb case and it seemed to me that it had gotten worse since when I'd read it earlier. I think the FAR may have been a bit premature since the arb case may have changed the balance of forces in the content dispute enough to be able to fix the problems through normal discussions. Maybe it's possible now to salvage stuff from the edit history that got corroded in later versions prior to the arb case.

    FWIW, I prefer less technical approach in the lead; for example, I had no trouble understanding "chooses randomly" (Donald O. Granberg's review in Science of Rosenhouse's book uses the same word in its own lead paragraph). "Uniformly" (while more precise) is IMO likelier to confuse a non-mathematical reader. In the arb case discussions I mostly agreed with Glkanter's content preferences, and found it sad that he was so terrible at collaborative editing that he had to be banned (there was really no choice about that).

    That Granberg (the book reviewer) is a sociologist rather than a mathematician gives me the idea of trying to "user test" the article, by going over to a non-mathematical wikiproject (like sociology) and asking for volunteers to read various versions of the articles and say which parts they found understandable. As math nerds we all understand the subject too well to put ourselves in the heads of the non-math people we are trying to communicate it to. 69.111.194.167 (talk) 08:34, 21 April 2011 (UTC)

I don't feel strongly about that, but random[ly] is wp:jargon used in probability problems to signify uniform. Ref [10] Tijfo098 (talk) 10:00, 21 April 2011 (UTC)

FARC commentary

Featured article criteria of concern mentioned in the review section include prose, referencing, stability, POV and MOS compliance. Dana boomer (talk) 15:23, 25 April 2011 (UTC)
  • I've not followed the article's talk page closely, but upon reading it today, it seems to me that the developments in the past few weeks can be summarized as "we agree to disagree" among the regulars. Tijfo098 (talk) 11:33, 26 April 2011 (UTC)
As one of the editors working on this, I would like to report that we are currently making excellent progress toward a restructured and greatly improved article. Content disputes are being handled the proper way, consensus is being sought, and pretty much everyone is working together in a friendly and productive manner. There are still disagreements over content, but there is also a plan to move forward and resolve those disagreements. I will report back here when the restructuring and editing is done, and that would be a good time for a featured article review. BTW, we really could use some more eyes looking at this and making comments or editing the page. It is easy to miss the obvious if you stare at the same material again and again. Guy Macon (talk) 17:24, 27 April 2011 (UTC) See comment below Guy Macon (talk) 19:18, 1 June 2011 (UTC)
I'm glad to hear some good news. Although I have to say I wonder how the POV tag currently placed on the article jibes with that. Tijfo098 (talk) 17:51, 27 April 2011 (UTC)
As I understand Wikipedia policy, in a situation like we have here (one editor who is under 1RR for ownership thinks there is a POV problem, everyone else thinks there isn't) the POV tag stays. The consensus is to finish the restructuring, examine the result, ask the dissenting editor exactly what changes would address his concerns, look at both versions, and then seek consensus as to which way we should go. Yes, there is still a content dispute, but everybody appears to be committed to following Wikipedia policy regarding resolving that dispute. Unless I misunderstand policy, leaving the POV tag is the right thing to do in this situation. Guy Macon (talk) 18:52, 27 April 2011 (UTC)

Comment This article has been at FARC for over two weeks, with little discussion in this section. Could we please get some comments on whether the interested editors believe this article should be kept or delisted, or whether additional work is needed and ongoing? Thanks, Dana boomer (talk) 14:36, 10 May 2011 (UTC)

  • Delist. I've caught up with the recent talk page comments (including the WP:LAME award), and I don't see substantive improvements or a consensus how to achieve that. Tijfo098 (talk) 09:27, 12 May 2011 (UTC)
  • Does anyone else have comments? Tijfo, if you have the time, pinging all of the editors who commented above and asking them to return and enter a declaration or further comments would be a huge help. Thanks, Dana boomer (talk) 17:01, 31 May 2011 (UTC)

Delist. After months of working out a plan to resolve the longstanding content dispute that resulted in an arbcom decision and major changes to the article that pretty much everyone agrees made it worse than it was when it was made an FA, I have pretty much given up (I did ask to be notified if some day they are ready to try my proposed solution.) I can no longer advocate keeping this a FA in its present state, and I no longer believe that any real progress is being made toward resolving the issues. The individual editors are doing av good job, but they are working at cross purposes because of the longstanding content dispute. Guy Macon (talk) 19:18, 1 June 2011 (UTC)

Has the situation deteriorated since your optomistic April 27 report? Can the 'working at cross purposes' be resolved? 166.216.194.65 (talk) 06:23, 2 June 2011 (UTC)

Template:Od

User behavior is still fine - no edit wars, no personal attacks, etc. Following Wikipedia policy is also good - everybody is citing reliable sources, no original research, etc. What has deteriorated is the chances of resolving the basic problem of different statistics professors being fairly evenly divided between two quite different and incompatible ways of explaining the Monty hall problem, with each group insisting that their way of explaining it must be in the lead and the other way of explaining it not be in the lead. I had come up with a plan which involved getting both sides to agree on creating two versions in talkspace that differ only where the content dispute required. Then I planned on shepherding the dispute through content dispute resolution and attaining consensus among a wider group of editors (nobody wants to wade though page after page of talkpage arguments about statistics) Alas, one of the most vocal proponents of one of the two sides refuses to cooperate with my plan. There is no requirement that he cooperate, of course, but without everyone agreeing on an easy to understand document showing exactly how the two sides of this highly technical mathematical dispute will look when translated to a Wikipedia article, I just don't see how I can expect any editors who don't happen to be statistics experts can judge the two sides of the dispute properly. So we are left with a good-faith content dispute, with both sides having quite reasonable - but highly technical - arguments as to why their POV should prevail. All efforts at compromise have failed. The arbcom action was a huge success at fixing the misbehavior issues, but of course the arbcom does not rule on content disputes.

Perhaps someone else might want to take a shot at being a neutral voice that does not take sides in the content dispute. I am getting a bit burnt out and am taking a break from it. Guy Macon (talk) 14:37, 2 June 2011 (UTC)


The above discussion is preserved as an archive. Please do not modify it. No further edits should be made to this page.
Retrieved from "https://en.wiki.beta.math.wmflabs.org/w/index.php?title=Testwiki:Featured_article_review/Monty_Hall_problem/archive3&oldid=33355"