Political Dichotomy Assessment: Project Journal and Development Timeline
Here's a quick insight into the progress I've made since I've started this project, hopefully it gives you a better idea of how something like this comes about, and the level of effort I've put into making this project.
Disclaimer: All sections before October 2019 were written in retrospect.
As part of some of the changes I was rolling out to an old Discord server of mine, I had the idea of creating a test to help people choose roles. The two main differences from other tests at this point were:
- The test is integrated into the Discord server's custom bot I made, so there's no need for an external site.
- Instead of approximating through a score, the test was purely operating on Boolean logic in an attempt to guess someone's beliefs through process of elimination.
While this proved accurate for more common beliefs and had a clean front-end, behind the scenes it was scaling horribly: the amount of questions and possibilities to be accounted for grew exponentially as the scope of ideology broadened.
Following some discussions and the retiring of the old bot, I returned to the drawing board and began drafting a chart in an attempt to categorize ideology; the main purpose at the time was to prove that a political center did not exist, and that ideology was not a left/right gradient.
I used the Political Triangle as a model, basically drawing on top of it in OneNote. As you can see, it was incredibly crude and inaccurate in a lot of places. A lot would have to be fixed, but you can begin to see concepts being ironed out such as the usage of a table before the logic gates.
Unsurprisingly, it got posted on /r/badpolitics; mocking aside, I actually did end up getting important feedback on the design from a few people in the thread.
The triangle itself was rather confusing without a proper explanation, and up until this point I had simply gave the explanation on the spot whenever I posted it; I felt it was more efficient and sensible to bundle the explanation with it onto a “poster” of sorts. It came with a warning and a guide to what each of the various colors represented.
The chart design badly needed an update, not just due to how ungodly ugly it was, but also because making changes was difficult when all the layers get merged on a standard image.
So I decided to sit down and remake the chart from scratch, but as an SVG this time. This would allow me to change colors, text, and positioning easily without affecting the rest of the image. This ended up being a good move, because over the next few months, the chart would go through countless minor revisions here and there.
I concluded my design was too complex for solely a chart, not just in a matter of execution, but also due to the nature of the topic itself. Too simple and it becomes just as inaccurate as the other charts I intended to replace, too dense and it's incomprehensible.
So, a new idea came to me: the other charts are known to use websites which are able to set up a test of sorts to walk people through the process. With a test I'd be able to convey these concepts without overwhelming the user with the information density.
At this point in time, I had no idea what I wanted the site to look like, so I focused on the more mechanical parts of the test with minimal CSS.
I settled on a model that fused the table-chart with the flowchart questions of May's design. The chart would narrow down the person to a specific “leaf” or sector of ideology, and then the flowchart would narrow it down to something specific.
In order to do this, I drafted a list of final results, or “branches”, and began making questions to bridge the leaf to the branch.
I eventually got around to setting up the site's CSS, using the aesthetic theme of another website I was running as a model. Things felt a lot more organized now that I could visualize the direction the test was taking and I had a clearer idea of how I wanted to execute the project.
After an extended hiatus, I returned back to work on the site, finally creating the page for the second question and changing up the color palette.
With the site coming further underway, and the SVG going through its countless revisions, I decided to shift my strategy. If I were to work on the questions first and make the results later, any changes I'd make to the results would force me to rework the questions that are dependent on them too.
I decided to go back and rework the branches themselves, since a lot of them were mostly either filler or too similar to others. I took notes until I settled on a balance of around two to three per leaf. This greatly helped me narrow things down and get a better understanding of exactly what I was adding to the test instead of just taking everything I could find off of Wikipedia.
Now that I had narrowed down the branches and started to get a more consistent idea of my plan for the leaves, I began my strategy of working backwards. In order to better complement the quiz portion and make the whole project more accessible to people newer to political theory, I'd shape the info/results pages for the leaves and branches around the goal of giving people a primer on the various ideologies that are being discussed in the test.
My plan was to provide a definition for each one, some key facts to clear misconceptions, and also a list of historical figures who would fall under each category.
By this point, the project had been moved to GitLab for a few months. It was around this time I began taking my planning from July and began on the execution. The list of historical figures got scrapped: the difficulty of accurately pigeonholing was not worth the effort when a list of names didn't exactly serve as a useful reference for those who weren't already familiar with their work.
The development of the “primer” aspects of the leaf and branch pages involved the following:
- The aforementioned definitions and key facts were kept, as they still served a use.
- An FAQ page was added, it explains the process of the test, and acts as a “wiki” of sorts for the whole site.
- Each leaf was given a key term that showed what concept is at the core of their worldview, and also a few statements explaining the leaf's relation to the previously answered questions.
- Reading lists are given for both leaves and branches, with the consultation of various people across the political landscape to ensure they are both introductory and also comprehensive. I put priority towards primary sources and texts that covered historiography, theory, and action.
As per the feedback I was receiving, I went ahead and made cosmetic changes to the site. Icons were added for each leaf, the triangle was completely recolored with contrast toned down, and the site was minimized, with unnecessary patterns/effects removed and the theme reduced to three main colors.
I started by sketching the icons, then eventually found similiar ones online that matched what I was going for. I edited them to match the color scheme of the site, then imported them in.
A large chunk of October was dedicated to finishing up the bio pages for the various branches. As of the 31st of October, 19 out of the 34 pages have been completed, with 87 books being linked as further reading material. Some of these books were more difficult to get than others, especially the Futurist Cookbook, which I am still working on getting together.
The Futurist Cookbook by F.T. Marinetti is an incredibly fascinating work, with one caveat. Digital copies are region-locked to the United Kingdom. So instead, I have taken to manually transcribing a physical copy I found at my library using LaTeX. Here's a preview of what I've done so far:
On top of that I've been experimenting with a few ways to improve the bio pages.
One of them being a “metadata” block for each book, detailing important information such as length and difficulty:
Another one was having specific, traceable schools of thought mentioned on the branches with a short definition, to give people something to connect to. Not sure if I'm going to keep this idea or not, but it's still a possibility for the time being.
By the end of November, I had finished up all the pages in the Radical and the Reformist clusters. A lot of this month was dedicated to cleaning up the various reading materials and grinding out a lot more of the boring stuff as we approach the end of the results pages. As for transcribing The Futurist Cookbook, currently I have completed a third of the whole thing.
Behind the scenes I have been consulting others for advice and beginning work on preparing for manually testing this with focus groups. I'll most likely begin putting this into action when the results are more concretely laid out.
The months of December and January were mostly dedicated to finishing up the results pages. As of today, only two results pages have to be done and I have already begun work on the next step in the process, which is creating the questions to filter from the leaves to the results respectively.
The methods of filtering vary depending on how many branches are contained in each respective leaf, but the overall format is the same. At first, I was doing logic-gates to filter; however, I scrapped that idea as the ordering of questions would simply introduce too much bias. Instead, I've decided to do it in terms of meters; each meter contains a principle which distinguishes the specific branch from the other ones within the leaf.
The core concept is similar to the meters present in 8Values, but the execution is very different, as you'll see very soon.
The most basic implementation of this is on leaves with two branches, such as the Moderate leaf. Within the Moderate leaf, the two branches are Syncretic and Pluralist.
I started by asking myself about what fundamentally separates a syncretist from a pluralist. What I decided is that they are both varying approaches to reconciling ideas; the syncretist resolves differences in opinion by fusing elements of ideas while the pluralist believes in the mutual conflict of ideologies and interests in order to keep each other in check.
We'll label our principles as Competition and Cooperation for the sake of simplicity. And this is where the inspiration from 8Values comes in; a set of ten binary questions will be asked, each contributing varying amounts to either Convergence or Coexistence, depending on their individual importance. For example, if I asked: “Are the best solutions typically found in the middle?”, that would give some points to the Cooperation but not to the Competition side of the meter.
And because we only have to filter between two, evaluating these results is as simple as comparing which one has the majority of points at the end.
The next level of complexity occurs if we have four branches on our leaf, such as what occurs in the Communist leaf. Within the Communist leaf, we have Autonomist, Councilist, Syndicalist, and Vanguardist as branches.
- Now that there is more than one, let us introduce the concept of matchups. Our meter system gives us the ability to evaluate one branch against the other, but we won't be able to add any more into each comparison. We will have to break this down into one versus one.
- Since there is an even number of branches, we can make our first matchup a two versus two. We will need a shared characteristic for each pair however, as to fit it into the 1v1 nature of the matchup. These shared characteristics we will label Democratic and Organic respectively.
- We can do this because both the Syndicalist and the Vanguardist branches ascribe to democratic forms of organization while both the Councilist and Autonomist branches ascribe to organic forms of organization. Being able to know which the test-taker fits into eliminates half the field.
- Because half of four is two, we can simply use the same process we used for the two-branch for each remaining pair, to get our final result.
And finally, let us discuss what is by far the most complex implementation, the three-branch. The reason I made it more complex is because we deal with an issue known as the Condorcet Paradox. In simple terms, it just means that when applying a matchup method to a field of three candidates, there is a specific scenario in which no winner can be determined. Due to the way we're applying the matchup method, this scenario is a lot more likely to occur than it normally should be, so we will have to find a way around this.
To demonstrate this, I'll use the Anarchist leaf, which happens to have three branches (Pacifist, Intersectionalist, and Communalist).
I begin by deciding that we will need every combination of matchups between each of the three candidates for the results to be evaluated fairly. The required combinations and their principles are as follows:
Pacifist v. Intersectionalist – Clemency/Retribution
Pacifist v. Communalist – Nonviolence/Insurrection
Communalist v. Intersectionalist – Majority/Minority
In the situations where a candidate wins twice, the dilemma is quickly solved; however, what happens if each one wins once? A possible fallback could be to check margins of victory: perhaps one victory is stronger than the other. However, that would introduce another issue, which is: what if the margins of victory are equal? We'll end up back where we started.
So, instead I have devised a definitive fallback, known as disqualifying matchups. To explain, let assume our winning principles are as follows: Clemency, Insurrection, and Minority. Insurrection as a principle is inherently antithetical to Pacifism, outright disqualifying it from being a possibility. This leaves Communalist and Intersectionalist remaining, and we have the Majority/Minority matchup to tell us which one wins. In this case, the “tie” would result in Intersectionalist being the final result.
April was an unexpectedly productive month for the project. Starting with the minor changes:
- Post-Left branch got reworked into New Left
- References for the majority of branches were redone
- The Monarchist branch was completed
- The “schools of thought” idea has been implemented; branch pages have been given a section listing real-world ideological currents
- Cleaned out junk files and properly organized the resources folder
The two major changes for this month is the overhaul to the manual/FAQ and the progress made on the filter questions.
Beginning with the manual:
- The link to the quiz itself has been removed, as it was causing too much confusion. The quiz is long outdated and it remaining accessible misleads people into thinking otherwise.
- I've begun rolling the manual into a GitLab wiki-page. It'll be easier to edit and will create less clutter to the site itself.
- Another reason I am doing this is because one of the long-term plans is to rewrite the HTML from scratch once the methodology is fully laid out. Currently the code is rather messy and chaotic.
- I may or may not self-host the rewritten site, depending on whether or not a static page (which GitLab offers) is a hindrance to the site.
The filter questions have been developed for both the Progressive and Proprietarian branches, and the layout should follow the below template:
As for other long-term goals, I'm planning to implement the metadata pages I was considering following the site's overhaul. For now the main priority is finishing the filter questions.
May and June were mostly wrapping things up with the branch pages and making actual progress on filter questions:
- It is now official that all thirty-six branch pages are done, marking off a major milestone for the project.
- I've gone ahead and made a directory for the various filter questions which can be found here.
- The divisions for each filter set has been decided (barring the Egoist vs. Post Left division). Currently, 11 out of the 32 expected sets are completed.
These filter questions are subject to revision as I am constantly testing them with various people I know from online political communities.
One of the things I didn't expect to complete by now was the metadata, but it has been added for every branch page. I put these inside drop-downs as not to overwhelm the reader with information. Check the wiki to learn more about how it works.
When creating these drop-downs, I had to consider how I wanted to split up the information. At first I was considering setting up a section for Importance, but eventually decided to roll that into a Seminal type for the key readings.
Even after I decided on the list of tags, there were still things I had to reconsider. A rating for Length is one of these, as the site currently contains documents ranging from 1 to 1000 pages. The original plan was to make every star worth 200 pages and every half-star worth 100, but ultimately this left a lot of resources with either a one or two-star length rating. Eventually, I decided to take into account what the most common page ranges are and developed a solution: ranges would get wider on the fringes (with incredibly short or incredibly long documents) and narrow towards the middle. This means that a two-star rating might cover a range of about 50 pages (books between 250 and 300 pages long), while a five-star rating would cover triple that (books between 850 and 1000 pages long). This would allow for nuance between the ratings of books of average lengths.
Currently, the Difficulty rating I'm doing rather loosely: I read random passages in the book, and judge its difficulty. However, I am considering a rubric of sorts in the future, it would just take some more brainstorming.
Genre was initially limited to a set of about five options (the way Type is handled), but eventually I decided the library is too diverse to keep it limited. Readers should be able to understand what each Genre tag means anyways, so this should not be a big deal.
Type, on the other hand, is using terms with a very particular meaning in this context, so it has to remain limited. I might add definitions for each type in the wiki later.
- Various reference files have been replaced with ones that are smaller and easier to read.
- We are now up to 15 out of 32 sets of filter questions completed.
I'm getting busier, so expect updates every few months from here on out as opposed to every month. So far, no new filter questions have been completed (although there's a few sets which I have started on and am hoping to finish by the next update).
However, during this time, there have been two ideas floating in the back of my head, it's just a question of how and whether I should implement them.
Starting with the simpler one, it would be adding a sort of “Radical Democracy” branch to the Populist leaf. This branch would cover things such as consensus democracy, participatory democracy, and so on, filling the space that used to be covered by the Third Position leaf. Question is what to call it, and whether or not it truly even warrants its own branch, as opposed to just implementing some of the stuff into other branches.
Second is a lot more of a drastic change, and that would be overhauling how the filter questions are done. What I've noticed is that a lot of these sets are me rephrasing the same question and it makes it difficult to come up with them. My thought was that instead of doing a combination of Yes/No questions, I present two passages for people to read (each providing their respective branch's stance on a central question) and asking the user which one they agree with more. However, while I think this could solve the issues that are causing progress on the filter questions to come to a crawl, I think it also has the potential to exacerbate them:
- Difficulty with applications to certain branches
- Ensuring the questions are something the end-user can be expected to have developed an opinion on
First update of 2021, here's a list of minor stuff to report.
- Proprietarian has been renamed to Libertarian following my discovery that the former term has a very specific meaning within anarcho-capitalist circles.
- Some of the references in the results pages have been redone, I won't list them all here, look if you want.
- The Regenerate cluster has been renamed to “Restore”, which I think gets across the general idea better.
- It was brought to my attention that someone went ahead and made a test which uses an approach identical to that of my original model from the May 2018 section. While this approach has been long abandoned by me for numerous reasons, if you're curious to see what it may have turned out like, check it out here.
The big thing is that I went ahead and overhauled the branch questions format (pictured below) as per the aforementioned plans. I have now completed 14 out of 32 of the pages in the new style. Below is the one I completed as part of an Assimilation versus Liberation question.
Some stuff I added which weren't mentioned in the last post:
- A slider is being used for the user input. The slider can be moved into one of five positions, depending on what you agree with more. The CSS is a pain to work with, so I'll add stuff like notches and labels later.
- Passages are paraphrased in order to ensure they're both of an equivalent reading level and also for the sake of keeping things concise.
- The overarching question is explicit, rather than implicit with the hopes of making the activity more accessible.
Overall, I'm very satisfied with the change. This method strikes at the core of the distinctions between each branch while also providing the end-user the necessary information to make a choice.
Concurrent to this, I'm doing some light brainstorming regarding how to rework the canopy questions and also how to deal with an issue I've anticipated for a long time but constantly kicked down the road.
Starting with the issue, it's the fact that there exists branches which can completely spill into different leaves/clusters if the test-taker approaches the questions from a different angle. The two most obvious offenders to me are Utopian and Anarcho-capitalist, but there is something to be said about how the Populist leaf overlaps with the Nationalist leaf.
Starting with Utopian, the lines between it and something like Technocracy/Meritocracy, while existing, are blurred. Spitballing for a second here, I guess it could be argued that Technocrats begin from the present and progress from there, while utopian socialists start from an ideal and work backward. Issue is that this distinction has to be resolved at the canopy, not in filter questions.
Same applies with anarcho-capitalism, ancaps' rhetorical focus on stuff like coercion and statism could lead them to fall into the “Political” focus on the triangle, either placing them as left-anarchists or even worse, Internationalist assuming they don't fall in the Abolish cluster. And that speaks to something else about right-anarchists/libertarians. It occupies this level of ambiguity between Abolish and Advance; normally this'd warrant it being placed in between the two on the triangle, but that position is already occupied by Progressive, which IMO is a better fit. In addition, within the same branch ARE reformist libertarians, so it's not like I can exactly take it out either.
To me, I can justify the placement in Advance (even for ancaps), in that they operate on what are fundamentally classical conceptions of property rights and political power. However, it's not just sufficient to consider my perspective, I have to also ensure the test-taker's perspective is accounted for in designing this test. I don't have a solution to this yet, but I figured it was worth acknowledging so I can begin figuring out one.
As far as the actual design of the canopy goes, the current design I feel has to be overhauled. The sorting into base/superstructure and hegemony assumes way too much self-awareness on part of the test-taker, and a test for those who already know the answer is pointless. As a matter of fact, during the course of making this test, I think I've developed a law: there is an inverse relationship between the burden on the test-taker and the burden on the test-maker when designing stuff like this.
An idea I had was to use a series of word clouds, have the test-taker select words which represent concepts in their world-view. Because these concepts are more concrete, it'll take the burden off of the test-taker. Here's a quick barebones sketch of what it may look like:
Perhaps it'd help if I ran key ideological texts or archives of online communities through a wordcloud generator for inspiration. Here's a wordcloud of a chapter ripped from The Communist Manifesto:
Regardless, the main priority right now is getting these filter questions done. I've begun periodically streaming my progress of it on Twitch, which has definitely helped to motivate me, since the process is generally tedious as all hell.
Was originally planning to wait until the end of June to write this, but I realized there's probably enough I've completed since March to warrant a relatively minor update.
- We are now up to 21 out of 32 completed sets of the filter questions. Currently a good chunk of the remaining are half-done, where I'm simply stuck on finding a final lead in order to complete them.
- The more important news is that I finally ported the site and its manual to my own personal domain/server. I most likely will permanently migrate to here once the site's eventual code overhaul is completed, since using GitLab for all the hosting is rather unwieldy.
- The manual now runs on MkDocs, there's a nice selection of extensions which I may look into implementing should I ever need it.
Strap in, because this is probably the biggest update in almost a year. Right off the bat, here's a minor status update:
- We are now up to 25 out of 32 completed sets of filter questions.
- I'm considering renaming the test to the BrownTest so it's easier to remember and there's less semantics about whether or not “dichotomy” is accurate. This is named because... the site is brown.
- I've been having conversations with the creator of the Three Telos Model. Despite the common triangular-ness, we had quite a few (constructive) disagreements in our approaches on multiple fronts. Having that sort of differing perspective should prove quite useful in reminding me of what I'm looking to accomplish here.
- Might look into adding a glossary to the wiki given how much I'm having to use neologisms to describe the function of this model.
The big one though is that I finally moved onto working on the canopy questions. And yes, the whole thing has been entirely reworked from the ground up. Here were the major issues with the previous canopy design, stuff I've been meaning do deal with for like over a year by now:
- The questions were far too direct, expecting the end-user to comprehend and process an incredibly abstract concept. Anyone who is capable of this probably already knows where they fall.
- Both activities required a paragraph or two of instructions which based themselves on a set of jargon that once again, wasn't something you could just expect the user to grasp. It's very likely their eyes would just gloss over the details.
One of the ideas I had at first was to replace the old two-tiered drag-and-drop instructions with a sort of grammatical approach, so what the user needed to do was more self-evident. Apart from the hell that was getting the CSS for it to work, the bigger issue was that there was just no real way to phrase the sentence in any way which made sense. The idea was quickly abandoned.
I wasn't sure how to else to proceed until I saw a Discord comment which just sort of kick-started a chain of eureka moments for me:
Taking this into account, I broke up all the Focuses into these sort of more specific topics to see what I'd get out of it.
As a reminder, the way the canopy presently works is that its divided into two sections, one for the Focus and one for the Approach. In theory, the Focus questions should determine what is most important to you, while the Approach determines what you plan to do about it.
Yet it occurred to be that I could simplify things down by combining both sections into one, each element being represented by one aspect of the same activity.
How exactly to go about this was unclear, but given the pains I was having attempting to fix up the CSS for the “sentence activity”, I decided to see what parts of the discarded filter questions format I could reuse.
It was around this time I was talking to the Three Telos guy, and I decided to take a look at an online test someone made based on his model. Overall it was about what I expected, but for some reason the way its questions were structured with the dropdowns gave me an idea.
Rather than just let the user select the answer via drop-down, I'd also force them to select the question via drop-down.
Let's assume that we have one of these drop-down menus. In this drop-down there's three questions which the user can select. Each one of these three options represents a topic within a given Focus. So this drop-down will contain a question on Tradition, a question on Distribution, and a question on Conflict.
Doing the math, we will end up with 24 drop-downs needed for the user to get to answer every question. But what happens when we give the user only say, six drop-downs they can answer? They pick the questions that are the most important to them. Thus we have dealt with the matter of Focus.
From there, we'll have the multiple choice of the answers vary with whichever question is selected. Each of these options still follow a common pattern in that they expess either a reactionary, radical, or liberal perspective on the issue. And there's the matter of the Approach handled.
I've slapped together a bunch of generic questions to show you the template of what this will look like:
A decent amount of progress has been made since the last update: starting off, a Social Democrat branch has been added, which doing so this late into development was not at all an easy choice to make. But there was a sufficient amount of information and distinctions which neither the Social Liberal nor Democratic Socialist branches covered. This also means I'm gonna have to get around to rewriting some of the branch questions for the Progressive leaf (much to my dismay).
As for the Canopy questions, they're done and fully written for the most part. This took quite a while, since I was conducting interviews and messing around with phrasing quite a bit to make it more accessible. Although I suppose I can fix up readability later. The main thing that still needs to be decided is how to distinguish the mixed leaves from the leaves fully within a single cluster.
The first question I was able to address a couple months ago using a decent amount of math, but unfortunately I don’t remember the specifics of what I did.
The matter of how many slots and how to allocate questions to slots was resolved a couple months ago using quite a bit of math (unfortunately I don't remember the specifics of how I did it):
- There are 6 slots, each one holding a question the user gets to select
- Each slot contains 3 selectable questions; one per Focus
- There are 12 questions in the pool total; 4 per each Focus
- In total, there are 24 unique combinations a slot can have
It has to be this way because:
- Three questions per slot means that each Focus has equal opportunity to be picked
- Six is a good number of slots: if it’s too small, we won’t have enough info, if it’s too large we’ll run out of questions
- Six is divisible by both 24 combinations and 12 questions
It has been quite a while since the last update, but I have good reasons. For you see, the test’s design has been pretty much finished. It is literally now just a matter of rewriting the code for the site, which I’ll be collaborating with a friend on. We’ve already begun on that, will be proceeding with the same sort of bottom-up approach to making it that I did for the test design itself.
As for some minor notes:
- As a page, Social Democrat has been added. I’m going to have to rework the interbranch questions for the Progressive leaf as a result.
- I’m debating adding Agrarian to the Populist leaf but I’d rather not think about that now.
- I’ve been periodically going back and updating the sources on the reference pages. As I learn more about these perspectives, the more literature I’m getting which would fit.
- Presently the site is designed in a way where all text is hardcoded. A key part of the rewrite is going to be breaking information down into JSON form so it’s easier to load, edit, and query.
The main hurdle I had between October and now pertained to the canopy questions. What I figured out in the previous updates definitively gave me a way to return the user a Leaf in any of the three clusters. But for leaves in between, there was no real defined way to get those.
The obvious thing to do would be to have it so two-way ties in Focus would result in that, but there’s some issues with that approach. One, that would be far too narrow of a criteria and would disproportionately put people outside of those categories even though that shouldn’t necessarily be the case. Two, it would not properly account for the other forms of overlap or the ways in which those Other categories are specifically designed.
The solution I came up with is to have certain answers in combination set off a flag. For example:
- The flag for Accelerationist is raised if one selects either of the following:
- the History question (providing an Abolish answer)
- the Production question (providing either an Abolish or a Regenerate answer)
- The flag for Progressive is raised if one selects both of the following:
- the Distribution question (with an Abolish answer)
- the Power quesiton (with an Advance answer)
- The flag for Populist is raised if one selects both of the following:
- the History question (with a Regenerate answer)
- either the Conflict question (with an Abolish answer) or the the Identity question (with an Advance answer)
If any of these flags are raised, the test will proceed to a second stage of the Canopy in which there will be a single slot intended to weigh any leaves which had their flags raised against the closest result obtained the normal way.
As for three-way ties, there’s no real clean solution. The most obvious one is to have a prime number of slots but doing that will screw up the test design in one way or another. It has to be six. My current plan is to have a drop-down of options with whichever set of leaves can be narrowed down to and then allow the user to continue the test from there. It’s rare enough of a case where it shouldn’t be a major issue.
There are two other ideas I’ve been juggling, but haven’t made a priority of implementing.
- A confidence meter which displays how confident the site can be in the user’s result. Would not be too difficult to implement and could come up with a scoring system to track uncertainty across tests. If it’s getting added, it’ll occur during the code rewrite.
- Including links to publications or think-tanks promoting certain views. Would like to do this since not many people have time or motivation to read books, but would require a great deal of effort to gather for every page. It’d be like doing references all over again, which is why I’m hesitant. Also I’m still not sure where on the webpage I’d put it.