A THIRD Survey for NuRav
I took the “readability” survey. If you have any concerns about the site causing you harm be warned the whole thing is pretty much screenshots. And it’s looking at some really subtle differences between contrast and line spacing. 1/x https://t.co/noNKJDiEv1
— Carrie CraftGeek (@knitswhereitsat) July 8, 2020
NuRav Surveys: A Brief History of Preference Gathering
While the survey was up several concerns were expressed. The main ones I saw:
1. As it was only on Ravelry, it was inaccessible to people harmed by NuRav and Classic Mode Skins.
2. The majority of the survey asked about reaction to NuRav aesthetics with only a small section on accessibility.
3. It asked people to disclose disabilities
4. The survey was attached to user accounts, so it was not anonymous which was of particular concern for people with disability.
Survey Two was a targeted survey sent to Ravelers who had expressed harm from the site. In there July 8, 2020 update, Ravelry described it as
“…a qualitative open question readability survey with a population of Ravelers, with a particular focus on reaching out to folks who had selected vision-related accessibility options in their preferences.” Ravelry Update 7/08/2020
I can’t speak to the Second Survey as it was not generally avaialbe.
Now we have the SURVEY THREE! According to Ravelry this third survey is built from feedback received from Survey Two.
Third Survey: The Experience
The entirety of the survey is made up of closed, multiple choice questions meaning there is no opportunity to provide further feedback beyond the multiple choices. The survey begins with some questions you would expect: how often do you use Ravelry, what device do you use, are you on the forums, which skin are you using, etc. That’s all fine in my non expert opinion.
Then we get to the bulk of the survey. 22 pages of questions asking participants to look at variations of NuRav forum pages, buttons and text line height. The questions are all formatted thusly:
-Which of these do you prefer?
2 – 4 Radio button options that require looking at screenshots of different design options
1 radio button option for “I see no difference”
Remember when I said in the first paragraph that Survey Three was at least accessible since it’s on Google and not Ravelry. Well, Most of the survey is asking users, many of whom have reported adverse reaction to looking at screenshots of NuRav, to look at somewhere between 35 – 80 screenshots of NuRav. (I didn’t actually count the number of screenshots I looked at. This is an estimate based on the average number of screenshots per questions. Let’s just say it’s a lot)
That’s Not Accessible!
Even if you do not have an accessibility issue looking at the screenshots, it’s cumbersome. Most of the screenshots you have to click on a PDF to open in another window so you can see it. Some questions you must open as many as four different windows. (Clearly a non-professional created survey.)
But, if you choose to go through the hassle of opening up all these screenshots, and can manage to look at them without experiencing eye strain, migraine, headache, vertigo or a seizure what you are being asked to evaluate in many cases is … subtle.
There’s basically three categories of screenshots: buttons, forum pages, and text line spacing.
Each category has 5 -7 different questions about them with a different set of screenshots. (Note: this is all per my memory. I didn’t screenshot or record my screen during any of this so I may not be completely accurate about all of this)
The buttons are the easiest to assess because there’s no need to open a separate window. Basically the button options are looking at different shades of cyan, coral and a really hideous grey. I mean hideous. I think it has some yellow or green in it. It reminds me of a garbage heap for some reason..
Truthfully, all of the button options I wanted to say “None of these” But, that was not an option. The only option other than choosing a screenshot was “I see no difference.” Well there were clear color differences, even if I thought all of them were awful for different reasons.
But, you know what was REALLY problematic about the button options. EVERY SINGLE ONE HAD THE SAME FUCKING DROP SHADOW THAT HAS CAUSED ACCESSIBILITY PROBLEMS.
Seriously Ravelry, let the drop shadow go.
The screenshots around the Forum pages and text line spacing are the hardest to deal with.
Assessing text line height differences was at least perceptible, but I did take time to flip between the two screenshots to see it. But, the text was a dummy text which I found odd. It wasn’t exactly Lorem Ipsum, as the text was in English, but it was also nonsensical and didn’t encourage me to read the text, just scan it. This was a readability survey. I don’t know maybe I should be encouraged to read the text to determine if I found the text line height readable.
As discussed above, you have to open a PDF for each option to actually see them. And often the differences are almost imperceptible. There’s a reason Ravelry gave “See no difference” as an option.
One of the MAJOR complains about NuRav is the white background is too white. (This is probably one of the easiest things to fix by the way.). Unsurprisingly, there is a whole series of screenshots of the forum pages that are assessing reaction to the background color. Some of the variations are clear and obvious – oh that’s blue! oh that’s grey! And some the variations to choose between are oh… that’s white and oh….that’s white, too.
If you have a really good eye for contrast and/or color you may be able to detect that one white background has a drop more warmth than the other option. I was able to do so. I think. Maybe I just imagined it. But, if I do see a squidge more Red in one white background, I doubt most people are able to detect that difference.
Honestly, I felt like I was being pranked. I don’t think that was Ravelry’s intention, but that was the impact. It was made worst knowing they should have figured out the best, accessible background white before they even went into Beta Testing.
What is really maddening to me, is this survey is not even necessary if Ravelry would just hire a web accessibility consultant. Yes, that job exists, and it exists for a reason. Design decisions around accessibility should not be put up to a popular vote which is what this survey is doing. Accessibility is not about what most people prefer. Accessibility is about reducing harm.
But, Ravelry to the best of my knowledge is not hiring a consultant. If they have, they really shouldn’t keep that a secret. They would go a long way in showing they are taking EFFECTIVE steps in addressing their accessibility problem by announcing they are growing their “small team”
The Third Survey: Let’s Roast This Bitch
I think this survey is highly, highly problematic. I already discussed that the survey is still not accessible because of all the screenshots. I have already read accounts on twitter of people trying to take the survey and experiencing headaches, eye strain and vertigo.
But, even the framing for the survey is problematic. First Ravelry keeps calling this a “Readability” survey. The problem is not that NuRav is hard to read – well there’s that but that’s not why there is a huge shitshow now occurring. The problem is NuRav is causing HARM.
The other issue with the framing is Ravelry discusses this as a matter of preference, ie what do you like better. But, issues around NuRav is not about personal preferences. They are about accessibility issues adversely effecting people’s health.
I don’t eat shellfish because I’m Jewish and I follow some kosher mitzvot. But, if I’m in an environment where my only access to food is shellfish, I eat the shellfish. For me not eating shellfish is a preference.
If someone is allergic to shellfish, not eating shellfish is not a preference. They cannot survive in an environment where only shellfish is available. That’s a matter of accessibility.
|
Based completely on my own speculation, I believe Ravelry did not offer a “None of the Above” response because for aesthetic reasons. We know from Livia’s post “The Story of the New Look” that Ravelry staff struggled with choosing colors to match with the Coral color logo, oh wait sorry cinnamon color logo. (Yes, I’m being petty.)
|
Ravelry is people, NuRav is a Beta Test
Instead of investing resources into an accessibility expert, Ravelry is again turning to the members to do unpaid work for them. And, what’s worst is they started this giant Beta Test with no warning. Nope. They just drafted the entire 9 million members into it.
It was one thing for Ravelry to use volunteers to beta test the first site when they really had no income and were living on a dream. It was one thing to rely on unpaid members to moderate forums, and build the pattern database in exchange for a mostly free site that offered access to amazing patterns and an active knitting community. It was a barter between Ravelry business and the members and people felt like it was a square deal. Win-win for everyone.
But, now the balance is out of whack. Ravelry is asking some members who cannot use the site to put themselves in harms way to stare at screenshots of the site that has caused them harm. And within the survey, they are not offering real opportunity for people to give full, honest feedback because, once again, the questions are a closed. There is no option to say “None of these, they all suck let me tell you why” The only choice is pick one of these screenshots or “I see no difference”
People who want can try to reach out to Ravelry through email, but then the anonymity is gone. (Also, very likely will never receive a response) And, there’s a real trust issue now that Ravelry will even listen to one on one feedback. Time and again Ravelry has closed posts on For Love of Ravelry that raise accessibility concerns.
Good intentions and All that Jazz
There’s an old saw that the road to hell is paved with good intentions. The Ravelry’s teams’ intentions may be mitigating in the court of public opinion. But, their intentions don’t matter a hill of beans in the face of people being harmed by their site. (Yes, I just mixed a bunch of cliches. It was a style choice. I’m sticking with it.)
I saw your thread on Twitter and have read your blog post. I didn’t have accessibility issues with the site, but the survey gave me a terrible headache. Also, halfway through I thought to myself “what is the point?”
I think whoever designed the site is so married to the design, which is a bad thing for a designer, that they literally cannot open their minds to changing it.
The lack of response on this very important issue will be the death of their business if they are not careful. I said on Twitter and I’ll say here, someone is going to swoop in with a new, better site and steal their gig.
Thank you for this! For advocating where I can’t because I don’t have the spoons for it. I would prefer a livestream on YouTube on a hour I could join (I’m from the Netherlands)
💜 Marieke
Yes please do livestream. YouTube more familiar but either fine
Great! Right now I’m planing to do the Livestream this coming Saturday at 11:00 pm PDT. If you’d like and email reminder click on this link. https://mailchi.mp/977582323dae/livestream