I love that data can change my perspective or illuminate a topic. In the last couple weeks some new data has dropped about a couple topics I care a bit about: Accessibility and Mobile Web Performance. These are industry-impacting studies and I hope you’ll read and consider the findings yourself, but here’s a list of key datapoints that stood out to me and changed my perspective a bit.

Deque’s Accessibility Coverage Report

At axe-con this year, Glenda Sims (aka The Goodwitch) gave a talk where she released some findings from Deque’s Accessibility Coverage Report. Here’s some of the discoveries:

  • 57% of accessibility errors are detectable by automated tools
  • Upwards of 80% of accessibility errors found using guided testing

This is great news and here’s how it changed my perspective: I have often repeated the idea that automated accessibility testing caught around 20% of errors. I think that datapoint was something I picked up from a Karl Groves post from 2011. Based on Deque’s new lens for understanding the volume of issues, that number is much higher: 57%! How much more encouraging is it that tools like axe DevTools, Accessibility Insights, or Lighthouse can catch the majority of issues!?

The next level up from the basic tests are the “guided tests” provided by axe DevTools or Accessibility Insights which can catch upwards of 80% of all the major issues. It makes web accessibility feel more like a solveable problem than a looming debt. I see a future where the Web can be more accessible than inaccessible and that is something to strive for, users are asking for it. You are one button click away from knowing your site is 57% accessible. Isn’t that a great feeling!

Mobile Performance Inequality Gap 2021

Alex Russel (Google) collects a lot of data on Javascript performance, network conditions, and median to low-end devices. His 2017 report had a bittersweet pill to swallow that your critical path needed to be under 170kb. Alex is back with the 2021 edition of the Mobile Performance Inequality Gap, this year with some good news and some thought-provoking insights. Here are my favorite parts:

  • New recommended budget is ~100KiB (gzipped) of HTML/CSS/fonts and 300-350KiB of JavaScript on the wire (compressed).
  • The performance gulf between high-end and mid-end devices
    • 2020’s high-end Androids sport the single-core performance of an iPhone 8, a phone released in Q3’17
    • Mid-priced Androids were slightly faster than 2014’s iPhone 6
  • Device replacement timeframes are extending to around 33 months

How this changes my perspective: First of all, 100kb of HTML and CSS and 350kb of JS is a lot to build a page. Less of a backflip to have a fast experience. High-end phones getting faster are partly responsible, but the flipside of that boon is that the gulf between a high-end and mid-tier devices is widening. A 2020 high-end Android is like a five year old iPhone 8. And mid-priced Androids are more like a seven year old iPhone 6. Phenomenal to see it in that perspective. Two weeks ago I went through the process of decommissioning these exact old iPhone models, I understand the gulf more now. The baseline for me isn’t latest iPhone, it’s seven year old iPhone.

The fact that device replacement timeframes are extending was not lost on me either. My wife and I have lost the desire to update our phones every year. And my iPhone XR is 25% slower than the latest model iPhone. That’s something to think about when evaluating our sites. When you feel like a site is “fast”, you must check your device context. Testing on the latest phones may be “skating to where the puck is going to be”, but it also creates a dangerous blindspot in your product’s customer base.