Khan Engineering

Khan Engineering

We're the engineers behind Khan Academy. We're building a free, world-class education for anyone, anywhere.

Subscribe

Latest posts

How to upgrade hundreds of React components without breaking production

Jangmi Jo on September 23

How Engineering Principles Can Help You Scale

Marta Kosarchyn on August 21

Making Websites Work with Windows High Contrast Mode

Diedra Rater on March 21

Kotlin for Python developers

Aasmund Eldhuset on Nov 29, 2018

Using static analysis in Python, JavaScript and more to make your system safer

Kevin Dangoor on Jul 26, 2018

Kotlin on the server at Khan Academy

Colin Fuller on Jun 28, 2018

The Original Serverless Architecture is Still Here

Kevin Dangoor on May 31, 2018

What do software architects at Khan Academy do?

Kevin Dangoor on May 14, 2018

New data pipeline management platform at Khan Academy

Ragini Gupta on Apr 30, 2018

Untangling our Python Code

Carter J. Bastian on Apr 16, 2018

Slicker: A Tool for Moving Things in Python

Ben Kraft on Apr 2, 2018

The Great Python Refactor of 2017 And Also 2018

Craig Silverstein on Mar 19, 2018

Working Remotely

Scott Grant on Oct 2, 2017

Tips for giving your first code reviews

Hannah Blumberg on Sep 18, 2017

Let's Reduce! A Gentle Introduction to Javascript's Reduce Method

Josh Comeau on Jul 10, 2017

Creating Query Components with Apollo

Brian Genisio on Jun 12, 2017

Migrating to a Mobile Monorepo for React Native

Jared Forsyth on May 29, 2017

Memcached-Backed Content Infrastructure

Ben Kraft on May 15, 2017

Profiling App Engine Memcached

Ben Kraft on May 1, 2017

App Engine Flex Language Shootout

Amos Latteier on Apr 17, 2017

What's New in OSS at Khan Academy

Brian Genisio on Apr 3, 2017

Automating App Store Screenshots

Bryan Clark on Mar 27, 2017

It's Okay to Break Things: Reflections on Khan Academy's Healthy Hackathon

Kimerie Green on Mar 6, 2017

Interning at Khan Academy: from student to intern

Shadaj Laddad on Dec 12, 2016

Prototyping with Framer

Nick Breen on Oct 3, 2016

Evolving our content infrastructure

William Chargin on Sep 19, 2016

Building a Really, Really Small Android App

Charlie Marsh on Aug 22, 2016

A Case for Time Tracking: Data Driven Time-Management

Oliver Northwood on Aug 8, 2016

Time Management at Khan Academy

Several Authors on Jul 25, 2016

Hackathons Can Be Healthy

Tom Yedwab on Jul 11, 2016

Ensuring transaction-safety in Google App Engine

Craig Silverstein on Jun 27, 2016

The User Write Lock: an Alternative to Transactions for Google App Engine

Craig Silverstein on Jun 20, 2016

Khan Academy's Engineering Principles

Ben Kamens on Jun 6, 2016

Minimizing the length of regular expressions, in practice

Craig Silverstein on May 23, 2016

Introducing SwiftTweaks

Bryan Clark on May 9, 2016

The Autonomous Dumbledore

Evy Kassirer on Apr 25, 2016

Engineering career development at Khan Academy

Ben Eater on Apr 11, 2016

Inline CSS at Khan Academy: Aphrodite

Jamie Wong on Mar 29, 2016

Starting Android at Khan Academy

Ben Komalo on Feb 29, 2016

Automating Highly Similar Translations

Kevin Barabash on Feb 15, 2016

The weekly snippet-server: open-sourced

Craig Silverstein on Feb 1, 2016

Stories from our latest intern class

2015 Interns on Dec 21, 2015

Kanbanning the LearnStorm Dev Process

Kevin Dangoor on Dec 7, 2015

Forgo JS packaging? Not so fast

Craig Silverstein on Nov 23, 2015

Switching to Slack

Benjamin Pollack on Nov 9, 2015

Receiving feedback as an intern at Khan Academy

David Wang on Oct 26, 2015

Schrödinger's deploys no more: how we update translations

Chelsea Voss on Oct 12, 2015

i18nize-templates: Internationalization After the Fact

Craig Silverstein on Sep 28, 2015

Making thumbnails fast

William Chargin on Sep 14, 2015

Copy-pasting more than just text

Sam Lau on Aug 31, 2015

No cheating allowed!!

Phillip Lemons on Aug 17, 2015

Fun with slope fields, css and react

Marcos Ojeda on Aug 5, 2015

Khan Academy: a new employee's primer

Riley Shaw on Jul 20, 2015

How wooden puzzles can destroy dev teams

John Sullivan on Jul 6, 2015

Babel in Khan Academy's i18n Toolchain

Kevin Barabash on Jun 22, 2015

tota11y - an accessibility visualization toolkit

Jordan Scales on Jun 8, 2015

Meta

How to upgrade hundreds of React components without breaking production

by Jangmi Jo on September 23

Background

In 2018, we started renewing our color system using our new shared design system, Wonder Blocks (“WB” in short). WB reduces site colors from 48 to 18 to improve accessibility and consistency. It also includes one primary color to draw attention to actionable objects. We decided to remove and replace the old “domain colors”, which were specific to particular domains of content, such as “math”, “science”, etc. As part of this update, I was tasked with replacing our old buttons and links that were using those domain colors with our new WB components.

Strategies

Make a prop conversion guide from the target components to the new Wonder Blocks components

As I converted old components, I kept track of each old prop and how to achieve parity with the new component. This way, I could refer to the guide to make future conversions quite simple. In addition, anyone else helping out could also use the guide. Using it ensured all conversions were consistent.

The following is a small excerpt from the guide:


...
If the Button has these props, just remove them.
juicy
hoverColor
domain
type=”primary” (primary is the new default)
color (Use color=”destructive” if it’s destructive button. Otherwise, remove color prop)

If the Button has these props, consider the following:
width (Remove it from props and use minWidth in style)
...


Categorize the changes by team, keep the changes very small, and deploy often

By doing this, I could reduce the chances of merge conflicts and make deployment more painless. At Khan Academy, we organize the code by packages, each of which is owned by one team. By keeping each commit within one package, I could request reviews and ask design questions from the correct team and it was straightforward to write test plans since the change only impacted one area of the site. In general, it’s easier for reviewers to provide detailed reviews when the changes are small. I tried to keep changes limited to only touching 10 files. Some might say, “ain’t nobody got time for monster diffs”.

Commits shown categorized by package
Keeping number of files touched and lines of code changed small

Write a detailed test plan with before and after screenshots

Normally we would have a mockup for a new change but for this renewal, we had to do an entire codebase sweep without new mockups. Since these changes crossed so many areas of the site, I asked each team to assign a designer and code reviewer to validate the changes. We also needed to make changes in lesser-known areas of the codebase, so I wrote test plans including steps to navigate to where the component is rendered and screenshots at various screen widths (for responsive views) to give reviewers an at-a-glance summary of the change.

URLs and screenshots for responsive views in commit description

In practice

Having decided upon appropriate guidelines, I started the component conversion one by one. I made a list of all the components that needed conversion, and just picked one and went for it. Since our web application has been around for a long time, there are so many product areas that I didn’t know about. Still, I needed to touch code all over the place and I didn’t want to break anything accidentally. Naturally, having a good testing strategy was very important. Depending on the parent component of the target, I did the following:

Components in commonly used parts of the site

This type of component is mostly modern and relatively easy to test. They usually are specific enough that I can figure out where they are in the application and see it rendered before and after conversion to test. I included a before and after screenshot to simplify the review process.

Before and after screenshots in commit description

It became tricky when the component was shared across the entire app, like the Breadcrumb component, which is used frequently in different contexts. It takes more time and effort to thoroughly test each place. This called for a more detailed test plan. Even though the change was small, there were many places where this component was rendered that had to be tested. Again, screenshots helped reviewers out here.

Before and after screenshots for shared component

Components in less commonly used parts of the site

As a ten-year-old site, we have some older components in our code which aren’t changed often. In some cases, I felt lucky to find anyone who knew where these were used. Some of these older components didn’t even make good use of our system of “fixture tests” which render components outside of their normal context in the site (we use a custom framework for this, but React Storybook is another popular solution). The lack of good tests for those made safely making changes more difficult.

Example of when we used a fixture to verify the results

For those unlucky cases, I wrote a temporary fixture or injected that component in a known area of my development server with fake props just to see how the component looks. I then took screenshots and removed my temporary work. This was extra difficult because many of the old components have Backbone mixed in. The following is an example of a test plan using a temporary hack in order to verify the conversion.


Test Plan:

Comment out mixins: [BackboneMixin], in
javascript/exercises-legacy-package/components/exercise-footer.jsx
Make const maybeConfetti = null in render function.
Add return "" in javascript/tasks-package/completion-criteria-formatter.js:26
Uncomment javascript/exercises-legacy-package/components/exercise-footer.jsx.fixture.js
Visit fixture http://localhost:8081/react-sandbox/exercises-legacy-package/components/exercise-footer.jsx.fixture.js
Change the props to see various buttons.


Challenges

It took me a little over a month, totalling 68 commits and touching 174 files. During this procedure, here are some things that made this task challenging:

1. Lots of infrequently touched product area

Many changes touched older code which hadn’t been updated for years and didn’t have the tests we require now. That was tough but the only thing I could do was to try my best. It really reinforced the importance of writing tests. Writing good tests ensures future engineers can modify old code comfortably even when the original authors are gone.

2. Time difference and communication

During this project, I worked from South Korea, which is 16 hours ahead of Mountain View. That made communication harder since I was asleep during normal office hours. To combat this, I wrote detailed test plans with fool-proof reproduction steps so that I could minimize the number of questions and reduce back and forth on code reviews.

3. Button and Link components can cause either client- or server-side navigation

We started converting our site to a single page app (SPA) last year. Some parts had already been converted, but others still used the old architecture. Sometimes, it wasn’t clear whether I should use client-side navigation or trigger navigation to a new page. This led to an unexpected error after deployment when one of the converted Links used client side navigation to a page that wasn’t in the SPA. This caused the page to go blank for that route, and we had to quickly change the Link to use server-side navigation (This unfortunately happened while I was sleeping - thanks to John Resig for the fix!)

Summary

Converting a variety of older, inconsistent components to a single new component takes time, effort, patience, and good tests.

Thanks

Special thanks to John Resig and Kevin Barabash who supported the decision to convert links and buttons as part of the color system rollout and all the designers and engineers who reviewed the changes.