KA Engineering

KA Engineering

We're the engineers behind Khan Academy. We're building a free, world-class education for anyone, anywhere.

Subscribe

Latest posts

Let's Reduce! A Gentle Introduction to Javascript's Reduce Method

Josh Comeau on July 10

Creating Query Components with Apollo

Brian Genisio on June 12

Migrating to a Mobile Monorepo for React Native

Jared Forsyth on May 29

Memcached-Backed Content Infrastructure

Ben Kraft on May 15

Profiling App Engine Memcached

Ben Kraft on May 1

App Engine Flex Language Shootout

Amos Latteier on April 17

What's New in OSS at Khan Academy

Brian Genisio on April 3

Automating App Store Screenshots

Bryan Clark on March 27

It's Okay to Break Things: Reflections on Khan Academy's Healthy Hackathon

Kimerie Green on March 6

Interning at Khan Academy: from student to intern

Shadaj Laddad on Dec 12, 2016

Prototyping with Framer

Nick Breen on Oct 3, 2016

Evolving our content infrastructure

William Chargin on Sep 19, 2016

Building a Really, Really Small Android App

Charlie Marsh on Aug 22, 2016

A Case for Time Tracking: Data Driven Time-Management

Oliver Northwood on Aug 8, 2016

Time Management at Khan Academy

Several Authors on Jul 25, 2016

Hackathons Can Be Healthy

Tom Yedwab on Jul 11, 2016

Ensuring transaction-safety in Google App Engine

Craig Silverstein on Jun 27, 2016

The User Write Lock: an Alternative to Transactions for Google App Engine

Craig Silverstein on Jun 20, 2016

Khan Academy's Engineering Principles

Ben Kamens on Jun 6, 2016

Minimizing the length of regular expressions, in practice

Craig Silverstein on May 23, 2016

Introducing SwiftTweaks

Bryan Clark on May 9, 2016

The Autonomous Dumbledore

Evy Kassirer on Apr 25, 2016

Engineering career development at Khan Academy

Ben Eater on Apr 11, 2016

Inline CSS at Khan Academy: Aphrodite

Jamie Wong on Mar 29, 2016

Starting Android at Khan Academy

Ben Komalo on Feb 29, 2016

Automating Highly Similar Translations

Kevin Barabash on Feb 15, 2016

The weekly snippet-server: open-sourced

Craig Silverstein on Feb 1, 2016

Stories from our latest intern class

2015 Interns on Dec 21, 2015

Kanbanning the LearnStorm Dev Process

Kevin Dangoor on Dec 7, 2015

Forgo JS packaging? Not so fast

Craig Silverstein on Nov 23, 2015

Switching to Slack

Benjamin Pollack on Nov 9, 2015

Receiving feedback as an intern at Khan Academy

David Wang on Oct 26, 2015

Schrödinger's deploys no more: how we update translations

Chelsea Voss on Oct 12, 2015

i18nize-templates: Internationalization After the Fact

Craig Silverstein on Sep 28, 2015

Making thumbnails fast

William Chargin on Sep 14, 2015

Copy-pasting more than just text

Sam Lau on Aug 31, 2015

No cheating allowed!!

Phillip Lemons on Aug 17, 2015

Fun with slope fields, css and react

Marcos Ojeda on Aug 5, 2015

Khan Academy: a new employee's primer

Riley Shaw on Jul 20, 2015

How wooden puzzles can destroy dev teams

John Sullivan on Jul 6, 2015

Babel in Khan Academy's i18n Toolchain

Kevin Barabash on Jun 22, 2015

tota11y - an accessibility visualization toolkit

Jordan Scales on Jun 8, 2015

Meta

Kanbanning the LearnStorm Dev Process

by Kevin Dangoor on Dec 7, 2015

In his New Employee's Primer post from July, Riley Shaw wrote a little about our Teams Initiatives Projects (TIP) style of thinking about our development work. Here's a quick summary of TIP:

  • Teams represent areas of deep domain expertise like mobile, design, data science,
  • Initiatives are cross-functional, multi-project efforts that the organization believes are worth a significant investment, and
  • Projects are specific pieces of work that are scoped to take 2-5 weeks for 1-3 people.

A five-week, three-person project is substantial, so that gives you an idea of where an initiative starts in terms of scope.

TIP was adopted in February, so 2015 is the first year in which we've had initiatives. We don't have a specific process that each initiative uses to manage itself, which gives each initiative the freedom to figure out its best path forward.

LearnStorm

I'm working on the LearnStorm initiative. LearnStorm is a math challenge for students in 3rd-12th grades (and the Irish equivalents) in the Bay Area, Chicago, Idaho and Ireland. This initiative includes a collection of features that we're building into Khan Academy and a large amount of work from our programs team to organize things like helping teachers register students and in-person events. There are a bunch of us working over several months to make this challenge the best it can be.

Our initial development process

When we started development at the beginning of September, we used an approach similar to what other initiatives were doing at the time: we imagined the initiative as a collection of projects. We had rough ideas about how long each project would take and put the projects on a first draft timeline in a Google Sheets spreadsheet. We also had a Trello board that we used to show the projects in progress and the projects that were coming up. This was a useful way to think about the project at this early stage, but it didn't take long for us to start feeling limitations in how we could adapt our plans to handle new information.

Around the beginning of November, we switched from a general "working on projects" mode to a "need to launch signups" mode. It became important to track many small and medium sized tasks that needed to be done before we could open up signups. Over a crazy few days we moved from Trello checklists to a Google Doc and finally to a list in Asana.

That was a bumpy time.

Smoothing the process for the second phase

LearnStorm is different from other Khan Academy initiatives to date in that we have some hard deadlines. The challenge will start January 29th, 2016 for example. When it comes to getting a project out, there's the idea that you've got three levers to play with: scope, time and resources. For LearnStorm, the time is fixed. We also have the whole team in place that we're going to have, so the resources lever is also fixed (plus: Mythical Man-Month). Scope is the only thing we have to play with in making sure that the challenge is ready on time.

Two to five week-long projects can contain quite a bit of scope and don't lay that scope out in a way that makes it easy to choose between alternatives. Larger projects also make it harder to change priorities as we learn new things about the project.

What if there was a process that allowed us to easily see how the project is progressing and reprioritize features based on new information?

Enter Kanban

I had experience with Kanban in the past and thought it seemed like a good fit for our needs. Here is Wikipedia's description of Kanban for software development:

Kanban is a method for managing knowledge work with an emphasis on just-in-time delivery while not overloading the team members. In this approach, the process, from definition of a task to its delivery to the customer, is displayed for participants to see. Team members pull work from a queue.

Just-in-time delivery of the most important work is what makes Kanban work well for us. Each work item that we put into our queue in Trello is no more than a few days long. If needed, the priorities for the whole team can be changed in just a few days without interrupting the work that's already in progress.

Plus, these small work items also provide a lot of choices for prioritization. When working through a couple of big features, there might be pieces of each that are lower priority and might be skipped entirely.

But where's the big picture?

Since the beginning of the initiative, we've been having regular retrospectives to improve how we work. We recently had our first retro since moving to Kanban and there was a lot of positive feedback. Developers on the team found the process of grabbing the next work item and running with it to be a good way to focus on the most important work. The flip side is that it's hard to see the big picture progress we're making on the overall project.

We think a partial fix for this is easy: just make a document or diagram that groups the cards on the Kanban board with the overall project goals that they're bringing to life. I say that this is a partial fix, because there is a tradeoff in adopting small units of work as we have. It's much easier to see how 5 bigger projects become a whole than 25 smaller work items. And some of those small work items may never even be done.

Choice of tools

We write automated tests and try things out along the way, but we're still planning to have a focused testing and bug fixing period at the end of the project.

When we were getting ready to launch signups, Asana's straightforward list view worked well for rapidly collecting and sorting all of the feedback we got from the testers. I've found it much harder to work with Trello boards that have a large number of items on them, so we'll probably switch back to Asana for that phase at the end of the Initiative.

Trello and Asana store similar kinds of data, but the views and UI features make a huge difference in how you approach and use the tools. Unsurprisingly, people have used their APIs to present different views on top of the same data, but we haven't explored those third party tools yet.

Pick a process and improve

Kanban is not the "one process to rule them all". In fact, Kanban isn't a single process at all, but rather a way to think about and evolve the process you have. Continuous improvement is the name of the game, and that's what we've been going for. Our use of Kanban has helped us to collect, visualize and work on the most important things as our initiative evolves.