KA Engineering

KA Engineering

We're the engineers behind Khan Academy. We're building a free, world-class education for anyone, anywhere.

Subscribe

Upcoming fortnightly post

Migrating to a Mobile Monorepo for React Native

by Jared Forsyth on May 29

Latest posts

Memcached-Backed Content Infrastructure

Ben Kraft on May 15

Profiling App Engine Memcached

Ben Kraft on May 1

App Engine Flex Language Shootout

Amos Latteier on April 17

What's New in OSS at Khan Academy

Brian Genisio on April 3

Automating App Store Screenshots

Bryan Clark on March 27

It's Okay to Break Things: Reflections on Khan Academy's Healthy Hackathon

Kimerie Green on March 6

Interning at Khan Academy: from student to intern

Shadaj Laddad on December 12

Prototyping with Framer

Nick Breen on October 3

Evolving our content infrastructure

William Chargin on September 19

Building a Really, Really Small Android App

Charlie Marsh on August 22

A Case for Time Tracking: Data Driven Time-Management

Oliver Northwood on August 8

Time Management at Khan Academy

Several Authors on July 25

Hackathons Can Be Healthy

Tom Yedwab on July 11

Ensuring transaction-safety in Google App Engine

Craig Silverstein on June 27

The User Write Lock: an Alternative to Transactions for Google App Engine

Craig Silverstein on June 20

Khan Academy's Engineering Principles

Ben Kamens on June 6

Minimizing the length of regular expressions, in practice

Craig Silverstein on May 23

Introducing SwiftTweaks

Bryan Clark on May 9

The Autonomous Dumbledore

Evy Kassirer on April 25

Engineering career development at Khan Academy

Ben Eater on April 11

Inline CSS at Khan Academy: Aphrodite

Jamie Wong on March 29

Starting Android at Khan Academy

Ben Komalo on February 29

Automating Highly Similar Translations

Kevin Barabash on February 15

The weekly snippet-server: open-sourced

Craig Silverstein on February 1

Stories from our latest intern class

2015 Interns on December 21

Kanbanning the LearnStorm Dev Process

Kevin Dangoor on December 7

Forgo JS packaging? Not so fast

Craig Silverstein on November 23

Switching to Slack

Benjamin Pollack on November 9

Receiving feedback as an intern at Khan Academy

David Wang on October 26

Schrödinger's deploys no more: how we update translations

Chelsea Voss on October 12

i18nize-templates: Internationalization After the Fact

Craig Silverstein on September 28

Making thumbnails fast

William Chargin on September 14

Copy-pasting more than just text

Sam Lau on August 31

No cheating allowed!!

Phillip Lemons on August 17

Fun with slope fields, css and react

Marcos Ojeda on August 5

Khan Academy: a new employee's primer

Riley Shaw on July 20

How wooden puzzles can destroy dev teams

John Sullivan on July 6

Babel in Khan Academy's i18n Toolchain

Kevin Barabash on June 22

tota11y - an accessibility visualization toolkit

Jordan Scales on June 8

Meta

No cheating allowed!!

by Phillip Lemons on August 17

The problem

Recently, a number of students on Khan Academy found a way to cheat by taking hints offline and not having them counted towards their online profile. When going through exercises on Khan Academy you answer the problems given to you and receive feedback on whether your answer was correct or incorrect. If you get stuck on a problem you are able to take hints and have that problem counted as incorrect. Check out this exercise if you want to try it yourself. The images below show the user getting a correct answer and taking a hint respectively.

Correct answer screenshot Taking a hint screenshot

The cheaters realized that if they disconnected from the internet, took the hints, and reconnected, they would still have a problem counted as correct. Taking offline hints worked this way because our servers expect a request from the client when users take a hint or answer a problem. If the users were disconnected from the internet the server would never see the request and the request was not stored anywhere on the client so it would be lost.

How did we fix this?

In order to address the offline cheating, we decided to change how the client sends requests to the server. By utilizing the client’s local storage, we could store failed requests to be retried once the user reconnected to the internet. This solution has the added benefit of removing the need for the client to be connected to the server all the time. Users with a spotty internet connection would have a better experience because everything would work even if the internet cut out for a short period of time.

In our new architecture, anytime a user performs an action a string representing that action is stored in a queue that is saved to localstorage. When the queue is consumed, each action is mapped to a function that implements the action. This approach allows us to have more control over what happens when a request is not received by the server. The new queue retries any actions that fail and implements a linear backoff function so as not to be constantly sending requests when the user is not connected to the internet.

Below is an image that shows the old architecture (left) and the new architecture (right). If the old client never received a response from the server the request would never be retried. In the new architecture the request is retried until it reaches the server and we get a response.

Architecture screenshot

A nice consequence of this architecture is that it can be generalized to work in other parts of our system. Code that deals with sending requests to the server can be updated to use this architecture and work more consistently even with a bad internet connection. Supporting an offline mode also becomes a possibility because you can just save all of the actions the user makes and send them to the server at a later time when the user has reconnected to the internet.

The downside

One of the biggest downsides with this implementation is that with some editing of the user’s local storage, a hint request can be erased from the action queue. We decided this was acceptable for a couple of reasons. First, our typical classroom user is unlikely to know how to edit their local storage. Second, even if the user edits their local storage, it is visually obvious to those in the same room that they are up to something. A teacher can easily see students messing around with the chrome devtools and act accordingly.

Conclusion

A client based architecture makes for a much better user experience because a spotty connection does not create a barrier to using our application. In our case, it also made it much harder to cheat on exercise problems and was a great way to make server requests more reliable. Additionally, this architecture makes having an offline mode more feasible.