Andrew Morgan on Databases

At AWS re:Invent, we demonstrated how MongoDB Mobile, MongoDB Stitch, and AWS services could be used to build a cloud-controlled Mars rover – read the overview post for the setup. This post focuses on how Stitch Mobile Sync synchronizes the user commands written to MongoDB Atlas by the Mission Control app with the MongoDB Mobile database embedded in the rover (and how it syncs the data back to Atlas when it’s updated in MongoDB Mobile).

Many people who took part in the demo at re:Invent asked why Mission Control wrote the commands to Atlas rather than sending them directly to the rover. The reason is that you can’t guarantee that the rover will always have network access (and when thousands descended on the expo hall for the Monday evening social, we learned that maintaining a connection over conference WiFi can be just as tricky as maintaining one to Mars). The commands are stored in Atlas and then synchronized to the rover whenever it’s online.

Each rover has a single document in the rover.rovers collection, and each command is stored as an element in the moves array:

{
  "_id" : "5bee1053fdc728f2623e20eb",
  "moves" : [
    {
      "_id" : "5c0e4db5119f6e36c7d06f55",
      "angle" : 90,
      "speed" : 2
    },
    {
      "_id" : "5c0e4dbc119f6e36c7d06f56",
      "angle" : 118,
      "speed" : 3
    },
    {
      "_id" : "5c0e4dc3119f6e36c7d06f57",
      "angle" : 45,
      "speed" : -1
    }
  ],
  "__stitch_sync_version" : {
    "spv" : 1,
    "id" : "2f704b04-2338-4c75-a7cf-d555c94cb556",
    "v" : NumberLong(10313)
  }
}

Stitch Mobile sync automatically pushes the document to the rover’s MongoDB Mobile database, and the rover’s Android app moves the rover in response. Once a move command has been acted on, the app removes it from the array and updates the document in MongoDB Mobile. Stitch Mobile Sync then automatically pushes the change back to Atlas:

doMove(move);
final Document update = new Document("$pull", new Document("moves",
                new Document("_id", move.getId())));
rovers.sync().updateOne(getRoverFilter(), update).addOnCompleteListener(task -> {
  if (!task.isSuccessful()) {
    Log.d(TAG, "failed to update rover document", task.getException());
  }
  try {
    Thread.sleep(MOVE_LOOP_WAIT_TIME_MS);
  } catch (InterruptedException e) {
    e.printStackTrace();
  }
  moveLoop();
}

You may have noticed that the same document is being updated in both Atlas and MongoDB Mobile, raising the possibility of conflicts. A conflict occurs when the document is updated in one database and, before that change has been synced, the same document is updated in the second. Sketchy network connectivity increases the size of the window where this can happen and that makes conflicts more likely.

Fortunately, Stitch Mobile Sync allows the client application to register a conflict handler function. The function is passed both versions of the document and can decide what the ‘winning’ document should look like. The conflict handling could be as simple as “local wins”, but in this case, we take the remote document (which contains at least one command not yet added to the local database) and remove any commands that have already been acted on:

public class RoverActivity extends Activity 
  implements ConflictHandler<Rover> {

  @Override
  public Rover resolveConflict(
    final BsonValue documentId,
    final ChangeEvent<Rover> localEvent,
    final ChangeEvent<Rover> remoteEvent
    ) {
    if (localEvent.getFullDocument().getLastMoveCompleted() == null) {
      return remoteEvent.getFullDocument();
    }
    // Given this sync model consists of a single producer and a
    // single consumer, a conflict can only occur when a production
    // and consumption happens at the same "time". That means
    // that there should always be an overlap of moves during a 
    // conflict and that the last move completed is always present
    // in the remote. Therefore we should trim all moves up to
    // and including the last completed move.
    final Rover localRover = localEvent.getFullDocument();
    final String lastMoveCompleted = localRover.getLastMoveCompleted();
    final Rover remoteRover = remoteEvent.getFullDocument();
    final List<Move> nextMoves = new ArrayList<>(remoteRover
      .getMoves().size());
    boolean caughtUp = false;
    for (final Move move : remoteRover.getMoves()) {
      if (move.getId().equals(lastMoveCompleted)) {
        caughtUp = true;
      } else 
      {
        if (caughtUp) {
          nextMoves.add(move);
        }
      }
    }
    return new Rover(localRover, nextMoves);
  }
}

The next post in this series looks at how Stitch Functions can be used to provide an aggregated view of the rover’s sensor data.

If you can’t wait then you can find all of the code in the Stitch Rover GitHub repo.

At AWS re:Invent, we demonstrated how MongoDB Mobile, MongoDB Stitch, and AWS services could be used to build a cloud-controlled Mars rover – read the overview post for the setup. This post focuses on how the mission control app records the user commands in MongoDB Atlas.

Mission Control is a simple web application that we ran on an iPad. The web app takes commands from the user through its UI which displays directions for the rover to travel in. Each command sets the rover off in that direction for a short fixed amount of time.

Rather than sending the command directly to the rover, and as Mars is a long way away and network connections are not always reliable, the app stores the commands in an array within the rover’s document in Atlas – in that way, commands can be queued up and sent to the rover as soon as it’s back online.

This is what the document looks like after a few commands have been submitted (but not yet acted on by the rover):

{
    "_id" : "5bee1053fdc728f2623e20eb",
    "moves" : [
        {
            "_id" : "5c0e4db5119f6e36c7d06f55",
            "angle" : 90,
            "speed" : 2
        },
        {
            "_id" : "5c0e4dbc119f6e36c7d06f56",
            "angle" : 118,
            "speed" : 3
        },
        {
            "_id" : "5c0e4dc3119f6e36c7d06f57",
            "angle" : 45,
            "speed" : -1
        }
    ],
    "__stitch_sync_version" : {
        "spv" : 1,
        "id" : "2f704b04-2338-4c75-a7cf-d555c94cb556",
        "v" : NumberLong(10313)
    }
}

A traditional way of access the database from the frontend app would be to stand up an app server, implement a custom REST API and data access control layer, and then send the commands to it from the frontend app. MongoDB Stitch massively simplifies that workflow by letting a web (or mobile) app execute MongoDB Query Language operations directly – removing the need for thousands of lines of boilerplate code.

When we explained this access model to demo visitors, some were nervous about this approach as we’ve been taught that the schema and database access should be hidden from the frontend (remember SQL injection attacks?). Fortunately, Stitch QueryAnywhere is made up of two components – the first is the Stitch SDK that enables direct access to the database, the second is the sophisticated, fine-grained user access controls provided by the Stitch service. If that doesn’t allay your fears then you have the option to obfuscate the schema by accessing the database through Stitch Functions.

Updating the document from the web app is a cinch with Stitch. The first step is to import the Stitch SDK:

<head>
  <script src="https://s3.amazonaws.com/stitch-sdks/js/bundles/4.0.15-0/stitch.js"></script>
  <script src="https://s3.amazonaws.com/stitch-sdks/bson/bson.bundle.js"></script>
</head>

The second step is to create the object/document for the command and push it onto the moves array within this rover’s document in the rover.rovers collection:

function pushMoveToRover(roverId) {
    const client = stitch.Stitch.defaultAppClient;
    const coll = client.getServiceClient(stitch.RemoteMongoClient.factory,
        'mongodb-atlas').db('rover').collection('rovers');
    const angle = parseInt($("#angle").val());
    const speed = parseInt($("#speed").val());
    if (isNaN(angle) || isNaN(speed)) {
        return;
    }
    const move = {"_id": new BSON.ObjectID().toHexString(), angle, speed};
    coll.updateOne({"_id": roverId}, {"$push": { "moves": move }, 
        "$inc": {"__stitch_sync_version.v": 1}});
}

If you are used to working with the MongoDB Query Language from an app server, that should seem very familiar. The only thing that might catch your eye is that __stitch_sync_version.v. That’s part of how this update will get to the MongoDB Mobile database embedded in the rover. We’ll explain that in the next part.

If you can’t wait then you can find all of the code in the Stitch Rover GitHub repo.