When building a serverless first platform it’s very hard to ignore the compelling feature set offered by AWS AppSync and DynamoDB. Each service has their own list of great features, but the magic lies when you combine these two services together. I have often said that the sweet spot when working with AWS is when you start to play AWS lego; use the building blocks provided by AWS to “click” the services together using configuration rather than code.
We are using these two services to help build a fully serverless insurance platform and in this article I’m going to share some lessons we have learnt along the way.
1. Determine if you want to use single or multi table design with DynamoDB
One thing the serverless community can agree on is that Alex DeBrie’s DynamoDB book is a must read if you are starting your journey with DynamoDB. DeBrie lays out the best practices for working with DynamoDB, single table design and modelling your tables. He also gives advice on when it may not be appropriate to use single table design, including when you are working with GraphQL.
This is a decision that you will want to make early on in your project. At this stage it's almost universally accepted that single table design is the right approach for most use cases.
For our project we decided on a mixed approach. Where appropriate, we will use single table design to group closely related items together. Closely related items could be defined as items that are likely to be queried together. Determining your access patterns before working on a feature will help you to better understand the relationship between your entities and the types of queries you are going to use.
For loosely related items that sometimes are queried together but also separately, we typically store those entities in separate tables. We rely on AppSync to resolve our entities and aggregate the data into a single response for the client.
We would also use separate tables if the entity stored in a table is versioned. Rather than storing multiple different entities in a table, we would instead store multiple versions of an entity. Keeping the versioned entity in its own table helps to reduce complexity of that table and makes it easier to work with when resolving data from the table.
2. Model your tables following single table design best practices
This point applies regardless if you are using a single or multiple tables!
Part of single table design is decoupling your partition key and sort key from the actual record you are trying to store.
Let's take a record that has an ID, it would be tempting to make this field your partition key and perhaps Date Created as your sort key.
This approach makes your table a lot less flexible for the future. Perhaps you will realise that storing multiple entities in this table is a good idea, by having a partition key named ID it makes it harder to put something else in that field, for example a username.
Instead store the entity type and identifier in a field called partitionKey and sortKey. This is commonplace in single table design, but is still a good practice to follow when using multiple tables.
Some developers prefer to use PK and SK as the names for these fields. At Instil, we tend to avoid acronyms where possible and prefer to use clear and unambiguous naming conventions. At scale this might be something to consider when you want to shorten your attribute names to save on storage but that isn’t an issue for our particular project.
One side point is to always include a sort key field even if you don’t need it right away. You can't add a sort key after a table has been created!
3. Use AppSync DynamoDB resolvers
A game changing moment for our platform was when we switched from using Lambda resolvers to DynamoDB resolvers. The ability to connect your API directly to your database delivers performance that a Lambda cannot match. It also enables developers to quickly add new features to your API whilst adding very little "code".
Multi-table design makes this approach much simpler, it removes the need for complex VTL resolver templates that transform data returned from a table following single table design. AppSync will effortlessly query your tables in parallel and aggregate the responses into the format that matches your GraphQL schema.
4. Be wary of the N+1 Problem
The N+1 problem isn’t specific to AppSync, it can occur in any GraphQL system when one top level query produces N items whose type contains a field that must also be resolved.
For example, in the query below, listOrders returns N items, and each item has a reviews field that must be resolved in turn.
query getOrders {
listOrders { // 1 top level query
items { // N items
id
reviews { // N additional queries
author
}
}
}
}AWS has some advice on how to work around this - https://aws.amazon.com/blogs/mobile/introducing-configurable-batching-size-for-aws-appsync-lambda-resolvers/
But I think this should instead be factored into your GraphQL schema and DynamoDB table design to eliminate the N+1 problem where possible. Perhaps there is an argument here to store the items and reviews in a single table to aid with querying that data, but other entities can still live in different tables.
Hopefully you have found these points useful as you continue your development journey with AppSync and DynamoDB. We have found that the combination of these two services has provided a flexible architecture that we have been able to extend and refactor over time as we deliver new features.
We develop custom cloud and mobile software products for some of the world's leading brands. If you would like to learn more get in touch.
