Fixing a Performance Anti-pattern With Amazon DynamoDB and Lessons Learned | by Amanda Quint | Jul, 2022

My first steps with Amazon DyamoDB

Picture by Waren Brasse on Unsplash

I’ve not too long ago been engaged on a private challenge the place I’ve determined to make use of and study extra about DynamoDB. Whereas I’ve used Dynamo a bit for small issues prior to now, the vast majority of my previous expertise has been with relational databases, and it’s taken me a while to get my head round working with NoSQL.

DynamoDB is Amazon Internet Providers’ absolutely managed NoSQL database.

Not like relational databases, which retailer their knowledge in structured rows and columns, Dynamo knowledge is saved in key-value pairs. Which means that the information you retailer in Dynamo is unstructured in comparison with what you’d often see in a relational database. The one schema that it’s a must to outline is your desk keys: the partition key and the kind key. Past that, your knowledge could be schemaless (though your utility nonetheless has to know deal with it).

In my challenge, I’m utilizing Python and accessing Dynamo by way of the boto3 library. That is operating on AWS Lambda. In my code, I’ve a knowledge mannequin, assortment , which I wanted to do primary CRUD (Create, Learn, Replace, Delete) operations on.

My code appeared one thing like this:

Do you see my mistake?

With each database operation I used to be doing, I used to be re-initializing the boto3 useful resource and the Dynamo Desk.

It was simple to do (particularly with GitHub Copilot principally writing these features for me), and I didn’t actually give it some thought on the time. Nonetheless, whereas reviewing some code, my fiancé, Myles Loffler, identified to me that that is actually an anti-pattern when working with boto3 clients, particularly on Lambda.

Observe: Whereas I hit this anti-pattern whereas working with Dynamo, because of the nature of the CRUD operations — you’ll be able to run into it any time you’re re-initializing the identical boto3 useful resource time and again!

Fortunately, this was each simple to repair, and fixing it made my code higher general!

I created a brand new utilities file to deal with my database connections and added the next features:

On this instance, the Dynamo desk title is now retrieved by way of a separate perform saved in settings.

Then, I refactored my CRUD features to only use get_dynamo_table as an alternative of utilizing boto3 instantly.

Now I can simply name the brand new get_dynamo_table perform.

You could have seen the usage of world variables in my database utility. Whereas that is typically thought of a foul follow, I used them primarily based on the Lambda greatest follow that implies taking advantage of execution environment reuse.

This made three main variations to my code:

Efficiency

That is clearly the large one!

To check, I deployed two variations of the applying, one with the outdated code and one with the brand new, and ran by a number of the workflows that resulted in CRUD operations.

Listed here are some examples of traces by my utility with out this alteration utilized to it. This motion included deleting an merchandise and a question:

A Delete and Question motion prior to creating this alteration, taking 84ms and 76ms, respectively.

And listed here are traces of the identical two actions (delete and question) by the applying with this alteration utilized:

The identical Delete and Question motion after making this alteration, taking 7ms and 12ms, respectively.

Though we’re solely speaking about fractions of a second, these delays add up over time! The applying feels notably quicker and extra responsive after this alteration was utilized.

Cleanliness

This strategy is each simpler to learn and, in the long run, will certainly be much less code in my challenge. I don’t should import boto3and entry the setting variables every time I must entry the database. There’s a single perform to entry Dynamo now, so if I want to alter one thing, I’ll solely have to alter it in a single place.

Testability

The exams are cleaner too. Earlier than, I needed to mock boto3 in each location, which was additionally pretty difficult, as I used to be mocking the return worth of a return worth. Now I can simply mock get_dynamo_table , which is much more easy and straightforward to learn.

Reusability

The brand new features in utils.db don’t care concerning the desk title in any respect — it’s retrieved by the get_dynamo_table_name perform in settings. As my challenge grows, this code is reuseable throughout different Lambdas that will use totally different Dynamo tables.

More Posts