One of the things many developers love about MongoDB and other NoSQL databases is the ability to store flexible objects in their native format. When you walk around the office singing its praises, experienced developers chime in and tell you that this is a quick path to a hard to maintain application, where changes to the schema in one part of the application break the rest of the application (an overly coupled application). This tends to mean that the more mature a product, the less NoSQL features we can use, and the more it feels like a traditional databases with worse schema validation.

There is better way though, where we can use the flexibility and power of our NoSQL databases, and still have a loosely coupled and easy to maintain application. To do this we need to build services that can handle data dynamically, rather than using typed data everywhere.

The Technology

First up for these conversations we need a modern programming language. While all these tricks work in any language, a modern language with tools that allow us to dynamically set and access properties of object will make our lives easier, and helps with the goal of making our application stack easier to maintain. In my experience Python, NodeJS, and C# all do a good job at this, while Java addicts might have to look elsewhere.
Having a database whose drives/connectors and SDK’s allow for easier access to object properties dynamically, and allow querying without providing a schema are more good-to-haves.

For example if we have MongoDB and C#, the C# driver for Mongodb allows us to query in a number of different ways, including setting our own custom query properties. We can also use the returned documents as a BsonDocument type which allows us to query and access properties of the document dynamically. If push comes to shove, or we need to add compatibility for other components C#’s dynamic objects and generic typing will still allow us to be as flexible as possible.

If we were more in a NodeJS and MongoDB mood we can have even more fun with Typescript/Javascript being very much setup for objects having dynamic access to their properties. The MongoDB driver is also a very dynamic setup with the queries and data access being as dynamic as the language. With this approach to handling data in MongoDB dynamically libraries like mongoose won’t be that helpful, so best get acquainted with the Mongodb NodeJS native driver.

An Idea: NoSQL Insert/Update Service

The easiest thing you can do generically is create a service that insert/updates objects. This is easy to do dynamically as we don’t need to care about anything in the object at all. The inputs for this kind of service is an object, the primary key (if its an update), and a target collection/database/table/… While this service seems of little use, it’s quite helpful if you have an asynchronous microservice architecture where this object can be queued up for inserting without worrying about the fail-safe logic inside your core application code. It also has the nice side effect of allowing us to keep the number of things needing to connect to MonogDB to a minimum, which is great if you are running a small MongoDB Atlas cluster, or another NoSQL tool that has a connection limit.

Let’s Stop Using ReplaceOne

In MongoDB, ReplaceOne is the method we use where we want to replace an existing document with the document we provide. This is great if we are changing a majority of a documents properties or have an unknown number of changes that we want to submit but, many developers (myself included) can get lazy with ReplaceOne and start using it every time we want to update anything about a document. This habit limits us in our ability to use the great dynamic and flexible interface NoSQL. Update operators like MongoDB’s $inc (increment) and others allows us to interact with objects in their native place without having to understand the full object, just the components we need to care about.

Only Get What We Need

Writing apps that have good performance when the data needed for those apps is somewhere else can be quite challenging. Part of getting there is to have good code for talking to your database. MongoDB’s project operator and other NoSQL’s similar features you can only return the components of the object you need. This means that when a schema is changed elsewhere, so long as the components of the objects you need remain your code can still function. We also keep the amount of data needed to be sent down the wire to a minimum, so our database queries return faster

These are just a few ideas of good ways to start getting into handling your data in its most native form.
I will have a few more coming soon.

Until then…
Tim Gray
Coffee to Code