JanusGraph Resources and Scaling

🚧

JanusGraph Deprecation

As of September 2019, Compose will soon no longer support JanusGraph on the platform. Provisioning of new JanusGraph deployments is disabled.

Resources in a JanusGraph deployment are allocated in two dimensions. The first is the JanusGraph Engine Nodes, which perform the graph functions and queries. The second is the Storage nodes, which are Scylla databases set up in a manner for redundancy and high-availability. You may scale these two dimensions independently of each other.

For example, if you have a relatively small data set and need to execute complex queries, you may keep the storage nodes at the default size of 5GB, but choose to scale the memory of the Engine nodes up to make your queries more performant. Alternatively, if your queries are performing well, but your data set has grown, you may increase the size of the storage nodes with out touching the Engine nodes.

Current size and scaling options are found on the Resources page of your JanusGraph deployment.

Vertical Scaling

Both the Engine nodes and the Storage nodes may be vertically scaled with more memory and storage.

🚧

Compose auto-scaling

Compose auto-scaling feature is not in place for the beta release of JanusGraph on Compose. Your deployment will not automatically add resources. All scaling will have to be done manually.

Horizontal Scaling

Additional JanusGraph Engine nodes can be added to your deployment.

🚧

Rolling Restart

All Scaling operations (both horizontal and vertical) in JanusGraph will trigger a rolling restart of your database. There will be some disruption to connections and availability.

Adding Memory to JanusGraph Engine Nodes

In the Resources view of your JanusGraph deployment, the top panel shows a list of configured nodes, backend nodes, and TCP/HAProxy portals. Under JanusGraph Nodes each row displays the name, location, and IP address of the node.

In the lower right of the JanusGraph Nodes panel is a Scale button. To add or reduce memory allocated to the nodes in your deployment, click on it and you will see the Scale JanusGraph Nodes page.

By adjusting the slider, you can raise or lower the memory allocated to the all the JanusGraph nodes. The memory and storage allocation is 1:1 and is available in 256MB memory/256MB storage units.

When the slider is moved, the view will update with the new cost per month for the deployment. Note that the slider will not let you move to a position that is less than your currently used memory.

Clicking Confirm triggers the rescaling of the deployment.

Adding a JanusGraph Engine Node

To add extra nodes, the Resources panel, the Engine Nodes pane has an Modify Nodes button. Click on it and the panel will show the current number of nodes, their combined storage and memory, and a slider for you to increase to your desired number of nodes. Additional nodes will have the same resources as the current nodes in the deployment.

As you move the slider the description of the resources configured to your deployment reflects the current selection.

Click on Confirm to trigger the scaling operation. When the operation is finished your additional engine nodes will appear in the Deployment Topology for your deployment.

Adding Storage to JanusGraph Storage Nodes

Storage Nodes lists your deployment's Scyalla nodes and the storage size of each node. Under that is the location where the nodes reside followed by the name and IP address of each node.

In the lower right is a Scale button. Click on it to change the amount of storage allocated to each node on the Scale Storage Nodes page. By adjusting the slider you can raise or lower the storage allocated to each node. The storage to memory allocation is 10:1 and is available in 5GB storage/510MB memory units.

When the slider is moved, the view will update with the new cost per month for the deployment. Note that the slider will not let you move to a position that is less than your currently used storage.

Clicking Confirm triggers the rescaling of the deployment.

Adding Memory to HAProxy Portals

JanusGraph is configured with two HAProxy capsules to manage and handle incoming connections. These can also be scaled to allow them to handle traffic more efficiently. HAProxy portals are scaled entirely in memory.

Clicking Scale takes you to a slider where you can increase or decrease the memory allocated to the HAProxy portals and see the projected monthly cost of those changes.

Clicking Confirm puts those changes in place.

Adding Additional HAProxy Portals

From the side-bar, in the Security panel, there is a list of the currently configured HAProxy portals.

To add additional portals, click on the Add TCP portal button. A confirmation will appear and clicking on Create will start provisioning the router. Once the process is complete, the Deployment Overview will be updated to show the new router's connection strings.

Scaling Down

Just like you can increase the amount of resources to your deployment, you may also decrease them. Since the scaling of the JanusGraph Engine nodes and the Scylla Storage nodes operate independently, it is possible to shrink one without affecting the other. For example, if you have trimmed the size of your data, but are still running complex queries, you can reduce the Storage node size without reducing the Engine node memory. Or if you have the same amount of data, but have simplified the graph operations, then you can reduce the amount of memory in the Engine nodes while keeping the Storage nodes the same.

The process is much the same as scaling up. On the Deployment Resources panel, click the Scale button, and in the JanusGraph Nodes panel and lower the slider. Note that the slider will not let you move to a position that is less than your currently used memory.

Removing a JanusGraph Engine Node

If you wish to remove an Engine node entirely, click on the Modify Nodes button. Click on it and the panel will show the current number of nodes, their combined storage and memory. You may move the slider to decrease the number of nodes on your deployment.

---

Still Need Help?

If this article didn't solve things, summon a human and get some help!