sed in progress / thoughts are provided "as is", without warranty of any kind

Network Storage Considerations (2023)

2023-04-01

The Storage

As an avid photographer and less avid videographer, I have found a file workflow that works for me. And at the heart of that workflow is a lot of storage and file organization. One of the end results for that workflow is that I am able to quickly and easily find my historical captures in explorer.exe, Finder, Lightroom, or even Apple Photos.

Until recently, that workflow has worked. But times change, and homelab electricity draw has increased–rather significantly. I’ve measured my oldest FreeNAS server (hereinafter referred to as “8-disk”) and it was often seen idling at 170W–ramping up to a sustained 230W when in the middle of a simple rsync task. As a point of comparsion, my newer TrueNAS server (hereinafter referred to as “6-disk”) hovers around 70-90W with similar activity.

I’m now looking at retiring my power-hungry 8-disk server and its dual Xeon motherboard socketed with 2x E5-2630L v2 60W TDP CPUs.

The complication, here, is that the 8-disk serves as my primary server which is exposing SMB shares to my Windows and macOS clients. It’s also replicating its (read-only) snapshots to the 6-disk server.

After some minor changes in order to facilitate its retirement, I effectively now have the below (where I replaced the read-only snapshot replications with weekly rsync tasks):

sequenceDiagram participant B2 participant Clients participant 8-disk participant 6-disk Clients->8-disk: SMB client R/W Note over Clients,8-disk: Ex: Lightroom, DaVinci 8-disk->>B2: Cloud Sync photo dataset 8-disk->>6-disk: rsync all datasets

In the diagram above, it’s more more obvious that the 8-disk is a central component to my general file storage workflow. That’s where files are first written to. That’s where files are first read from. That server is also scheduling the various backup tasks to both the remote Backblaze B2 bucket as well as the local 6-disk TrueNAS).

So, the initial reaction was to swap the 8-disk with a Synology NAS (or similar appliance) as I now have a bias for optimizing on power draw. In comparison to the observed 170-230W, a DS1621+ sits around 26-52W. That would fit comfortably in the role of a local-secondary, promoting the 6-disk TrueNAS with its Xeon D-1521. And in that role, the Synology has no need to be fast. We can leave that to the 6-disk TrueNAS and its 10 Gbps interfaces.

That setup would look something like the below diagram, where the 6-disk TrueNAS is promoted to the role of local-primary, with the Synology as the (new) local-secondary server:

sequenceDiagram participant Archival participant Clients participant 6-disk participant Synology Clients->6-disk: SMB client R/W Note over Clients,6-disk: Ex: Lightroom, DaVinci 6-disk->>Archival: Cloud Sync photo dataset 6-disk->>Synology: rsync all datasets

Note: The write to cloud-based Archival storage could be initiated by the Synology via the Cloud Sync plugin, if I wanted to make this workflow more waterfall.

In the middle of these changes, I would also reconsider using B2, in favor of the GCS Archive class due to the pricing. It’s similar to the more popular AWS Glacier. Of course, the underlying assumption is that the cloud backup is used for disaster recovery purposes only. To date, there’s only been one instance where I’ve needed to recover files from B2, and that was when I did not have a local-secondary storage to recover from.

And while I want to remain a TrueNAS household, the draw of purpose-built, lower power draw NAS appliances is difficult to say no to.

The deltas are projected to be:

  1. Replace: 170-230W 8-disk FreeNAS -> 26-52W DS1621+
  2. Replace: Backblaze B2 @ $5/TB/mo) -> Archive class storage @ $1.20/TB/mo (Note: this does not include retrieval costs)

What’s Next

I’ll move forward with the above plan: Replacing the 8-disk FreeNAS with a Synology NAS (and B2 with Archive-class storage). Afterwards, I can look look into separating the server and app-hosting functionality out of the 6-disk, into a dedicated low power server of its own–treating the NAS as “just a NAS”.